JP4519364B2 - Panorama image display method, software and apparatus - Google Patents

Description

[0001]
[Technical field to which the invention belongs]
The present invention relates to a panoramic image display method, software, and apparatus that combine a plurality of image data to generate and display a panoramic image with a wide field of view, and in particular, a technique suitable for correcting a measurement defect portion of a panoramic image. It is about.
[0002]
[Prior art]
A conventional panoramic image display method and apparatus is disclosed in Japanese Patent Laid-Open No. 2000-217815 as a method and apparatus for synthesizing a panoramic image by synthesizing ultrasonic images.
Here, an imaging unit that converts ultrasonic data acquired from the probe into an image, a moving image display function that displays the image obtained by the imaging unit as a moving image, and a movement amount between images generated by the probe operation are shown. A computer with a function to detect the amount of movement to detect and a function to detect the amount of rotation between the tomographic images generated by the probe operation, and a movement in which the tomographic image sequence being diagnosed is detected by the computer in parallel with the video display By providing a panoramic image display means for displaying a panoramic image connected in a plane according to the amount and the rotation amount, a panoramic image can be created in real time from the tomographic image sequence obtained by the probe operation.
[0003]
In addition, the panorama image creation status can be confirmed every moment on the display, and the failure of the probe operation can be corrected.
[0004]
[Problems to be solved by the invention]
However, since the conventional technology does not have a function for editing the panorama image after creation, even if there is a defective portion with respect to the measurement of the panorama image, the entire measurement area of the panorama image is measured again from the beginning. There was a need. In addition, the re-measurement included a possibility of failing again (resulting in measurement failure). In order to obtain a panoramic image that is totally clear from the above, it may be necessary to measure again and again, which means that it takes longer time to examine one patient. However, there is a problem that the efficiency is extremely low as a diagnostic imaging apparatus for a hospital that has to examine a patient.
[0005]
In a specific example, the cyst of the chest not only moves the probe, but also due to patient movement due to patient breathing and heartbeat, making it difficult to set the probe position properly. There is also a clinical result that the measurement was repeated ten times or more to measure the panoramic image.
[0006]
In addition, in order to create a clear panoramic image, it is necessary to move the probe at an appropriate speed so that the computer can obtain image information that makes it easy to calculate the amount of movement and rotation of the probe. There is also.
[0007]
In addition, if the strength with which the probe is brought into contact with the diagnostic part changes, the image may be distorted, which may result in measurement failure. Must be kept. In addition, the appropriate strength with which the probe is applied varies depending on the patient's physique and the diagnosis site, so it is not easy for operators such as doctors and clinical technologists who are unaccustomed to using the diagnostic device to know the appropriate strength. Absent.
[0008]
Even an operator who knows the appropriate speed to move the probe and the appropriate strength to bring the probe into contact with the diagnostic part covers a wide range of the diagnostic part and takes a long panoramic image measurement time. If so, there is a high probability that measurement errors will occur due to patient movement or hand shake with the probe, etc., and there is a risk that the advantage of panoramic images that can create diagnostic images over a wide range may not be fully utilized. .
[0009]
In addition, when the diagnosis site is a panoramic image over a wide area and measurement is partially defective, it can be said that taking the time from the beginning again and measuring the whole again wastes valuable examination time.
[0010]
Therefore, a first object of the present invention is to provide a panoramic image display method capable of efficiently correcting a measurement defect portion of a panoramic image.
[0011]
A second object of the present invention is to provide panoramic image display software capable of efficiently correcting a measurement defect portion of a panoramic image.
[0012]
A third object of the present invention is to provide a panoramic image display device capable of efficiently correcting a measurement defect portion of a panoramic image.
[0013]
[Means for Solving the Problems]
The first object is to obtain image data of a plurality of image frames that are close in position, to detect positional relationship data between the plurality of image frames, and to use the detected positional relationship data The step of combining the plurality of image frames to form panoramic image data and the step of obtaining the image data to form the panoramic image data at the same position to the step of forming the panoramic image data are repeated a plurality of times. A step of displaying individual panoramic image data obtained by the repetition on a display device, a step of setting missing information for the displayed individual panoramic image, and excluding the set missing information A step of synthesizing the panorama image data; and It is achieved by the panorama image display method comprising the steps of displaying on the display device as Ma image.
[0014]
The second object is to obtain image data acquisition means for obtaining image data of a plurality of image frames, position relation data detection means for detecting position relation data between the plurality of image frames, and the detected position relation. Panorama image data forming means for combining the plurality of image frames using relational data to form panoramic image data, display means for displaying the formed panoramic image data as a panoramic image, and the image data acquiring means through display A computer software used in a panorama display device comprising: control means for controlling the means, wherein the image data acquisition means obtains image data of a plurality of image frames in close proximity; and the position A procedure in which relationship data detection means detects positional relationship data between the plurality of image frames, and the panoramic image; A data forming unit that combines the plurality of image frames using the detected positional relationship data to form panoramic image data; and the control unit forms the panoramic image data at the same position. A procedure for obtaining image data or a procedure for forming the panorama image data a plurality of times, a procedure for displaying individual panorama image data obtained by repeating the procedure on the display means, and the display A procedure for setting missing information for each individual panoramic image, a procedure for synthesizing panoramic image data excluding the set missing information, and a procedure for causing the display means to display the synthesized panoramic image data as a panoramic image; , it is achieved by the panorama image display software consisting program for realizing on a computer That.
[0015]
Further, the third object is to obtain means for obtaining image data of a plurality of image frames that are close in position, means for detecting position relation data between the plurality of image frames, and the detected position relation data. Means for combining the plurality of image frames to form panoramic image data, and obtaining the image data to form the panoramic image data at the same position to forming the panoramic image data a plurality of times. Means for repeating, means for displaying individual panoramic image data obtained by the repetition on a display device, means for setting missing information for each displayed panoramic image, and the set missing information Means for synthesizing panoramic image data excluding the image, and displaying the synthesized panoramic image data as a panoramic image It is achieved by a panoramic image display apparatus characterized by comprising a means for displaying the location.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Preferred embodiments of a panoramic image display method, software and apparatus of the present invention will be described with reference to the drawings.
[0017]
Moreover, although the apparatus which obtains the moving image employ | adopted here is an ultrasonic diagnostic apparatus, the domestic animal which uses ultrasonic waves, X-rays, etc. in addition to various medical image diagnostic apparatuses, such as X-ray apparatus which can collect a moving image similarly It may be an image diagnosis apparatus for livestock that diagnoses the inside of an animal.
[0018]
FIG. 2 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus in which the panoramic image display method of the present invention is employed.
As shown in FIG. 2, the ultrasonic diagnostic apparatus includes a probe 201, an ultrasonic transmission / reception unit 202, a received wave phasing unit 203, a digital scan converter (DSC) unit 204, a panoramic image processing unit 205, An image display unit 206, a print output unit 207, an image memory 208, an overall control unit 209, an operation panel 210, and an interface (I / F) unit 211.
[0019]
The probe 201 transmits an ultrasonic wave to the patient's diagnostic site and receives a reflected echo from the diagnostic site. The ultrasonic transmission / reception unit 202 drives the probe 201 and amplifies the reflected echo received by the probe 201. The phasing unit 203 phasings the reflected echo signal amplified by the ultrasonic transmission / reception unit 202. The DSC unit 204 converts the phased reflected echo signal into an ultrasonic image of at least one mode such as B mode, M mode, and Doppler mode. The panorama image processing unit 205 has a configuration newly added to the present invention, obtains panorama image data obtained by combining a plurality of frames of ultrasonic image data a plurality of times, and synthesizes the panorama image data. The image display unit 206 displays the synthesized panoramic image data as a panoramic image. The print output unit 207 is a video printer that prints a display image on film or paper. The storage device 208 is any storage medium such as a magnetic disk or a magneto-optical disk that stores ultrasonic images and panoramic images. The I / F unit 209 is connected to a network facility such as a telephone line provided outside, and various types of data including ultrasonic image data, panoramic image data, and programs are transmitted from the ultrasonic diagnostic apparatus to the outside or externally via the network facility. Can communicate with the ultrasonic diagnostic apparatus. The operation panel 210 is an input device including a keyboard and a pointing device for inputting various parameters set by the ultrasonic diagnostic apparatus for measurement. The overall control unit 211 transmits / receives ultrasonic waves of the ultrasonic transmission / reception unit 202, phasing of the phasing unit 203, selection of a mode of the DSC 204, synthesis of the panoramic image processing unit 205, and image display unit 206 according to the input of the operation panel 210. Display, printing of the print output unit 207, storage of the storage device 208, and communication of the I / F unit 209 are controlled.
[0020]
The panorama image processing unit 205 further includes a panorama image storage unit 205a, a comparison unit 205b, a panorama image composition processing unit 205c, a composite panorama image storage unit 205d, and a panorama image composition control unit 205e.
[0021]
The panoramic image unit 205a creates and stores panoramic image data a plurality of times (here, three times) for the same diagnosis part of the patient. As a method for creating this panoramic image, any of the techniques described in Japanese Patent Laid-Open Nos. 8-280688 and 2000-217815 may be used. The panorama image data created in this way is obtained three times, and the panorama image data created in the first, second and third times are stored in the frame memories 205a-1, 205a-2 and 205a-3, respectively. . The comparison unit 205b determines whether there is a protruding pixel value among the pixels at the same position in the three panoramic image data. As a result, if there is a protruding pixel value, the non-protruding pixel value of other panoramic image data is used for the subsequent synthesis process, and if it is not a protruding pixel value, the pixel value is used for the subsequent synthesis process. Output the pixel value as possible. This comparison is performed for all the pixels constituting the panoramic image data. The panorama image composition processing unit 205c collects the non-projecting pixels, synthesizes the panorama image data, and outputs composite panorama image data. If the combined panorama image data is not a pixel value with a protruding pixel value at the same position, they may be averaged and output. By outputting this addition average, synthesized panoramic image data with little correlation noise can be obtained. The composite panorama image storage unit 205d stores the composite panorama image output to the panorama image composition processing unit 205c. The panorama image composition control unit 205e controls storage of the panorama image storage unit 205a, comparison operation of the comparison unit 205b, composition of the panorama image composition processing unit 205c, and storage of the composite panorama image storage unit 205d.
[0022]
Next, a procedure for synthesizing a panoramic image will be described with reference to FIGS. FIG. 1 is a flowchart for explaining an embodiment of a panoramic image display method of the present invention, and FIG. 3 is a diagram showing a display example of the panoramic image display method of FIG.
[0023]
A panoramic image measurement count N (here, 3 times) is set (step 101). The loop counter I related to the number of times of measurement N is set to 0 and initialized (step 102). A panoramic image is measured (step 103). The measured panoramic image is stored in the frame memory. The first time is stored in the frame memory 205a-1, the second time is stored in the frame memory 205a-2, and the third time is stored in the frame memory 205a-3 (step 104). The loop counter I is updated by adding 1 to the loop counter I (step 105). The measurement number N is compared with the loop counter I. If the loop counter I has reached the set measurement number N, the process branches to Yes and proceeds to the next step 107. If not, the process branches to No and repeats steps 103 to 105. (Step 106). Next, the total number of pixels M for reading each panoramic image measured three times is set. This setting may be input by the operator to the operation panel 210. However, if the panoramic image matrix size is predetermined such as 256 × 128, the number of pixels may be set to 32768 ( Step 107). The loop counter J related to the number of pixels M is set to 0 and initialized (step 108). The pixel data of the panoramic image measured at the first time, the second time, and the third time are read as D1, D2, and D3, respectively (step 109). The pixel values D1, D2, and D3 are respectively compared to determine whether or not there is something significantly different from the other two. If there is, the process branches to step 111, and if not, the process branches to step 112 (step 110). If it is determined that there are different pixel values, the pixel position (address) Ae of the panoramic image of the measurement is stored as an error pixel. If the pixel values of the remaining two panoramic images are substantially the same, one of them is set as the pixel value. Further, if the pixel values of the two panoramic images are subjected to an addition averaging process, and the value obtained by the addition averaging process is used as a pixel value, the influence of the correlation noise can be removed (step 111). When it is determined that there is no different pixel value, the pixel value is representative of any one of the three panoramic images. Further, if the pixel values of the three panoramic images are subjected to an addition averaging process, and the value obtained by the addition averaging process is used as a pixel value, the influence of correlation noise can be removed. Then, the pixel value or the addition average value obtained in this step or step 111 is stored in the composite panorama image storage unit 205d (step 112). The loop counter J is updated by adding 1 to the loop counter J (step 113). The number of pixels M is compared with the loop counter J. If the loop counter J has reached the set number of pixels M, the process branches to Yes and proceeds to the next step 115. If not, the process branches to No and repeats steps 109 to 113. (Step 114). Finally, the composite panorama image data is displayed on the image display unit 206 as shown in FIG. Here, a panorama image 31 measured first time, a panorama image 32 measured second time, and a panorama image 33 measured third time are displayed, and measurement-defective areas 34, 35, and 36 are respectively displayed at different positions. It is displayed superimposed on the panoramic images 31-33. These measurement defect areas 34 to 36 are created based on Ae in step 111. In addition, the cursor 37 of the pointing device can arbitrarily correct the position of each measurement failure. Then, the composite panoramic image 38 is displayed through the procedures of steps 101 to 114. In this example, the panorama image measured at each time and the composite panorama image are displayed side by side. However, the panorama image measured at each time and the composite panorama image may be switched and displayed. If display of the process in the middle is unnecessary, only the composite panorama image may be displayed. These side-by-side display and switching display can be arbitrarily set by the operator from the operation panel 210 (step 115).
[0024]
In the determination of the protruding value, the measured panoramic image data is arranged or switched and displayed, and broken lines or the like surrounding the plurality of pixel areas protruding by the operator are pointed to the mouse, trackball, joystick or the like. The “manual” method that enables input by the device and the average position of the pixels of the measured panoramic image data are added and averaged to obtain the average value of each pixel value, and the value is projected from the average value. Obtained by the "Full Auto" method and the "Full Auto" method of displaying an image by superimposing a broken line or the like surrounding the region formed by the selected pixel value on the panoramic image data having the protruding pixel value. There is a “semi-automatic” method of correcting a protruding pixel area of a panoramic image in a “manual” manner. These “manual”, “fully automatic”, and “semi-automatic” methods can be arbitrarily selected. If the pixel value is a protruding pixel value, the pixel value is recorded and the pixel value is excluded from the synthesis process. This operation is performed on all the pixels constituting the panoramic image. In this way, a panoramic image that can be used for diagnosis from the synthesized panoramic image data can be displayed on the display device.
[0025]
In addition, even if the panorama image measurement failure portion is not a panorama image measurement, either the measurement failure portion is newly obtained as moving image data, or the moving image data obtained in the past is read out, and the measurement failure portion of the panorama image is selected. Such moving image data can also be synthesized.
[0026]
Further, the above method can be realized by software in which the above procedure is described in any program language if the overall control unit is a CPU. This software can be stored in a storage medium, and can also be distributed by a network facility conforming to any communication standard such as the Internet.
[0027]
Further, although the present embodiment has been described by taking a moving image as an example, it goes without saying that a so-called still image is also included.
[0028]
As described above, according to the present embodiment, the effects of the following items can be exhibited.
[0029]
(1) As a result of creating a panorama image several times, even if a measurement failure occurs in part, if the parts are different from each other, combining the respective panorama images saves the trouble of re-measurement. The time spent on creation is reduced and the throughput of the examination is improved.
[0030]
(2) Even when trying to create a panoramic image over a wide range, the result of creating multiple (3) times is combined, so the probability of measurement failure is reduced, so the operator is proficient in operating the panorama meter. Even if not, it does not reduce the efficiency of measurement.
[0031]
(3) By creating a plurality of panoramic images and displaying them side by side on the screen or switching them, the comparison of the respective images becomes concise and clear, and the most clearly measured panoramic image can be selected. Further, it becomes possible to obtain a panoramic image with higher image quality and higher reliability.
[0032]
(4) By displaying a broken line over the image range to be corrected, the operator can see which range is combined and how to check it, so it can be seen that there are no artifacts and prevent misdiagnosis. it can.
[0033]
(5) If the “semi-automatic” method is used, the image range to be corrected obtained by the “full-automatic” method can be changed “manually”, so that it is possible to cope with complicated cases where automatic determination is difficult.
[0034]
【The invention's effect】
The present invention has an effect of providing a panoramic image display method, software, and apparatus capable of efficiently correcting a measurement defect portion of a panoramic image.
[Brief description of the drawings]
FIG. 1 is a flowchart illustrating an embodiment of a panoramic image display method according to the present invention.
2 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus in which the panoramic image display method of FIG. 1 is adopted.
FIG. 3 is a view showing a display example of the panoramic image display method of FIG. 1;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 201 ... Probe, 202 ... Ultrasonic transmission / reception part, 203 ... Received wave phasing part, 204 ... DSC part, 205 ... Image processing part, 206 ... Image display part, 207 ... Print output part, 208 ... Memory | storage device, 209 Reference control unit 210 Control panel 211 I / F unit 205a Wide field processing unit 205a-1 to 205a-3 Frame memory 205b Correlation processing unit 205c Wide field synthesis processing unit 205d: Composite wide field frame memory unit, 205e: Wide field image control unit

Claims (7)

  Obtaining image data of a plurality of image frames that are close in position, detecting positional relationship data between the plurality of image frames, and using the detected positional relationship data to determine the plurality of image frames. A step of combining to form panoramic image data, a step of obtaining the image data to form the panoramic image data at the same position, or a step of repeating the step of forming the panoramic image data a plurality of times. Displaying the individual panoramic image data obtained by the above-mentioned steps on the display device, setting missing information on the displayed individual panoramic images, and synthesizing the panoramic image data excluding the set missing information. And the synthesized panoramic image data as a panoramic image Panorama image display method comprising the steps of displaying the shown device.   The panoramic image display method according to claim 1, further comprising a step of arranging or switching the panoramic image data before and after the composition on the same screen and displaying them as a panoramic image.   3. The panorama image display method according to claim 2, wherein the panorama image data before synthesis is displayed by superimposing the missing information of the panorama image data on the panorama image data.   In the superposition of the panoramic image data before synthesis and the missing information, the step of obtaining new image data of an image frame for at least a portion matching the missing portion, and the newly obtained image frame The panoramic image display method according to claim 3, wherein the panoramic image is generated by combining the image data of the above and the panoramic image data before the addition averaging.   The step of reading out image data of image frames obtained in the past for at least a portion that matches the missing portion in displaying the image by superimposing the panoramic image data before synthesis and the missing information, and the read image 4. The panoramic image display method according to claim 3, wherein a panoramic image is generated by combining image data of a frame and panoramic image data before the averaging. Image data acquisition means for obtaining image data of a plurality of image frames, positional relation data detection means for detecting positional relation data between the plurality of image frames, and the plurality of images using the detected positional relation data Panoramic image data forming means for combining frames to form panoramic image data; display means for displaying the formed panoramic image data as a panoramic image; and control means for controlling the image data acquisition means to display means. Computer software used for a panoramic display device comprising:
A procedure in which the image data acquisition means obtains image data of a plurality of image frames close in position;
A procedure in which the positional relationship data detecting means detects positional relationship data between the plurality of image frames;
The panorama image data forming means combining the plurality of image frames using the detected positional relationship data to form panorama image data;
A procedure in which the control means obtains the image data in order to form the panoramic image data at the same position, or a procedure in which the procedure to form the panoramic image data is repeated a plurality of times, and the individual obtained by repeating the procedure. Displaying the panorama image data on the display means, a procedure for setting missing information for each displayed panoramic image, a procedure for synthesizing panoramic image data excluding the set missing information, and the synthesis Panorama image display software comprising a program for causing a computer to display the panorama image data as a panorama image on the display means .   Means for obtaining image data of a plurality of image frames that are close in position; means for detecting positional relationship data between the plurality of image frames; and using the detected positional relationship data to determine the plurality of image frames. Means for combining and forming panoramic image data; means for obtaining the image data to form the panoramic image data at the same position to repeating the step of forming the panoramic image data a plurality of times; Means for displaying the individual panorama image data obtained by the above-mentioned display device, means for setting missing information for the displayed individual panorama image, and means for synthesizing the panorama image data excluding the set missing information And means for displaying the synthesized panoramic image data on the display device as a panoramic image. Panoramic image display apparatus characterized by was e.

Images