JP2003339703A - Apparatus and system for processing ultrasonic image signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that a conventional ultrasonic diagnostic apparatus can obtain information of the section of an object from an image, but the section becomes a front surface image so that the image might be not preferably different from that at an observer's viewing point and a system for conducting a stereoscopic reconstruction has a large apparatus scale to need huge calculations. <P>SOLUTION: An apparatus for processing an ultrasonic image signal comprises: a scanner having a three-dimensional information retrieving unit 2 mechanically coupled to an electronic scanning type probe 1; an electronic scanning type ultrasonic diagnostic unit 3; an A-D converter 11 for digitizing an obtained video signal; a switch 12 for selecting a retrieved image signal or an image signal from a magneto-optic disk 19; a perspective projection processing circuit 13 for processing to coordinate-convert to perspectively project the selected image signal; an MPEG encoder 17 for compressing the perspective projection processed or unprocessed image signal to an MPEG format; and a TV monitor 21 for displaying the processed and the unprocessed image signal, an FMD (face mounted display) 22 and the disk 19 for recording the compressed image signal. A system for processing the ultrasonic image signal is provided. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and its system for three-dimensionally displaying a tomographic plane of an object to be observed by using an ultrasonic image signal.

[0002]

2. Description of the Related Art Generally, an ultrasonic diagnostic apparatus can be roughly classified into linear, convex, and sector types according to the shape of its probe. However, any type of apparatus can non-invasively examine the inside of an observation object. It is characterized in that it can be observed, and has been widely used for diagnosis, particularly in the medical field.

Further, a video signal from an ultrasonic diagnostic apparatus obtained by a three-dimensional scanner using an optical encoder is digitally converted and taken into a memory board to select and calculate data necessary for creating a three-dimensional image. , There is a system that performs stereoscopic reconstruction and displays.

[0004]

However, in the above-mentioned conventional ultrasonic diagnostic apparatus, the information of the cross section of the object can be obtained from the image, but the image is just an image as if the cross section was seen from the front. However, it may be slightly different from what the observer sees, which is not desirable for diagnosis.

Further, in a system for performing three-dimensional reconstruction, a large amount of image data necessary for reconstruction must be stored, resulting in a large device scale, and a huge amount of calculation being required, resulting in processing time. It took a long time, and it was not possible to observe in real time.

Therefore, the present invention enables ultrasonic diagnostic images to be displayed three-dimensionally in real time, and ultrasonic image signal processing that does not increase the scale of the apparatus even if a large amount of image data necessary for reconstruction is stored. An object is to provide an apparatus and its system.

[0007]

In order to achieve the above object, the present invention is an apparatus for processing an ultrasonic image signal captured over a plurality of fields or a plurality of frames, which is generated from the image signal of each field or frame. Image having a plurality of pieces of frame data to be compressed and a plurality of pieces of data relating to the position of each field or frame in the three-dimensional space and a recording means for recording the compressed moving picture data A signal processing device is provided. The compression means performs moving image compression in the MPEG format.

In addition, a plurality of frames or a plurality of fields from the moving probe in close proximity to the object to be inspected 3
Scanner means for capturing an ultrasonic image signal together with position information in the dimensional space, ultrasonic diagnostic means for converting the ultrasonic image signal from the scanner means into an image signal, and image signals of each frame or field are generated. An ultrasonic image including a compression unit that compresses a plurality of frame data and a plurality of data relating to the position of each field or frame in the three-dimensional space, and a recording unit that records the compressed moving image data. A signal processing system is provided.

In the ultrasonic image signal processing apparatus having the above-mentioned structure, the position and tilt information of the probe or the information concerning the observation condition of the observer is added and digitally recorded.
That is, it is recorded by the MPEG of the international standard system, and the 3D coordinate data of each frame, that is, the upper left corner coordinate and the normal vector are coded and then recorded. An image subjected to perspective projection conversion can be recorded as a video signal as it is on a VTR or the like, but it is also recorded as digitized data in terms of searchability and the like.

[0010]

DETAILED DESCRIPTION OF THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 shows a first embodiment according to the present invention.
The schematic configuration of the ultrasonic image signal processing apparatus as the embodiment of will be shown and described. This ultrasonic image signal processing device,
It is roughly divided into a scanner section, an electronic scanning ultrasonic diagnostic section, an image processing section, a recording / reproducing section, and a display section.

The scanner unit comprises an electronic scanning probe 1.
And a three-dimensional information fetching section 2 mechanically connected to each other. Here, the three-dimensional information capturing unit 2 is
In the case of the three-dimensional scanner described in the related art, it is an optical encoder and can know the position of each frame in the three-dimensional space and the inclination of the surface.

The electronic scanning ultrasonic diagnostic unit comprises an electronic scanning ultrasonic diagnostic apparatus 3, which supplies a pulse to the electronic scanning probe 1 and converts the received echo wave into an analog video signal and outputs it. The input unit 4 is used to set or change parameters used in the present system and input text and the like, and is connected with a pointing device such as a mouse. The image processing apparatus interfaces the recording / reproducing apparatus and the display apparatus after performing the processing described later on the analog video signal from the ultrasonic diagnostic apparatus.

First, an analog video signal obtained by the electronic scanning ultrasonic diagnostic apparatus 3 is converted into digital data by an A / D converter 11, and is transmitted to a perspective projection processing circuit 13 including an image memory via a switch 12. Is entered. The switching of the switch 12 is controlled by the CPU 14 and switches whether the subsequent processing is performed on the input signal from the A / D converter 11 or on the image signal read from the recording / reproducing unit described later.

Coordinate conversion processing for perspective projection by the perspective projection processing circuit 13 is performed on the image memory. This coordinate conversion processing is performed by the CPU 14 based on the three-dimensional position and angle information output from the three-dimensional information fetching unit 2, and is controlled by a control signal from the perspective projection controller 15.

The switch 1 controlled by the CPU 14
By switching 6, the selection is made through the perspective projection processing circuit 13 or not, that is, whether the perspective projection processing is performed or not, and then the processing for transmission to the recording / reproducing unit and the display unit is performed. When recording next, the MPEG encoder 17 uses MP, which is an international standard compression method for moving images.
After compression to EG format, the bus controller 18
Then, the data from the CPU 14 is added and recorded on the magneto-optical disk (MO) 19 of the recording / reproducing section.

The data display is converted into an analog signal by the D / A converter 20, and then the TV monitor 21 or F
It is displayed on the MD (Face Mounted Display) 22. Furthermore, when reading the data recorded on the magneto-optical disk 19, the header data is read by the bus controller 18 and then the MPEG decoder 2 is used.
The data is decompressed by 3 and transmitted to the perspective projection processing circuit 13 via the switch 12.

In this embodiment, the ultrasonic diagnostic apparatus and the image processing apparatus have been described as separate components, but they can be integrated, and in that case, the electronic scanning ultrasonic diagnostic apparatus 3 can be used as a digital signal. Because the signal can be taken out directly, A
The / D converter 11 is unnecessary and the device becomes simpler.

Next, a series of ultrasonic images obtained by the above-mentioned three-dimensional scanner will be described. For example, the device shown in FIG. 2 is medical, has a movable probe as shown by a broken line inside, and the bottom surface is a portion to be brought into close contact with the body of the patient to be observed and has a spherical shape. . An ultrasonic image is captured while moving this probe in the direction of the arrow at a constant speed.

At this time, since the position of the probe is accurately known, the position of the cross section cut out from each image can also be accurately known. The probe of this device is tilted back and forth from the normal position at an angle of 30 degrees, and scans 60 degrees from end to end at 30 frames per second for 4 seconds.

The 120 images obtained in this way are recombined into 3
FIG. 3 is a bird's-eye view of a rearranged object in a three-dimensional space with thinned frames for easy viewing. As described above, in the present invention, in order to facilitate the sensory grasp of the positional relationship between the respective frames arranged in the three-dimensional space, the perspective projection is performed so that the frames are displayed as if they were observed from an appropriate position. .

FIG. 4 shows information when performing perspective projection for changing the viewpoint. Considering an ultrasonic image of one frame in FIG. 4, the equation of the plane of the image is known as data for specifying the coordinates of the ultrasonic image in the three-dimensional space. It is necessary that the position of at least one point is specified.
Therefore, the coordinate 31 in the three-dimensional space of the upper left corner in the image,
The normal vector 32 of the plane containing the image is given.

On the other hand, as conditions for observing the surface, it is necessary to determine the position of the observer, the line of sight, and the viewing angle. In addition, it is necessary to determine the upward direction at the time of observation in order to enable display when the observer tilts his or her head, but in the present invention, the naturalness of the image is required. Therefore, the upward direction at the time of observation coincides with the upward direction of the ultrasonic image.

Therefore, the viewpoint 33 of the observer, the point 34 of interest of the observer for determining the line of sight, and the viewing angle 35.
give. However, the viewing angle 35 is an angle indicating how far in the horizontal direction the point of interest is from the point of interest. Here, as the coordinate system, what is generally called a right-handed system is used, that is, the horizontal right direction is x.
The axis, the upward direction in the vertical direction, and the forward direction are defined as the positive y axis and the positive z axis, respectively. Given these conditions,
It is possible to perform perspective projection of an image, and it becomes possible to reproduce an image as if it were observed from a designated observation position. The concept of projection will be briefly described below.

First, consider a projection plane 36 positioned at a right angle to the line of sight in the space from the viewpoint 33 into the field of view, and divide the part of the projection plane 36 into the field of view according to the number of pixels of the image. Then, a straight line that connects each image in the field of view and the viewpoint is obtained, and the intersection of each straight line and the plane of the ultrasonic image is obtained. The value of the image at each intersection is taken as the value of the image after projection.

Next, as an example of a device for displaying an ultrasonic image in the present invention, a spectacle type display (hereinafter referred to as FMD)
Will be described. For example, the FMD disclosed in "Image Observation Device" of Japanese Patent Application No. 3-295874 has a structure as shown in FIG. In this figure, the user's head 4
1 shows the eyeballs 42a and 42b, and the parts 43a and b
48a and 48b are glasses-type headphone-equipped displays to be held on the face. In the signal processing units 43a and 43b with built-in speakers, while reproducing the audio signals, the image signals are displayed on the self-luminous two-dimensional display elements 44a and 44b.
It is projected inside the eyeball using the imaging lenses 45a and 45b, the optical axis bending wedge prisms 46a and 46b, the aperture diaphragms 47a and 47b, and the aspherical concave mirrors 48a and 48b.

Since this image and audio signal processing is performed independently for right and left, it can be reproduced as a stereoscopic image and stereo sound. This device is designed to match the diopter of the eyeball with an aspherical concave mirror 4
In order to set the position of the virtual image created by 8a and b to an appropriate position,
It has a function of adjusting the positions of the self-luminous two-dimensional display elements 44a, 44b in the optical axis direction. Further, the aspherical concave mirrors 48a, 48b can change their reflectances, and it is also possible to perform so-called see-through observation in which the outside of the FMD is viewed through the switch through a switch.

Next, a method of determining the viewing angle for determining the observation magnification will be described. First, suppose that the relationship between the apparent size in the ultrasonic image and the actual size of the object is obtained. Figure 6
Consider a case where an object 51 having a size h is displayed as shown in FIG. When an image of this object is displayed using the FMD 53 of FIG. 6B, as shown in the figure, when the one-sided viewing angle θ of the FMD is 16 degrees and 360 pixels are present in the display area 54, When the object 51 is displayed on the display in the size of the region 55 with the number of pixels represented by x in the following equation, the apparent sizes match.

X = (360 × h) / (tan 16 ° × d) In other words, the size felt when actually observing an object having the size of h from the distance of d, and displayed by x pixels on the display. It matches the size you feel when you see what you see.

For example, if the number of pixels taken in at the time of shooting is equal to the number of pixels in the display when displaying at the same size, the viewpoint is set so that the viewing angle in perspective projection is 16 degrees and the observation distance is d. You just have to set it.

On the other hand, when it is desired to display the image at a higher magnification than the actual observation or to emphasize the perspective, it is necessary to display the image with a different apparent size. The angle and the observation distance should be set to appropriate values. In other words, by performing projection conversion while manipulating the parameters of the viewing conditions used in the perspective projection process, it is possible to change and display the magnification at the same time, which makes it possible to combine hardware and electronic zoom-in / zoom-out. Therefore, it is not necessary to newly provide means.

In the present invention, the CPU 14 shown in FIG. 1 can be set to arbitrary values by the input unit 4. Next, a method of generating an output image to a device capable of reproducing a stereoscopic image will be described.

Prepare viewpoints for both the left and right eyes,
A left image and a right image obtained by perspective projection are respectively created from them, and displayed so that the left and right eyes can observe them separately. For example, when using the FMD, when the images viewed by the left and right eyes are assigned to the even field and the odd field, the left image and the right image are recorded in the even field and the odd field, respectively. In the FMD, the even field image is output to the display element for the left eye, and the odd field image is output to the display element for the right eye. At this time, the line of sight of each of the left eye and the right eye becomes a different one because it connects the point of interest and the different viewpoints of both eyes, and thus naturally becomes different, and the reproduced image is stereoscopically viewed due to having parallax,
It becomes easier to understand the spatial arrangement.

In the case of outputting to a three-dimensional image display device such as an FMD, the left position is the left eye viewpoint and the right position is the right eye viewpoint from the above-mentioned viewpoint by a distance of half the eye width. Then, perspective projection is performed with a left visual line and a right visual line that connect these two viewpoints and the point of interest, and the parallax at the time of stereoscopic viewing can be changed by changing the pupil distance. Rather than the positional relationship in the three-dimensional space
It is also possible to make a display that emphasizes the sense of depth.

This depth enhancement can also be realized only by changing the parameters of perspective projection, and it is not necessary to newly provide means for this, as in the case of the electronic zoom-in / zoom-out described above. Therefore, in the present invention, the pupil distance can be changed by the input unit 4 described above according to the preference of the operator.

In the present invention, in order to make it easier to grasp the three-dimensional sensation, the scale (3
Dimension scale) can be displayed together. A second embodiment having a function of displaying the following three-dimensional scale will be described with reference to the drawings.

For example, in FIG. 7 (a), the images of four frames are displayed three-dimensionally, and actual scales are added to the coordinate axes of the x-axis 61, the y-axis 62a, and the z-axis 63a. And is displayed as a scale. This coordinate axis is designed to be created with an arbitrary position in space as the origin. In the figure, only the coordinates that have positive values for the three axes are displayed, but it is also possible to display negative values. It is possible to display only specified coordinate axes.

As a method of determining the axis to be displayed among these coordinate axes, the effect of displaying each coordinate axis depending on the position of the viewpoint is judged in addition to the input by the operator, and it is judged to be effective. It is possible to display only what you have. For example, in the case of FIG. 7A, the x-axis 61 overlaps the image, and it can be determined that it is preferable not to observe it. Moreover, it is desirable that the range of the values of the y-axis and the z-axis is large enough to cover the range where the ultrasonic image exists.

An example in which the display of the coordinate axes is determined in this way is shown in FIG. 7 (b). Regarding the method of actually displaying the coordinate axes determined as described above as shown in FIG. 7B,
This will be described below using the configuration shown in FIG.

First, the origin of the coordinate axes is determined in order to determine where on the screen the coordinate axes are arranged. Next, the coordinate creation unit 71 sets the coordinate axes of the size determined as described above.
Is created as a binary image by, and the image is perspectively projected by the projection conversion processing unit 72 in the same manner as the ultrasonic image.
The synthesizing unit 73 overlays the ultrasonic image.
Then, the combined image signal is recorded in the recording unit 74 provided with a compression unit and the like, and is sent to the display unit 75 and displayed.

This coordinate axis may be that of a multivalued image, and C
In accordance with an instruction from the PU 76, the synthesizing unit 73 displays this scale with a designated color, displays an exclusive OR (XOR) with the ultrasonic image, blinks it according to the observer's preference, and stops the display. It can be done. Further, the scale can be moved to any position on the screen by the mouse 77 connected to the CPU 76.

By displaying this scale,
In addition to facilitating measurement of the size of the object and the position of each frame, when observing with an FMD, left and right reference points of parallax are formed in the frame, which makes it easier to obtain a stereoscopic sensation. The correspondence between the function described in the present embodiment and the embodiment shown in FIG. 1 is that the scale creating unit 71, the projection conversion processing unit 72, and the synthesizing unit 73 shown in FIG. 8 are the perspective projection processing circuit 13 in FIG. include. In addition, the recording unit 74 and the display unit 75 are the MPEG encoder 17 in FIG.
It corresponds to the bus controller 18, the magneto-optical disk 19, the D / A converter 20, the TV monitor 21, the FMD 22 and the MPEG decoder 23, respectively.

Next, a method of recording an image signal by such an apparatus will be described.

First, FIG. 9 shows an example of the MPEG data structure, which will be briefly described. The sequence layer is composed of a set of a plurality of image data called GOP layers. The GOP layer is composed of a certain number of frames of data called a picture layer. At this time, each layer records information regarding the data of each layer as a header at the head of the layer. In FIG. 9, reference numeral 81 is a sequence header in which data such as image size and quantization matrix is written.
A GOP header 82 contains a time code and the like. 8
Reference numerals 3, 85, and 87 are picture headers, in which information regarding the coding type and motion vector is written. And 8
4, 86 and 88 are frame data,
Each of the above headers includes a user-definable area.

Next, an actual recording operation by the ultrasonic image signal processing apparatus of the present invention will be described. Since the original image before perspective projection conversion is output as a video signal, it can be recorded as it is on a VTR or the like, but three-dimensional data is recorded, searchability and recordability make it possible to save the probe image. It is desirable that the position and tilt information or information relating to the viewing condition of the observer be added and digitally recorded. That is, the sequence is recorded by MPEG, which is an international standard method of moving image compression recording, and the 3D coordinate data of each frame, that is, the upper left corner coordinate and the normal vector are encoded and then recorded in the picture header.

On the other hand, the image subjected to the perspective projection conversion can be recorded as a video signal on the VTR or the like as it is, but it is desirable that the image is also recorded as digitized data from the viewpoint of searchability. At the time of this recording, the information about the device that captured the original image, the flag indicating that the image is after perspective projection, and the information about the viewpoint at the time of projection, the point of interest, and the viewing angle should be recorded in the sequence header. There is.

Switching between the pre-conversion image and the post-conversion image to be recorded is performed by the switch 16 shown in FIG. Further, during reproduction, the switch 12 is switched to send the read signal to the perspective projection processing unit 13. Then, when the recorded signal is the original image before conversion, the switch 16 is switched to send the output from the perspective projection processing unit 13 to the display unit,
When the signal is the converted signal, the switch 16 is switched so that the signal is displayed without passing through the perspective projection processing unit 13.

Although the above-mentioned three-dimensional scale can record an overlayed image, only an ultrasonic image is recorded as image data without overlaying, and the scale is encoded to be a user of the sequence header. It is also possible to record in the data area. At this time, the synthesizing unit 73 shown in FIG. 8 outputs the overlaid image to the display unit 75, and outputs the converted ultrasonic image without overlay to the recording unit 74. And C
The PU 76 creates a signal in which the scale-related information is encoded and sends it to the recording unit 74. The recording unit 74 creates a header from the transmitted signal, encodes the image signal, and outputs M
It is recorded on a medium such as O or a magnetic tape.

In the above-described scale encoding method, the start point, the end point, and the scale interval on the screen are recorded.
Any method may be used as long as the reproduction apparatus records information that can reconstruct the scale.

As described above, the function of recording the images before and after the perspective projection, respectively, is that if the images before projection are recorded, the original image with the highest resolution from the ultrasonic image capturing device remains. By using the perspective projection reproducing apparatus of the present invention, it is possible to obtain a perspective projection image in which the observation position is arbitrarily changed.

On the other hand, if the image after perspective projection is recorded, it can be reproduced by a normal reproducing apparatus having no perspective projection function.

Next, let us consider a frame to be recorded in this ultrasonic image signal processing apparatus. Normally, an ultrasonic image has a finite resolution in the direction perpendicular to the image, and the minimum thickness value that can be resolved in one frame is r [mm], and the probe has a width w [mm]. Time t
Multiply by [sec], and s frames [frame /
If an image of [second] is captured, r> w / (st) [mm / frame], then oversampling. Here, in the present invention, when the movement of the observation region is small, it is possible to reduce the amount of recorded information by resampling the frame at a ratio of w / (rst).

FIG. 10 shows a flow chart for making the determination, which will be described. First, it is judged whether the recording medium saving mode is on (step S1), and if it is on (YE
S), and then determine whether r> w / (st) (step S
2) When the result is true (YES), (rst)
/ W is truncated to the right of the decimal point to obtain an integer value i (step S4). If it is off in step S1 and false in step S2 (NO), the value i is set to 1 (step S3), and the process proceeds to step S6 described later.

Next, based on the value i obtained in step S4,
The minimum value m of the frame rate is calculated from the speed of movement of the observed image, and it is determined whether or not the value is s / i or less (step S5). If true (YES), s / i [frame / Sec] frame rate recording (step S
6). If false (NO), i subtracts 1 from i to obtain the maximum integer that satisfies the condition (step S
7) and returns to step S5. Here, the minimum frame rate m may be input from the keyboard of the input unit 4 in FIG. 1, but for example, a motion vector is obtained for each block by using the motion vector detection circuit of the compression circuit, and those motions are calculated. Of the vectors, the largest one may be used to automatically determine the minimum frame rate.

An example of the case where the probe is mechanically moved to continuously capture an ultrasonic image has been shown so far, but as a third embodiment according to the present invention, the ultrasonic image signal processing apparatus is used. Another example of capturing an ultrasonic image will be described.

A 3D (3D) mouse will be described as an example of the 3D information capturing unit 2 shown in FIG.

FIG. 11 shows a conceptual configuration of a magnetic type 3D mouse.

In this 3D mouse, the source 91 is an orthogonal coil in which coils are wound around x, y, and z axes, and is driven by the drive circuit 92 to generate a predetermined magnetic field.
When a sensor 95 composed of a plurality of orthogonal coils is placed in opposition to the source 91 in this magnetic field, a current is induced in each coil of the sensor 95. Since the induced current is proportional to the strength of the magnetic field, it is converted into a voltage by the detection circuit 94, and
It is output as distance / angle information by the CPU 93.

As shown in FIG. 12, the distance / angle information is the position represented by x, y, z and Q _x ,
It is an angle on each axis represented by Q _y and Q _z . Conventionally,
The three-dimensional position sensor was used to capture the three-dimensional information, but this can be realized with one 3D mouse, and the device can be designed very compactly.

When this 3D mouse is attached to the ultrasonic probe, it is possible to capture the ultrasonic image while moving the probe and simultaneously obtain the reference coordinates and the normal vector for each frame. With this probe,
Needless to say, it is possible to perform perspective projection observation even on an ultrasonic image captured while the operator freely moves or tilts the probe, and as described above, recording is performed with 3D coordinate data added to the header. By doing so, you can see how the image was captured while moving the probe even after the examination.

The ultrasonic images captured by moving the probe with the 3D mouse described above in the same plane as the cross section by the ultrasonic waves are arranged with some overlapping portions.
An image with a continuous wide angle of view can be obtained based on the position information of the probe.

For example, two images 9 as shown in FIG.
If it is taken from the place where 6,97 are lined up,
In normal observation, only one of them could be displayed. However, by creating one image 98 from these two images, it has become possible to widen the range of the observation region that was conventionally limited by the size of the probe. In other words, by capturing the required number of images according to the range to be observed, it has become possible to display a large organ such as the liver in one ultrasonic image.

Further, it is possible to obtain a wide-angle ultrasonic image by using a plurality of probes and by making the signal from each probe the same as in the above method. Therefore, the time required for moving the probe can be shortened, so that the image does not shift even when observing a moving portion.

Next, as a fourth embodiment, the images disclosed in the "image pickup device" of Japanese Patent Application No. 4-89092 and the "electronic camera" of Japanese Patent Application No. 4-94473 proposed by the present applicant will be described. An example of applying the synthesis algorithm to the present invention will be described.

As shown in FIG. 14, the image 1 of the two image signals captured so that the areas partially overlap each other.
And the image 2 are temporarily stored in the image storage unit 101. Based on the image signal of the overlapping area read from the image storage unit 101, the relative pixel position of each image is specified by the conversion coefficient (rotation amount R, displacement amount S) detected by the displacement amount detection unit 102. To be done. The interpolation unit 103 sequentially corrects the pixel values by sequentially using the interpolation calculation on the pixel values of the plurality of images, and obtains an interpolated image signal for superposition. This interpolated image signal,
The signals in the image storage unit 101 are combined by the combining unit 104 and output as a wide-angle image.

According to this method, 3D by 3D mouse
Even if the accuracy of the coordinate information value is not very high, the relative positional relationship between the two images can be accurately obtained from the signal of the overlapping portion, so that the connecting portion of the combined images does not become discontinuous. With characteristics. Although the case where two input images are used has been described here, it is possible to obtain a projection image with a wider view angle by using more images.

Further, when the above-mentioned 3D mouse is used for detecting the movement of the observer, the 3D mouse and the FMD are combined to detect the movement of the head of the observer and interlock with the movement of the head. It is also possible to have a function of scrolling the screen by using. In other words, when observing the above-mentioned wide-angle image, a part of it is displayed on the screen, and the part that does not completely fill the screen is turned to face by turning the face to enter the screen. To be displayed. As a result, an image having a size that does not fit in the field of view of the FMD can be handled by virtually expanding the image space.

Next, as a fifth embodiment of the ultrasonic image signal processing apparatus of the present invention, an example in which a specific organ is extracted from an ultrasonic image to measure the shape and a special observation is performed will be described.

FIG. 15 shows an example in which an area of an organ such as the gallbladder is extracted by performing general boundary extraction processing or outline extraction processing as a method used for image shape analysis on an ultrasonic image. The boundary image 112 is obtained by searching for a discontinuous portion of the texture in the ultrasonic image 111. This boundary line is a part of the surface of the organ, and the boundaries extracted from multiple frames are displayed in a wire frame, or the surfaces of the organs are taken out by connecting them while performing interpolation, and are displayed by performing a hatching process, etc. You can

FIG. 16 shows a configuration example of a circuit for carrying out such processing. In the example using the above-mentioned three-dimensional scanner, 12 captured from the scanner unit 113
The ultrasonic image sequence of 0 frame is temporarily recorded in the memory unit 114. This signal is divided into two and one is projected and converted by the perspective projection unit 115 described so far. The other signal is the organ surface extraction processing unit 116.
The surface of the organ is extracted for each frame, and the three-dimensional reconstruction unit 117 forms a wire frame in the three-dimensional space, and the perspective projection unit 118 performs projection conversion.

These two post-projection converted signals are combined by the signal combining section 119 and then the FMD of the display section 120.
Is displayed in. The organ information in the wire frame is unchanged in the sequence, but the position where it intersects with the cross section of the ultrasonic image is different in each frame. Therefore, when synthesizing, the wire frame of the portion closer to the observer than the ultrasonic image plane is displayed by a solid line, and the portion on the other side is displayed by a broken line.

In this method, the display is made so that the sequence of 120 frames is repeatedly reproduced.
Even if the three-dimensional reconstruction of the organ portion cannot be performed in real time, it is synthesized as soon as the reconstruction is completed. At this time, for the memory unit 114 that stores the repeatedly read sequence, the medium may be a magneto-optical disk or the like in addition to the semiconductor, and a medium having an image compression / expansion circuit is used. The signal to be stored may be subjected to data compression processing.

In the ultrasonic image signal processing apparatus according to the sixth embodiment of the present invention, a process for adding an afterimage effect by adding signals of a frame preceding in time to a further input sequence. You can also do Figure 1
Reference numeral 7 denotes a signal processing when the image after the afterimage processing is displayed by perspective projection. Image input unit 121
After being converted by the perspective projection unit 122, the sequence from is multiplied by the coefficient a in the multiplier 123,
The signal of 5 is multiplied by the coefficient b in the multiplier 126 and added in the adder 124.

The result is divided into one that is written in the memory unit 125 to be added to the signal of the next frame and one that is sent to the display unit 127. However, the coefficients a and b used here represent frame weights, and the larger the value of b, the stronger the afterimage effect.
Both are set to be 1 when added. By using this configuration, it is possible to view a change between frames of an image as an afterimage, and it becomes easier to grasp the spatial arrangement.

Up to now, it has been explained that it is possible to create an image as if observed in various states from one original sequence by changing the parameters of the projection transformation, but there is also a desire to see these at the same time. is there.

As a seventh embodiment, for example, in FIG. 18, the input image is sent to the perspective projection processing unit 131, where the front,
Create observation images from four viewpoints from left, right, and top,
The four screens are sent to the display unit 133 by the multi-screen creating unit 132 as a multi-screen 140 for displaying each of the four images from the first quadrant to the fourth quadrant as shown in FIG. At this time, the display screen is divided into four, so 1
One screen has been reduced to a quarter size,
As described above, it is possible to simultaneously reduce the images by setting the parameters of the perspective projection conversion to appropriate values by the input unit 134, and thus a circuit for creating the reduced image is not necessary. Then, the CPU 135 also calculates parameters for setting conversion for viewing from an appropriate position, sets these parameters, and provides information on the layout of the multi-screen. Further, since the perspective projection processing is processed independently for each parameter, if the number of divisions increases and the calculation cannot be completed with only one perspective projection processing unit, a plurality of perspective projection processing units are connected in parallel. It is also possible to make each calculate. In such a case, the portions surrounded by broken lines in FIG. 18 are perspective projection processing units 131a and 1a.
This is an example of the case where the number is increased like 31b, ...

An example of the case where the ultrasonic image signal processing apparatus of the present invention described above is used for treatment while observing an ultrasonic image in the medical field will be described. For example, an injection needle-shaped device is inserted inside the abdomen, cells of the target tissue are collected and tested while confirming the position on the ultrasound image at the same time, or the anticancer drug is directly injected into the cancer tissue. Has been. In such a procedure, it is desired to obtain the positional relationship between the position of the tip of the needle and the tissue in real time. In such a case, the image of the ultrasonic image signal processing device of the present invention is displayed by F
By observing with MD, you can work while seeing the tip of the needle and the target tissue three-dimensionally, so it is very accurate,
The cells of the tissue can be safely collected.
At this time, if a foot switch or the like is used as the see-through changeover switch of the FMD, the see-through function is used to see the ultrasonic image and the hand at which the actual work is performed.
It is possible to switch without removing D and without interrupting work, which is very safe and convenient.

Further, the present invention is not limited to the above-mentioned embodiment, and it goes without saying that various modifications and applications can be made without departing from the scope of the invention.

[0078]

As described above in detail, according to the present invention, a plurality of frame data generated by an image signal of each field or frame and three fields of each field or frame.
Since the plurality of data relating to the position in the dimensional space is configured to be compressed and recorded, the ultrasonic image signal processing device and its device size do not increase even if a large amount of image data necessary for reconstruction is stored. A system can be provided. Further, the present invention provides an ultrasonic image signal processing device and an ultrasonic image signal processing system, which can display an ultrasonic diagnostic image three-dimensionally and can easily acquire a three-dimensional shape and position sensuously. be able to.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of an ultrasonic image signal processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration example of a three-dimensional scanner.

FIG. 3 is a bird's-eye view showing a scanning state by the three-dimensional scanner shown in FIG.

FIG. 4 is a diagram showing information when performing perspective projection for changing a viewpoint.

FIG. 5 is a diagram showing a configuration example of an image observation apparatus (glasses type display).

FIG. 6 is a diagram showing a relationship between an apparent size of an ultrasonic image and an actual size of an object.

FIG. 7 is a diagram for explaining a function of displaying a three-dimensional scale as a second embodiment of the present invention.

FIG. 8 is a diagram showing a schematic configuration of an ultrasonic image signal processing apparatus as a third embodiment of the present invention.

FIG. 9 is a diagram showing an example of a data structure of MPEG.

FIG. 10 is a flowchart for explaining the operation of the ultrasonic image signal processing device according to the third embodiment.

FIG. 11 is a diagram showing a specific configuration of a magnetic 3D mouse used in an ultrasonic image signal processing apparatus.

FIG. 12 is a diagram for explaining distance / angle information in a 3D mouse.

FIG. 13 is a diagram for explaining a state of an image captured by using a 3D mouse.

FIG. 14 is a diagram showing a configuration of an ultrasonic image signal processing device according to a fourth embodiment.

FIG. 15 is a diagram showing an example in which a specific area is extracted by performing boundary extraction processing or contour line extraction processing by an ultrasonic image signal processing apparatus according to a fifth embodiment of the present invention.

FIG. 16 is a diagram showing a configuration of an ultrasonic image signal processing device according to a fifth embodiment.

FIG. 17 is a diagram showing a configuration of an ultrasonic image signal processing device as a sixth embodiment according to the present invention.

FIG. 18 is a diagram showing a configuration of an ultrasonic image signal processing device as a seventh embodiment according to the present invention.

FIG. 19 is a diagram showing an example of an image observed according to the seventh embodiment.

[Explanation of symbols]

1 ... Electronic scanning probe, 2 ... three-dimensional information capturing unit,
3 ... Electronic scanning type ultrasonic diagnostic apparatus, 4 ... Input section, 11 ... A
/ D converter, 12, 16 ... Switch, 13 ... Perspective projection processing circuit, 14 ... CPU, 15 ... Perspective projection controller,
17 ... MPEG encoder, 18 ... Bus controller,
19 ... Magneto-optical disk (MO), 20 ... D / A converter,
21 ... TV monitor, 22 ... FMD (face mounted display), 23 ... MPEG decoder.

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Hiroyuki Watanabe             2-43 Hatagaya, Shibuya-ku, Tokyo Ori             Inside Npus Optical Industry Co., Ltd. (72) Inventor Hiroyuki Fukuda             2-43 Hatagaya, Shibuya-ku, Tokyo Ori             Inside Npus Optical Industry Co., Ltd. F-term (reference) 4C601 BB03 BB16 EE12 JC25 KK21                       LL13

Claims (4)

[Claims] 1. An apparatus for processing an ultrasonic image signal captured over a plurality of fields or a plurality of frames, wherein a plurality of frame data generated from the image signals of the respective fields or frames and 3 of each field or frame.
An ultrasonic image signal processing apparatus, comprising: a compression unit that compresses a moving image of a plurality of pieces of data relating to a position in a dimensional space; and a recording unit that records the compressed image data. 2. The ultrasonic image signal processing apparatus according to claim 1, wherein the compression means performs the moving image compression in MPEG format. 3. Scanner means for capturing as an ultrasonic image signal together with position information in a three-dimensional space over a plurality of frames or a plurality of fields from a moving probe which is close to an object to be inspected, and an ultrasonic image signal from the scanner means. Ultrasonic diagnostic means for converting the image into an image signal, a plurality of frame data generated from the image signal of each frame or field, and 3 of each field or frame
An ultrasonic image signal processing system, comprising: a compression unit for compressing a moving image of a plurality of data relating to a position in a dimensional space; and a recording unit for recording the compressed moving image data. 4. The ultrasonic image signal processing system according to claim 3, wherein the compression means performs the moving image compression in an MPEG format.