JP6945334B2 - Ultrasound diagnostic equipment and medical image processing equipment - Google Patents

Description

Embodiments of the present invention relate to an ultrasonic diagnostic apparatus and a medical image processing apparatus.

In the medical field, an ultrasonic diagnostic apparatus is used that images the inside of a subject by using ultrasonic waves generated by a plurality of vibrators (piezoelectric vibrators) of an ultrasonic probe. The ultrasonic diagnostic apparatus transmits ultrasonic waves into a subject from an ultrasonic probe connected to the ultrasonic diagnostic apparatus, generates a received signal based on the reflected wave, and obtains a desired ultrasonic image by image processing.

The operator may collect a plurality of two-dimensional image data in chronological order while moving the ultrasonic probe, and may collect a plurality of position information of the ultrasonic probe. In that case, the ultrasonic diagnostic apparatus generates and displays three-dimensional image data by arranging and three-dimensionally reconstructing a plurality of two-dimensional image data based on a plurality of position information.

Japanese Unexamined Patent Publication No. 8-332187

An object to be solved by the present invention is to provide an ultrasonic diagnostic apparatus and a medical image processing apparatus capable of appropriately correcting the position information of a plurality of two-dimensional image data.

The ultrasonic diagnostic apparatus according to the present embodiment is a transmission / reception means for transmitting ultrasonic waves to an ultrasonic probe and receiving a signal based on the ultrasonic waves received by the ultrasonic probe, and based on the signal. A generation means for generating a plurality of two-dimensional image data in a time series, a collection means for collecting a plurality of position information in three dimensions of the ultrasonic probe, and the plurality of position information in the plurality of two-dimensional image data, respectively. It has an associating means for associating and a correction means for correcting the plurality of position information associated with the associating means.

The schematic diagram which shows the structure of the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. (A) to (C) are diagrams for explaining the scanning surface of the ultrasonic probe. The conceptual diagram for demonstrating the outline of the correction method of the position information attached to the 2D image data in the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. (A) and (B) are diagrams for explaining the alignment process. The figure for demonstrating the alignment process. The figure for demonstrating the alignment process. The flowchart which shows the operation of the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. The schematic diagram which shows the structure of the ultrasonic diagnostic apparatus which concerns on 2nd Embodiment. The conceptual diagram for demonstrating the outline of the position information correction method in the ultrasonic diagnostic apparatus which concerns on 2nd Embodiment. (A) is a diagram showing a histogram of two-dimensional image data before the histogram conversion processing, and (B) is a diagram showing a histogram of two-dimensional image data after the histogram conversion processing. (A) is a diagram showing an example of displaying a three-dimensional image by a conventional ultrasonic diagnostic apparatus, and (B) is a diagram showing an example of displaying a three-dimensional image by the ultrasonic diagnostic apparatus according to the second embodiment. The flowchart which shows the operation of the ultrasonic diagnostic apparatus which concerns on 2nd Embodiment. The figure for demonstrating the linear transformation process of 2D image data. (A) and (B) are diagrams for explaining the distortion correction process. The figure for demonstrating the distortion correction processing. The figure for demonstrating the projective transformation. The conceptual diagram for demonstrating the outline of the method of using the correction of the position information attached to the 2D image data and the correction of the distortion of the 2D image data in combination in the ultrasonic diagnostic apparatus according to the 1st embodiment. The conceptual diagram for demonstrating the outline of the correction method of the position information attached to the volume data in the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. (A) and (B) are conceptual diagrams for explaining the alignment process for synthesizing two volume data. The schematic diagram which shows the structure of the medical image processing apparatus which concerns on this embodiment. The block diagram which shows the function of the medical image processing apparatus which concerns on this embodiment.

The ultrasonic diagnostic apparatus and the medical image processing apparatus according to the present embodiment will be described with reference to the accompanying drawings.

1. 1. The ultrasonic diagnostic apparatus according to the first embodiment FIG. 1 is a schematic view showing the configuration of the ultrasonic diagnostic apparatus according to the first embodiment.

FIG. 1 shows an ultrasonic diagnostic apparatus 10 according to the first embodiment. The ultrasonic diagnostic apparatus 10 includes an ultrasonic probe 11, a magnetic field transmitter 12, a sensor 13, and an apparatus main body 14. In some cases, only the device main body 14 is referred to as an ultrasonic diagnostic device, in which case the ultrasonic diagnostic device is connected to an ultrasonic probe, a magnetic field transmitter, and a sensor provided outside the ultrasonic diagnostic device.

The ultrasonic probe 11 transmits and receives ultrasonic waves to a subject (for example, a patient). The ultrasonic probe 11 transmits and receives ultrasonic waves by bringing the front surface of the subject into contact with the surface of the subject, and a plurality of minute vibrations arranged in one dimension (1D) or two dimensions (2D). It has a child (piezoelectric element) at its tip. This vibrator is an electroacoustic conversion element, and has a function of converting an electric pulse into an ultrasonic pulse at the time of transmission and converting a reflected wave into an electric signal (received signal) at the time of reception.

The ultrasonic probe 11 is small and lightweight, and is connected to the device main body 14 via a cable. Examples of the type of ultrasonic probe 11 include a 1D array probe, a mechanical 4D probe, a 2D array probe, and the like. The 1D array probe has a configuration in which a plurality of vibrators are arranged one-dimensionally. Here, the 1D array probe also includes a configuration in which a small number of vibrators are arranged in the elevation direction. The mechanical 4D probe has a configuration in which an array in which a plurality of vibrators are arranged one-dimensionally is mechanically swung in an elevation direction, that is, in a direction intersecting a scanning surface. The 2D array probe has a configuration in which a plurality of vibrators are arranged two-dimensionally.

2 (A) to 2 (C) are views for explaining the scanning surface of the ultrasonic probe 11.

FIG. 2A shows the movement of the scanning surface P when the operator moves the 1D array probe as the ultrasonic probe 11. In this case, since the positions of both the sensor 13 and the scanning surface P are fixed with respect to the 1D array probe 11, if the geometrical positional relationship from the sensor 13 to the scanning surface P is converted, scanning is performed from the position information of the sensor 13. The position information of the surface P can be obtained. When the operator moves the 1D array probe 11 in the direction intersecting the scanning surface P, so-called three-dimensional scanning is performed. The movement (movement) of the 1D array probe 11 includes parallel movement, fanning, rotation, and the like, and the same applies hereinafter.

2 (B) and 2 (C) show a case where the scanning surface P is moved by a mechanical 4D probe or a 2D array probe as the ultrasonic probe 11. FIGS. 2B and 2C show a normal usage of the mechanical 4D probe 11 and the two-dimensional array probe 11, that is, a case where the tilt angle of the scanning surface is automatically changed.

As shown in FIG. 2B, the mechanical 4D probe 11 mechanically swings the 1D array in a direction intersecting the scanning surface P while repeatedly scanning the inside of the scanning surface P with the 1D array, so-called 3 A dimensional scan is performed.

As shown in FIG. 2C, in the 2D array probe 11, so-called three-dimensional scanning is performed by electronically moving the ultrasonic beam in the direction intersecting the scanning surface P while scanning the inside of the scanning surface P. Will be.

In the cases shown in FIGS. 2B and 2C, the position of the sensor 13 is fixed with respect to the ultrasonic probe 11, and the inclination angle of the scanning surface P changes with the passage of time. Since the inclination angle of each scanning surface P with respect to the ultrasonic probe 11 is obtained by calculation, the geometrical positional relationship of each scanning surface P can be converted from the position information and the inclination angle of the ultrasonic probe 11. Therefore, even when a mechanical 4D probe or a 2D array probe is adopted as the ultrasonic probe 11, if the scanning surface P is fixed to one tilt angle, the same thing as the 1D array probe can be performed.

The mechanical 4D probe or 2D array probe as the ultrasonic probe 11 can also perform the same three-dimensional scanning as when the operator moves the 1D array probe. In that case, the operator moves the mechanical 4D probe 11 or the like that repeatedly scans the inside of the scanning surface P with the scanning surface P fixed at one tilt angle.

As described above, the ultrasonic probe 11 can be handled in the same manner regardless of the type of the ultrasonic probe 11. Hereinafter, a case will be described in which the 1D array probe repeatedly scans the inside of one scanning surface P and the operator manually moves the 1D array probe 11 to perform three-dimensional scanning.

Returning to the description of FIG. 1, the magnetic field transmitter 12 is arranged in the vicinity of the ultrasonic probe 11 so that the sensor 13 is within the effective range of the magnetic field generated from the magnetic field transmitter 12. The magnetic field transmitter 12 generates a magnetic field under the control of the device main body 14.

The sensor 13 detects a plurality of position information of the ultrasonic probe 11 in time series and outputs the information to the apparatus main body 14. The sensor 13 includes a type of sensor attached to the ultrasonic probe 11 and a type of sensor provided separately from the ultrasonic probe 11. The latter sensor is an optical sensor, which photographs the feature points of the ultrasonic probe 11 to be measured from a plurality of positions and detects each position of the ultrasonic probe 11 by the principle of triangulation. Hereinafter, the case where the sensor 13 is the former sensor will be described.

The sensor 13 is attached to the ultrasonic probe 11, detects its own position information, and outputs it to the device main body 14. The position information of the sensor 13 can also be regarded as the position information of the ultrasonic probe 11. The position information of the ultrasonic probe 11 includes the position and orientation (tilt angle) of the ultrasonic probe 11. For example, the posture of the ultrasonic probe 11 can be detected by the magnetic field transmitter 12 sequentially transmitting a three-axis magnetic field and sequentially receiving the magnetic field by the sensor 13. Further, the sensor 13 is a 3-axis gyro sensor that detects the 3-axis angular velocity in the 3-dimensional space, a 3-axis acceleration sensor that detects the 3-axis acceleration in the 3-dimensional space, and 3 that detects the 3-axis geomagnetism in the 3-dimensional space. It may be a so-called 9-axis sensor including at least one of the axial geomagnetic sensors.

The apparatus main body 14 includes a transmission / reception circuit 21, a two-dimensional image generation circuit 22, a two-dimensional memory 23, a position information collection circuit 24, a position information mapping circuit 25, an alignment circuit 26, a correction circuit 27, a volume generation circuit 28, and three dimensions. It includes a memory 29, a three-dimensional image generation circuit 30, a control circuit 31, a storage circuit 32, an input circuit 33, and a display 34. The transmission / reception circuit 21, the two-dimensional image generation circuit 22, the position information collection circuit 24, the position information mapping circuit 25, the alignment circuit 26, the correction circuit 27, the volume generation circuit 28, and the three-dimensional image generation circuit 30 are field programmable gate arrays. (FPGA: field programmable gate array) and the like.

The transmission / reception circuit 21 transmits ultrasonic waves to the ultrasonic probe 11 according to the control signal from the control circuit 31, and also receives a signal (received signal) based on the ultrasonic waves received by the ultrasonic probe. The transmission / reception circuit 21 includes a transmission circuit that generates a drive signal for radiating a transmission wave from the ultrasonic probe 11 and a reception circuit that performs phase adjustment addition to the reception signal from the ultrasonic probe 11.

The transmission circuit includes a rate pulse generator, a transmission delay circuit, and a pulser. The rate pulse generator generates a rate pulse that determines the repetition period of the transmitted wave by dividing the continuous wave or square wave supplied from the reference signal generation circuit, and supplies this rate pulse to the transmission delay circuit. .. The transmission delay circuit is composed of the same number of independent delay circuits as the transducers used for transmission, and has a predetermined delay time and a predetermined delay time for converging the transmitted wave at a predetermined depth in order to obtain a narrow beam width in transmission. The delay time for radiating the transmitted wave in the direction of is given to the rate pulse, and the rate pulse is supplied to the pulser. The pulsar has an independent drive circuit and generates a drive pulse for driving the vibrator built in the ultrasonic probe 11 based on the rate pulse.

The reception circuit of the transmission / reception circuit 21 includes a preamplifier, an A / D (analog to digital) conversion circuit, a reception delay circuit, and an adder circuit. The preamplifier amplifies a minute signal converted into an electric reception signal by the vibrator to secure a sufficient S / N. The received signal amplified to a predetermined magnitude by the preamplifier is converted into a digital signal by the A / D conversion circuit and sent to the reception delay circuit. The reception delay circuit outputs a focusing delay time for focusing the reflected wave at a predetermined depth and a deflection delay time for setting the reception directivity in a predetermined direction from the A / D conversion circuit. Give to the received signal. The adder circuit adds the received signal from the reception delay circuit in phase-aligned addition (adds the received signals obtained from a predetermined direction in phase with each other).

The two-dimensional image generation circuit 22 relates to a plurality of two-dimensional image data in chronological order, that is, two related to a plurality of frames, based on the reception signal input from the reception circuit of the transmission / reception circuit 21 according to the control signal from the control circuit 31. Generates 2D image data. Examples of the type of two-dimensional image data include B-mode image data, color-mode image data, application-mode image data such as elastography, and the like.

Normally, the color mode image and the application mode image are superimposed and displayed on the B mode image as the background image, so that the B mode image data is generated even in the mode of generating these images. Since the data area of the color mode image data and the application mode image data is limited, the color mode image data and the application mode image data are not suitable for the alignment process by the alignment circuit 26 described later. Therefore, it is preferable to perform the alignment process using the B mode image data even in the mode of generating these images.

As the form of the two-dimensional image data, raw data composed of a plurality of raster data in the scanning surface P (shown in FIGS. 2A to 2C) related to a certain time phase, or raw data is scanned. SC data after conversion (SC: Scan Conversion) processing can be mentioned. Hereinafter, unless otherwise specified, a case where the two-dimensional image data is raw data will be described as an example.

The two-dimensional memory 23 is a storage circuit having a plurality of memory cells in two axial directions per frame and having the plurality of frames. The two-dimensional memory 23 stores a plurality of raw data in a time series generated by the two-dimensional image generation circuit 22. Since the ultrasonic probe 11 is moved and operated by the operator, the plurality of raw data in the time series, that is, the data at the plurality of positions. Time data related to raster data collection is attached to each raster data constituting each raw data by using a system timer.

The position information collecting circuit 24 controls the magnetic field transmitter 12 to transmit a magnetic field from the magnetic field transmitter 12, and collects a plurality of position information of the ultrasonic probe 11 from the sensor 13 in a time series. The position information collecting circuit 24 collects each position information as raw data position information, that is, position information of the scanning surface related to the raw data. The position information of the scanning surface includes the position and orientation of the scanning surface.

The position information collecting circuit 24 can convert the position information of the sensor 13 into the position information of the scanning surface related to the raw data based on the geometrical positional relationship up to each point of the scanning surface related to the raw data. Further, when the ultrasonic probe 11 is a mechanical 4D probe (shown in FIG. 2B), the position information of the scanning surface is obtained based on the position information of the sensor 13 and the tilt angle (swing angle) of the scanning surface. Be done. When the ultrasonic probe 11 is a 2D array probe (shown in FIG. 2C), the position information of the scanning surface is obtained based on the position information of the sensor 13 and the tilt angle of the scanning surface (angle of the ultrasonic beam). Be done.

The position information associating circuit 25 associates the position information collected by the position information collecting circuit 24 with each of the plurality of raw data generated by the two-dimensional image generation circuit 22. The position information associating circuit 25 compares the time data attached to each of the plurality of raw data with the time data attached to each of the plurality of position information, and is closest to, immediately before, or immediately before the time of each raw data. The position information having the time immediately after is associated with the raw data. Here, the time of each raw data may be the time attached to the first raster data among the plurality of raster data constituting each raw data, or the time attached to the central raster data. It may be the average time of a plurality of raster data.

The method of adjusting the time between the plurality of raw data and the plurality of position information is not limited to the above case. For example, by synchronizing the collection of position information by the sensor 13 and the position information collection circuit 24 with the collection of raw data, the position information may be associated with the corresponding raw data.

The position information associating circuit 25 can attach the position information to each of the plurality of raw data in order to associate the position information with each of the plurality of raw data. For example, the position information associating circuit 25 writes the position information in the header, footer, or the like of each raw data. A plurality of raw data with position information are stored in the two-dimensional memory 23.

Alternatively, the position information associating circuit 25 may write the raw data and the position information in the corresponding table in order to associate the position information with each of the plurality of raw data. Hereinafter, in order to associate the position information with each of the plurality of raw data, a case where the position information is attached to each of the plurality of raw data will be described as an example.

The alignment circuit 26 and the correction circuit 27 function as "correction means". The alignment circuit 26 and the correction circuit 27 perform spatial alignment of the plurality of raw data by correcting the position information attached to the plurality of raw data by the position information associating circuit 25. The alignment circuit 26 performs alignment processing on a plurality of raw data arranged according to the incidental position information, and calculates a plurality of position correction information. The alignment processing method will be described later with reference to FIGS. 4 to 6. Then, the correction circuit 27 corrects the position information attached to each raw data with each position correction information calculated by the alignment circuit 26, and replaces the position information with the corrected position information.

FIG. 3 is a conceptual diagram for explaining an outline of a method for correcting position information attached to two-dimensional image data in the ultrasonic diagnostic apparatus 10.

As shown on the left side of FIG. 3, position information is attached to each two-dimensional image data (raw data or SC data). The position correction information is calculated for each of the two-dimensional image data by performing the alignment process on each of the two-dimensional image data arranged according to the incidental position information. By correcting the position information attached to each two-dimensional image data by each position correction information, the position information attached to each of the plurality of two-dimensional image data is replaced with the corrected position information.

4 to 6 are diagrams for explaining the alignment process.

For alignment, SC data whose images directly correspond to the spatial shape of the subject is used. However, if the coordinate conversion of the alignment processing described below is stored in a table including the conversion from raw data to SC data, it is not always necessary to use SC data for alignment, and raw data may be used. It is advantageous in terms of processing speed to tabulate the conversions in this way. In the following explanation, it is assumed that the conversion is tabulated using raw data, but as described above, the actual alignment is performed by SC data, so in order to facilitate understanding, the SC data shape is shown in the figure. Is used.

Two frames of raw data are selected from a plurality of raw data. As shown in FIG. 4A, one ROI (Region Of Interest) is set as a reference ROI on one of the raw data (reference raw data) of the raw data for two frames. FIG. 4A shows a case where the reference ROI is rectangular, but the reference ROI is not limited to that case.

As shown in FIG. 5, the scanning surface related to the reference raw data and the scanning surface related to the other raw data (moving raw data) are arranged based on the position information attached to each. A virtual projection plane is set between the position of the scanning surface related to the reference raw data and the position of the scanning surface related to the moving raw data. Then, the point of contact of the vertical line of the reference ROI with the projection plane is set as the projection ROI on the projection plane. Subsequently, a straight line perpendicular to the scanning surface is drawn from the projected ROI toward the scanning surface related to the moving raw data, and the contact point between the straight line and the scanning surface related to the moving raw data is set as the moving ROI. The mobile ROI exists on the mobile raw data.

A moving surface whose position and angle match is set on the scanning surface related to the moving raw data. With the moving ROI fixed to the moving surface, the moving raw data is moved (translated and rotationally moved) on the moving surface, so that the image in the moving ROI fixed to the moving surface changes. That is, by moving the moving raw data on the moving surface while the moving ROI is fixed to the moving surface, a plurality of moving raw data portions related to a plurality of positions of the moving ROI (FIG. 6) is obtained.

As shown in FIG. 6, reference projection image data can be obtained by projecting the reference raw data portion in the reference ROI based on the reference raw data onto the projection plane. A plurality of moving projection image data can be obtained by projecting a plurality of moving raw data portions in the moving ROI based on the moving raw data onto the projection surface. For each of the plurality of moving projection image data, the degree of agreement with the reference projection image data is calculated, and the degree of agreement is compared. Then, the moving projection image data when the degree of coincidence is the maximum (the value is the minimum) is considered to match the reference projection image data. Then, the amount of movement from the initial movement row data before the movement to the movement row data related to the movement projection image data when the degree of coincidence is maximum is obtained as the position correction information of the movement row data.

For comparison of the degree of agreement, for example, the correlation of both projected image data is used. As an example, as shown in the following equation (1) or (2), the absolute value of the difference between the pixel values of the projection plane is added over all the pixels in the projection plane, and the position where the addition value is minimized is determined. Therefore, there is a method of assuming that the position is correct. In the following equation (1) or (2), aj indicates the pixel value of the reference projected image data, bj indicates the pixel value of the moving projected image data, j indicates the pixel in the projected ROI, and N Indicates the total number of pixels in the projected ROI. This method is advantageous in terms of processing speed because the processing is simple.

Here, it is desirable that the raw data for the two selected frames are adjacent in time or position. This is because the frame rate is as high as several tens of Hz, and the time difference between the raw data of two adjacent frames is small, so it can be considered that there is no large-scale change in the movement between the two frames. For this reason, the movement of the tilt angle with respect to the moving surface of the moving raw data at the time of alignment has already been omitted and simplified. Further, when the moving raw data is moved on the moving surface, the rotational movement may be omitted and simplified, which improves the processing time. For the same reason, the positions of the scanning surfaces of the raw data for two adjacent frames are already approximately matched by the position information, so that the range in which the moving raw data is moved on the moving surface does not have to be so large. By doing so, the processing time can be considerably shorter than in the case where the alignment processing is performed only with raw data without using the sensor 13.

As described above, the position correction information of the moving raw data is obtained by performing the alignment processing based on the degree of coincidence between the reference projected image data based on the reference raw data and the moving projected image data based on the moving raw data. Is calculated. Further, it is assumed that the above-mentioned calculation process of the position correction information is sequentially performed by using the moving raw data for which the position correction information is calculated as the reference raw data and another raw data as the moving raw data. As a result, for example, when the ultrasonic probe 11 is operated to move the surface of the subject, the alignment is sequentially performed, and the alignment as a whole is performed. Further, by setting the ROI inside the raw data, it is possible to perform the alignment with respect to the inside of the living body without being affected by the unevenness of the surface of the subject.
If the frame interval is narrower than the spatial resolution of ultrasonic waves, the frames may be thinned out. As a result, the total processing time of the plurality of reference raw data can be shortened. In this case, the two adjacent frames refer to the two adjacent frames after being thinned out.

Further, the present invention is not limited to the case where one reference ROI is set on the reference raw data. As shown in FIG. 4B, a plurality of reference ROIs (reference ROI_1 to reference ROI_4) may be set on the reference raw data. When a plurality of reference ROIs are set, an element of position correction information is obtained for each reference ROI. The representative values of these elements are used as the overall position correction information. Here, as a representative value, in addition to a simple average, various measures can be taken to obtain an appropriate value, such as averaging by excluding a value that is separated from another value by a certain value or more, taking a median, and the like.

Returning to the description of FIG. 1, the correction circuit 27 corrects the position information attached to each of the plurality of raw data stored in the two-dimensional memory 23 based on the position correction information obtained by the alignment circuit 26. , Replace the position information with the corrected position information.

The volume generation circuit 28 arranges a plurality of raw data stored in the two-dimensional memory 23 in the three-dimensional memory 29 according to the corrected position information, and performs three-dimensional reconstruction in which interpolation processing is performed as necessary. As a result, volume data is generated in the three-dimensional memory 29. A known technique is used as the interpolation processing method. Known techniques are described, for example, in non-patent documents (Trobaugh, JW, Trobaugh, DJ, Richard WD "Three-Dimensional Imaging with Stereotactic Ultrasonography", Computerized Medical Imaging and Graphics, 18: 5, 315-323, 1994.). The technology that has been done is mentioned.

The technology of the non-patent document is to arrange raw data for two adjacent frames in space using position information, and to obtain the pixel value on the surface between them from the value of the proximity point (pixel) to the nearest neighbor (nearest neighbor). It is obtained by interpolation such as nearest neighbor), bilinear interpolation, and bicubic interpolation. The volume generation circuit 28 corrects the collected plurality of position information according to the correction by the correction circuit 27, and uses the technique of the non-patent document based on the plurality of raw data arranged according to the corrected plurality of position information. And generate volume data.

It should be noted that the two-dimensional image data collection, the position information collection, the position information attachment, the alignment, the position information correction, and the volume data generation may be performed in sequence. Here, it is preferable that all or part of the alignment, the position information correction, and the volume data generation are performed in parallel with the two-dimensional image data collection, the position information collection, and the position information attachment. From the processing contents shown above, the alignment, the position information correction, and the volume data generation can be started from the time when several frames are collected even if all the two-dimensional image data are not prepared. If the processing is performed in parallel in this way, the position information correction or the volume data generation is completed almost at the same time when all the two-dimensional image data are collected, so that the waiting time from the end of data collection to the volume data generation is significantly shortened. ..

The three-dimensional memory 29 is a storage circuit including a plurality of memory cells in the three-axis directions (X-axis, Y-axis, and Z-axis directions). The three-dimensional memory 29 stores the volume data generated by the volume generation circuit 28. Considering the case where the ultrasonic probe 11 is translated in the direction intersecting the scanning surface, the ultrasonic probe 11 is usually used with the X-axis and the Y-axis as the scanning surface and the Z-axis as the moving direction in order to secure spatial resolution. The scanning surface is set so as to fill the X-axis and Y-axis surfaces.

The three-dimensional image generation circuit 30 uses the volume data stored in the three-dimensional memory 29 as a three-dimensional image such as MPR (Multi-Planar Reconstruction) processing, volume rendering processing, surface rendering processing, and MIP (Maximum Intensity Projection) processing. Apply processing. Further, the three-dimensional image generation circuit 30 generates three-dimensional image data such as MPR image data, volume-rendered image data, surface-rendered image data, and MIP image data by performing three-dimensional image processing on the volume data. Then, the three-dimensional image generation circuit 30 displays the three-dimensional image data as a three-dimensional image on the display 34.

The three-dimensional image displayed on the display 34 by the three-dimensional image generation circuit 30 is equivalent to that shown in FIG. 11 (B). The three-dimensional image shown in FIG. 11 (B) is significantly improved in tissue distortion as compared with the conventional three-dimensional image shown in FIG. 11 (A).

The control circuit 31 includes a dedicated or general-purpose CPU (central processing unit), MPU (micro processor unit), GPU (Graphics Processing Unit), an application specific integrated circuit (ASIC), and programmable. It means a logical device or the like. Examples of the programmable logic device include a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). Can be mentioned. The control circuit 31 comprehensively controls the processing operations of the respective units 21 to 30, 32 to 34 by reading and executing a program stored in the storage circuit 32 or directly incorporated in the control circuit 31.

Further, the control circuit 31 may be composed of a single circuit or a combination of a plurality of independent circuit elements. In the latter case, the storage circuit 32 for storing the program may be provided individually for each circuit element, or the single storage circuit 32 may store the program corresponding to the functions of the plurality of circuit elements. good.

The storage circuit 32 is composed of semiconductor memory elements such as RAM (random access memory) and flash memory, a hard disk, an optical disk, and the like. The storage circuit 32 may be composed of a USB (universal serial bus) memory and a portable medium such as a DVD (digital video disk). The storage circuit 32 stores various processing programs (including an OS (operating system) and the like in addition to the application program) used in the control circuit 31 and data necessary for executing the program. In addition, the OS may include a GUI (graphical user interface) that makes extensive use of graphics for displaying information on the display 34 to the operator and allows basic operations to be performed by the input circuit 33.

The input circuit 33 is a circuit for inputting a signal from an input device that can be operated by an operator, and here, the input device itself is also included in the input circuit 33. Input devices include pointing devices (eg, mice), keyboards, trackballs, and various buttons. When the input device is operated by the operator, the input circuit 33 generates an input signal corresponding to the operation and outputs the input signal to the control circuit 31. The device main body 14 may include a touch panel in which the input device is integrally configured with the display 34.

The input circuit 33 outputs the transmission conditions set by the operator to the control circuit 31. The transmission condition is, for example, the center frequency of ultrasonic waves transmitted via the ultrasonic probe 11. The center frequency is the scanning method (linear, convex, sector, etc.), the part to be diagnosed of the subject, the mode of ultrasonic diagnosis (B mode, Doppler mode, color Doppler mode, etc.), from the surface of the subject to the part to be diagnosed. It depends on the distance etc.

Further, the input circuit 33 includes a data collection start button that can be operated by the operator, a data collection end button, a switch for switching whether or not to correct the position information, and the like.

The display 34 is composed of a general display output device such as a liquid crystal display or an OLED (organic light emitting diode) display. The display 34 displays the three-dimensional image data generated by the three-dimensional image generation circuit 30 or the like as a three-dimensional image under the control of the control circuit 31.

Subsequently, the operation of the ultrasonic diagnostic apparatus 10 will be described with reference to FIGS. 1 and 7.

FIG. 7 is a flowchart showing the operation of the ultrasonic diagnostic apparatus 10.

When the button for starting data collection as the input circuit 33 is pressed by the operator, the transmission / reception circuit 21 controls the ultrasonic probe 11 to execute transmission / reception of ultrasonic waves, and collects data related to a plurality of frames ( Step ST1). The two-dimensional image generation circuit 22 generates a plurality of raw data in chronological order based on the data collected in step ST1 (step ST2).

The position information collecting circuit 24 collects a plurality of position information of the ultrasonic probe 11 from the sensor 13 as position information of each raw data (step ST3). The position information associating circuit 25 attaches the position information collected in step ST3 to each raw data generated in step ST2 (step ST4). The plurality of raw data to which the position information is attached by step ST4 is stored in the two-dimensional memory 23.

The alignment circuit 26 and the correction circuit 27 adjust the spatial positional relationship by correcting the position information attached to each of the plurality of raw data. Specifically, the alignment circuit 26 performs alignment processing on a plurality of raw data arranged according to the position information attached in step ST4, and calculates a plurality of position correction information. Then, the correction circuit 27 corrects the position information attached to each raw data with each position correction information calculated by the alignment circuit 26, and replaces the position information with the corrected position information.

The alignment circuit 26 sets the reference raw data and the corresponding moving raw data from the plurality of raw data (step ST5). As shown in FIG. 4A, the alignment circuit 26 sets the reference ROI on the reference raw data set in step ST5 (step ST6).

As described with reference to FIG. 5, the alignment circuit 26 sets the moving ROI on the scanning surface related to the moving raw data based on the reference ROI of the reference raw data (step ST7). As described with reference to FIG. 5, the alignment circuit 26 moves (translates and rotates) the moving raw data on the moving surface in a state where the moving ROI is fixed to the moving surface (step ST8). As a result, a plurality of moving raw data portions related to a plurality of positions of the moving ROI can be obtained based on the moving raw data.

As described with reference to FIG. 6, the alignment circuit 26 acquires the reference projected image data by projecting the reference raw data portion in the reference ROI based on the reference raw data onto the projection plane, and is based on the moving raw data. The moving projection image data is acquired by projecting the moving raw data portion in the moving ROI onto the projection plane. Then, the alignment circuit 26 calculates the degree of coincidence with the reference projection image data for the moving projection image data as described with reference to FIG. 6 (step ST9). The alignment circuit 26 determines whether or not to calculate another degree of coincidence, that is, whether or not to further move the moving raw data on the moving surface (step ST10).

If the judgment in step ST10 is YES, that is, if it is determined to move the moving data on the moving surface, the alignment circuit 26 moves the moving raw data on the moving surface with the moving ROI fixed to the moving surface. Move (step ST8).

On the other hand, when it is determined in step ST10 that NO, that is, the movement data is not moved on the moving surface, the alignment circuit 26 compares the plurality of matching degrees calculated in step ST9 and moves. The amount of movement from the position of the previous initial movement raw data to the position of the movement raw data related to the movement projection image data when the degree of coincidence is maximum is calculated as position correction information (step ST11).

The alignment circuit 26 determines whether or not to calculate the position correction information from other raw data (step ST12). If YES in step ST12, that is, if it is determined that the position correction information is calculated from other raw data, the alignment circuit 26 sets the reference raw data and the corresponding moving raw data in the other raw data. (Step ST5).

On the other hand, if NO in step ST12, that is, the position correction information is not calculated from other raw data, that is, it is determined that the position correction information has been calculated for the desired raw data, the correction circuit 27 is determined by step ST4. The position information attached to each of the plurality of raw data is corrected based on the position correction information calculated in step ST11, and the position information is replaced with the corrected position information (step ST13). When the button for ending data collection as the input circuit 33 is pressed by the operator, the data collection started in step ST1 is ended at an arbitrary timing.

According to the ultrasonic diagnostic apparatus 10, a plurality of raw data are arranged based on the position information associated with each of the plurality of raw data, and the position information is appropriately adjusted by aligning the two arranged raw data with each other. Can be corrected to.

2. Ultrasonic diagnostic device according to the second embodiment In addition to the above-mentioned ultrasonic diagnostic device 10, the ultrasonic diagnostic device according to the second embodiment has a plurality of rows to be aligned for the purpose of improving the accuracy of alignment. Properly image the data.

FIG. 8 is a schematic view showing the configuration of the ultrasonic diagnostic apparatus according to the second embodiment.

FIG. 8 shows the ultrasonic diagnostic apparatus 10A according to the second embodiment. The ultrasonic diagnostic apparatus 10A includes an ultrasonic probe 11, a magnetic field transmitter 12, a sensor 13, and an apparatus main body 14A. In some cases, only the device main body 14A is referred to as an ultrasonic diagnostic device, in which case the ultrasonic diagnostic device is connected to an ultrasonic probe, a magnetic field transmitter, and a sensor provided outside the ultrasonic diagnostic device.

In FIG. 8, the same members as the members of the ultrasonic diagnostic apparatus 10 shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.

The apparatus main body 14A includes a transmission / reception circuit 21, a two-dimensional image generation circuit 22, a two-dimensional memory 23, a position information collection circuit 24, a position information mapping circuit 25, an alignment circuit 26, a correction circuit 27, a volume generation circuit 28, and three dimensions. It includes a memory 29, a three-dimensional image generation circuit 30, a control circuit 31, a storage circuit 32, an input circuit 33A, a display 34, and an image processing circuit 35. The image processing circuit 35 is composed of a field programmable gate array or the like.

The input circuit 33A has the same configuration as the input circuit 33 shown in FIG. Further, the input circuit 33A includes a button for starting data collection, a button for ending data collection, a switch for switching whether or not to correct the position information, and whether to perform image processing on a plurality of raw data to be aligned. Includes an operator-operable switch to switch between. Further, a switch such as a rotary switch may be provided that can input whether or not to correct the position information and whether or not to perform image processing on a plurality of raw data to be aligned.

The image processing circuit 35 performs image processing on the two-dimensional image data (raw data or SC data) generated by the two-dimensional image generation circuit 22. Examples of the image processing performed by the image processing circuit 35 include (1) histogram conversion processing, (2) speckle reduction processing, and (3) structure enhancement processing.

FIG. 9 is a conceptual diagram for explaining an outline of a method for correcting position information according to the ultrasonic diagnostic apparatus 10A.

As shown in FIG. 9, position information is attached to each two-dimensional image data. Image processing is applied to each two-dimensional image data. Subsequently, the position correction information is calculated for each of the two-dimensional image data by performing the alignment processing on each of the two-dimensional image data after the image processing arranged according to the incidental position information. By correcting the position information attached to each two-dimensional image data by each position correction information, the position information attached to each of the plurality of two-dimensional image data is replaced with the corrected position information.

As the frame serving as the starting point in the position correction processing, the raw data corresponding to the central frame may be used among the plurality of raw data. Alternatively, raw data corresponding to a frame designated by the operator via the input circuit 33 may be used. The correction circuit 27 starts from the frame that is the starting point, and sequentially calculates the position correction information of the raw data corresponding to the adjacent frames.

Here, the image processing performed by the image processing circuit 35, that is, (1) histogram conversion processing, (2) speckle reduction processing, and (3) structure enhancement processing will be described in order.

(1) Histogram conversion process In two-dimensional image data, when the distribution of pixel values (for example, 100-200) is narrower than the display gradation (for example, 0-255), the image becomes difficult to identify due to a small difference in shading. In such a case, the correction of the position information may fail, and the position shift may become large. In order to improve this, the original two-dimensional image data is subjected to a histogram conversion process to optimize the histogram of the two-dimensional image data. 10 (A) and 10 (B) are diagrams showing histograms of two-dimensional image data before and after the histogram conversion process.

Histogram conversion processing uses a polygonal tone curve conversion (a graph representing the gradation conversion function that converts pixel values is called a tone curve), a gamma conversion using a gamma curve, and an S-shaped tone curve. There are conversions and so on. Since it is not practical to manually set and convert this tone curve for each image, it is automated.

One method of automation is to increase the interval between the minimum and maximum values of the pixel value distribution in which the frequency of the histogram shown in FIG. 10 (A) exists, as shown in FIG. 10 (B). It's a way to spread it. More preferably, the minimum value and the maximum value of the pixel value distribution are made to substantially match the minimum value and the maximum value of the display gradation, respectively. Alternatively, another method of automation is to convert the histogram shown in FIG. 10 (A) into a specified distribution. For example, the changed distribution shape is an arbitrary distribution shape including a uniform distribution and a Gaussian distribution over the entire display gradation. The histogram conversion process is not limited to these two methods.

By optimizing the histogram by performing histogram conversion on the two-dimensional image data, the shading difference of the image is easily emphasized, it becomes easy to identify the structure, etc., and the accuracy of the subsequent correction of the position information is improved. NS. As a result, the distortion of the displayed three-dimensional image is further reduced, so that a three-dimensional image having high diagnostic ability can be provided to the diagnostician.

(2), (3) Speckle reduction treatment and structure emphasis treatment Speckle reduction treatment and structure emphasis treatment are performed using known techniques, for example, the techniques described in JP-A-2009-153918. This is at least a nonlinear anisotropic diffusion filter, multi-resolution decomposition / synthesis, high-frequency component control processing, and a method of using these in combination.

The non-linear anisotropic diffusion filter performs isotropic diffusion processing (smoothing processing) on each bright spot of two-dimensional image data when non-directional bright spots are distributed as in speckle. Reduce the speckle of 2D image data. On the other hand, in the case of a linear structure with high brightness such as a blood vessel wall or diaphragm, diffusion is performed in the direction along the line, and diffusion is hardly performed in the direction perpendicular to the line or the edge is emphasized. , Improve the connection without blurring the lines and emphasize the structure.

Multiple resolution decomposition typically uses wavelet transform to decompose the entire image data as two-dimensional image data into partial image data having a high frequency component in the spatial frequency and multiple partial image data having a low frequency component. do. For example, when the size of the original whole image data is 512 pixels × 512 pixels, it can be decomposed into four partial image data (LL, LH, HL, HH) of 256 pixels × 256 pixels by multi-resolution decomposition. LL has low frequency components both vertically and horizontally. HH has high frequency components both vertically and horizontally. In LH and HL, one of the vertical and horizontal sides has a high frequency component, and the other has a low frequency component. Each partial image data is also a 1/2 reduced image (whole image) of the entire original two-dimensional image data. Further, the LL component can be decomposed into multiple resolutions to obtain four partial image data (LLLL, LLLH, LLHL, LLHH) of 128 pixels × 128 pixels. Furthermore, multi-resolution decomposition can also be performed.

Here, consider four partial image data (LL, LH, HL, HH) of 256 pixels × 256 pixels. The LL component is subjected to a non-linear anisotropic diffusion filter treatment to reduce speckles and emphasize the structure. In addition, high frequency component control processing is performed on the LH, HL, and HH components by some method. The purpose of the high frequency component control process is to reduce the speckle of the soft tissue excluding the structural part. In order to exclude the part of the structure from the processing target, the edge information extracted by performing the nonlinear anisotropic diffusion filter processing on the LL component is used. After performing a series of processing on the four partial image data, the speckle is reduced and the structure is reduced by returning to the entire image data of 512 pixels × 512 pixels by multi-resolution composition, typically using the wavelet inverse transform. An image with emphasized objects is obtained.

By using the nonlinear anisotropic diffusion filter, the multi-resolution decomposition / synthesis, and the high frequency component control processing in combination, it is expected that a higher effect can be obtained than when the nonlinear anisotropic diffusion filter is used alone. Further, since the nonlinear anisotropic diffusion filter processing takes time, the processing time is improved by using the LL image having an image size smaller than that of the original overall image data.

In this way, at least by using a nonlinear anisotropic diffusion filter, multiple resolution decomposition / synthesis, high-frequency component control processing, and a combination of these, the speckle is reduced and the structure is emphasized. The effect of alignment fluctuations due to the above is reduced, and the accuracy of alignment (calculation of the degree of coincidence between the reference projection image data and the moving projection image data) is improved by clarifying the structure. As a result, the accuracy of subsequent correction of position information is improved, distortion of the displayed 3D image can be further reduced, high-quality 3D image data can be obtained, and an ultrasonic diagnostic apparatus with high diagnostic ability can be obtained. Can be provided.

FIG. 11A is a diagram showing an example of displaying a three-dimensional image by a conventional ultrasonic diagnostic apparatus. FIG. 11B is a diagram showing an example of displaying a three-dimensional image by the ultrasonic diagnostic apparatus 10A.

11 (A) and 11 (B) show the MPR image of the A surface based on the volume data in the upper left, the MPR image of the B surface based on the volume data in the upper right, and the MPR of the C surface based on the volume data in the lower left. The image is shown, and the volume rendered image based on the volume data is shown in the lower right. In particular, focusing on the MPR image of the B surface shown in the upper right of FIG. 11 (B), the strain of the tissue is significantly improved as compared with the MPR image of the B surface shown in the upper right of FIG. 11 (A). Diagnostic ability is improved.

Subsequently, the operation of the ultrasonic diagnostic apparatus 10A will be described with reference to FIGS. 8 and 12.

FIG. 12 is a flowchart showing the operation of the ultrasonic diagnostic apparatus 10A.

In FIG. 12, the same steps as those of the ultrasonic diagnostic apparatus 10 shown in FIG. 7 are designated by the same reference numerals, and the description thereof will be omitted.

When the switch for switching whether to perform image processing or not as the input circuit 33A is turned on by the operator, after step ST4, the image processing circuit 35 receives the raw data to which the position information is attached by step ST4. Image processing is performed (step ST23). Examples of the image processing performed in step ST23 include (1) histogram conversion processing, (2) speckle reduction processing, and (3) structure enhancement processing.

According to the ultrasonic diagnostic apparatus 10A, a plurality of raw data are arranged based on the position information associated with each of the plurality of raw data, and the position information is appropriately adjusted by aligning the two arranged raw data with each other. Can be corrected to. Further, according to the ultrasonic diagnostic apparatus 10A, the accuracy of correction of position information can be improved by appropriately performing image processing of a plurality of raw data to be aligned.

3. 3. First Modification Example of Ultrasonic Diagnostic Device According to the ultrasonic diagnostic device 10 shown in FIG. 1, the incidental position information is corrected and the tissue portion included in each two-dimensional image data is slid to obtain appropriate position. Can be achieved. In addition, the ultrasonic diagnostic apparatus 10 can also optimize the distortion of the tissue portion included in each two-dimensional image data. In that case, the correction circuit 27 corrects the position information described above and also corrects the distortion of each two-dimensional image data. Specifically, the correction circuit 27 corrects the position information using the position correction information, while correcting the distortion using the distortion correction information for each two-dimensional image data. During the movement operation of the ultrasonic probe 11, the force of pressing the ultrasonic probe 11 against the subject is not constant, so that the biological tissue at the examination site moves, and the biological tissue which is a soft tissue may be deformed. In such a case, the correction circuit 27 corrects the distortion. In particular, it is effective because the image obtained by photographing the mammary gland tends to be distorted.

There are two methods for correcting the distortion: a method using a linear transformation, a method using a distortion correction vector, and a method using image mosaicing. Further, there are two methods using the distortion correction vector. There are also two methods using image mosaicking.

First, a method using a linear transformation will be described.

The correction circuit 27 performs linear conversion on the two-dimensional image data of the second frame arranged according to the incidental position information, and the image data after the linear conversion related to the second frame is the image before the deformation of the soft structure. Consider it as data. Since the frame rate is as fast as several tens of Hz and the time difference between the two frames is small, and the deformation is not intentionally applied, it is considered that the degree of deformation of the soft tissue between the images of the two adjacent frames is small. , The distortion may be corrected by a linear transformation.

That is, the correction means includes the projected image data based on the two-dimensional image data related to the first frame (corresponding to the reference projected image data shown in FIG. 6) and the plurality of linearly converted image data related to the second frame (FIG. 6). The degree of coincidence of the above formula (1) or (2) is calculated with (corresponding to the moving projection image data shown in 6).

Here, examples of the linear conversion process include enlargement / reduction, rotation, skew, and a composite conversion process thereof. Further, as described above, since the degree of deformation of the soft tissue between the two-dimensional image data of two adjacent frames is considered to be small, the linear conversion process can be simplified by, for example, focusing on enlargement / reduction. ..

FIG. 13 is a diagram for explaining a linear conversion process of two-dimensional image data.

The left side of FIG. 13 shows an image based on the original two-dimensional image data. The right side shows an image based on the image data after the original two-dimensional image data is enlarged or reduced.

It should be noted that the above linear transformation and the movement of the moving raw data on the moving surface described with reference to FIG. 5 can also be used in combination.

Further, the linear transformation may be performed a plurality of times by changing the coefficient of the linear transformation and the processing format. For example, trials including the movement of raw data are performed for a plurality of types of linear transformations, the degree of matching in each trial is compared, and the linear transformation and the movement of raw data that maximize the degree of matching are adopted. As another example, the state without linear transformation is set as the initial state, the first linear transformation is performed, the trial including the movement of raw data is performed, the degree of agreement between the two is compared, and the coefficient and processing of the next linear transformation are performed. Linear transformation and low that maximize the degree of agreement by using optimization methods such as the steepest descent method and the downhill simplex method, which gradually increase the degree of agreement while deciding the format and retrying. Adopt data movement.

By linearly transforming the two-dimensional image data in this way, it is possible to correct the position information and the distortion information.

Next, a method using the distortion correction vector will be described.

The correction circuit 27 calculates the position correction information as described with reference to FIGS. 4 to 6. The position correction information is the magnitude and direction of movement between the position of the movement raw data before the movement and the position of the movement raw data when the degree of agreement is high, and may be called a displacement vector. As shown in FIG. 4B, a plurality of reference ROIs may be set on the reference raw data, and when a plurality of reference ROIs are set, the element of position correction information, that is, each ROI, is set for each reference ROI. The displacement vector is obtained. Each of the plurality of ROIs is called an ROI element. The position correction information, that is, the displacement vector of the entire ROI subtracted from the displacement vector of the ROI element is called the distortion correction vector of the ROI element. When the distortion correction processing is performed by the first method and the second method described below, a plurality of reference ROIs are set. The number, position, shape, etc. of the reference ROI when obtaining the position correction information does not necessarily match the number, position, shape, etc. of the reference ROI when performing the distortion correction processing. For example, the number of reference ROIs for obtaining position correction information may be one, and the number of reference ROIs for performing distortion correction processing may be 20.

First, a first method for correcting distortion using a distortion correction vector for each two-dimensional image data will be described. In the first method, the displacement vector pi of each point (pixel) is calculated, and the displacement vector P of the entire ROI is subtracted from these displacement vectors to obtain the distortion correction vector di of each point.

14 to 15 are diagrams for explaining the distortion correction process.

Two frames of raw data are selected from a plurality of raw data. As shown in FIG. 14A, one ROI is set on one of the raw data for two frames. FIG. 14A shows a case where the ROI is rectangular, but the ROI is not limited to that case.

Further, a plurality of ROI elements, for example, 20 ROI elements are set in the ROI. Then, a displacement vector (shown in FIG. 14B) is obtained for each of the 20 ROI elements. These displacement vectors are regarded as the displacement vectors of the center points of each ROI element.

FIG. 15 shows four ROI elements R11, R12, R21, and R22 extracted from the 20 ROI elements shown in FIG. 14 (B). In the four ROI elements R11, R12, R21, and R22, the displacement vector of a point (pixel) other than the center point is obtained by interpolation from a plurality of displacement vectors related to a plurality of surrounding ROI elements. For example, the displacement vector related to a point other than the center point in the ROI element R11 (triangular mark in the figure) is the four center points in the four ROI elements R11, R12, R21, R22 (circle marks in the figure). ) Is calculated by interpolation based on the displacement vector. As the interpolation, first-order interpolation (bilinear interpolation) and third-order interpolation (bicubic interpolation) are adopted.

In this way, the correction circuit 27 sets the displacement vector related to the center point (circle mark in FIG. 15) in each ROI element and the displacement vector related to the point other than the center point (triangle mark in FIG. 15). , Calculated as the displacement vector pi of each point in the raw data. Then, the distortion correction vector of each point of the raw data is calculated as the distortion correction information of the raw data by di = pi-P and applied to the raw data.

The correction circuit 27 may omit the calculation of the displacement vector related to a point other than the center point, if necessary, in order to shorten the calculation time. For example, the correction circuit 27 calculates one displacement vector for each ROI element as a whole of a plurality of adjacent points, not in units of points (pixels). Alternatively, when the magnitudes of all the displacement vectors related to all the center points are equal to or less than the threshold value (or less than) in the plurality of continuous ROI elements, the correction circuit 27 is other than the center points for the plurality of ROI elements. Do not calculate the displacement vector for the point. Then, the value of the displacement vector of each point in each ROI element is the value of the displacement vector related to the center point of the ROI element. Alternatively, if the magnitude of all the displacement vectors related to the center points in all the ROI elements is equal to or less than the threshold value in the same frame, the correction circuit 27 does not calculate the displacement vectors related to the points other than the center point for the frame. Then, the value of the displacement vector of each point in each ROI element is the value of the displacement vector related to the center point of the ROI element.

According to the first method, since interpolation is performed with a displacement vector having a value larger than that of the distortion correction vector, noise resistance can be improved.

Subsequently, a second method for correcting the distortion using the distortion correction vector for each two-dimensional image data will be described. The correction circuit 27 calculates the position correction information, that is, the displacement vector P of the entire ROI. Next, the displacement vectors p1, p2, ..., P20 of the 20 ROI elements shown in FIG. 14B are calculated. From both, the distortion correction vector which is the distortion correction information of 20 ROI elements is calculated as d1 = p1-P, d2 = p2-P, ..., D20 = p20-P. The distortion correction vectors d1, d2, ..., D20 are regarded as distortion correction vectors at the center points of each ROI element. The interpolation for the displacement vector performed by the first method is applied to each of the distortion correction vectors, and the distortion correction vector di of the point (pixel) other than the center point is calculated from the distortion correction vectors d1, d2, ..., D20. ..

According to the second method, it is not necessary to perform the calculation of the distortion correction vector for each point, so that the time required for the correction process can be shortened.

In the first method and the second method, the position information may be corrected again using the distortion-corrected two-dimensional image data.

Next, a method using image mosaicking will be described.

In image mosaicking, the two-dimensional image data is geometrically transformed by the following steps (a) to (c). Various methods related to image mosaicking have been developed.
(A) Detection and matching of feature points (b) Geometric transformation estimation (c) Geometric transformation of images There are various possible geometric transformations, but here, the method using projection transformation and the displacement vector and distortion. A method using a correction vector will be described.

In the method using the projective transformation, in the above (a), the correction circuit 27 detects a plurality of feature points from each of the two raw data, the reference raw data and the moving raw data. Next, the correction circuit 27 associates (matches) the detected plurality of feature points between the raw data.

In the (b) following the (a), the correction circuit 27 uses a projective transformation as a geometric transformation, which is a more general transformation than the linear transformation described above. As shown in FIG. 16, the projective transformation is a transformation in which an arbitrary quadrangle composed of points Q1 to Q4 is transferred to another arbitrary quadrangle. The projective transformation also includes the movement of image data, and the linearity of the line segment is maintained, but the parallelism is lost. The correction circuit 27 can obtain the projective transformation (value) from the coordinates of the corresponding feature points of the two raw data.

In the (c) following the (b), the correction circuit 27 converts the image data with respect to the moving raw data for which the distortion is corrected by using the projective transformation.

On the other hand, in the method using the displacement vector or the distortion correction vector, the correction circuit 27 associates the plurality of feature points by detecting and matching the plurality of feature points of the reference raw data and the moving raw data. This association can be regarded as the above-mentioned displacement vector. Therefore, the correction circuit 27 uses the displacement vector and the distortion correction vector of points other than the plurality of feature points as the displacement vector and the distortion correction vector of the plurality of feature points in the vicinity, as in the method using the displacement vector and the distortion correction vector described above. The image data is converted by obtaining from.

According to the method using image mosaicking, the processing is more complicated than the method using linear transformation, but distortion correction with high accuracy is possible.

By applying image mosaicing to the two-dimensional image data in this way, it is possible to correct the position information and the distortion information.

As described above, since the distortion of the 3D image displayed by the method using the linear transformation, the method using the distortion correction vector, or the image mosaicking is reduced, it is necessary to provide the diagnostician with a 3D image having high diagnostic ability. Can be done.

FIG. 17 is a conceptual diagram for explaining an outline of a method in which the correction of the position information attached to the two-dimensional image data and the correction of the distortion of the two-dimensional image data are used together in the ultrasonic diagnostic apparatus 10.

As shown in FIG. 17, position information is attached to each two-dimensional image data. The position correction information is calculated for each of the two-dimensional image data by performing the alignment process on each of the two-dimensional image data arranged according to the incidental position information. By correcting the position information attached to each two-dimensional image data by each position correction information, the position information attached to each of the plurality of two-dimensional image data is replaced with the corrected position information.

Distortion correction is performed for each 2D image data by performing distortion correction processing on each 2D image data arranged according to the incidental position information after the correction of the position information or in parallel with the correction of the position information. Information is calculated. Each two-dimensional image data is corrected by the distortion correction information.

As the frame that becomes the starting point in the distortion correction processing, the raw data corresponding to the center frame is used among the plurality of raw data. Alternatively, as the starting frame, raw data corresponding to the frame designated by the operator via the input circuit 33 is used. The correction circuit 27 starts from the frame that is the starting point, and sequentially calculates the distortion correction information of the raw data corresponding to the adjacent frames.

Although the distortion correction process has been described above as being applied to the ultrasonic diagnostic apparatus 10 shown in FIG. 1, the strain correction process can also be applied to the ultrasonic diagnostic apparatus 10A shown in FIG. Specifically, the alignment processing and the distortion correction processing shown in FIG. 17 may be performed on each of the two-dimensional image data after the image processing, which are arranged according to the incidental position information.

4. Second Modified Example of Ultrasonic Diagnostic Device According to the ultrasonic diagnostic device 10 shown in FIG. 1, the operator moves the 1D array probe (shown in FIG. 2A) as the ultrasonic probe 11 in two dimensions. Position correction processing can be performed on the position information of each two-dimensional image data obtained by transmitting and receiving ultrasonic waves to the photographing area. However, the ultrasonic diagnostic apparatus 10 uses a mechanical 4D probe (shown in FIG. 2B) or a 2D array probe (shown in FIG. 2C) capable of photographing a plurality of scanning surfaces as the ultrasonic probe 11. You may.

In that case, the ultrasonic diagnostic apparatus 10 corrects the position of each volume data obtained by transmitting and receiving ultrasonic waves to a three-dimensional imaging region while the operator moves the mechanical 4D probe or 2D array probe. Performs processing and generates composite volume data from multiple volume data. In addition, there may be a case where a plurality of volume data are generated while the operator moves the 1D array probe (shown in FIG. 2A).

FIG. 18 is a conceptual diagram for explaining an outline of a method for correcting position information attached to volume data in the ultrasonic diagnostic apparatus 10.

As shown on the left side of FIG. 18, position information is attached to each two-dimensional image data (raw data or SC data) related to the first frame. Position information is attached to each two-dimensional image data (raw data or SC data) related to the second frame.

By performing interpolation processing on each of the two-dimensional image data related to the first frame, the first volume data related to the first frame is generated. By performing interpolation processing on each of the two-dimensional image data related to the second frame, the second volume data related to the second frame is generated.

Here, the position information of the first volume data is a representative value of a plurality of position information attached to each of the plurality of two-dimensional image data that is the basis of the first volume data. The position information attached to each two-dimensional image data that is the basis of the first volume data is geometrically calculated from the position information of the ultrasonic probe 11. Further, as a representative value, in addition to a simple average, various measures can be taken to obtain an appropriate value, such as averaging by excluding a value that is separated from another value by a certain value or more, taking a median, and the like. The same applies to the position information of the second volume data.

Position correction information is calculated for each volume data by performing alignment processing on each volume data arranged according to the incidental position information. By correcting the position information attached to each volume data by each position correction information, the position information attached to each of the plurality of volume data is replaced with the corrected position information.

Here, the alignment process for synthesizing two volume data for generating the composite volume data will be described.

19 (A) and 19 (B) are conceptual diagrams for explaining the alignment process for synthesizing two volume data.

FIG. 19A shows a side view of the 2D array probe 11 before and after movement by the operator. FIG. 19B is a front view of the 2D array probe 11 before and after movement by the operator. 19 (A) and 19 (B) show the first volume data V1, the second volume data V2, and their three-dimensional superposed region V.

One (or a plurality of) three-dimensional ROIs are set in the three-dimensional superimposition region V in the first volume data V1. When the first volume data V1 is used and the three-dimensional ROI is set in the three-dimensional superposition region V, the three-dimensional ROI is also set in the second volume data V2.

When the second volume data V2 is moved three-dimensionally (translation and rotation) while the three-dimensional ROI is fixed, the image in the fixed three-dimensional ROI changes. A plurality of moving side partial data in the three-dimensional ROI based on the second volume data V2 can be obtained. For each of the plurality of moving-side partial data, the degree of coincidence with the reference-side partial data in the three-dimensional ROI of the first volume data V1 is calculated, and the degree of coincidence is compared. Then, the partial data on the moving side when the degree of matching is the maximum (the minimum value) is considered to match the partial data on the reference side. Then, the amount of movement from the initial second volume data before the movement to the second volume data related to the partial data on the moving side when the degree of coincidence is maximum is obtained as the position correction information of the second volume data V2.

As the frame that becomes the starting point in the position correction process, the volume data corresponding to the central frame is used among the plurality of volume data. Alternatively, as the starting frame, volume data corresponding to the frame designated by the operator via the input circuit 33 is used. The correction circuit 27 starts from the frame that is the starting point, and sequentially calculates the position correction information of the volume data corresponding to the adjacent frames. Further, the correction circuit 27 is not limited to the case where the composite volume data is generated by using the entire plurality of volume data. When the moving speed of the probe by the operator is slower than the scanning of the volume, there are three or more volume data at one point, but the correction circuit 27 uses two adjacent volume data among them. Synthetic volume data may be generated.

In that case, the correction circuit 27 uses the two-dimensional image data corresponding to the central scanning surface among the plurality of two-dimensional image data that are the basis of each volume data. The correction circuit 27 performs the alignment processing of two adjacent volumes by the method according to FIGS. 19A and 19B. The correction circuit 27 obtains the data between the two two-dimensional image data corresponding to the two central scanning surfaces after the alignment by interpolation, and obtains the data between them by interpolation, and the correction circuit 27 obtains the data between them by interpolation, and the correction circuit 27 obtains the data corresponding to the two central scanning surfaces after the alignment. Synthetic volume data is generated based on the two-dimensional image data and the data obtained by interpolation. Since the moving speed of the probe is slow and the two central scanning planes are close to each other, such interpolation is possible.

Alternatively, the correction circuit 27 combines the data between the two central scanning surfaces after alignment with the first volume data V1, the second volume data V2, or the first volume data V1 and the second volume data V2. May be obtained by.

Furthermore, if the probe movement speed by the operator is slower than the volume scanning, there is a large amount of volume data at one point, but not all volume data is required for one point, at least. It suffices to have two volume data. In such a case, the transmission / reception circuit 21 reduces the tilt angle (swing width) of the scanning surface of the 2D array probe (or mechanical 4D probe) to include a range of 2 to a few volume data in which one point is adjacent to each other. It can also be controlled to fit in. By the control, the volume rate for generating the composite volume data is increased, so that the real-time property can be improved. The tilt angle of the scanning surface may be determined from the normal moving speed obtained from clinical experience, may be specified by the operator via the input circuit 33, and may be a position detected by the sensor 13. It may be automatically set from the moving speed of the 2D array probe obtained from the information. By combining the control by the transmission / reception circuit 21 with the generation of the composite volume data described above, the composite volume data can be efficiently generated.

The above description has been made assuming that the position information correction process attached to the volume data is applied to the ultrasonic diagnostic apparatus 10 shown in FIG. 1, but the ultrasonic diagnostic apparatus 10A shown in FIG. 8 is attached to the volume data. It is also possible to apply the position information correction process.

5. Medical image processing device according to this embodiment FIG. 20 is a schematic view showing a configuration of a medical image processing device according to this embodiment.

FIG. 20 shows a medical image processing device 50 according to the present embodiment. The medical image processing device 50 is a medical image management device (image server) (not shown), a workstation, an image interpretation terminal (not shown), or the like, and is provided on a medical image system connected via a network. Further, the medical image processing device 50 may be an offline device.

The medical image processing device 50 includes a control circuit 51, a storage circuit 52, an input circuit 53, a display 54, a communication control circuit 55, a two-dimensional memory 56, and a three-dimensional memory 57.

The control circuit 51 has the same configuration as the control circuit 31 shown in FIG. The control circuit 51 comprehensively controls the processing operations of the respective units 52 to 57 by reading and executing a program stored in the storage circuit 52 or directly incorporated in the control circuit 51.

The storage circuit 52 has a configuration equivalent to that of the storage circuit 32 shown in FIG. The storage circuit 52 stores various processing programs used in the control circuit 51 and data necessary for executing the programs. In addition, the OS may include a GUI that makes extensive use of graphics for displaying information on the display 54 to the operator and allows basic operations to be performed by the input circuit 53.

The input circuit 53 has the same configuration as the input circuit 33 shown in FIG. When the input device is operated by the operator, the input circuit 53 generates an input signal corresponding to the operation and outputs the input signal to the control circuit 51. The medical image processing device 50 may include a touch panel in which the input device is integrally configured with the display 54.

The display 54 has the same configuration as the display 34 shown in FIG. The display 54 displays the three-dimensional image data or the like generated under the control of the control circuit 51 as a three-dimensional image.

The communication control circuit 55 is composed of connectors that match the parallel connection specifications and the serial connection specifications. The communication control circuit 55 has a function of performing communication control according to each standard and being able to connect to a network through a telephone line, whereby the medical image processing device 50 is connected to the network.

The two-dimensional memory 56 has the same configuration as the two-dimensional memory 23 shown in FIG. The two-dimensional memory 56 stores each two-dimensional image data with position information transmitted via the communication control circuit 55.

The three-dimensional memory 57 has a configuration equivalent to that of the three-dimensional memory 29 shown in FIG. The volume data generated by the control circuit 51 is stored.

Subsequently, the function of the medical image processing device 50 according to the present embodiment will be described.

FIG. 21 is a block diagram showing the functions of the medical image processing device 50 according to the present embodiment.
When the control circuit 51 executes the program, the medical image processing device 50 functions as an image processing function 61, an alignment function 62, a correction function 63, a volume generation function 64, and a three-dimensional image generation function 65. The case where the functions 61 to 65 function as software will be described as an example, but some or all of the functions 61 to 65 are provided in the medical image processing apparatus 50 in terms of hardware. You may.

The image processing function 61 has a function equivalent to that of the image processing circuit 35 shown in FIG.

The alignment function 62 has a function equivalent to that of the alignment circuit 26 shown in FIG. 1 or FIG. The alignment function 62 performs alignment based on a plurality of raw data processed or unprocessed by the image processing function 61.

The correction function 63 has a function equivalent to that of the correction circuit 27 shown in FIG. 1 or FIG.

The volume generation function 64 has a function equivalent to that of the volume generation circuit 28 shown in FIG. 1 or FIG.

The three-dimensional image generation function 65 has a function equivalent to that of the three-dimensional image generation circuit 30 shown in FIG. 1 or FIG.

According to the medical image processing apparatus 50, a plurality of raw data are arranged based on the position information associated with each of the plurality of raw data, and the position information is appropriately adjusted by aligning the two arranged raw data with each other. Can be corrected to. Further, according to the medical image processing apparatus 50, the accuracy of correction of position information can be improved by appropriately performing image processing of a plurality of raw data to be aligned.

According to the ultrasonic diagnostic apparatus and the medical image processing apparatus of at least one embodiment described above, the position information of a plurality of two-dimensional image data can be appropriately corrected.

Although some embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

10, 10A ... Ultrasonic diagnostic device 21 ... Transmission / reception circuit 22 ... Two-dimensional image generation circuit 23 ... Two-dimensional memory 24 ... Position information collection circuit 25 ... Position information mapping circuit 26 ... Alignment circuit 27 ... Correction circuit 28 ... Volume generation Circuit 29 ... 3D memory 30 ... 3D image generation circuit 34 ... Display 35 ... Image processing circuit 50 ... Medical image processing device 51 ... Control circuit 54 ... Display 56 ... 2D memory 57 ... 3D memory 61 ... Image processing function 62 … Alignment function 63… Correction function 64… Volume generation function 65… Three-dimensional image generation function

Claims (23)

A transmission / reception means for transmitting ultrasonic waves to the ultrasonic probe and receiving a signal based on the ultrasonic waves received by the ultrasonic probe.
A generation means for generating a plurality of two-dimensional image data in chronological order based on the signal, and
A collecting means for collecting a plurality of position information of the ultrasonic probe in three dimensions, and
Associating means for associating the plurality of position information with the plurality of two-dimensional image data, and
A correction means for correcting the plurality of position information associated with the matching means, and a correction means for correcting the plurality of position information.
Have a,
The correction means is
A projection plane is set between two 2D image data among the plurality of 2D image data, and a projection plane is set.
The two two-dimensional image data are arranged according to the corresponding position information among the plurality of position information, and the two-dimensional image data is arranged.
The two two-dimensional image data are projected onto the projection surface,
The correction is performed by performing an alignment process of two two-dimensional image data on the projection surface.
Ultrasonic diagnostic equipment. Further, it has an alignment means for calculating a plurality of position correction information by performing the alignment process on the plurality of two-dimensional image data arranged according to the plurality of position information.
The correction means corrects the plurality of position information associated with the plurality of two-dimensional image data with the plurality of position correction information.
The ultrasonic diagnostic apparatus according to claim 1. The associating means attaches the plurality of position information to the plurality of two-dimensional image data in order to associate the plurality of position information with the plurality of two-dimensional image data.
The ultrasonic diagnostic apparatus according to claim 1 or 2. A volume generation means that generates volume data by arranging the plurality of two-dimensional image data based on the position information after correction by the correction means, and
A three-dimensional image generation means that generates three-dimensional image data based on the volume data, and
The ultrasonic diagnostic apparatus according to any one of claims 1 to 3, further comprising. The correction means sets one or a plurality of ROIs (Region Of Interest) in the two two-dimensional image data, and performs the alignment process based on the data in the ROI.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 4. The ultrasonic diagnostic apparatus according to any one of claims 1 to 5 , further comprising the ultrasonic probe including a 1D array in which a plurality of vibrators are arranged one-dimensionally. The ultrasonic wave according to any one of claims 1 to 5 , further comprising a 1D array in which a plurality of vibrators are arranged one-dimensionally, and further having the ultrasonic probe for mechanically swinging the 1D array. Diagnostic device. The ultrasonic diagnostic apparatus according to any one of claims 1 to 5 , further comprising the ultrasonic probe including a 2D array in which a plurality of vibrators are two-dimensionally arranged. Further having an image processing means for performing image processing for converting a histogram of each of the two-dimensional image data of the plurality of two-dimensional image data.
The correction means corrects the position information based on the plurality of two-dimensional image data after the image processing.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 8. The image processing means expands the range of pixel values in which the frequency of the histogram exists, or transforms the histogram into a specified distribution.
The ultrasonic diagnostic apparatus according to claim 9. Further having an image processing means for performing image processing for reducing speckle on the plurality of two-dimensional image data.
The correction means corrects the position information based on the plurality of two-dimensional image data after the image processing.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 10. Further, it has an image processing means for performing image processing for emphasizing a structure on the plurality of two-dimensional image data.
The correction means corrects the position information based on the plurality of two-dimensional image data after the image processing.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 10. The image processing means performs, as the image processing, a nonlinear anisotropic diffusion filter, multiple resolution decomposition / synthesis, high frequency component control processing, or a combination thereof.
The ultrasonic diagnostic apparatus according to claim 11 or 12. Each of the plurality of two-dimensional image data is B mode image data.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 13. The ultrasonic diagnostic apparatus according to any one of claims 1 to 14 , further comprising a switch for switching whether or not to correct the position information. All or part of the processing by the correction means is performed in parallel with the processing by the transmission / reception means, the generation means, the collection means, and the association means.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 15. The correction means corrects the position information and also corrects the distortion of the plurality of two-dimensional image data.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 16. The correction means uses a linear transformation to correct the distortion on the plurality of two-dimensional image data.
The ultrasonic diagnostic apparatus according to claim 17. The correction means uses a distortion correction vector to correct the distortion of the plurality of two-dimensional image data.
The ultrasonic diagnostic apparatus according to claim 17. The correction means uses image mosaicking to correct the distortion of the plurality of two-dimensional image data.
The ultrasonic diagnostic apparatus according to claim 17. The collecting means collects the three-dimensional position information from a sensor attached to the ultrasonic probe.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 20. The correction means selects, as the two two-dimensional image data among the plurality of two-dimensional image data, image data corresponding to the corrected position information and image data corresponding to the position information before correction. do,
The ultrasonic diagnostic apparatus according to any one of claims 1 to 21. A medical image processing device that processes a plurality of two-dimensional image data associated with a plurality of three-dimensional position information based on the transmission and reception of ultrasonic waves.
Wherein the alignment process for each sequence said plurality of 2-dimensional image data according to a plurality of position information, it has a correction means for correcting said plurality of position information associated with said plurality of two-dimensional image data,
The correction means is
A projection plane is set between two 2D image data among the plurality of 2D image data, and a projection plane is set.
The two two-dimensional image data are arranged according to the corresponding position information among the plurality of position information, and the two-dimensional image data is arranged.
The two two-dimensional image data are projected onto the projection surface,
The correction is performed by performing an alignment process of two two-dimensional image data on the projection surface.
Medical image processing equipment.

Images