CN114680928A - Ultrasonic scanning feedback system - Google Patents

Abstract

The invention provides an ultrasonic scanning feedback system, which comprises: the ultrasonic probe is used for scanning the scanned part of the detection object to obtain an ultrasonic image of the scanned part of the detection object; a memory storing a computer program; a processor for running the computer program, the computer program when running executing an ultrasound scanning feedback method; the display component correspondingly displays according to the steps needing to be displayed in the ultrasonic scanning feedback method; the ultrasonic scanning feedback method comprises the following steps: acquiring and displaying an ultrasonic image of a scanned part of a detected object in real time; distinguishing and displaying the scanned parts; and identifying whether a focus exists in the ultrasonic image acquired in real time, and if so, distinguishing and displaying a focus area. The invention provides the feedback to the ultrasonic image and/or the operator obtained by the ultrasonic probe, thereby avoiding the omission and improving the efficiency of ultrasonic scanning.

Description

Ultrasonic scanning feedback system

Technical Field

The invention belongs to the technical field of ultrasonic imaging, and particularly relates to an ultrasonic scanning feedback system.

Background

During a clinical ultrasound examination or an ultrasound examination training process, an operator needs to operate an ultrasound probe with one hand and control a control panel of an ultrasound device with the other hand to adjust imaging parameters so as to obtain an ultrasound image meeting diagnosis requirements. All operations and judgment depend on the personal clinical experience of an operator, and the ultrasonic probe does not have corresponding scanning feedback after acquiring the ultrasonic image.

For example, when scanning a breast, an operator needs to move the ultrasonic probe in multiple directions to scan the whole breast; possible problems are:

firstly, if an operator does not know which places the ultrasonic probe has scanned, scanning missing can be caused, and the scanning missing occurs;

secondly, the focus is found in the previous scanning process, but the position corresponding to the original focus is not known after the scanning task is continuously executed, and the scanning needs to be repeatedly carried out, so that the efficiency is low.

Disclosure of Invention

The invention aims to overcome the defects in the prior art and provides an ultrasonic scanning feedback system which is convenient for feeding back ultrasonic images acquired by an ultrasonic probe and/or an operator.

The embodiment of the invention provides an ultrasonic scanning feedback system, which comprises:

the ultrasonic probe is used for scanning the scanned part of the detection object to obtain an ultrasonic image of the scanned part of the detection object;

a memory storing a computer program;

a processor for running the computer program, the computer program when running executing an ultrasound scanning feedback method;

the display component correspondingly displays according to the steps needing to be displayed in the ultrasonic scanning feedback method;

the ultrasonic scanning feedback method comprises the following steps:

acquiring and displaying an ultrasonic image of a scanned part of a detected object in real time;

distinguishing and displaying the scanned parts;

and identifying whether a focus exists in the ultrasonic image acquired in real time, and if so, distinguishing and displaying a focus area.

Further, if a lesion area is identified in the ultrasound image, the identified lesion area and a corresponding lesion category are stored.

Further, the identifying whether a lesion exists in the ultrasound image acquired in real time specifically includes: inputting the real-time acquired ultrasonic image into a pre-trained recognition neural network model, and outputting a focus area and a corresponding focus category through the recognition neural network model.

Further, after identifying the lesion region and the lesion category in the ultrasound image, at least one frame of similar sample ultrasound image is matched from the ultrasound image database, and displayed or a prompt capable of being displayed is given.

Further, the focus area output by the neural network model is identified as an adjustable area, the focus area and the focus category output by the neural network model and the focus area and the focus category determined after adjustment are processed to obtain error information, corresponding sample resampling weights are formed according to the error, and resampling training is carried out on the neural network model according to the obtained sample resampling weights.

Furthermore, the adjusted focus area is marked, and the marked ultrasonic image is stored in an ultrasonic image database and used as sample ultrasonic image data for matching.

Further, the ultrasonic scanning feedback system further comprises an ultrasonic probe position acquisition module, which is used for acquiring the position information of the ultrasonic probe;

the ultrasonic scanning feedback method further comprises the following steps: and acquiring and storing the position information of the ultrasonic probe corresponding to the ultrasonic image containing the focus.

Further, the ultrasound scanning feedback method further comprises the following steps: and (4) giving a guiding prompt to a scanning path from the initial focus section to the target focus section by matching the three-dimensional ultrasonic model.

Further, the three-dimensional ultrasonic model is established in advance; and continuously self-updates in the ultrasonic scanning process;

when the three-dimensional ultrasonic model is established, on one hand, ultrasonic scanning is carried out on a scanned part of a detected object through an ultrasonic probe to obtain an ultrasonic image of each section of a focus of the detected object, and on the other hand, relative spatial position information of the ultrasonic probe relative to the detected object is obtained in real time; and inputting the ultrasonic image of each section and the relative spatial position information of the corresponding ultrasonic probe relative to the detection object into the deep neural network model to obtain a three-dimensional ultrasonic model.

Furthermore, the guidance mode includes one or more modes of image guidance, video guidance, identification guidance, character guidance, light guidance and projection guidance.

Compared with the prior art, the ultrasonic scanning feedback system provided by the application can be used for respectively displaying the scanned part and the identified focus area on the display component during ultrasonic scanning; the scanning path of the ultrasonic probe can be guided and prompted; the missing scanning of operators is avoided, and the efficiency of ultrasonic scanning is also improved.

Drawings

Fig. 1 is a schematic diagram of an ultrasound scanning feedback system in an embodiment of the present invention.

Fig. 2 is a flowchart of an ultrasound scanning feedback method in an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The embodiment of the invention provides an ultrasonic scanning feedback system, which comprises:

the ultrasonic probe is used for scanning the scanned part of the detection object to obtain an ultrasonic image of the scanned part of the detection object;

a memory storing a computer program;

a processor for running the computer program, the computer program when running executing an ultrasound scanning feedback method;

the display component correspondingly displays according to the steps needing to be displayed in the ultrasonic scanning feedback method;

in addition, an ultrasonic wave transmitting channel and an ultrasonic wave receiving channel can be further included, which are not the key points of the present application, and many documents are introduced in the prior art, and are not described herein again;

in some embodiments, an ultrasound scanning feedback method comprises:

step S10, acquiring and displaying an ultrasonic image of a scanned part of the detection object in real time;

when ultrasonic scanning is carried out, an ultrasonic image of a scanned part is obtained through an ultrasonic probe, the ultrasonic probe is excited by a transmitting pulse to transmit ultrasonic waves to a target tissue (for example, organs, tissues, blood vessels and the like in a human body or an animal body), an ultrasonic echo with information of the target tissue reflected from a target area is received after a certain time delay, the ultrasonic echo is converted into an electric signal again to obtain an ultrasonic image, and then the ultrasonic image can be displayed through a display component; display components include, but are not limited to, touch screens;

the detection object can comprise a human, an animal, even an artificial model and the like; scanning sites may include, for example, liver, heart, uterus, brain, chest, abdomen, etc.; the ultrasonic image comprises an ultrasonic image and/or an ultrasonic video; the ultrasonic probe can be manually operated by an operator, or can be operated by an auxiliary device (such as a mechanical arm);

step S20, distinguishing and displaying the scanned parts;

in the real-time ultrasonic scanning process, the scanned parts can be displayed in a highlight mode or in different colors through the display component;

step S30, identifying whether a focus exists in the real-time acquired ultrasonic image, and if so, distinguishing and displaying a focus area; optionally, storing the identified lesion area and the corresponding lesion category;

specifically, the ultrasound image acquired in real time can be input into a pre-trained recognition neural network model, and a focus area and a corresponding focus category are output through the recognition neural network model;

identifying that the neural network model comprises an input layer, a plurality of hidden layers and an output layer; identifying a plurality of hidden layers of the neural network model to automatically extract the characteristics of different focuses in the ultrasonic image; the hidden layer comprises a plurality of convolution layers, a plurality of pooling layers and the like; identifying that all hidden layers in the neural network model, the input layer and the hidden layers, and the hidden layers and the output layer are connected through weight parameters; the hidden layer also comprises some settings for preventing overfitting, such as randomly inactivating some weight parameters between the input layer and the hidden layer or between the hidden layer and the output layer, i.e. the back propagation algorithm does not adjust these inactivation weights;

fixing the real-time acquired ultrasonic image to the same size matched with the input layer of the recognition neural network model, and normalizing the ultrasonic image; inputting the normalized ultrasonic image into a trained recognition neural network model, and outputting all bounding boxes which represent the prediction of the focus in the ultrasonic image; screening a bounding box to obtain a focus area and a focus category;

in order to improve the accuracy, an ultrasonic image acquired by an ultrasonic probe in real time is input into a trained recognition neural network model, after a focus area and a focus category in the ultrasonic image are recognized, at least one frame of similar sample ultrasonic image is matched from an ultrasonic image database (storing ultrasonic image data with a plurality of marked focus information), and is displayed or a prompt capable of being displayed is given so as to assist an operator to determine a focus in the ultrasonic image;

the focus area obtained after the ultrasonic image is input into the neural network model for recognition is an adjustable area, such as an adjustable rectangular frame or an elliptical frame; the operator may adjust the adjustable region, for example, by adjusting the operation of the touch display unit, or may adjust the adjustable region by using another algorithm, for example, to obtain an adjusted lesion region; when the focus area output by the neural network model is identified and adjusted by an operator, the focus area and the focus category output by the neural network model and the focus area and the focus category determined by the operator after adjustment are processed to obtain error information, corresponding sample resampling weights are formed according to the error, and the neural network model is identified and resampled according to the obtained sample resampling weights, so that the accuracy of identifying the neural network model is improved; marking the adjusted lesion area, and storing the marked ultrasonic image into an ultrasonic image database as sample ultrasonic image data for matching;

in a better embodiment, the ultrasonic scanning feedback system further comprises an ultrasonic probe position acquisition module for acquiring the position information of the ultrasonic probe; the ultrasonic scanning feedback method can further comprise the following steps:

step S40, acquiring and storing the azimuth information of the ultrasonic probe corresponding to the ultrasonic image containing the focus;

the azimuth information includes position information and angle information;

in some embodiments, the ultrasonic diagnostic apparatus is equipped with a three-dimensional camera, and the three-dimensional camera is used for acquiring relative spatial position information of the ultrasonic probe relative to the detection object so as to obtain azimuth information of the ultrasonic probe corresponding to the ultrasonic image;

in some embodiments, an inertial sensor (IMU) is disposed within the ultrasound probe that can acquire relative spatial position information of the ultrasound probe with respect to the test object; the ultrasonic diagnosis device is provided with the three-dimensional camera, and the inertial sensor is combined with the three-dimensional camera, so that the relative spatial position information of the ultrasonic probe relative to the detection object can be more accurately determined;

in some embodiments, the ultrasonic probe can also obtain relative spatial position information of the ultrasonic probe relative to the detection object through the infrared positioning device; for example, at least 1 infrared transmitter is respectively arranged at four corners of the shell of the ultrasonic probe and used for transmitting infrared light, and meanwhile, an infrared sensor is arranged in a room where the ultrasonic equipment is located and used for receiving the infrared light transmitted by the infrared transmitter, and the infrared transmitter can transmit the infrared light to all directions, so that the relative spatial position information of the ultrasonic probe relative to a detection object can be determined according to the received infrared light;

therefore, when the ultrasonic image of the focus area is identified, the corresponding position information of the ultrasonic probe can be stored at the same time; when an operator wants to perform accurate positioning scanning later, the stored information can be called to quickly position;

in a more preferred embodiment, the method further comprises the following steps:

step S50, a guiding prompt is given to the scanning path from the initial focus section to the target focus section by matching the three-dimensional ultrasonic model;

it can be understood that: if the focus is a sphere, the focus section obtained by scanning with the ultrasonic probe is a process from small to large and then small, or a process from large to small directly, for a certain focus, one or more optimal focus sections for determining the disease condition may exist, namely, the target focus section which the operator wants to obtain from the ultrasonic image; accurate positioning is needed to obtain an ultrasonic image meeting the diagnosis requirement, and an operator is guided to reasonably move an ultrasonic probe to obtain a target focus section;

the three-dimensional ultrasound model can be pre-established; and continuously self-updates in the ultrasonic scanning process;

when the three-dimensional ultrasonic model is established, on one hand, the ultrasonic probe is used for carrying out ultrasonic scanning on the scanned part of the detected object to obtain an ultrasonic image of each section of the focus of the detected object, and on the other hand, the relative spatial position information of the ultrasonic probe relative to the detected object is obtained in real time and can be obtained through the embodiments; inputting the ultrasonic image of each section and the relative spatial position information of the corresponding ultrasonic probe relative to the detection object into a depth neural network model to obtain a three-dimensional ultrasonic model;

then, acquiring ultrasonic image characteristics of the scanned part through a first convolution neural network; the first convolution neural network inputs an ultrasonic image obtained after an ultrasonic probe scans a scanned object, then two layers of convolution and pooling modules are used, the size of a convolution kernel is 3 multiplied by 3, the step length is 1, the number of the convolution kernels is increased by multiples of 32, the size of the kernel of the pooling layer is 2 multiplied by 2, the step length is 2, the number of the modules is consistent with that of the subsequent bilinear interpolation and convolution modules, and the number of the modules can be increased or reduced according to the training test effect; two layers of convolution (convolution kernel 3 multiplied by 3, step length 1) are arranged between the two modules for connection, and feature extraction is enhanced; the number of channels output by the bilinear interpolation layer and the convolution layer is used as an image after feature enhancement and extraction, and a ReLU activation function is added after convolution for relieving the problem of gradient disappearance; a convolution layer is connected behind the front pooling layer, the size of the convolution kernel is 1 multiplied by 1, the purpose is to fuse and extract features, meanwhile, nonlinearity is increased, the fitting capacity of the network is increased, the part can be added with the former to be used as the input of next up-sampling, and the capability of improving network classification is achieved; in the final bilinear interpolation and convolution layer, performing convolution on the output channel number, and outputting the extracted characteristic image data with the same size as the input ultrasonic image;

then, aligning the relative spatial position information of the ultrasonic probe relative to the detection object and the characteristic image to a three-dimensional data model; inquiring the spatial position corresponding to the relative spatial position information of the ultrasonic probe relative to the detection object in the three-dimensional data model of the characteristic image output in the first step, and inserting the characteristic image data to the corresponding spatial position if the three-dimensional data model does not have the characteristic image data at the corresponding spatial position; if existing characteristic image data exist in the corresponding space position, extracting the existing characteristic image data of the three-dimensional data model at the corresponding space position, and obtaining new characteristic image data after fusion by inputting the current characteristic image data output by the first neural network model together with the existing characteristic image data output by the first neural network; inserting the new feature image data after fusion to the corresponding spatial position for updating the original feature image data; specifically, the second convolutional neural network is used for fusing new characteristic image data into the three-dimensional data model and updating the existing characteristic image data at the corresponding spatial position; the network input comprises two input loops, namely a current characteristic image data loop and an existing characteristic image data loop; 1) the current characteristic image data loop (upper loop) is used for inputting characteristic image data of an image processed by a first neural network by an ultrasonic probe; 2) and an existing characteristic image data loop (a lower loop) is used for inquiring existing characteristic image data obtained at a corresponding spatial position in the three-dimensional ultrasonic model through the relative spatial position information of the ultrasonic probe. The second convolution neural network model copies and fuses the two paths of characteristic image data after the first convolution to form a fusion data processing loop of the middle layer; the three data processing loops are processed in the same processing mode, two layers of convolution and pooling modules are respectively adopted, the size of a convolution kernel is 3 x 3, the step length is 1, the number of the convolution kernels is increased by multiples of 32, the size of the kernel of the pooling layer is 2 x 2, the step length is 2, the number of the modules is consistent with that of the subsequent bilinear interpolation and convolution modules, and the number of the modules can be increased or reduced according to the training test effect. The difference of the three processing loops is that the current characteristic image data loop processes current characteristic image data output by the first neural network, the existing characteristic image data loop processes existing characteristic image data in the three-dimensional data model, the middle layer fuses the current characteristic image data and the existing characteristic image data, and the model finally fuses a fused image from extracted characteristics by using bilinear interpolation and convolution; the second convolutional neural network model adopts a multi-loop form, so that the feature extraction is enhanced; and the multi-scale features are respectively fused and are respectively added to the middle loop at different resolutions, and finally, a comprehensive multi-scale information fusion feature image is formed and is used for updating the original ultrasonic image in the three-dimensional data model and finally generating a matched three-dimensional model.

For various focuses, one or more optimal focus sections for determining the disease condition, namely a target focus section, can be predefined in the three-dimensional ultrasonic model; identifying the ultrasonic image acquired in real time, acquiring an initial focus section after determining that a focus exists, matching a three-dimensional ultrasonic model according to the focus category, further acquiring a reasonable scanning path from the initial focus section to a target focus section, and giving a guidance prompt; the guiding mode can comprise one or more modes of image guiding, video guiding, identification guiding, character guiding, light guiding and projection guiding; after the ultrasonic probe reaches the position corresponding to the target lesion section, prompting can be performed in various ways, for example, a prompt tone of a drop is emitted.

The embodiment of the present invention further provides an ultrasound scanning feedback system, including:

a memory storing a computer program;

a processor for executing the computer program, the computer program executing the steps of the method as described hereinbefore.

An embodiment of the present invention further provides a storage medium, in which a computer program is stored, where the computer program is configured to execute the steps of the method as described above when running.

Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to examples, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.

Claims (10)

1. An ultrasound scanning feedback system, comprising:

the ultrasonic probe is used for scanning the scanned part of the detection object to obtain an ultrasonic image of the scanned part of the detection object;

a memory storing a computer program;

a processor for running the computer program, the computer program when running executing an ultrasound scanning feedback method;

the display component correspondingly displays according to the steps needing to be displayed in the ultrasonic scanning feedback method;

the ultrasonic scanning feedback method comprises the following steps:

acquiring and displaying an ultrasonic image of a scanned part of a detected object in real time;

distinguishing and displaying the scanned parts;

and identifying whether a focus exists in the ultrasonic image acquired in real time, and if so, distinguishing and displaying a focus area.

2. The ultrasound scanning feedback system of claim 1,

if a lesion area is identified in the ultrasound image, the identified lesion area and a corresponding lesion category are stored.

3. The ultrasound scanning feedback system of claim 1,

the identifying whether a focus exists in the ultrasonic image acquired in real time specifically comprises: inputting the real-time acquired ultrasonic image into a pre-trained recognition neural network model, and outputting a focus area and a corresponding focus category through the recognition neural network model.

4. The ultrasound scanning feedback system of claim 3,

after the lesion area and the lesion category in the ultrasonic image are identified, at least one frame of similar sample ultrasonic image is matched from the ultrasonic image database, and is displayed or a prompt capable of being displayed is given.

5. The ultrasound scanning feedback system of claim 3 or 4,

recognizing a focus area output by the neural network model as an adjustable area, processing the focus area and the focus category output by the neural network model and the focus area and the focus category determined after adjustment to obtain error information, forming corresponding sample resampling weight according to the error, and performing resampling training on the neural network model according to the obtained sample resampling weight.

6. The ultrasound scanning feedback system of claim 5,

and marking the adjusted lesion area, and storing the marked ultrasonic image into an ultrasonic image database to serve as sample ultrasonic image data for matching.

7. The ultrasound scanning feedback system of claim 1 or 2,

the ultrasonic scanning feedback system also comprises an ultrasonic probe position acquisition module which is used for acquiring the position information of the ultrasonic probe;

the ultrasonic scanning feedback method further comprises the following steps: and acquiring and storing the position information of the ultrasonic probe corresponding to the ultrasonic image containing the focus.

8. The ultrasound scanning feedback system of claim 1 or 2, further comprising:

the ultrasonic scanning feedback method further comprises the following steps: and (4) giving a guiding prompt to a scanning path from the initial focus section to the target focus section by matching the three-dimensional ultrasonic model.

9. The ultrasound scanning feedback system of claim 8,

the three-dimensional ultrasonic model is established in advance; and continuously self-updates in the ultrasonic scanning process;

when the three-dimensional ultrasonic model is established, on one hand, ultrasonic scanning is carried out on a scanned part of a detected object through an ultrasonic probe to obtain an ultrasonic image of each section of a focus of the detected object, and on the other hand, relative spatial position information of the ultrasonic probe relative to the detected object is obtained in real time; and inputting the ultrasonic image of each section and the relative spatial position information of the corresponding ultrasonic probe relative to the detection object into the deep neural network model to obtain a three-dimensional ultrasonic model.

10. The ultrasound scanning feedback system of claim 8,

the guiding mode comprises one or more modes of image guiding, video guiding, identification guiding, character guiding, light guiding and projection guiding.

Images