WO2015078148A1

WO2015078148A1 - Ultrasound-assisted scanning method and system

Info

Publication number: WO2015078148A1
Application number: PCT/CN2014/077325
Authority: WO
Inventors: 温博; 邹耀贤
Original assignee: 深圳迈瑞生物医疗电子股份有限公司
Priority date: 2013-11-28
Filing date: 2014-05-13
Publication date: 2015-06-04
Also published as: CN104680481B; CN104680481A

Abstract

Disclosed in the present invention are an ultrasound-assisted scanning method and system. On the basis of generating a real-time ultrasound image, carrying out matching operations on the acquired real-time image and the library image in the image library, so that the correctness of the current scanning operations is immediately fed back to the user. The method can further output image-text help information according to a matching result. If the current real-time image is the standard section image of an tissue or an organ, the relevant image-text information of the standard section is automatically exported, thereby improving the ease of the operations. If failing to obtain the standard section image, prompting the user on how to adjust the position of the probe, thereby quickly and accurately obtaining the desired image. The method and system of the present invention improve the user interaction and user guidance, and can significantly improve the assisted scanning effect.

Description

Description: An ultrasonic assisted scanning method and system

The invention belongs to the field of ultrasonic imaging technology, and relates to an ultrasonic assisted scanning method and system. Background technique

Ultrasonic instruments are generally used by doctors to observe the internal structure of the human body. The doctor places the operating probe on the surface of the skin corresponding to the human body, and the ultrasound image of the part can be obtained. Ultrasound has become a major aid for doctors' diagnosis because of its safety, convenience, losslessness and low cost. Due to the complexity of the operation of the ultrasonic instrument, it is necessary to operate the doctor to have a very clear understanding of the spatial structure of various organs and tissues of the human body in order to produce standard cut surfaces of various organs and tissues. Newly-introduced ultrasound doctors, clinicians in emerging fields, private clinics Doctors, some nursing staff, and many other people have the need to improve their knowledge and skills in ultrasound, and they all face the reality of lack of training resources.

The advantage of integrated teaching software on the ultrasound system is that the user can learn while practicing, and not only learn the theoretical knowledge through the book, but also greatly improve the training efficiency. However, although the current ultrasound-assisted teaching system can realize the text and text and the teaching according to the steps, but only when the user selects a certain standard aspect, the system displays the standard image of the cut surface and the related graphic interpretation, while scanning The doctor needs to hold the probe in one hand, making it inconvenient to choose the desired cut surface. In addition, the existing ultrasound teaching system has poor operation interaction, and the system cannot give feedback according to the user's real-time operation. Therefore, the user cannot judge the correctness of his operation well, and can not obtain the combination of his own situation in the actual operation. Effective guidance. Therefore, it is necessary to help users learn and improve the ultrasonic scanning technology faster and better. Summary of the invention

In view of the above technical problems existing in the teaching system of the existing ultrasonic instrument integration, the present invention provides a real-time feedback to the user whether the operation is correct, and automatically according to the operation result, the graphic under the standard cut surface is explained in detail or how to adjust the operation. An ultrasound-assisted scanning system and method for obtaining standard cut surfaces.

According to a first aspect of the invention, an ultrasound assisted scanning method is provided. The method includes transmitting ultrasonic waves to a body under test at a probe position, receiving an echo signal reflected by the measured body and generating Pre-real time image;

The method also includes:

Retrieving a plurality of library images from a pre-established image library, the plurality of library images including a standard image reflecting a clinical standard cut surface of the measured body;

Performing similarity matching on the generated real-time image with the plurality of library images;

A matching result of the real-time image and the plurality of library images is output, and the standard image is retrieved according to the matching result or an instruction is provided on how to adjust the probe position to obtain a real-time image that matches the standard image.

In accordance with the field of the invention, an ultrasound assisted scanning system is provided. The system includes: a probe for transmitting ultrasonic waves to a body under test at a probe position and receiving an echo signal reflected by the body to be tested;

a signal processor, configured to process the echo signal and generate a current real-time image according to the display; a display, configured to display the real-time image generated by the output;

The ultrasound-assisted scanning system further includes:

An image library, configured to store a plurality of pre-established library images, the plurality of library images including a standard image reflecting a clinical standard cut surface of the tested body;

An image matching module, configured to perform similarity matching between the generated real-time image and the plurality of library images;

An output configuration module, configured to enable the display to output a matching result of the real-time image and the plurality of library images, and retrieve the standard image according to the matching result or assist in explaining how to adjust the probe position to obtain the standard The image matches the real-time image.

The invention can obtain the following beneficial effects: Through the similarity matching between the real-time image and the library image, the present invention can automatically feedback the correctness of the current scanning operation of the user, and determine whether the obtained image is a standard image of a certain tissue or organ; Further, based on the above matching result, the present invention can automatically call up the graphic information corresponding to the clinical standard cut surface of the image when obtaining the standard image, overcoming the cumbersome operation of the prior art requiring the user to manually select the scanning cut surface; When the standard image is not obtained, the user can provide real-time guidance on how to adjust the position of the probe, which greatly improves the learning efficiency of the user, and enables the user to quickly improve the image scanning level. DRAWINGS In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work. In the figure:

1 is an exemplary flow chart of an ultrasonic scanning method of the present invention;

2 is an exemplary flow chart of an ultrasonic scanning method in the first embodiment of the present invention;

3 is an exemplary flow chart of an ultrasonic scanning method in a second embodiment of the present invention;

4 is a schematic diagram of a position of a feature point on a library image and a corresponding matching point position in a real-time image; FIG. 5 is an exemplary flowchart of an ultrasonic scanning method in a third embodiment of the present invention;

Figure 6 is an exemplary block diagram of an ultrasonic scanning system of the present invention;

Figure 7a is an exemplary block diagram of an ultrasonic scanning system in accordance with a particular embodiment of the present invention;

Figure 7b is another exemplary block diagram of an ultrasound scanning system in accordance with a particular embodiment of the present invention. detailed description

The invention will now be described in detail by way of specific embodiments. It should be understood that although the term "image" is commonly used hereinafter, that is, the specific type of image is not described, since the invention belongs to the field of ultrasonic imaging technology, "image" refers in particular to an ultrasound image.

The invention provides a system scheme for intelligently assisting an ultrasound device for beginners to rapidly improve the ultrasonic scanning technology and knowledge level, and particularly to provide an ultrasound-assisted scanning method and system. 1 is an exemplary flow chart of an ultrasonic scanning method according to the present invention. The basic steps of the method are as follows: The delayed focus pulse is sent to the probe through the transmitting circuit, and the probe transmits ultrasonic waves to the tested body, and receives the signal after a certain delay. Ultrasound reflected from the body under test. The echo signal enters the beam synthesizer, completes focus delay, weighting and channel summation, and obtains real-time ultrasonic image (hereinafter referred to as real-time image) through signal processing, and then performs image on the currently acquired real-time image and pre-established library information. Matching, outputting image help information required for real-time images according to the matching result, and displaying the information and the real-time image on the display.

The "image matching" described above refers specifically to the similarity matching between the real-time image and the gallery information. The process may include the following two steps: A. Building a library and B, similarity matching calculation.

The step A belongs to the offline step, that is, the image library is established before the scan, and only the corresponding library image needs to be retrieved from the image library during the actual scan process. The image library to which the present invention relates includes a detailed human face slice ultrasound library. In the normal human body, the lateral, longitudinal, and oblique directions of each organ or tissue are respectively Fixed a small distance interval to determine a probe placement position. At each probe position, with the probe vertical body reference as the reference, swing to a fixed angle at both sides, and store one ultrasound image per swing as a library image in the image library. The above library image should include, in particular, a standard image that reflects the clinical standard section. Each library image in the gallery contains information on the direction of the facet and the angle of the probe's swing. Among them, the standard image used in clinical practice (especially the standard sonogram), and a series of library images corresponding to the position of the standard image probe need to be specially marked, and the position of the probe corresponding to the clinical standard section and the contour model of the current probe position are recorded. .

This step B is based on the unique features of each library image in the image library, such as a high echo of what shape has a region of a certain image, and the like. According to the present invention, the correlation calculation is performed according to the pattern matching algorithm, the image characteristics based on the library image and the real-time image, and the highest correlation is considered to belong to the corresponding matching image. The present invention can implement image matching based on image blocks or feature point based pattern matching methods. The above-described step B is developed in detail in the specific embodiment below.

The image matching result of the present invention may include: whether the matching is successful; whether the matching real-time image reflects the cut surface corresponding to the cut surface at the standard probe position; and the real-time image corresponding to the standard probe position reflects whether the cut surface corresponds to the standard probe position Clinical standard section. According to the above image matching result, the "image help information" described above may include the following classifications:

Unmatched success: If the real-time image obtained by the user scan cannot match any library image, indicating that the current real-time image does not have a corresponding slice in the library image, the prompt information is given while displaying the reference phantom of the measured body. Prompt that the current scan operation is incorrect and prompt the user how to move the probe in the reference phantom. Gp, the image help information at this time includes reference phantom information, matching failure information, and adjustment prompt information.

Successful match but non-standard probe position: If the real-time slice reflected by the matching real-time image does not correspond to the cut surface at the standard probe position (ie corresponds to the cut surface at the non-standard probe position), then the reference phantom at the output is displayed. Above, the probe position mark (unlike the standard probe position mark) appears at the real-time cut surface position, and the user is informed that there is no standard cut surface at the probe placement position, and the user is prompted to adjust according to the standard probe position on the reference phantom. Gp, the image help information at this time includes reference phantom information, real-time & standard probe position information, and adjustment prompt information.

Non-clinical standard cut in standard probe position: If the real-time image reflected by the real-time image corresponding to the standard probe position does not correspond to the clinical standard cut at the standard probe position (ie, meets the non-clinical standard cut at the standard probe position), this Highlighted on the reference phantom that displays the output (such as highlighting or Special color mark, etc.) The current probe position, prompting the user how much the probe should be left or right, and can feedback to the user whether the current probe is placed in the position of the clinical standard cut surface, so that the user can adjust the probe angle according to the feedback. . Gp, the image help information at this time includes reference phantom information, highlighted probe position information, adjustment prompt information, and clinical standard cut surface judgment information.

At the standard probe position and the clinical standard cut surface: The matching result indicates that the user's scan operation is correct. At this time, not only the probe mark corresponding to the standard probe position is highlighted on the reference phantom, but also displayed (for example, in the help information area). The graphic information corresponding to the clinical standard section, such as standard sonogram, anatomical diagram, scanning technique map, scanning technique, etc. Gp, the image help information at this time includes reference phantom information, highlighted probe position information, and clinical standard cut surface corresponding help information.

The detailed implementation flow of the ultrasonic assisted scanning of the present invention will be described below by way of specific embodiments.

Embodiment 1 : Image block-based matching method (similarity calculation)

As shown in Fig. 2, it discloses an ultrasonic assisted scanning method according to a first embodiment of the present invention, which implements image matching based on a pattern matching method of an image block. This embodiment can be divided into two phases: a pattern matching phase of determining the matching point (step S12) and a search phase for screening the optimal matching library image (step S13). The matching point matching all the feature points is determined on the real-time image by the similarity calculation, and the library image with the highest similarity is selected according to the matching degree of all the feature points in each library image and the corresponding matching points, where the matching is performed. The degree refers especially to the similarity between the two.

The method further includes establishing a preparation phase of the library image feature points before determining the matching points. In order to improve the processing efficiency, the preparation phase is usually completed before the start of the matching, and only the determined feature points need to be called during the real-time operation (step S11).

A number of feature points are determined for each library image in the image library, and the number of feature points can be set as needed, for example, 20 feature points are selected for each image. The manner in which the feature points are established may be various; for example, a plurality of points with relatively obvious features may be manually determined as feature points, such as edge contour points of various organs in the image or intersections of tissues. Since the data volume of the image library is usually large, the feature points of each library image can be automatically calculated by a specific program. One method for automatically calculating image feature points is as follows:

Step 1: Divide the library image into several sub-areas.

Ho step 2: establishing a characteristic point in each sub-region Ρ _¾, Ρ _¾ represents a j-th feature point image. The method for confirming the sub-region feature points can be selected as needed. For example, a gradient or a point with the highest gray scale in the sub-region can be selected as the feature point of the sub-region. Determining the pattern matching phase of the matching point (step S12) requires separately determining a matching point of each feature point of each library image on the real-time image, which includes the following sub-steps:

S121: Take each feature point of each library image in the image library, and determine the search range of each feature point on the current real-time image. The search range is an empirical parameter and can be set as needed. For example, the neighborhood of the corresponding point on the real-time scan image is taken as the search range according to the coordinates of the current feature point. For example, the neighborhood size may be 200*200.

S122: Take a feature point Ρ _3⁄4 , and determine a neighborhood block of size W*H as a template on the library image centering on the current feature point.

S123: Determine a neighboring block with the same size (W*H) as the template size as the center of the plurality of pixels in the region within the determined search range, for subsequent similarity calculation. In order to ensure the matching accuracy, each pixel in the selected area is centered, and each feature point of each library image is matched one by one. However, considering the problem of the amount of calculation, it is also possible to determine a neighboring block at intervals of N points when searching for matching points, thereby improving the overall calculation speed. After searching for the advantages, check them one by one near the point.

S124: Calculate the similarity value of the template of the library image and the neighborhood block of the real-time image, and take the center of the most similar neighborhood block as the matching point of the template feature point in the real-time image. Specifically, first, a library image is selected, and then a similarity value between a template of each feature point of the image library image and each neighborhood block of the real-time image is calculated, and a template with the highest similarity value is determined for each template of the feature point template. The neighborhood block, according to which the matching points corresponding to all the feature points in each library image are obtained. Reselect another library image Repeat the above steps until you get the matching points for all feature points of all library images. There are many methods for measuring the similarity between templates and neighborhood blocks, such as Euclidean distance, Cosme similarity, cumulative pixel difference, cumulative correlation coefficient, etc., and different metrics have different definitions of similarity. In other words, the meaning of "the highest similarity value" described above actually means that the template is most similar to the neighborhood block.

One method of measuring the similarity between a template and a neighborhood block is to calculate the sum of the absolute values of the pixel differences of the template and the neighborhood block. which is

In the above formula, El represents the sum of the absolute values of the pixel differences between the template and the neighborhood block, and is the neighborhood, and I and 1⁄2 respectively represent the gray values of the pixels in the template and the neighborhood block. It can be seen from equation (1) that the smaller the similarity value, the better the similarity. The sum of the number of relationships. which is

In the above formula, E2 represents the sum of the correlation coefficients of the template and the pixels of the neighborhood block, Ω is the neighborhood, and I and 1⁄2 respectively represent the gray values of the pixels of the template and the neighborhood block. It can be seen from equation (1) that the greater the similarity value, the better the similarity.

When the similarity value is calculated by the methods of equations (1) and (2), the matching point most similar to a feature point is the point where the E1 value is the smallest or the E2 value is the largest. It should also be understood that not every feature point will have a corresponding matching point. According to the characteristics of the pixel difference method and the correlation coefficient method, when there is no matching point in a certain feature point, a fixed similarity value is assigned to the feature point, and a larger value is given under the pixel difference method, and a correlation coefficient method is assigned to Smaller value.

S125: The size of the similarity value most similar to a certain feature point is used as a criterion for determining whether or not a matching point is found. This step is a preferred step. An empirical parameter E_Thre (also called the first matching standard value) can be further defined to determine whether the center of the most similar neighborhood block calculated is indeed the matching point of the feature point in the real-time image. For example, in the case of formula (1) as a metric, if the similarity value of a point is greater than E-Thre, it means that the feature point does not have a corresponding matching point in the real-time image, and formula (2) is used as the metric formula. In the case, if the similarity value of the corresponding point is smaller than E-Thre, it indicates that the feature point does not have a corresponding matching point in the real-time image.

The search phase for screening the optimal matching library image (step S13) requires screening a library image that is most similar to the real-time image in the image library, and the first embodiment is based on all feature points of the single-image image and its real-time image. The similarity of the matching points achieves the above screening. The theoretical basis is based on the fact that if the real-time image and the library image are not the same slice, then the library image will have many feature points in the real-time image without matching points; and if it is the same slice, the library image Most feature points have matching points in the live image. Therefore, a method for searching the library image for the most similar image to the current real-time image is to calculate a sum of similarity values of respective feature points in each library image and corresponding matching points in the real-time image (step S131), that is,

^E - =∑ ^E y (3 ) where is the total similarity value of the first library image, which is the Jth feature point in the first library image The similarity value can be calculated by the method exemplified by the formula (1) or (2). As described above, if the jth feature point does not have a matching point on the real-time image, a fixed constant can be used as its similarity value (ie, the above fixed similarity value) to compensate for the feature point where the matching point does not exist. For example, for equation (1), a larger value can be set as the similarity value of the feature point. For equation (2), a smaller value can be set instead of the similarity value of the feature point, for example, set to zero. All library images ^£ After ten operator to select the optimum value when the image similarity database corresponding to the best similarity to the current real-time image of the image library.

Step S132: another empirical parameter threshold (ie, the first slice matching threshold) may be set to determine whether the current library image with the best similarity between the current real-time image and the search is the same slice, and the empirical parameter may be based on the similarity adopted. The metric formula is determined. For example, when the formula (1) is used, if the optimal similarity value is greater than the set matching threshold, the current real-time image is considered to have no image of the same slice in the image library; if the optimality is similar when using equation (2) If the degree value is less than the set face matching threshold, it is considered that the current real-time image does not have the same facet image in the image library.

Embodiment 1 of the present invention can screen the most similar library images in the image library by the similarity calculations of steps S12 and S13. However, this method relies too much on the calculation of similarity. Due to the high noise of the ultrasound image, it may affect the screening of the most similar library images in some cases.

Embodiment 2: Image block-based matching method (topological property)

As shown in FIG. 3, it discloses an ultrasonic assisted scanning method according to a second embodiment of the present invention. This embodiment also implements image matching based on a pattern matching method of an image block, and is also divided into two stages: determining a pattern of matching points. The matching phase (step S22) and the search phase of screening the optimal matching library image (step S23). The specific process for determining the matching point is the same as that of Embodiment 1, and the description will not be repeated here. In the embodiment, when the optimal matching library image is screened, the similarity calculation result is no longer used, but the topological property of the feature point is used for screening. For the feature points of each library image, its relative positional relationship on the plane is fixed. If the current real-time image and the library image are the same slice, although there are certain translation, rotation and scaling, the feature points are These relative positional relationships should be approximately maintained at the corresponding points of the real-time image (as shown in Figure 4). Step S23 includes the following sub-steps:

S231: calculating an angle ^{θ between} each feature point of each library image and its adjacent feature points;

S232: When there is a corresponding matching point in a certain feature point and its adjacent feature points, the angle between the matching point of the feature point on the real-time image and the matching point of the adjacent feature point is calculated, and the corresponding matching is missing at the feature point. When the point is called, a preset fixed angle % (for example, set to 360°) is used as the angle between the matching point of the feature point on the real-time image and the matching point of the adjacent feature point. S233: Calculate the sum of the differences between the angles of all the feature points of each library image and the corresponding matching points, and determine the library image corresponding to the minimum angle difference. which is

S234: Compare the determined minimum angle difference with a second slice matching threshold. If the minimum angle difference is smaller than the second slice matching threshold, use the corresponding library image as the optimal matching library image of the real-time image. This step is also a preferred step to improve the accuracy of the matching judgment.

Alternative embodiment: Reduce the amount of calculation of the matching point confirmation process

In the above embodiments 1 and 2, when the matching point is determined, if the template size is large, a large amount of calculation is caused. In another alternative embodiment, the present invention determines the matching points of the feature points of each library image based on the following steps. When screening the optimal matching library image, the specific method described in any one of Embodiments 1 and 2 may be adopted. . First, the search range corresponding to each feature point is still determined on the real-time image, and the specific process is the same as the above two embodiments.

Then, a template of a certain size is determined centering on each feature point on the library image. After the template image A is taken out, A is a matrix of m*n (m, n is the size of the template), and the template of A'A is calculated. The characteristic value A = 1⁄2, '", A„] (Α' is the transpose of matrix A). Similarly, in the determined search range, the image B corresponding to the neighboring block of the same size is taken as the center of the plurality of pixels in the region. Here, each pixel may be selected as the center, or one pixel may be selected as a neighborhood every N pixels. Block center. Then calculate the neighborhood block eigenvalue of B'B ^{= [} A, '", AJ _(Β ' is the transpose of the matrix )).

Then, the similarity between the template feature value and the neighborhood block feature value ⁷ is measured by the method of the above formula (1) or (2), and the similarity calculation result is used as the template of the library image and the similarity value of the neighborhood block of the real-time image. , and the point with the best similarity can be selected as the matching point of the specific feature point. For example, when calculating using equation (1), the point with the smallest similarity value is selected. Also preferably, a matching standard value may be preset; after obtaining the optimal similarity value, it is determined whether the value meets the range defined by the matching standard value, thereby further increasing the accuracy of the matching operation.

In this embodiment, the template of m*n is simplified to be represented by n eigenvalues, and the eigenvalues can basically express the main characteristics of the template image, and the calculation can be greatly simplified when the similarity calculation is performed by formula (1) or formula (2). the amount.

The image block-based matching method only calculates feature points on the library image, but passes the phase in the real-time image. Similarity matches the matching points of the search feature points. Due to the uncertainty of the target position in the real-time image, the search range is generally large, which results in a relatively large amount of calculation. Another method for correlating real-time images and library images is feature point-based image matching. See Example 3 for a detailed description.

Embodiment 3: Feature point based matching method

As shown in Fig. 5, it discloses an ultrasonic assisted scanning method according to a third embodiment of the present invention, which implements image matching based on a pattern matching method of feature points. The embodiment can be divided into three stages: determining a feature point of the library image and the real-time image (step S31), establishing a feature point correspondence relationship (step S32), and searching a search phase of the optimal matching library image (step S33) . Wherein, the correspondence between the real-time image and the library image feature points is established by the similarity calculation, according to the number of corresponding feature points of each library image, or the similarity of the feature points corresponding to each library image and the real-time image. The sum of the degrees is used to filter out the optimal matching library image. Here, the "corresponding feature point" refers to a point in the library image that has a correspondence with a feature point of the real-time image.

Step S31: acquiring feature points of each of the library images and the real-time images of the plurality of library images respectively. Specific implementations include simultaneous calculation of feature points: The same method is used to calculate feature points for real-time images and library images. Commonly used feature points include corner points, inflection points, edge points, etc. Common calculation methods include selecting the point with the largest gradient in a sub-area as the feature point; or calculating the local autocorrelation of four directions for each point in the image, and then selecting The minimum value taken from the correlation result is taken as the characteristic value of the point, or it is further determined whether the value is greater than the experience threshold, and the point is regarded as a feature point only when the experience threshold is exceeded. The specific implementation method further includes calculating the feature points of the library image first, and then calculating the feature points of the real-time image in real time by using the same calculation method. The feature points of the library image can be calculated and stored preferentially, thereby reducing the amount of calculation during the scanning process.

Step S32: A large number of feature points are obtained in the real-time image and each library image, but not all feature points have a corresponding relationship. A large part of the real-time image feature points may not have corresponding points in the library image. The purpose of this step is to establish a correspondence between the feature points of the real-time image and the feature points of each library image by the similarity calculation. specifically:

Step S321: First, a neighborhood block of the same size is determined centering on each feature point on the real-time image and each library image.

Step S322: Calculate similarity values of the two neighborhood blocks of the library image and the real-time image, and use the library image feature points with the similarity value as the corresponding feature points of the real-time image. Specifically, first, a feature point is selected on the real-time image, and then a library image is selected, and the similarity between the neighborhood block of each feature point of the image library image and the neighborhood block of the feature point of the real-time image is calculated. Value, the best value of the similarity value on the library image The sign is the corresponding feature point of the feature point of the real-time image; reselecting another library image repeats the above steps until the corresponding feature point of the feature point on all the library images is obtained. Then another feature point of the real-time image is re-selected, and corresponding feature points on all the library images are obtained according to the above process. Here, it is preferable to refer to feature points on the library image and the real-time image having the corresponding relationship as feature point pairs.

The following two methods can be used to calculate the similarity value. One is to calculate the similarity values of the two by calculating the sum of the absolute values of the pixel differences of the neighborhood blocks of the library image and the real-time image:

£3 = ∑\ Il -Ir \ where E3 represents the sum of the absolute values of the pixel differences of the neighborhood image of the library image and the real-time image, and II and Ir represent the grayscale of the pixel points in the neighborhood block of the library image and the real-time image, respectively. value.

The other is to calculate the similarity between the two by calculating the sum of the correlation coefficients of the pixels of the neighborhood block of the library image and the real-time image:

Where E4 represents the sum of the correlation coefficients of the pixels of the library image and the neighborhood block of the real-time image, and II and Ir respectively represent the gray values of the pixel points in the neighborhood block of the library image and the real-time image.

S323: This step is a preferred step. The optimal similarity value obtained in step S322 is compared with a predetermined second matching standard value. If the range defined by the matching standard value is exceeded, it is considered that there is virtually no corresponding feature point in the image of the library that matches the feature point of the real-time image. By step S32, it can be determined whether each feature point of the real-time image has corresponding feature points in each image, and in particular, corresponding feature point pairs and their similarities can be determined.

Step S33: The optimal matching library image is selected according to the number of corresponding feature points of each library image, or the sum of the similarity values of the feature point pairs corresponding to each library image and the real-time image. An optimal matching library image screening method is to select a library image corresponding to the largest number of feature points as a matching image. At the same time, the experience threshold can be set. If the number of feature points matching the real-time image is less than the threshold, the match is considered unsuccessful, and the current real-time image does not have a corresponding slice in the library image. Another optimal matching library image screening method is to select the optimal library image of the sum of the similarity values of the feature point pairs between the current real-time image and the single-array image as the matching image, and the empirical threshold can also be set to Match the judgment of success. As described above, when a feature point of a real-time image does not have a corresponding feature point, a fixed similarity value is assigned. The above embodiments 1-3 are all searching for the entire image library. Usually to improve the effectiveness of the auxiliary scan, the number of library images in the image library is large, including ultrasound images of various specific tissues or organs under various orientations and imaging angles. In view of the comprehensive consideration of scanning accuracy and scanning speed, the ultrasonic assisted scanning method of the present invention can receive user input before the start of scanning, and determine the organ & tissue name of the tested body to be scanned, thereby calling the library image. Only for multiple library images that satisfy the organ name, the calculation range of image matching is significantly reduced without affecting the accuracy of image matching calculation.

Referring to Figure 6, the present invention also provides an ultrasound-assisted scanning system that provides real-time feedback of the user's operational correctness through real-time image matching, and guides the user to improve scanning techniques by providing detailed graphic information. Specifically, the ultrasound assisted scanning system includes an imaging subsystem, a scan assist subsystem, and a display subsystem. The imaging subsystem includes a probe 11 and an imaging module 12; wherein the probe 11 is directly in contact with the body to be tested, and is configured to emit ultrasonic waves to the body to be tested and receive echo signals reflected by the body under test at a certain probe position, and the imaging module 12 Signal processing of the echo signal to obtain a real-time image at the current probe position. The scan assistant subsystem includes an image library 13 and an image matching module 14; the image library 13 is configured to store a plurality of library images that are pre-established, and the image matching module 14 is configured to use the real-time image generated by the imaging module 12 and the image library 13 The library image is similarly matched so that the correctness of the ultrasonic scanning operation at the current probe position can be instantly judged or fed back. The display subsystem includes a display 15 and an output configuration module 16; the output configuration module 16 is communicatively coupled to the image matching module 14 and the library image 13 to enable the display 15 to output various matching results after the image matching module 14 obtains the determined matching result. The corresponding graphic information, such as but not limited to the current real-time image, the graphic interpretation of adjusting the position of the probe, and the graphic information corresponding to the standard image, and the like.

Referring to FIGS. 7a and 7b, the image matching module 14 of the above-described ultrasonic assisted scanning system includes a library image feature point acquiring unit 141, a real-time image matching point determining unit 142, and an optimal matching library image screening unit 143. The library image feature point acquisition unit 141 is configured to retrieve feature points of each of the library images of the plurality of predetermined library images. The library image feature points described herein are pre-determined to improve the computational speed of the actual scan, but it is not excluded that the present invention can be implemented by determining the library image feature points in real time. The real-time image matching point determining unit 142 is configured to determine a matching point corresponding to each feature point of each library image in the real-time image by the similarity calculation. The invention adopts image block or feature point based pattern matching method to realize similarity matching calculation. The optimal matching library image screening unit 143 selects the library image with the highest similarity among the plurality of library images according to the matching degree of all the feature points in the single-array image and the corresponding matching points, as the optimal matching library image of the current real-time image. . In an embodiment, the real-time image matching point determining unit 142 first determines a search range corresponding to each feature point on the real-time image according to each feature point determined by the library image feature point acquiring unit 141, for example, taking a neighborhood of the corresponding coordinate point. For the search scope. Then, a template of a certain size is determined centering on each feature point on the library image, and a neighborhood block having the same size as the template is determined centering on a plurality of pixels within the search range. After determining the template of each feature point and the plurality of neighborhood blocks corresponding thereto, the real-time image matching point determining unit 142 calculates the similarity value of each template of the library image and all neighborhood blocks in the real-time image, which can be based on The method of pixel difference or correlation coefficient described in the paper is performed, and the center of the neighborhood block with the template similarity value is taken as the matching point corresponding to the feature point.

In an embodiment, the real-time image matching point determining unit 142 first determines a search range corresponding to each feature point on the real-time image according to each feature point determined by the library image feature point acquiring unit 141, for example, taking a neighborhood of the corresponding coordinate point. For the search scope. Then, a template of a certain size is determined centering on each feature point on the library image, and a neighborhood block having the same size as the template is determined centering on a plurality of pixels within the search range. After the corresponding template image A and the neighborhood block image B are taken out (A and B are both matrices of m*n), the real-time image matching point determining unit 142 calculates the template feature value of A'A ⁼ , ''', «] ( Α ' is the transpose of matrix A) and B'B's neighborhood block eigenvalues ^ ^{= [} A, '", AJ _{( B} ' is the transpose of matrix B), and according to the pixel difference or correlation coefficient described above The method calculates the similarity between the template feature value and the neighborhood block feature value. The similarity calculation result may reflect the similarity value between each template of the library image and the neighborhood block of the real-time image, and the real-time image matching point determining unit 142 accordingly The center of the neighborhood block with the similarity of the template similarity value is used as the matching point corresponding to the feature point.

In an embodiment, the real-time image matching point determining unit 142 also presets a matching standard value. After obtaining the neighborhood block most similar to a template, the similarity values of the two are compared with the matching standard values. Only when the optimal similarity value satisfies the range defined by the matching standard value, it is considered that the current neighborhood block does correspond to the template, and the center thereof is indeed the matching point of the feature points of the template. Different similarity calculation methods have different metrics. For example, when using the pixel difference method, the optimal similarity value should not exceed the matching standard value. On the contrary, when the correlation coefficient method is used, the optimal similarity value should not be less than the matching standard value. .

In an embodiment, the optimal matching library image screening unit 143 calculates the sum of the similarity values of all the feature points of each library image and the corresponding matching points, thereby determining the total similarity value of the image of the library and the real-time image. It should be noted that not every feature point has a matching point in the real-time image, so a fixed similarity value is assigned to the feature point when the feature point lacks the corresponding matching point. Total similarity in all library images The most optimal value is the optimal matching library image of the current real-time image.

In an embodiment, the optimal matching library image screening unit 143 performs the filtering based on the relative positional relationship between the feature points. First, the optimal matching library image screening unit 143 calculates the angle between each feature point of each library image and its adjacent feature points, which is recorded as the angle of the feature point; and then calculates the matching point of the feature point on the real-time image. The angle between the matching points of the adjacent feature points is recorded as the angle of the matching points. Similarly, when there is no matching point in a feature point, a preset fixed angle is called as the angle between the matching point of the feature point on the real-time image and the matching point of the adjacent feature point. The optimal matching library image screening unit 143 then calculates the sum of the differences between the angles of all the feature points of each library image and the corresponding matching points, and determines the library image corresponding to the minimum angle difference, and the library image is The optimal matching library image of the current real-time image.

In an embodiment, the optimal matching library image screening unit 143 also presets a face matching threshold. After the minimum angle difference or the optimal similarity value is obtained, the threshold matching threshold is compared. Only when the minimum angle difference or the optimal similarity value satisfies the range defined by the slice matching threshold, the corresponding library images are selected as the optimal matching library images.

In an embodiment, the image matching module 14 of the ultrasonic assisted scanning system includes a feature point acquisition unit 141', a feature point correspondence establishing unit 142', and an optimal matching library image screening unit 143'. The feature point acquisition sheet 141 'obtains the feature points of each library image and real-time image respectively, and pays particular attention to determining the feature points of the two methods by the same method. The feature point correspondence establishing unit 142' establishes a correspondence relationship between the feature points of the real-time image and the feature points of each library image by the similarity calculation, thereby determining whether each feature point of the real-time image has a corresponding correspondence in each library image. Feature points, and two feature points with corresponding relationships are recorded as feature point pairs. The optimal matching library image screening unit 143' selects the optimality according to the number of corresponding feature points of each library image, or the sum of the similarity values of the feature point pairs corresponding to each library image and the real-time image. Match the library image.

In an embodiment, the feature point correspondence establishing unit 142' first determines a neighborhood block of the same size centering on each feature point on each of the real-time image and each library image, and then calculates each neighborhood block of the real-time image. The similarity value with all neighborhood blocks of each library image is obtained as the neighborhood block in each library image that is most similar to a neighborhood block of the real-time image. The center of the two most similar neighborhood blocks in the real-time image and the library image is the feature point pair, and the feature point on the library image is especially called the corresponding feature point.

In an embodiment, the feature point correspondence establishing unit 142' compares the calculated optimal similarity value with a predetermined matching standard value. If the optimal similarity value satisfies the range defined by the matching standard value, the center of the neighborhood block of the corresponding library image is used as the feature point of the real-time image in the library map. Corresponding feature points on the image.

In an embodiment, the output configuration module 16 configures the display 15 to output different graphics help information based on the specific matching results of the image matching module 14. The image library 13 is pre-stored with the corresponding data (such as images, probe marks, text guidance information, etc.) required to assist the user in performing an efficient scan. The output configuration module 16 recalls the corresponding data information from the image library 13 based on the received matching result, and displays it on the display 15 in real time to form a good interaction with the user. The specific matching results and graphic help information have been detailed in the above, and will not be repeated here.

It will be understood by those skilled in the art that although the ultrasonic assisted scanning system described above includes an image library, an image matching module, and an output configuration module, the above components may not be integrated in the ultrasonic diagnostic apparatus, but function as an ultrasonic diagnostic apparatus. The plug-in is connected to the instrument when the user requires the instrument to provide an auxiliary scan to form the described ultrasound-assisted scanning system.

The above detailed development of the ultrasound-assisted scanning method and system reveals the significant advantages of the present invention over the existing teaching system: 1. The real-time feedback mechanism enables the user to know whether the current scanning operation meets the clinical medical requirements; 2. Standard image The automatic call-out mechanism can save the user from manually selecting the required standard image operation, and the overall user-friendliness is stronger. 3. The probe adjustment prompt mechanism enables the user, especially the beginner user, to know how to properly adjust the probe position and improve Learning efficiency.

It can be understood by those skilled in the art that all or part of the steps of the various methods in the above embodiments may be completed by a program, and the program may be stored in a computer readable storage medium, and the storage medium may include: a read only memory , random access memory, disk or CD, etc.

The above is a detailed description of the present invention in connection with the specific embodiments, and it is not intended that the specific embodiments of the invention are limited to the description. For those skilled in the art, a number of simple deductions or substitutions may be made without departing from the inventive concept.

Claims

claims

1. An ultrasonic-assisted scanning method, which includes transmitting ultrasonic waves to the body under test at a probe position, receiving echo signals reflected by the body under test, and generating a current real-time image accordingly;

It is also characterized by:

Retrieve a plurality of library images from a pre-established image library, where the plurality of library images include standard images reflecting clinical standard sections of the tested body;

performing similarity matching between the generated real-time image and the plurality of library images; and

Output the matching results between the real-time image and the plurality of library images, and retrieve graphic and text information corresponding to the standard image according to the matching results or assist in explaining how to adjust the probe position to obtain a real-time image that matches the standard image. .

2. The ultrasound-assisted scanning method according to claim 1, characterized in that a pattern matching method based on image blocks or feature points is used to perform similarity matching on the real-time image and the plurality of library images.

3. The ultrasonic-assisted scanning method according to claim 2, wherein using an image block-based pattern matching method to perform similarity matching on the real-time image and the plurality of library images includes: retrieving predetermined Feature points of each library image of the plurality of library images;

Determine matching points corresponding to each feature point of each library image in the real-time image through similarity calculation; and

According to the matching degree of all feature points of each library image and the corresponding matching points, the library image with the highest similarity is selected from the multiple library images as the optimal matching library image of the current real-time image.

4. The ultrasound-assisted scanning method according to claim 3, wherein determining the matching point corresponding to each feature point of each library image through similarity calculation includes:

Determine the search range corresponding to each feature point on the real-time image;

Determine a template of a specific size centered on each feature point on the library image;

Determine multiple neighborhood blocks with the same size as the template within the search range, centered on multiple pixels within the search range; Calculate the similarity value between the template of the library image and the neighborhood block of the real-time image, and use the center of the neighborhood block with the optimal similarity value relative to a certain template as the feature point corresponding to the template matching point.

5. The ultrasound-assisted scanning method according to claim 3, wherein determining the matching point corresponding to each feature point of each library image through similarity calculation includes:

Determine a template of a specific size with each feature point as the center on the library image, and calculate the template feature values of the template and its transposed image;

Determine multiple neighborhood blocks with the same size as the template within the search range, centered on multiple pixels within the search range, and calculate the neighborhood block feature values of the neighborhood block and its transposed image;

Calculate the similarity between the template feature value and the neighborhood block feature value, and use the similarity calculation result as the similarity value between the template of the library image and the neighborhood block of the real-time image, relative to a certain The center of the neighborhood block with the best similarity value for the template is used as the matching point corresponding to the feature point of the template.

6. The ultrasound-assisted scanning method according to claim 4 or 5, characterized in that, after calculating the similarity value of the template of the library image and the neighborhood block of the real-time image, the method further includes: :

Compare the optimal similarity value with a predetermined first matching criterion value;

If the optimal similarity value meets the range defined by the first matching standard value, then the center of its corresponding neighborhood block is used as the matching point corresponding to the feature point.

7. The ultrasound-assisted scanning method according to claim 4 or 5, wherein calculating the similarity value between the template of the library image and the neighborhood block of the real-time image includes:

Calculate the similarity value between the template and the neighborhood block by calculating the sum of the absolute values of the pixel differences between the two:

Where El represents the sum of the absolute values of the pixel differences between the template and the neighborhood block, I and ½ represent respectively Indicates the grayscale value of the pixels in the template and the neighborhood block;

Or include:

Calculate the similarity value between the template and the neighborhood block by calculating the sum of the correlation coefficients between the two:

Wherein E2 represents the sum of correlation coefficients of the template and the pixels of the neighborhood block, and I and ½ represent the grayscale values of the pixels of the template and the neighborhood block respectively.

8. The ultrasound-assisted scanning method according to claim 4 or 5, wherein determining the neighborhood blocks of the real-time image includes:

Determine the neighborhood block as the center of each pixel within the search range; or select the center every N pixels in the search range to determine the neighborhood block.

9. The ultrasound-assisted scanning method according to claim 3, wherein screening the optimal matching library image of the real-time image includes: calculating the similarity value between all feature points of each library image and the corresponding matching point. and, the library image with the best sum of similarity values is used as the optimal matching library image of the real-time image: where is the total similarity value of the library image and the real-time image; there is a corresponding matching point at the feature point When _E is the similarity value between the j-th feature point and the corresponding matching point in the first image, when the feature point lacks a corresponding matching point, _E is the preset fixed similarity value of the j-th feature point.

10. The ultrasound-assisted scanning method according to claim 3, wherein screening the optimal matching library images of the real-time images includes:

Calculate the angle between each feature point of each library image and its adjacent feature points;

When the feature point has a corresponding matching point, calculate the angle between the matching point of the feature point on the real-time image and the matching point of its adjacent feature point, and call a preset when the feature point lacks a corresponding matching point. The fixed angle is used as the matching point of the feature point on the real-time image and the matching point of its adjacent feature points. the angle;

Calculate the sum of the differences between the angles of all feature points of each library image and the corresponding matching point angles, and determine the library image corresponding to the minimum angle difference based on this;

Compare the minimum included angle difference with a second slice matching threshold. If the minimum included angle difference is less than the second slice matching threshold, then use its corresponding library image as the optimal matching library for the real-time image. image.

11. The ultrasound-assisted scanning method according to claim 2, wherein using a pattern matching method based on feature points to perform similarity matching on the real-time image and the multiple library images includes: obtaining the multiple library images respectively. The feature points of each library image and the real-time image of each library image; establishing a correspondence between the feature points of the real-time image and the feature points of each library image through similarity calculation, thereby determining the feature points of the real-time image Whether each feature point has a corresponding feature point in each library image;

The optimal matching library image is selected based on the number of corresponding feature points of each library image, or based on the sum of the similarity values of the feature points that have a corresponding relationship between each library image and the real-time image.

12. The ultrasound-assisted scanning method according to claim 11, wherein establishing the correspondence between the feature points of the real-time image and the feature points of each library image through similarity calculation includes: in the real-time Neighborhood blocks of the same size are determined with each feature point as the center on the image and each library image;

Calculate the similarity value between a certain neighborhood block of the real-time image and all the neighborhood blocks of a certain library image to determine the neighborhood block with the highest similarity to the neighborhood block of the real-time image on the library image. ;

Compare the optimal similarity value with a predetermined second matching criterion value;

If the optimal similarity value satisfies the range defined by the second matching standard value, then the center of the neighborhood block of the corresponding library image is used as the corresponding feature point of the real-time image on the library image. Feature points.

13. The ultrasound-assisted scanning method according to claim 12, wherein calculating the similarity value between a certain neighborhood block of the real-time image and all neighborhood blocks of a certain library image includes:

Calculated by calculating the sum of the absolute values of the pixel differences of the neighborhood blocks of the library image and the real-time image The similarity value between the two:

E3 = ∑\ Il -Ir \ where E3 represents the sum of the absolute values of pixel differences of the neighborhood blocks of the library image and the real-time image, II and Ir represent the neighborhoods of the library image and the real-time image respectively The gray value of the pixels in the block;

Or include:

Calculate the similarity value of the library image and the real-time image by calculating the sum of the correlation coefficients of the pixels in the neighborhood blocks of the two:

E4 represents the sum of the correlation coefficients of the pixels in the neighborhood blocks of the library image and the real-time image, and II and Ir respectively represent the grayscale values of the pixels in the neighborhood blocks of the library image and the real-time image.

14. The ultrasonic-assisted scanning method according to claim 1, wherein the matching results include:

Whether the real-time image and the library image match successfully;

Whether the section reflected by the successfully matched real-time image corresponds to the section at the standard probe position; and whether the section reflected by the real-time image corresponding to the standard probe position conforms to the clinical standard section at the standard probe position.

15. The ultrasound-assisted scanning method according to claim 14, wherein the method further includes: if the real-time image and the library image do not match successfully, displaying and outputting the reference volume of the body under test. phantom, and prompts how to adjust the probe position on the reference phantom;

If the section reflected by the successfully matched real-time image corresponds to the section at the non-standard probe position, the reference phantom of the tested body is displayed and output, the probe position corresponding to the real-time image is marked on the reference phantom, and Prompt how to adjust the probe position on the reference phantom;

If the section reflected by the real-time image corresponding to the standard probe position meets the non-clinical standard section at the standard probe position, then the reference phantom of the tested body is displayed and output, and the standard probe position is highlighted on the reference phantom. , and prompts how to adjust the probe angle on the reference phantom to obtain the clinical standard section; If the section reflected by the real-time image corresponding to the standard probe position meets the clinical standard section at the standard probe position, then the reference phantom of the tested body is displayed and output, and the standard probe position is highlighted on the reference phantom, And display graphic information corresponding to the clinical standard section.

16. The ultrasound-assisted scanning method according to claim 1, characterized in that, before retrieving multiple library images, the method further includes:

Enter the name of the tissue or organ of the body to be scanned; and

A plurality of library images corresponding to the name of the tissue or organ are retrieved from a pre-established image library.

17. An ultrasound-assisted scanning system, including:

A probe, used for transmitting ultrasonic waves to the body under test at a probe position and receiving echo signals reflected by the body under test;

An imaging module, used to process the echo signal and generate the current real-time image accordingly;

A display for displaying and outputting the generated real-time image;

It is characterized in that the ultrasound-assisted scanning system also includes:

An image library, used to store a plurality of pre-established library images, the plurality of library images including standard images reflecting clinical standard sections of the tested body;

An image matching module, communicatively connected to the imaging module and the image library, for performing similarity matching between the generated real-time image and the plurality of library images; and

An output configuration module is used to enable the display to output the matching results of the real-time image and the plurality of library images, and to retrieve graphic and text information corresponding to the standard image according to the matching results or to assist in explaining how to adjust the probe position. Obtain a real-time image that matches the standard image.

18. The ultrasound-assisted scanning system according to claim 17, wherein the image matching module includes:

A library image feature point acquisition unit, configured to retrieve the predetermined feature points of each library image of the plurality of library images;

A real-time image matching point determination unit configured to determine matching points corresponding to each feature point of each library image in the real-time image through similarity calculation; and

The optimal matching library image screening unit is used to match each library image based on all feature points of the corresponding library image. Based on the matching degree of the points, the library image with the highest similarity among multiple library images is selected as the optimal matching library image for the current real-time image.

19. The ultrasound-assisted scanning system according to claim 17, characterized in that,

The real-time image matching point determination unit is used for:

Determine multiple neighborhood blocks with the same size as the template within the search range, centered on multiple pixels within the search range;

Calculate the similarity value between each template of the library image and all neighborhood blocks in the real-time image, and use the center of the neighborhood block with the optimal similarity value relative to a certain template of the library image as the Matching points corresponding to the feature points of the template;

or

The real-time image matching point determination unit is used for:

Calculate the similarity between the template feature value and the neighborhood block feature value, use the similarity calculation result as the similarity value of the template of the library image and the neighborhood block of the real-time image, and compare the The center of the neighborhood block with the best similarity value for a certain template of the library image is used as the matching point corresponding to the feature point of the template.

20. The ultrasonic-assisted scanning system according to claim 18, characterized in that,

The optimal matching library image screening unit is used for:

Calculate the sum of similarity values of all feature points of each library image and the corresponding matching points, and use the library image with the optimal sum of similarity values as the optimal matching library image of the real-time image;

or

The optimal matching library image screening unit is used for: Calculate the angle between each feature point of each library image and its adjacent feature points;

When the feature point has a corresponding matching point, calculate the angle between the matching point of the feature point on the real-time image and the matching point of its adjacent feature point, and call a preset when the feature point lacks a corresponding matching point. The fixed angle is the angle between the matching point of the feature point on the real-time image and the matching point of its adjacent feature point;

21. The ultrasound-assisted scanning system according to claim 17, wherein the image matching module includes:

A feature point acquisition unit, configured to acquire feature points of each library image of the plurality of library images and the real-time image;

A feature point correspondence establishment unit, configured to establish a correspondence between the feature points of the real-time image and the feature points of each library image through similarity calculation, thereby determining that each feature point of the real-time image is in each library image. Whether there are corresponding feature points in the image; and

The optimal matching library image screening unit is used to filter out the optimal match based on the number of corresponding feature points of each library image, or based on the sum of the similarities of the feature points that have a corresponding relationship between each library image and the real-time image. Library image.

22. The ultrasonic-assisted scanning system according to claim 21, characterized in that the feature point correspondence relationship establishment unit is used for:

Determine neighborhood blocks of the same size centered on each feature point on the real-time image and each library image;

Calculate the similarity value between a certain neighborhood block of the real-time image and all the neighborhood blocks of each library image to determine the optimal similarity of the neighborhood block with respect to the real-time image on the library image. neighborhood block of values;

Compare the optimal similarity value with a predetermined second matching criterion value; If the optimal similarity value satisfies the range defined by the second matching standard value, then the center of the neighborhood block of the corresponding library image is used as the corresponding feature point of the real-time image on the library image. Feature points.