TWI802243B - Ultrasonic image processing system and its operation method - Google Patents

Abstract

The present disclosure provides an operating method of an ultrasonic image processing system, which includes steps as follows. A region of interest is obtained from each of the plurality of ultrasound images, the region of interest being a tumor region; the region of interest normalized to obtain a normalized region of interest; a transfer learning is performed on the normalized region of interest by using different classifiers, so as to obtain a determination model to determine the status of the tumor region.

Description

Ultrasonic image processing system and its operation method

The present invention relates to a system and its operating method, and in particular to an ultrasonic image processing system and its operating method.

Thyroid tumors are very common diseases. Generally, it is discovered through palpation and examination by a doctor. The prevalence rate is about 65%, and women are more than men.

At present, the first choice for clinical diagnosis of thyroid tumors is ultrasonographic imaging, fine-needle aspiration combined with cytology report or histopathological diagnosis after sectioning with a thick needle. However, this method can only diagnose typical papillary carcinoma. Other classifications Follicular carcinoma, for example, appears similar in ultrasound images but does not have obvious or definite features like papillary carcinoma, and it is difficult to distinguish the abnormality of cell nucleus by cytology. There are software on the market that uses traditional machine learning methods to analyze thyroid tumors on ultrasound images using features such as "anechoic area", "high echoic point", "echo pattern", "texture", and "edge", but it is still not used Deep learning techniques such as convolutional neural networks are used to distinguish between benign and malignant.

The invention proposes an ultrasonic image processing system and its operation method, which improves the problems of the prior art.

In an embodiment of the present invention, the ultrasonic image processing system proposed by the present invention includes a storage device and a processor. The storage device stores at least one instruction, and the processor is electrically connected to the storage device. The processor is used to access and execute at least one instruction to: obtain a region of interest from each of the plurality of ultrasound images, the region of interest is a tumor region; standardize the region of interest to obtain a standardized region of interest; Transfer learning is performed on standardized regions of interest using multiple different classifiers to derive a judgment model to judge the status of the tumor region.

In an embodiment of the present invention, the processor is used for accessing and executing at least one instruction to perform histogram equalization and horizontal flipping on multiple original ultrasound images for data enhancement to obtain multiple ultrasound images.

In an embodiment of the present invention, the processor is configured to access and execute at least one instruction to: obtain a region of interest and other regions from each of a plurality of ultrasound images; The region of interest is preprocessed by dividing the image intensity value of each pixel in the region of interest by the average image intensity value of the rest of the region.

In an embodiment of the present invention, the processor is configured to access and execute at least one instruction to: convert the region of interest to a preset gray level to obtain a standardized region of interest, the standardized region of interest conforms to a plurality of different The input size of the classifier.

In an embodiment of the present invention, the plurality of ultrasonic images include a plurality of gray-scale ultrasonic images and a plurality of elastic ultrasonic images, and the processor is used to access and execute at least one instruction to: decompose the plurality of elastic ultrasonic images be a plurality of red ultrasound images, a plurality of green ultrasound images, and a plurality of blue ultrasound images; normalizing the interest corresponding to each of the plurality of grayscale ultrasound images using a plurality of different classifiers region to obtain a first judgment model; use a plurality of different classifiers to transfer the standardized region of interest corresponding to each of the plurality of elastic ultrasound images to obtain a second Judgment model; using a plurality of different classifiers to transfer the standardized region of interest corresponding to each of the plurality of red ultrasound images to obtain a third judgment model; using a plurality of different classifiers to Transfer learning is performed on the standardized region of interest corresponding to each of the plurality of green ultrasound images to obtain a fourth judgment model; using a plurality of different classifiers to correspond to each of the plurality of blue ultrasound images One of the standardized regions of interest is transferred to obtain the fifth judgment model; a judgment model is selected from the first judgment model, the second judgment model, the third judgment model, the fourth judgment model and the fifth judgment model The judgment model with the highest judgment accuracy.

In one embodiment of the present invention, the operation method of the ultrasonic image processing system proposed by the present invention includes the following steps: obtaining a region of interest from each of a plurality of ultrasonic images, and the region of interest is a tumor region; The region of interest is standardized to obtain a standardized region of interest; multiple different classifiers are used to perform transfer learning on the standardized region of interest to obtain a judgment model to judge the status of the tumor region.

In an embodiment of the present invention, the operation method further includes: performing histogram equalization and horizontal flipping on the plurality of original ultrasonic images for data enhancement to obtain a plurality of ultrasonic images.

In an embodiment of the present invention, the step of obtaining the region of interest from each of the plurality of ultrasonic images includes: obtaining the region of interest and the remaining regions from each of the plurality of ultrasonic images; Before normalization, the region of interest is preprocessed, and the preprocessing is to divide the image intensity value of each pixel in the region of interest by the average image intensity value of the rest of the region.

In an embodiment of the present invention, the step of normalizing the ROI includes: converting the ROI to a preset grayscale level to obtain a standardized ROI, and the standardized ROI conforms to a plurality of different classifications The input size of the device.

In an embodiment of the present invention, the plurality of ultrasonic images include a plurality of gray-scale ultrasonic images and a plurality of elastic ultrasonic images, and the step of using a plurality of different classifiers to perform transfer learning on standardized ROIs includes : Decompose a plurality of elastic ultrasonic images into a plurality of red ultrasonic images, a plurality of green ultrasonic images, and a plurality of blue ultrasonic images; using a plurality of different classifiers will correspond to a plurality of gray-scale ultrasonic images Perform transfer learning on the standardized ROI of each of the plurality of elastic ultrasound images to obtain a first judgment model; use a plurality of different classifiers to perform transfer learning on the standardized ROI corresponding to each of the plurality of elastic ultrasound images transfer learning to derive a second judgment model; performing transfer learning on the standardized region of interest corresponding to each of the plurality of red ultrasound images using a plurality of different classifiers to obtain a third judgment model ; using a plurality of different classifiers to transfer the standardized region of interest corresponding to each of the plurality of green ultrasound images to obtain a fourth judgment model; using a plurality of different classifiers to correspond to Perform transfer learning on the standardized ROI of each of the plurality of blue ultrasound images to obtain a fifth judgment model; from the first judgment model, the second judgment model, the third judgment model, and the fourth judgment model Select a judgment model with the highest judgment accuracy from the fifth judgment model.

In summary, compared with the prior art, the technical solution of the present invention has obvious advantages and beneficial effects. With the ultrasonic image processing system and its operation method of the present invention, the error between the image and the evaluator is removed by the judgment model, making it more objective and accurate.

The above-mentioned description will be described in detail in the following implementation manners, and further explanations will be provided for the technical solution of the present invention.

In order to make the description of the present invention more detailed and complete, reference may be made to the accompanying drawings and various embodiments described below, and the same numbers in the drawings represent the same or similar elements. On the other hand, well-known elements and steps have not been described in the embodiments in order to avoid unnecessarily limiting the invention.

Please refer to FIG. 1, the technical aspect of the present invention is an ultrasonic image processing system 100, which can be applied to computers or widely used in related technical links. The ultrasonic image processing system 100 of this technical aspect can achieve considerable technical progress, and has extensive industrial application value. The specific implementation of the ultrasonic image processing system 100 will be described below with reference to FIG. 1 .

It should be understood that various implementations of the ultrasonic image processing system 100 are described in conjunction with FIG. 1 . In the following description, for ease of explanation, numerous specific details are further set forth in order to provide a comprehensive illustration of one or more embodiments. However, the technology may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to effectively describe the embodiments. The term "for example" used here means "as an example, instance or illustration". Any embodiment described herein as "by way of example" is not necessarily to be construed as preferred or advantageous over other embodiments.

In practice, in an embodiment of the present invention, the ultrasonic image processing system 100 can be a server, a computer host or other computer equipment. In terms of servers, many technologies have been developed or are in development to manage the operation of computer servers, generally providing accessibility, consistency and efficiency. Remote management allows the removal of I/O interfaces for servers (eg display screen, mouse, keyboard, etc.) and the need for network administrator entities to access each server. For example, a large data center comprising many computer servers is typically managed using a variety of remote management tools to configure, monitor and debug server hardware and software.

It should be understood that the use of "about", "approximately" or "approximately" herein is used to modify any quantity that may vary slightly, but such slight changes will not change its essence. Unless otherwise specified in the embodiments, it means that the error range of the numerical value modified by "about", "approximately" or "approximately" is generally allowed within 20%, preferably within 10%. Within, and more preferably within five percent.

In practice, in an embodiment of the present invention, the ultrasonic image processing system 100 can selectively establish a connection with the ultrasonic scanning device 190 . It should be understood that, in the description of the implementation and the scope of the patent application, the description involving "connection" can generally refer to a component that indirectly communicates with another component through wired and/or wireless communication through other components, or a component does not need to go through other components. other elements while being physically connected to another element. For example, the ultrasonic image processing system 100 can indirectly perform wired and/or wireless communication with the ultrasonic scanning device 190 through other components, or the ultrasonic image processing system 100 can be physically connected to the ultrasonic scanning device without passing through other components 190. Those who are familiar with this skill should choose it flexibly according to the needs of the time.

FIG. 1 is a block diagram of an ultrasonic image processing system 100 according to an embodiment of the present invention. As shown in FIG. 1 , the ultrasonic image processing system 100 includes a storage device 110 , a processor 120 and a display 130 . For example, the storage device 110 can be a hard disk, a flash storage device or other storage media, the processor 120 can be a central processing unit, and the display 130 can be a built-in display or an external monitor.

In terms of architecture, the ultrasonic image processing system 100 is electrically connected to the ultrasonic scanning device 190 , the storage device 110 is electrically connected to the processor 120 , and the processor 120 is electrically connected to the display 130 . It should be understood that in the descriptions of the embodiments and the scope of the patent application, the description of "electrical connection" can generally refer to the indirect electrical coupling of one element to another element through other elements, or the direct electrical coupling of an element without passing through other elements. electrically connected to another component. For example, the storage device 110 may be a built-in storage device electrically connected to the processor 120 directly, or the storage device 110 may be an external storage device indirectly connected to the processor 120 through a network device.

In practice, for example, the ultrasonic scanning device 190 can capture the original ultrasonic image of the patient. In practice, for example, the ultrasonic scanning device 190 can capture the original ultrasonic image of the patient's thyroid gland. Although only one ultrasonic scanning device 190 is shown in Figure 1, this does not limit the present invention. In practice, the ultrasonic scanning device 190 can generally refer to one or more ultrasonic scanning devices. Those skilled in the art should Choose flexibly according to the needs at that time.

In an embodiment of the present invention, the storage device 110 stores photos and at least one instruction, and the processor 120 is used to access and execute at least one instruction to: perform histogram equalization (histogram equalization) and Horizontal flipping is used for data augmentation to obtain multiple ultrasound images, so as to avoid overfitting caused by the relatively small number of images in subsequent transfer learning.

Next, the processor 120 is used for accessing and executing at least one instruction to: obtain a region of interest from each of the plurality of ultrasonic images, and the region of interest is a tumor region. The aforementioned ROI can be automatically determined by the software executed by the processor 120 . Or, in other embodiments, the aforementioned ROI can be selected from user-preset areas.

Regarding the above-mentioned region of interest, specifically, in an embodiment of the present invention, the processor 120 is configured to access and execute at least one instruction to: obtain the region of interest and the remaining regions from each of a plurality of ultrasound images ; Before the region of interest is standardized, the region of interest is preprocessed. The preprocessing system divides the image intensity value of each pixel in the region of interest by the average value of image intensity in the rest of the region, so as to effectively reduce the frequency of different ultrasonic scanning devices. 190 or image errors caused by different photographers.

Next, the processor 120 is configured to access and execute at least one instruction to: standardize the ROI to obtain a standardized ROI.

Regarding the above-mentioned standardization, specifically, in an embodiment of the present invention, the processor 120 is used to access and execute at least one instruction to: convert the region of interest to a preset grayscale level (such as: 0-255 grayscale range) to obtain a normalized region of interest that fits the input size of multiple different classifiers in subsequent transfer learning. In this way, the standardized ROI is not only beneficial to subsequent transfer learning, but also can effectively reduce image errors caused by different ultrasonic scanning devices 190 or different photographers.

Next, the processor 120 is used to access and execute at least one instruction to: use a plurality of different classifiers to perform transfer learning on the standardized region of interest, so as to obtain a judgment model to judge the state of the tumor region (such as: benign or malignant). By judging the model, the error between the image and the evaluator is removed, making it more objective and accurate.

In an embodiment of the present invention, the plurality of ultrasonic images include a plurality of gray-scale ultrasonic images and a plurality of elastic ultrasonic (US Elastography) images. Gray-scale ultrasound images can represent the outline of the tumor. In practice, for example, if the outline of the tumor has irregular edges, it means that the chance of malignancy is high. Elasticity ultrasound images can reflect the softness and hardness of the tumor. If the tissue is hard, the elasticity is poor, which is represented by blue, the soft tissue is represented by red, and the part with moderate softness and hardness is represented by green. In practice, for example, if the overall color of the tumor is blue, the probability of malignancy is higher.

Regarding the transfer learning mentioned above, the processor 120 is used to access and execute at least one instruction to: decompose a plurality of elastic ultrasonic images into a plurality of red ultrasonic images, a plurality of green ultrasonic images, and a plurality of blue ultrasonic images image; using a plurality of different classifiers to perform transfer learning on the standardized region of interest corresponding to each of the plurality of gray-scale ultrasound images to obtain a first judgment model; using a plurality of different classifiers to Transfer learning is performed on the standardized region of interest corresponding to each of the plurality of elastic ultrasound images to obtain a second judgment model; using a plurality of different classifiers to correspond to each of the plurality of red ultrasound images Perform transfer learning on the standardized region of interest of the person to obtain the third judgment model; use a plurality of different classifiers to perform transfer learning on the standardized region of interest corresponding to each of the plurality of green ultrasound images , to obtain a fourth judgment model; using a plurality of different classifiers to perform transfer learning on the standardized region of interest corresponding to each of a plurality of blue ultrasound images, to obtain a fifth judgment model; from A judgment model with the highest judgment accuracy is selected from the first judgment model, the second judgment model, the third judgment model, the fourth judgment model and the fifth judgment model, so as to further improve the judgment accuracy.

In order to further explain the operation method of the above-mentioned ultrasonic image processing system 100, please refer to Figs. 1-2 at the same time. Fig. 2 is an operation method 200 of the ultrasonic image processing system 100 according to an embodiment of the present invention. flow chart. As shown in FIG. 2, the operation method 200 includes steps S201-S203 (it should be understood that the steps mentioned in this embodiment can be adjusted according to actual needs unless the order is specifically stated. may even be executed simultaneously or partially simultaneously).

The method of operation 200 may take the form of a computer program product on a non-transitory computer-readable recording medium having computer-readable instructions embodied on the medium. Suitable recording media may include any of the following: Non-volatile memory such as: Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM); Volatile Memory, such as: Static Access Memory (SRAM), Dynamic Access Memory (DRAM), Double Data Rate Random Access Memory (DDR -RAM); optical storage devices, such as: CD-ROM, DVD-ROM; magnetic storage devices, such as: hard disk drive, floppy disk drive.

In step S201, a region of interest is obtained from each of a plurality of ultrasound images, and the region of interest is a tumor region. In step S202, the ROI is standardized to obtain a standardized ROI. In step S203, a plurality of different classifiers are used to carry out transfer learning on the standardized ROI to obtain a judgment model for judging the state of the tumor region.

In an embodiment of the present invention, the operation method 200 further includes: performing histogram equalization and horizontal flipping on multiple original ultrasonic images for data enhancement to obtain multiple ultrasonic images.

In an embodiment of the present invention, step S201 includes: obtaining a region of interest and other regions from each of a plurality of ultrasonic images; before standardizing the region of interest, performing preprocessing on the region of interest, the preprocessing It divides the image intensity value of each pixel in the region of interest by the average image intensity value of the remaining regions.

In an embodiment of the present invention, step S202 includes: converting the ROI to a preset gray level to obtain a standardized ROI, and the standardized ROI conforms to the input sizes of a plurality of different classifiers.

In an embodiment of the present invention, the plurality of ultrasonic images include a plurality of gray-scale ultrasonic images and a plurality of elastic ultrasonic images, and step S203 includes: decomposing the plurality of elastic ultrasonic images into a plurality of red ultrasonic images, a plurality of green ultrasound images, and a plurality of blue ultrasound images; using a plurality of different classifiers to perform transfer learning on a standardized region of interest corresponding to each of the plurality of gray-scale ultrasound images to obtain A first judgment model is obtained; a plurality of different classifiers are used to transfer the standardized region of interest corresponding to each of the plurality of elastic ultrasound images to obtain a second judgment model; a plurality of different classifiers is used The classifier performs transfer learning on the standardized region of interest corresponding to each of the plurality of red ultrasound images to obtain a third judgment model; using a plurality of different classifiers will correspond to the plurality of green ultrasound images The standardized region of interest of each of them is transferred to obtain the fourth judgment model; the standardized region of interest corresponding to each of the plurality of blue ultrasound images is converted using a plurality of different classifiers Carry out transfer learning to obtain a fifth judgment model; select a judgment model with the highest judgment accuracy from the first judgment model, the second judgment model, the third judgment model, the fourth judgment model and the fifth judgment model.

In summary, compared with the prior art, the technical solution of the present invention has obvious advantages and beneficial effects. With the ultrasonic image processing system 100 and its operation method 200 of the present invention, the error between the image and the evaluator is removed by the judgment model, making it more objective and accurate.

Although the present invention has been disclosed above in terms of implementation, it is not intended to limit the present invention. Anyone skilled in this art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection of the present invention The scope shall be defined by the appended patent application scope.

In order to make the above and other objects, features, advantages and embodiments of the present invention more obvious and understandable, the accompanying symbols are explained as follows:

100: Ultrasonic image processing system

110: storage device

120: Processor

130: Display

190: Ultrasonic scanning device

200: How it works

S201～S203: Steps

In order to make the above and other objects, features, advantages and embodiments of the present invention more clearly understood, the accompanying drawings are described as follows: Figure 1 is a block diagram of an ultrasonic image processing system according to an embodiment of the present invention; and FIG. 2 is a flowchart of an operation method of an ultrasonic image processing system according to an embodiment of the present invention.

200: How it works

S201~S203: steps

Claims (8)

An ultrasonic image processing system, comprising: a storage device storing at least one instruction; and a processor electrically connected to the storage device, wherein the processor is used to access and execute the at least one instruction to: Each of the images obtains a region of interest, the region of interest being a tumor region; normalizing the region of interest to obtain a normalized region of interest; and using a plurality of different classifiers to the normalized A transfer learning is performed on the region of interest to obtain a judgment model to judge the state of the tumor region, wherein the plurality of ultrasonic images include a plurality of gray-scale ultrasonic images and a plurality of elastic ultrasonic images, and the processor is used for accessing and executing the at least one instruction to: decompose the plurality of elastic ultrasound images into a plurality of red ultrasound images, a plurality of green ultrasound images, and a plurality of blue ultrasound images; use the plurality of different classifications The device will perform the transfer learning corresponding to the standardized region of interest in each of the plurality of gray-scale ultrasonic images to obtain a first judgment model; using the plurality of different classifiers will correspond to the The transfer learning is performed on the standardized ROI of each of the plurality of elastic ultrasound images to obtain a second judgment model; using the plurality of different classifiers will correspond to the plurality of red ultrasound images The transfer learning is performed on the standardized ROI of each of them to obtain a a third judgment model; using the plurality of different classifiers to perform the transfer learning on the standardized region of interest corresponding to each of the plurality of green ultrasound images to obtain a fourth judgment model; using The plurality of different classifiers perform the transfer learning corresponding to the standardized ROI of each of the plurality of blue ultrasound images to obtain a fifth judgment model; and from the first judgment model, the second judgment model, the third judgment model, the fourth judgment model, and the fifth judgment model, select a judgment model with the highest judgment accuracy. The ultrasonic image processing system as described in Claim 1, wherein the processor is used to access and execute the at least one instruction to: perform histogram equalization and horizontal flipping on multiple original ultrasonic images for data enhancement to obtain the Multiple ultrasound images. The ultrasonic image processing system as claimed in claim 1, wherein the processor is configured to access and execute the at least one instruction to: obtain the region of interest and a remaining region from each of a plurality of ultrasonic images; and Before normalizing the ROI, a preprocessing is performed on the ROI. The preprocessing is to divide the image intensity value of each pixel in the ROI by the average image intensity value of the remaining regions. The ultrasonic image processing system as claimed in claim 1, wherein the processor is used to access and execute the at least one instruction to: Converting the ROI to a preset grayscale level to obtain the normalized ROI, the normalized ROI conforms to the input sizes of the plurality of different classifiers. An operation method of an ultrasound image processing system, the operation method comprising the following steps: obtaining a region of interest from each of a plurality of ultrasound images, the region of interest being a tumor region; standardizing the region of interest, To obtain a standardized region of interest; and use a plurality of different classifiers to perform a transfer learning on the standardized region of interest to obtain a judgment model to judge the state of the tumor region, wherein the plurality of super The audio image includes a plurality of gray-scale ultrasonic images and a plurality of elastic ultrasonic images, and the step of performing the transfer learning on the standardized region of interest using the plurality of different classifiers includes: using the plurality of elastic ultrasonic images decomposed into a plurality of red ultrasound images, a plurality of green ultrasound images, and a plurality of blue ultrasound images; using the plurality of different classifiers will correspond to the plurality of grayscale ultrasound images for each of the performing the transfer learning on a standardized region of interest to obtain a first decision model; using the plurality of different classifiers to perform the standardized region of interest corresponding to each of the plurality of elastic ultrasound images the transfer learning to obtain a second judgment model; using the plurality of different classifiers to convert the standardized value corresponding to each of the plurality of red ultrasound images performing the transfer learning on the region of interest to obtain a third judgment model; performing the transfer on the standardized region of interest corresponding to each of the plurality of green ultrasound images using the plurality of different classifiers formula learning to obtain a fourth judgment model; using the plurality of different classifiers to perform the transfer learning on the standardized region of interest corresponding to each of the plurality of blue ultrasound images to obtain Finding a fifth judgment model; and selecting a judgment model with the highest judgment accuracy from the first judgment model, the second judgment model, the third judgment model, the fourth judgment model and the fifth judgment model. The operation method described in Claim 5 further includes: performing histogram equalization and horizontal flipping on multiple original ultrasound images for data enhancement to obtain the multiple ultrasound images. The operation method as described in claim 5, wherein the step of obtaining the region of interest from each of the plurality of ultrasonic images comprises: obtaining the region of interest and a remaining region from each of the plurality of ultrasonic images and before normalizing the ROI, performing a preprocessing on the ROI, the preprocessing is dividing the image intensity value of each pixel in the ROI by the average image intensity value of the remaining regions. The operation method as described in claim item 5, wherein the step of standardizing the region of interest includes: Converting the ROI to a preset grayscale level to obtain the normalized ROI, the normalized ROI conforms to the input sizes of the plurality of different classifiers.

Images