WO2023173545A1 - 超声图像的基准图像的生成方法、装置、设备及存储介质 - Google Patents

超声图像的基准图像的生成方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023173545A1
WO2023173545A1 PCT/CN2022/090159 CN2022090159W WO2023173545A1 WO 2023173545 A1 WO2023173545 A1 WO 2023173545A1 CN 2022090159 W CN2022090159 W CN 2022090159W WO 2023173545 A1 WO2023173545 A1 WO 2023173545A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
ultrasound image
input
variable
original
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/090159
Other languages
English (en)
French (fr)
Inventor
郑喜民
胡浩楠
舒畅
陈又新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2023173545A1 publication Critical patent/WO2023173545A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/10—Texturing; Colouring; Generation of textures or colours
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G06T2207/10136—3D ultrasound image

Definitions

  • the present application relates to the field of image feature processing in artificial intelligence, and in particular to a method, device, computer equipment and storage medium for generating a reference image of an ultrasound image.
  • the main purpose of this application is to provide a method for generating a reference image of an ultrasound image and a screen projection connection control method, device, computer equipment and storage medium, aiming to solve the problem of low accuracy of feature representation of the ultrasound reference image.
  • this application proposes a method for generating a reference image of an ultrasound image, including:
  • a reference image is generated based on the shape features and the fine-grained features.
  • This application also provides a device for generating a reference image of an ultrasound image, including:
  • Configuration information module used to obtain preconfigured mask information
  • Original image module used to obtain original ultrasound images
  • An image processing module configured to perform product processing on the original ultrasound image and the mask information to obtain a characteristic area ultrasound image
  • a variable conversion module used to input the characteristic area ultrasound image into a deep learning network to obtain a first characteristic variable; input the original ultrasound image into a deep learning network to obtain a second characteristic variable;
  • a feature extraction module configured to obtain the shape features of the image based on the first feature variable, and obtain the fine-grained features of the image based on the second feature variable;
  • a reference image module configured to generate a reference image according to the shape features and the fine-grained features.
  • the present application also provides a computer device, including a memory and a processor.
  • the memory stores a computer program.
  • the processor executes the computer program, the steps of the method for generating a reference image of an ultrasound image are implemented, wherein the ultrasound
  • the method of generating the reference image of the image includes:
  • a reference image is generated based on the shape features and the fine-grained features.
  • the present application also provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the steps of a method for generating a reference image of an ultrasound image;
  • a method for generating a reference image of an ultrasound image includes the following steps:
  • a reference image is generated based on the shape features and the fine-grained features.
  • This application example provides a method for generating a reference image for the ultrasound image by stripping out the background, noise and other information in the ultrasound image.
  • the original ultrasound image is obtained and the preconfigured mask information is obtained.
  • the mask The information can process the area of interest in the original ultrasound image. You can randomly select one from the mask information set as the preconfigured mask information, or you can configure the generation rules of the mask information to randomly generate a mask information as preconfigured mask information, and then perform product processing on the original ultrasound image and the mask information.
  • the mask information is changed to the same size as the original ultrasound image, and then the original ultrasound image is The image and the mask information are multiplied to obtain a characteristic area ultrasound image.
  • the characteristic area ultrasound image is input to a deep learning network.
  • Feature extraction is performed on the characteristic area ultrasound image through the deep learning network.
  • the extracted features In order to be able to affect the features of the reference image, the extracted features are encoded and transformed to obtain the first feature variable.
  • the original ultrasound image is input to the deep learning network to obtain the second feature variable.
  • the feature variable obtains the shape characteristics of the image, and the fine-grained features of the image are obtained according to the second characteristic variable.
  • the shape features are low-dimensional features of the image, and the fine-grained features are high-dimensional features of the image.
  • the final generated benchmark image does not contain effective features, but only contains environmental features of background information and noise information in the original ultrasound image. It has the same environment as the original ultrasound image but does not provide effective Information is close to the ideal reference image, thereby accurately representing information other than effective features in the ultrasound image, and providing accurate reference images and accurate explanatory explanations for the prediction results of AI decision-making.
  • Figure 1 is a schematic flow chart of a method for generating a reference image of an ultrasound image according to an embodiment of the present application
  • Figure 2 is a schematic flowchart of an embodiment of the present application for calculating the increment of the original ultrasound image compared to the reference image;
  • Figure 3 is a schematic structural diagram of an embodiment of a device for generating a reference image of an ultrasound image according to the present application
  • Figure 4 is a schematic structural block diagram of an embodiment of the computer equipment of the present application.
  • an embodiment of the present application provides a method for generating a reference image of an ultrasound image.
  • the method of generating a reference image of an ultrasound image includes steps S10-S60. Each step of the method of generating a reference image of an ultrasound image is The details are as follows.
  • This embodiment is applied to the generation scenario of ultrasound reference images.
  • the biggest obstacle to the current penetration of artificial intelligence technology into the medical field is the "black box" problem of deep neural networks. Humans cannot believe that unexplainable AI (Artificial Intelligence, artificial intelligence) can do Therefore, it is necessary to explain the prediction results made by AI, and determine the explanation of the prediction results by calculating the contribution of different areas on the ultrasound image to the prediction results compared with the baseline image. sex.
  • this embodiment In order to accurately determine the contribution of different areas on the ultrasound image to the prediction result compared to the baseline image, this embodiment generates a corresponding baseline image for each ultrasound image, and the baseline image is the original The information remaining after the effective features are stripped from the ultrasound image, that is, the environmental features in the original ultrasound image.
  • the mask information is a mask.
  • the mask information is composed of a ⁇ a small pieces.
  • the preconfigured mask information is a mask.
  • One piece of the information set is randomly selected as the preconfigured mask information; in another implementation, a piece of mask information is randomly generated as the preconfigured mask information by configuring the generation rules of the mask information.
  • the original ultrasonic image is obtained.
  • other medical equipment can be connected.
  • other medical equipment captures the original ultrasonic image
  • the original ultrasound image is obtained.
  • the original ultrasonic image and the mask information are multiplied, wherein the The pixel value x ij obeys a uniform distribution, and x ij ⁇ ⁇ 0,1 ⁇ , so that the mask information can process the area of interest more accurately, and then the mask information is changed to be the same as the original ultrasound
  • the images are of the same size, and then the original ultrasound image and the mask information are multiplied to obtain a characteristic area ultrasound image.
  • the characteristic area ultrasound image is defined as source A, and the original ultrasound image is defined as source B.
  • the characteristic region ultrasound image is input to the deep learning network.
  • the deep learning network performs feature extraction on the ultrasound image of the characteristic area, and the extracted features are features that can affect the reference image, and then the extracted features are encoded and transformed to obtain the first feature variable; similarly, the original ultrasound image is Input to the deep learning network, perform feature extraction on the original ultrasound image through the deep learning network, the extracted features are features that can affect the reference image, and then encode and transform the features extracted from the original ultrasound image to obtain the second feature variable.
  • the ultrasonic image of the characteristic area is input into the deep learning network to obtain the first characteristic variable; after the original ultrasonic image is input into the deep learning network to obtain the second characteristic variable, according to the first characteristic
  • the variable obtains the shape feature of the image, and the fine-grained feature of the image is obtained according to the second feature variable.
  • the shape feature is the low-dimensional feature of the image, including the shape contained in the image and the position of the shape.
  • the fine-grained feature is The high-dimensional features of the image include the texture features of the image and the color features of the image.
  • a reference image is generated according to the shape characteristics and the fine-grained characteristics.
  • the generated benchmark image does not contain effective features, but only contains the background information and noise information in the original ultrasound image. It has the same environment as the original ultrasound image but does not provide effective information, and is close to the ideal benchmark image, thus accurately The background, noise and other information in the original ultrasound image are stripped out to generate a benchmark image.
  • the benchmark image can express rich environmental features, thereby accurately representing information other than effective features in the image, and providing accurate prediction results for AI decision-making. benchmark images and accurate explanatory instructions.
  • This embodiment provides a method for generating a reference image for the ultrasound image by stripping out the background, noise and other information in the ultrasound image.
  • the original ultrasound image is obtained and the preconfigured mask information is obtained.
  • the mask The information can process the area of interest in the original ultrasound image. You can randomly select one from the mask information set as the preconfigured mask information, or you can configure the generation rules of the mask information to randomly generate a mask information as preconfigured mask information, and then perform product processing on the original ultrasound image and the mask information.
  • the mask information is changed to the same size as the original ultrasound image, and then the original ultrasound image is The image and the mask information are multiplied to obtain a characteristic area ultrasound image.
  • the characteristic area ultrasound image is input to a deep learning network.
  • Feature extraction is performed on the characteristic area ultrasound image through the deep learning network.
  • the extracted features In order to be able to affect the features of the reference image, the extracted features are encoded and transformed to obtain the first feature variable.
  • the original ultrasound image is input to the deep learning network to obtain the second feature variable.
  • the feature variable obtains the shape characteristics of the image, and the fine-grained features of the image are obtained according to the second characteristic variable.
  • the shape features are low-dimensional features of the image, and the fine-grained features are high-dimensional features of the image.
  • the final generated benchmark image does not contain effective features, but only contains environmental features of background information and noise information in the original ultrasound image. It has the same environment as the original ultrasound image but does not provide effective Information is close to the ideal reference image, thereby accurately representing information other than effective features in the ultrasound image, and providing accurate reference images and accurate explanatory explanations for the prediction results of AI decision-making.
  • inputting the characteristic region ultrasound image into a deep learning network to obtain a first feature variable; inputting the original ultrasound image into a deep learning network to obtain a second feature variable includes:
  • the original ultrasound image is input to a deep learning network, and the latent encoding of the original ultrasound image is transformed based on the nonlinear mapping network and affine transformation in the deep learning network to obtain the second feature variable.
  • the characteristic variable in the process of inputting the ultrasonic image of the characteristic area into the deep learning network to obtain the first characteristic variable; and inputting the original ultrasonic image into the deep learning network to obtain the second characteristic variable, the characteristic variable is
  • the regional ultrasound image is input to a deep learning network, which contains a nonlinear mapping network, and then the potential encoding of the characteristic region ultrasound image is transformed based on the nonlinear mapping network and affine transformation in the deep learning network.
  • the variable that can characterize the corresponding feature in the characteristic area ultrasound image obtains the variable that can characterize the corresponding feature in the characteristic area ultrasound image, and define it as the first feature variable; similarly, for the original ultrasound image, the original ultrasound image is input to the deep learning network, and based on the deep learning network
  • the nonlinear mapping network and affine transformation transform the latent encoding of the original ultrasound image to obtain variables that can characterize the corresponding features in the original ultrasound image, which are defined as second feature variables, thereby accurately extracting the original ultrasound image and Image features of characteristic area ultrasound images, thereby improving the accuracy of reference image generation.
  • obtaining the shape features of the image based on the first feature variable and obtaining the fine-grained features of the image based on the second feature variable include:
  • the second feature variable is input to the high-resolution convolutional layer in the generation network to obtain the fine-grained features of the image.
  • the first feature variables are input into the generation network, so
  • the generation network includes multiple convolution layers, and the first feature variable is input to a low-resolution convolution layer in the generation network.
  • the low-resolution convolution layer can control the low-dimensional features of the generated image, that is, obtain the shape of the image.
  • the second feature variable is input to the high-resolution convolution layer in the generation network.
  • the high-resolution convolution layer can control the high-dimensional features of the generated image, that is, obtain the fine-grained features of the image.
  • the low-resolution convolution layer is a convolution layer with a resolution of 4 2 -32 2 ;
  • the high-resolution convolution layer is a convolution layer with a resolution of 64 2 -1024 2 .
  • the low-resolution convolution layer is a convolution layer with a resolution of 4 2 -32 2 ;
  • the high-resolution convolution layer is a convolution layer with a resolution of 64 2 -1024 2.
  • the method further includes:
  • the original ultrasound image and the reference image are input to the classification network, and the original ultrasound image and the reference image are used to perform Compare, and calculate the increment of the original ultrasound image compared to the reference image based on the classification network.
  • the increment represents the contribution of each different area in the original ultrasound image to the determination result, and then based on the increment
  • the attribution map is generated quantitatively.
  • the attribution map represents the contribution of different areas in the original ultrasound image to the determination result. Through the attribution map, it can be judged whether the classification network correctly focuses on the correct image features and ignores the image. environmental characteristics, thereby improving the interpretability of the judgment results.
  • the original ultrasound image and the reference image are input to a classification network, and an increase in the original ultrasound image compared to the reference image is calculated based on the classification network.
  • Quantity including;
  • S62 Input each of the regional sub-images and the reference image to a classification network respectively, and calculate the first prediction score of each of the regional sub-images for the respective results compared to the reference image based on the classification network. Amount of contribution;
  • the process of inputting the original ultrasound image and the reference image into a classification network and calculating the increment of the original ultrasound image compared to the reference image based on the classification network first the The original ultrasound image is divided into several regional sub-images, for example, 9*9 regional sub-images, and then each regional sub-image and the reference image are input to the classification network, and each region sub-image is calculated based on the classification network.
  • the contribution amount of the prediction scores of each of the regional sub-images to the respective results compared to the base image is defined as the first contribution amount, and then the second contribution amount of the environmental features in the base image is obtained, and then according to the third The first contribution amount and the second contribution amount determine the increment.
  • the first contribution amount is subtracted from the second contribution amount, that is, the real contribution amount of each of the regional sub-images is obtained, and then The true contribution of each regional sub-image is coordinated to obtain the increment of the original ultrasound image compared to the reference image, thereby accurately representing the contribution of each different region in the original ultrasound image to the determination result, improving Determine the interpretability of results.
  • the method further includes:
  • Effective feature information of the reference image is determined according to the target sub-region image.
  • the attribution map is generated based on the increment
  • the area with the highest contribution in the attribution map is obtained, defined as a target sub-area image
  • the target sub-area image is determined based on the target sub-area image.
  • the effective feature information of the reference image is determined to determine the effective feature information of the reference image. Manual comparison can be performed based on the effective feature information, thereby improving the accuracy of the reference image.
  • this application also provides a device for generating a reference image of an ultrasound image, including:
  • Configuration information module 10 used to obtain preconfigured mask information
  • Original image module 20 used to obtain original ultrasound images
  • the image processing module 30 is used to perform product processing on the original ultrasound image and the mask information to obtain a characteristic area ultrasound image;
  • the variable conversion module 40 is used to input the characteristic area ultrasound image into the deep learning network to obtain the first characteristic variable; input the original ultrasound image into the deep learning network to obtain the second characteristic variable;
  • the feature extraction module 50 is configured to obtain the shape features of the image based on the first feature variable, and obtain the fine-grained features of the image based on the second feature variable;
  • a reference image module 60 is configured to generate a reference image according to the shape features and the fine-grained features.
  • each component of the device for generating a reference image of an ultrasonic image proposed in this application can implement the functions of any of the methods for generating a reference image of an ultrasonic image as described above.
  • inputting the characteristic region ultrasound image into a deep learning network to obtain a first feature variable; inputting the original ultrasound image into a deep learning network to obtain a second feature variable includes:
  • the original ultrasound image is input to a deep learning network, and the latent encoding of the original ultrasound image is transformed based on the nonlinear mapping network and affine transformation in the deep learning network to obtain the second feature variable.
  • obtaining the shape features of the image based on the first feature variable and obtaining the fine-grained features of the image based on the second feature variable include:
  • the second feature variable is input to the high-resolution convolutional layer in the generation network to obtain the fine-grained features of the image.
  • the low-resolution convolution layer is a convolution layer with a resolution of 4 2 -32 2 ;
  • the high-resolution convolution layer is a convolution layer with a resolution of 64 2 -1024 2 .
  • the method further includes:
  • the input of the original ultrasound image and the reference image to a classification network, and calculating the increment of the original ultrasound image compared to the reference image based on the classification network includes;
  • Each of the regional sub-images and the reference image are respectively input to a classification network, and a first contribution amount of each of the regional sub-images to the respective results compared to the reference image is calculated based on the classification network ;
  • the increment is determined based on the first contribution amount and the second contribution amount.
  • the method further includes:
  • Effective feature information of the reference image is determined according to the target sub-region image.
  • an embodiment of the present application also provides a computer device.
  • the computer device may be a mobile terminal, and its internal structure may be as shown in FIG. 4 .
  • the computer equipment includes a processor, a memory, a network interface, a display device and an input device connected through a system bus.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the display device of the computer device is used to display offline applications.
  • the input device of the computer device is used to receive user input in offline applications.
  • the computer's processor is designed to provide computing and control capabilities.
  • the computer device's memory includes non-volatile storage media.
  • the non-volatile storage medium stores operating systems, computer programs and databases.
  • the computer device's database is used to store raw data.
  • the computer program when executed by a processor, implements a method for generating a reference image of an ultrasound image.
  • the above-mentioned processor executes the above-mentioned method for generating a reference image of an ultrasonic image.
  • the method includes: obtaining preconfigured mask information; obtaining an original ultrasonic image; performing product processing on the original ultrasonic image and the mask information to obtain Feature area ultrasound image; input the feature area ultrasound image into the deep learning network to obtain the first feature variable; input the original ultrasound image into the deep learning network to obtain the second feature variable; obtain according to the first feature variable
  • the fine-grained characteristics of the image are obtained according to the second characteristic variable; and a reference image is generated based on the shape characteristics and the fine-grained characteristics.
  • the computer device provides a method for generating a reference image for the ultrasonic image by stripping out background, noise and other information in the ultrasonic image.
  • the original ultrasonic image is obtained and preconfigured mask information is obtained.
  • the mask information can process the area of interest in the original ultrasound image. You can randomly select one from the mask information set as the preconfigured mask information, or you can randomly generate a mask information by configuring the mask information generation rules. As preconfigured mask information, the original ultrasound image and the mask information are then multiplied.
  • the mask information is changed to the same size as the original ultrasound image, and then the original ultrasound image is The ultrasonic image and the mask information are multiplied to obtain a characteristic area ultrasonic image, and the characteristic area ultrasonic image is input to a deep learning network.
  • Feature extraction is performed on the characteristic area ultrasonic image through the deep learning network, and the characteristic area ultrasonic image is extracted.
  • the features of are the features that can affect the reference image, and then the extracted features are encoded and transformed to obtain the first feature variable.
  • the original ultrasound image is input to the deep learning network to obtain the second feature variable.
  • the first feature variable obtains the shape features of the image, and the fine-grained features of the image are obtained according to the second feature variable.
  • the shape features are low-dimensional features of the image, and the fine-grained features are high-dimensional features of the image. Then according to the The above-mentioned shape features and the above-mentioned fine-grained features generate a benchmark image.
  • the finally generated benchmark image does not contain effective features, but only contains environmental features of the background information and noise information in the original ultrasound image. It has the same environment as the original ultrasound image but does not Provide effective information that is close to the ideal reference image, thereby accurately representing information other than effective features in the ultrasound image, and providing accurate reference images and accurate explanatory explanations for the prediction results of AI decision-making.
  • An embodiment of the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium may be non-volatile or volatile, and a computer program is stored thereon.
  • the computer program is executed by the processor.
  • a method for generating a reference image of an ultrasonic image is implemented, including the steps of: obtaining preconfigured mask information; obtaining an original ultrasonic image; performing product processing on the original ultrasonic image and the mask information to obtain a characteristic area ultrasonic image ; Input the characteristic area ultrasound image to the deep learning network to obtain the first feature variable; input the original ultrasound image to the deep learning network to obtain the second feature variable; obtain the shape characteristics of the image according to the first feature variable , obtain the fine-grained features of the image based on the second feature variable; generate a reference image based on the shape features and the fine-grained features.
  • the computer-readable storage medium provides a method for generating a reference image for the ultrasonic image by stripping out background, noise and other information in the ultrasonic image.
  • the original ultrasonic image is obtained and the preconfigured mask information is obtained.
  • the mask information can process the area of interest in the original ultrasound image. One can be randomly selected from the mask information set as the preconfigured mask information, or one can be randomly generated by configuring the generation rules of the mask information.
  • the mask information is used as preconfigured mask information, and then the original ultrasound image and the mask information are multiplied.
  • the mask information is changed to the same size as the original ultrasound image, and then The original ultrasonic image and the mask information are multiplied to obtain a characteristic area ultrasonic image, and the characteristic area ultrasonic image is input to a deep learning network, and the characteristic area ultrasonic image is characterized by the deep learning network.
  • the extracted features are features that can affect the reference image, and then the extracted features are encoded and transformed to obtain the first feature variable.
  • the original ultrasound image is input into the deep learning network to obtain the second feature variable, The shape features of the image are obtained according to the first feature variable, and the fine-grained features of the image are obtained according to the second feature variable.
  • the shape features are low-dimensional features of the image, and the fine-grained features are high-dimensional features of the image.
  • a benchmark image is generated based on the shape features and the fine-grained features.
  • the finally generated benchmark image does not contain effective features, but only contains the background information and noise information in the original ultrasound image. It has the same characteristics as the original ultrasound image.
  • the environment does not provide effective information, but is close to the ideal reference image, thereby accurately representing information other than effective features in the ultrasound image, and providing accurate reference images and accurate explanatory instructions for the prediction results of AI decision-making.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • SDRAM dual speed data rate SDRAM
  • SSRSDRAM dual speed data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请涉及图像特征处理领域，特别是涉及到一种超声图像的基准图像的生成方法、装置、计算机设备及存储介质，所述方法包括：获取预配置的掩膜信息；获取原始超声图像；将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；根据所述形状特征与所述细粒特征生成基准图像。本申请能够生成准确且具有可靠的可解释性的基准图，提高预测结果的可靠性。

Description

超声图像的基准图像的生成方法、装置、设备及存储介质
本申请要求于2022年03月15日提交中国专利局、申请号为202210253006.4，发明名称为“超声图像的基准图像的生成方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能中的图像特征处理领域，特别是涉及到一种超声图像的基准图像的生成方法、装置、计算机设备及存储介质。
背景技术
随着成像技术的发展，数字影像已成为医学的主要数据，通过人工智能进行图像识别以辅助医生临床决策。当前人工智能技术向医疗领域渗透的最大障碍源于深度神经网络的“黑箱”问题，人类无法相信不可解释的AI做出的决策，原因在于无法准确地对AI决策作出合理的解释。目前一种方式是通过零值图计算超声图像对网络的预测结果的贡献，但发明人意识到由于无法提供准确的基准图像，使得该方式无法判别网络关注的是该超声图像的形状特征还是纹理特征，从而无法判断网络是否关注了正确的特征，即目前的基准图不具备较好的超声基准图特性，无法准确表示图像中的特征信息。
技术问题
本申请的主要目的为提供一种超声图像的基准图像的生成方法及投屏连接控制方法、装置、计算机设备及存储介质，旨在解决超声基准图的特征表示准确性低的问题。
技术解决方案
为了实现上述发明目的，本申请提出一种超声图像的基准图像的生成方法，包括：
获取预配置的掩膜信息；
获取原始超声图像；
将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声 图像；
将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
根据所述形状特征与所述细粒特征生成基准图像。
本申请还提供一种超声图像的基准图像的生成装置，包括：
配置信息模块，用于获取预配置的掩膜信息；
原始图像模块，用于获取原始超声图像；
图像处理模块，用于将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
变量转换模块，用于将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
特征提取模块，用于根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
基准图像模块，用于根据所述形状特征与所述细粒特征生成基准图像。
本申请还提供一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现超声图像的基准图像的生成方法的步骤，其中，所述超声图像的基准图像的生成方法，包括：
获取预配置的掩膜信息；
获取原始超声图像；
将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
根据所述形状特征与所述细粒特征生成基准图像。
本申请还提供一种计算机可读存储介质，其上存储有计算机程序，所述 计算机程序被处理器执行时实现超声图像的基准图像的生成方法的步骤；
其中，一种超声图像的基准图像的生成方法包括以下步骤：
获取预配置的掩膜信息；
获取原始超声图像；
将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
根据所述形状特征与所述细粒特征生成基准图像。
有益效果
本申请例提供了一种通过将超声图像中的背景、噪声等信息剥离出来，从而为超声图像生成的基准图像的方法，首先获取原始超声图像以及获取预配置的掩膜信息，所述掩膜信息能够对原始超声图像中感兴趣的区域进行处理，可以从掩膜信息集合中随机选取一个作为预配置的掩膜信息，也可以通过配置掩膜信息的生成规则，随机生成一个掩膜信息作为预配置的掩膜信息，然后将所述原始超声图像与所述掩膜信息进行乘积处理，首先将所述掩膜信息变化为与所述原始超声图像相同的尺寸大小，再将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像，将所述特征区域超声图像输入至深度学习网络，通过所述深度学习网络对所述特征区域超声图像进行特征提取，提取的特征为能够影响基准图像的特征，再将提取到的特征进行编码转化，得到第一特征变量，同样的，将所述原始超声图像输入至深度学习网络，得到第二特征变量，根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，所述形状特征为图像的低维度特征，所述细粒特征为图像的高维度特征，然后根据所述形状特征与所述细粒特征生成基准图像，最终生成的基准图像就是不包含有效特征，仅仅包含原始超声图像中的背景信息和噪声信息的环境特征，有着与原始超声图像相同的环境但不提供有效信息，接近于理想的基准图像，从而准确表示超声图 像中除有效特征以外的信息，对AI决策的预测结果提供准确的基准图像以及准确的解释性的说明。
附图说明
图1为本申请超声图像的基准图像的生成方法的一实施例流程示意图；
图2为本申请计算所述原始超声图像相较于所述基准图像的增量的一实施例流程示意图；
图3为本申请超声图像的基准图像的生成装置的一实施例结构示意图；
图4为本申请计算机设备的一实施例结构示意框图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
本发明的最佳实施方式
参照图1，本申请实施例提供一种超声图像的基准图像的生成方法，所述超声图像的基准图像的生成方法包括步骤S10-S60，对于所述超声图像的基准图像的生成方法的各个步骤的详细阐述如下。
S10、获取预配置的掩膜信息。
本实施例应用于超声基准图像的生成场景中，当前人工智能技术向医疗领域渗透的最大障碍源于深度神经网络的“黑箱”问题，人类无法相信不可解释的AI(Artificial Intelligence，人工智能)做出的决策，因此需要对AI作出的预测结果进行解释性的说明，通过计算超声图像上的各个不同的区域相比于基准图像(baseline image)对预测结果作出的贡献，从而确定预测结果的解释性。为了准确地确定超声图像上的各个不同的区域相比于基准图像(baseline image)对预测结果作出的贡献，本实施例为每一张超声图像生成对应的基准图像，所述基准图像是将原始超声图像剥离有效特征后剩下的信息，即原始超声图像中的环境特征。首先，获取预配置的掩膜信息，所述掩膜信息为mask，所述掩膜信息由a×a的小片组成，在一种实施方式中，所述预配置的掩膜信息为从掩膜信息集合中随机选取一个作为预配置的掩膜信息；在另一种实施方式中，通过配置掩膜信息的生成规则，随机生成一个掩膜信息作为预配置的掩膜信息。
S20、获取原始超声图像。
本实施例中，在获取预配置的掩膜信息之后，为了对每一个超声图像生成对应的基准图像，获取原始超声图像，具体的，可以对接其他医疗设备，当其他医疗设备拍摄到原始超声图像后，从而获取所述原始超声图像。
S30、将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像。
本实施例中，在获取预配置的掩膜信息后以及获取原始超声图像之后，将所述原始超声图像与所述掩膜信息进行乘积处理，其中，所述掩膜信息的每个小片上的像素值x ij服从均匀分布，且x ij∈{0,1}，使得所述掩膜信息能够更加准确地对感兴趣的区域进行处理，然后将所述掩膜信息变化为与所述原始超声图像相同的尺寸大小，再将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像，定义所述特征区域超声图像为source A，所述原始超声图像为source B。
S40、将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量。
本实施例中，在获取原始超声图像以及将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像之后，将所述特征区域超声图像输入至深度学习网络，通过所述深度学习网络对所述特征区域超声图像进行特征提取，提取的特征为能够影响基准图像的特征，再将提取到的特征进行编码转化，得到第一特征变量；同样的，将所述原始超声图像输入至深度学习网络，通过所述深度学习网络对所述原始超声图像进行特征提取，提取的特征为能够影响基准图像的特征，再将原始超声图像提取到的特征进行编码转化，得到第二特征变量。
S50、根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征。
本实施例中，在将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量之后，根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，所述形状特征为图像的低维度特征，包括图像包含的形状以及所述形状的位置，所述细粒特征为图像的高维度特征，包括图像的 纹理特征以及图像的颜色特征，当得到所述第一特征变量与所述第二特征变量之后，将其组合作为图像的样式变量，定义所述样式变量y＝(y s,y b)，其中y s为第一特征变量，y b为第二特征变量，通过将第一特征变量与第二特征变量进行组合后输入至卷积层中，可以得到对应的形状特征与细粒特征。
S60、根据所述形状特征与所述细粒特征生成基准图像。
本实施例中，在根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征之后，根据所述形状特征与所述细粒特征生成基准图像，最终生成的基准图像就是不包含有效特征，仅仅包含原始超声图像中的背景信息和噪声信息的环境特征，有着与原始超声图像相同的环境但不提供有效信息，接近于理想的基准图像，从而准确地将原始超声图像中的背景、噪声等信息剥离出来后生成基准图像，所述基准图像能够表示出丰富的环境特征，从而准确表示图像中除有效特征以外的信息，对AI决策的预测结果提供准确的基准图像以及准确的解释性的说明。
本实施例提供了一种通过将超声图像中的背景、噪声等信息剥离出来，从而为超声图像生成的基准图像的方法，首先获取原始超声图像以及获取预配置的掩膜信息，所述掩膜信息能够对原始超声图像中感兴趣的区域进行处理，可以从掩膜信息集合中随机选取一个作为预配置的掩膜信息，也可以通过配置掩膜信息的生成规则，随机生成一个掩膜信息作为预配置的掩膜信息，然后将所述原始超声图像与所述掩膜信息进行乘积处理，首先将所述掩膜信息变化为与所述原始超声图像相同的尺寸大小，再将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像，将所述特征区域超声图像输入至深度学习网络，通过所述深度学习网络对所述特征区域超声图像进行特征提取，提取的特征为能够影响基准图像的特征，再将提取到的特征进行编码转化，得到第一特征变量，同样的，将所述原始超声图像输入至深度学习网络，得到第二特征变量，根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，所述形状特征为图像的低维度特征，所述细粒特征为图像的高维度特征，然后根据所述形状特征与所述细粒特征生成基准图像，最终生成的基准图像就是不包含有效特征，仅仅包含原始超声图像中的背景信息和噪声信息的环境特征，有着与原始超声图像相同的环境但不提供有效信息，接近于理想的基准图像，从而准确表示超声图 像中除有效特征以外的信息，对AI决策的预测结果提供准确的基准图像以及准确的解释性的说明。
在一个实施例中，所述将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量，包括：
将所述特征区域超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述特征区域超声图像的潜在编码进行变换，得到第一特征变量；
将所述原始超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述原始超声图像的潜在编码进行变换，得到第二特征变量。
本实施例中，在将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量的过程中，将所述特征区域超声图像输入至深度学习网络，所述深度学习网络中包含非线性映射网络，然后基于所述深度学习网络中的非线性映射网络和仿射变换对所述特征区域超声图像的潜在编码进行变换，得到能够表征特征区域超声图像中对应特征的变量，将其定义为第一特征变量；同样的，对于原始超声图像，将所述原始超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述原始超声图像的潜在编码进行变换，得到能够表征原始超声图像中对应特征的变量，将其定义为第二特征变量，从而准确地提取原始超声图像与特征区域超声图像的图像特征，从而提高基准图像生成的准确性。
在一个实施例中，所述根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，包括：
将所述第一特征变量输入至生成网络中的低分辨率卷积层，得到图像的形状特征；
将所述第二特征变量输入至生成网络中的高分辨率卷积层，得到图像的细粒特征。
本实施例中，在根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征的过程中，将所述第一特征变量输入至生 成网络中，所述生成网络包含多个卷积层，将所述第一特征变量输入至生成网络中的低分辨率卷积层，低分辨率卷积层可以控制生成图像的低维特征，即得到图像的形状特征，同时，将所述第二特征变量输入至生成网络中的高分辨率卷积层，高分辨率卷积层可以控制生成图像的高维特征，即得到图像的细粒特征，通过将不同的控制图像生成的变量输入至不同的卷积层，从而得到图像的高纬度特征以及低纬度特征，准确地提取原始超声图像与特征区域超声图像的图像特征，从而提高基准图像生成的准确性。
在一个实施例中，所述低分辨率卷积层为分辨率4 2-32 2的卷积层；所述高分辨率卷积层为分辨率64 2-1024 2的卷积层。
本实施例中，所述低分辨率卷积层为分辨率4 2-32 2的卷积层；所述高分辨率卷积层为分辨率64 2-1024 2的卷积层，通过配置多个不同分辨率的卷积层，将不同的控制图像生成的变量输入至不同的卷积层，从而得到图像的高纬度特征以及低纬度特征，准确地提取原始超声图像与特征区域超声图像的图像特征，从而提高基准图像生成的准确性。
在一个实施例中，所述根据所述形状特征与所述细粒特征生成基准图像之后，还包括：
将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量；
根据所述增量生成归因图。
本实施例中，在根据所述形状特征与所述细粒特征生成基准图像之后，将所述原始超声图像与所述基准图像输入至分类网络，通过所述原始超声图像与所述基准图像进行对比，并基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量，所述增量表示所述原始超声图像中各个不同区域对于判定结果的贡献，再根据所述增量生成归因图(attribution map)，归因图中表示原始超声图像中各个不同区域对于判定结果的贡献，通过归因图能够判断分类网络是否正确地关注了正确的图像特征，而忽略掉图像的环境特征，从而提高对判定结果的可解释性。
在一个实施例中，如图2所示，所述将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量，包括；
S61：将所述原始超声图像划分为若干个区域子图像；
S62：分别将每个所述区域子图像与所述基准图像输入至分类网络，基于所述分类网络计算每个所述区域子图像对分别结果的预测分数相较于所述基准图像的第一贡献量；
S63：获取所述基准图像中环境特征的第二贡献量；
S64：根据所述第一贡献量与所述第二贡献量确定所述增量。
本实施例中，在将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量的过程中，首先将所述原始超声图像划分为若干个区域子图像，例如划分为9*9个区域子图像，然后分别将每个所述区域子图像与所述基准图像输入至分类网络，基于所述分类网络计算每个所述区域子图像对分别结果的预测分数相较于所述基准图像的贡献量，定义为第一贡献量，再获取所述基准图像中环境特征的第二贡献量，然后根据所述第一贡献量与所述第二贡献量确定所述增量，具体的，将所述第一贡献量减去所述第二贡献量，即得到每个所述区域子图像的真实贡献量，再将每个区域子图像的真实贡献量进行统筹，得到所述原始超声图像相较于所述基准图像的增量，从而准确地表示所述原始超声图像中各个不同区域对于判定结果的贡献，提高判定结果的可解释性。
在一个实施例中，所述根据所述增量生成归因图之后，还包括：
获取所述归因图中贡献量最高的目标子区域图像；
根据所述目标子区域图像确定所述基准图像的有效特征信息。
本实施例中，在根据所述增量生成归因图之后，获取所述归因图中贡献量最高的区域，将其定义为目标子区域图像，然后根据所述目标子区域图像确定所述基准图像的有效特征信息，从而确定基准图像的有效特征信息，可以根据所述有效特征信息进行人工比对，从而提高对基准图像的准确性。
参照图3，本申请还提供一种超声图像的基准图像的生成装置，包括：
配置信息模块10，用于获取预配置的掩膜信息；
原始图像模块20，用于获取原始超声图像；
图像处理模块30，用于将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
变量转换模块40，用于将所述特征区域超声图像输入至深度学习网络， 得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
特征提取模块50，用于根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
基准图像模块60，用于根据所述形状特征与所述细粒特征生成基准图像。
如上所述，可以理解地，本申请中提出的所述超声图像的基准图像的生成装置的各组成部分可以实现如上所述超声图像的基准图像的生成方法任一项的功能。
在一个实施例中，所述将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量，包括：
将所述特征区域超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述特征区域超声图像的潜在编码进行变换，得到第一特征变量；
将所述原始超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述原始超声图像的潜在编码进行变换，得到第二特征变量。
在一个实施例中，所述根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，包括：
将所述第一特征变量输入至生成网络中的低分辨率卷积层，得到图像的形状特征；
将所述第二特征变量输入至生成网络中的高分辨率卷积层，得到图像的细粒特征。
在一个实施例中，所述低分辨率卷积层为分辨率4 2-32 2的卷积层；所述高分辨率卷积层为分辨率64 2-1024 2的卷积层。
在一个实施例中，所述根据所述形状特征与所述细粒特征生成基准图像之后，还包括：
将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量；
根据所述增量生成归因图。
在一个实施例中，所述将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量，包括；
将所述原始超声图像划分为若干个区域子图像；
分别将每个所述区域子图像与所述基准图像输入至分类网络，基于所述分类网络计算每个所述区域子图像对分别结果的预测分数相较于所述基准图像的第一贡献量；
获取所述基准图像中环境特征的第二贡献量；
根据所述第一贡献量与所述第二贡献量确定所述增量。
在一个实施例中，所述根据所述增量生成归因图之后，还包括：
获取所述归因图中贡献量最高的目标子区域图像；
根据所述目标子区域图像确定所述基准图像的有效特征信息。
参照图4，本申请实施例中还提供一种计算机设备，该计算机设备可以是移动终端，其内部结构可以如图4所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和显示装置及输入装置。其中，该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机设备的显示装置用于显示离线应用。该计算机设备的输入装置用于接收用户在离线应用的输入。该计算机设计的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质。该非易失性存储介质存储有操作系统、计算机程序和数据库。该计算机设备的数据库用于存放原始数据。该计算机程序被处理器执行时以实现一种超声图像的基准图像的生成方法。
上述处理器执行上述的超声图像的基准图像的生成方法，所述方法包括：获取预配置的掩膜信息；获取原始超声图像；将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；根据所述形状特征与所述细粒特征生成基准图像。
所述计算机设备提供了一种通过将超声图像中的背景、噪声等信息剥离出来，从而为超声图像生成的基准图像的方法，首先获取原始超声图像以及 获取预配置的掩膜信息，所述掩膜信息能够对原始超声图像中感兴趣的区域进行处理，可以从掩膜信息集合中随机选取一个作为预配置的掩膜信息，也可以通过配置掩膜信息的生成规则，随机生成一个掩膜信息作为预配置的掩膜信息，然后将所述原始超声图像与所述掩膜信息进行乘积处理，首先将所述掩膜信息变化为与所述原始超声图像相同的尺寸大小，再将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像，，将所述特征区域超声图像输入至深度学习网络，通过所述深度学习网络对所述特征区域超声图像进行特征提取，提取的特征为能够影响基准图像的特征，再将提取到的特征进行编码转化，得到第一特征变量，同样的，将所述原始超声图像输入至深度学习网络，得到第二特征变量，根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，所述形状特征为图像的低维度特征，所述细粒特征为图像的高维度特征，然后根据所述形状特征与所述细粒特征生成基准图像，最终生成的基准图像就是不包含有效特征，仅仅包含原始超声图像中的背景信息和噪声信息的环境特征，有着与原始超声图像相同的环境但不提供有效信息，接近于理想的基准图像，从而准确表示超声图像中除有效特征以外的信息，对AI决策的预测结果提供准确的基准图像以及准确的解释性的说明。
本申请一实施例还提供一种计算机可读存储介质，计算机可读存储介质可以是非易失性，也可以是易失性，其上存储有计算机程序，所述计算机程序被所述处理器执行时实现一种超声图像的基准图像的生成方法，包括步骤：获取预配置的掩膜信息；获取原始超声图像；将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；根据所述形状特征与所述细粒特征生成基准图像。
所述计算机可读存储介质提供了一种通过将超声图像中的背景、噪声等信息剥离出来，从而为超声图像生成的基准图像的方法，首先获取原始超声图像以及获取预配置的掩膜信息，所述掩膜信息能够对原始超声图像中感兴趣的区域进行处理，可以从掩膜信息集合中随机选取一个作为预配置的掩膜 信息，也可以通过配置掩膜信息的生成规则，随机生成一个掩膜信息作为预配置的掩膜信息，然后将所述原始超声图像与所述掩膜信息进行乘积处理，首先将所述掩膜信息变化为与所述原始超声图像相同的尺寸大小，再将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像，，将所述特征区域超声图像输入至深度学习网络，通过所述深度学习网络对所述特征区域超声图像进行特征提取，提取的特征为能够影响基准图像的特征，再将提取到的特征进行编码转化，得到第一特征变量，同样的，将所述原始超声图像输入至深度学习网络，得到第二特征变量，根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，所述形状特征为图像的低维度特征，所述细粒特征为图像的高维度特征，然后根据所述形状特征与所述细粒特征生成基准图像，最终生成的基准图像就是不包含有效特征，仅仅包含原始超声图像中的背景信息和噪声信息的环境特征，有着与原始超声图像相同的环境但不提供有效信息，接近于理想的基准图像，从而准确表示超声图像中除有效特征以外的信息，对AI决策的预测结果提供准确的基准图像以及准确的解释性的说明。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的和实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可以包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双速据率SDRAM(SSRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。

Claims (20)

  1. 一种超声图像的基准图像的生成方法，其中，包括：
    获取预配置的掩膜信息；
    获取原始超声图像；
    将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
    将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
    根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
    根据所述形状特征与所述细粒特征生成基准图像。
  2. 根据权利要求1所述的超声图像的基准图像的生成方法，其中，所述将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量，包括：
    将所述特征区域超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述特征区域超声图像的潜在编码进行变换，得到第一特征变量；
    将所述原始超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述原始超声图像的潜在编码进行变换，得到第二特征变量。
  3. 根据权利要求1所述的超声图像的基准图像的生成方法，其中，所述根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，包括：
    将所述第一特征变量输入至生成网络中的低分辨率卷积层，得到图像的形状特征；
    将所述第二特征变量输入至生成网络中的高分辨率卷积层，得到图像的细粒特征。
  4. 根据权利要求3所述的超声图像的基准图像的生成方法，其中，所述低分辨率卷积层为分辨率4 2-32 2的卷积层；所述高分辨率卷积层为分辨率64 2-1024 2的卷积层。
  5. 根据权利要求1所述的超声图像的基准图像的生成方法，其中，所述根据所述形状特征与所述细粒特征生成基准图像之后，还包括：
    将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量；
    根据所述增量生成归因图。
  6. 根据权利要求5所述的超声图像的基准图像的生成方法，其中，所述将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量，包括；
    将所述原始超声图像划分为若干个区域子图像；
    分别将每个所述区域子图像与所述基准图像输入至分类网络，基于所述分类网络计算每个所述区域子图像对分别结果的预测分数相较于所述基准图像的第一贡献量；
    获取所述基准图像中环境特征的第二贡献量；
    根据所述第一贡献量与所述第二贡献量确定所述增量。
  7. 根据权利要求5所述的超声图像的基准图像的生成方法，其中，所述根据所述增量生成归因图之后，还包括：
    获取所述归因图中贡献量最高的目标子区域图像；
    根据所述目标子区域图像确定所述基准图像的有效特征信息。
  8. 一种超声图像的基准图像的生成装置，其中，包括：
    配置信息模块，用于获取预配置的掩膜信息；
    原始图像模块，用于获取原始超声图像；
    图像处理模块，用于将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
    变量转换模块，用于将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
    特征提取模块，用于根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
    基准图像模块，用于根据所述形状特征与所述细粒特征生成基准图像。
  9. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程 序，其中，所述处理器执行所述计算机程序时实现超声图像的基准图像的生成方法的步骤：
    其中，所述超声图像的基准图像的生成方法，包括：
    获取预配置的掩膜信息；
    获取原始超声图像；
    将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
    将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
    根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
    根据所述形状特征与所述细粒特征生成基准图像。
  10. 根据权利要求9所述的计算机设备，其中，所述将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量，包括：
    将所述特征区域超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述特征区域超声图像的潜在编码进行变换，得到第一特征变量；
    将所述原始超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述原始超声图像的潜在编码进行变换，得到第二特征变量。
  11. 根据权利要求9所述的计算机设备，其中，所述根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，包括：
    将所述第一特征变量输入至生成网络中的低分辨率卷积层，得到图像的形状特征；
    将所述第二特征变量输入至生成网络中的高分辨率卷积层，得到图像的细粒特征。
  12. 根据权利要求11所述的计算机设备，其中，所述低分辨率卷积层为分辨率42-322的卷积层；所述高分辨率卷积层为分辨率642-10242的卷积层。
  13. 根据权利要求9所述的计算机设备，其中，所述根据所述形状特征与所述细粒特征生成基准图像之后，还包括：
    将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量；
    根据所述增量生成归因图。
  14. 根据权利要求13所述的计算机设备，其中，所述将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量，包括；
    将所述原始超声图像划分为若干个区域子图像；
    分别将每个所述区域子图像与所述基准图像输入至分类网络，基于所述分类网络计算每个所述区域子图像对分别结果的预测分数相较于所述基准图像的第一贡献量；
    获取所述基准图像中环境特征的第二贡献量；
    根据所述第一贡献量与所述第二贡献量确定所述增量。
  15. 根据权利要求13所述的计算机设备，其中，所述根据所述增量生成归因图之后，还包括：
    获取所述归因图中贡献量最高的目标子区域图像；
    根据所述目标子区域图像确定所述基准图像的有效特征信息。
  16. 一种计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现超声图像的基准图像的生成方法的步骤，其中，所述超声图像的基准图像的生成方法包括以下步骤：
    获取预配置的掩膜信息；
    获取原始超声图像；
    将所述原始超声图像与所述掩膜信息进行乘积处理，得到特征区域超声图像；
    将所述特征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量；
    根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征；
    根据所述形状特征与所述细粒特征生成基准图像。
  17. 根据权利要求16所述的计算机可读存储介质，其中，所述将所述特 征区域超声图像输入至深度学习网络，得到第一特征变量；将所述原始超声图像输入至深度学习网络，得到第二特征变量，包括：
    将所述特征区域超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述特征区域超声图像的潜在编码进行变换，得到第一特征变量；
    将所述原始超声图像输入至深度学习网络，基于所述深度学习网络中的非线性映射网络和仿射变换对所述原始超声图像的潜在编码进行变换，得到第二特征变量。
  18. 根据权利要求16所述的计算机可读存储介质，其中，所述根据所述第一特征变量得到图像的形状特征，根据所述第二特征变量得到图像的细粒特征，包括：
    将所述第一特征变量输入至生成网络中的低分辨率卷积层，得到图像的形状特征；
    将所述第二特征变量输入至生成网络中的高分辨率卷积层，得到图像的细粒特征。
  19. 根据权利要求18所述的计算机可读存储介质，其中，所述低分辨率卷积层为分辨率42-322的卷积层；所述高分辨率卷积层为分辨率642-10242的卷积层。
  20. 根据权利要求16所述的计算机可读存储介质，其中，所述根据所述形状特征与所述细粒特征生成基准图像之后，还包括：
    将所述原始超声图像与所述基准图像输入至分类网络，基于所述分类网络计算所述原始超声图像相较于所述基准图像的增量；
    根据所述增量生成归因图。
PCT/CN2022/090159 2022-03-15 2022-04-29 超声图像的基准图像的生成方法、装置、设备及存储介质 Ceased WO2023173545A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210253006.4 2022-03-15
CN202210253006.4A CN114708156B (zh) 2022-03-15 2022-03-15 超声图像的基准图像的生成方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023173545A1 true WO2023173545A1 (zh) 2023-09-21

Family

ID=82169344

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/090159 Ceased WO2023173545A1 (zh) 2022-03-15 2022-04-29 超声图像的基准图像的生成方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN114708156B (zh)
WO (1) WO2023173545A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080262354A1 (en) * 2006-01-10 2008-10-23 Tetsuya Yoshida Ultrasonic diagnostic apparatus and method of generating ultrasonic image
US20130070993A1 (en) * 2011-09-19 2013-03-21 Samsung Medison Co., Ltd. Method and apparatus for processing image, ultrasound diagnosis apparatus, and medical imaging system
CN110032985A (zh) * 2019-04-22 2019-07-19 清华大学深圳研究生院 一种血细胞自动检测识别方法
CN112001226A (zh) * 2020-07-07 2020-11-27 中科曙光(南京)计算技术有限公司 一种无人驾驶3d目标检测方法、装置及存储介质
WO2021169452A1 (zh) * 2020-09-27 2021-09-02 平安科技（深圳）有限公司 甲状腺结节图像的处理方法、装置和计算机设备

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110427970B (zh) * 2019-07-05 2023-08-01 平安科技（深圳）有限公司 图像分类方法、装置、计算机设备和存储介质
CN112967300B (zh) * 2021-02-23 2024-10-18 艾瑞迈迪医疗科技（北京）有限公司 基于多尺度融合网络的三维超声甲状腺分割方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080262354A1 (en) * 2006-01-10 2008-10-23 Tetsuya Yoshida Ultrasonic diagnostic apparatus and method of generating ultrasonic image
US20130070993A1 (en) * 2011-09-19 2013-03-21 Samsung Medison Co., Ltd. Method and apparatus for processing image, ultrasound diagnosis apparatus, and medical imaging system
CN110032985A (zh) * 2019-04-22 2019-07-19 清华大学深圳研究生院 一种血细胞自动检测识别方法
CN112001226A (zh) * 2020-07-07 2020-11-27 中科曙光(南京)计算技术有限公司 一种无人驾驶3d目标检测方法、装置及存储介质
WO2021169452A1 (zh) * 2020-09-27 2021-09-02 平安科技（深圳）有限公司 甲状腺结节图像的处理方法、装置和计算机设备

Also Published As

Publication number Publication date
CN114708156B (zh) 2025-04-18
CN114708156A (zh) 2022-07-05

Similar Documents

Publication Publication Date Title
CN113313085B (zh) 一种图像处理方法、装置、电子设备及存储介质
CN112837357B (zh) 医学影像配准方法、装置、计算机设备及存储介质
CN116883466A (zh) 基于位置感知的光学与sar图像配准方法、装置及设备
CN112818963B (zh) 人脸识别模型的训练方法、装置、设备及存储介质
CN113792682A (zh) 基于人脸图像的人脸质量评估方法、装置、设备及介质
US12511750B2 (en) Image processing method and apparatus based on image processing model, electronic device, storage medium, and computer program product
KR20210068077A (ko) 3d모델 생성 방법, 장치, 기기 및 저장매체
WO2021139340A1 (zh) 扩充数据的方法、装置和计算机设备
CN111583264A (zh) 图像分割网络的训练方法、图像分割方法和存储介质
CN115170632A (zh) 三维点云的点云表征生成方法、装置、设备及存储介质
CN113555086A (zh) 基于机器学习的辩证分析方法、装置、设备及介质
CN118799369B (zh) 一种基于迭代优化的多模态图像配准方法及系统
CN110807463B (zh) 图像分割方法、装置、计算机设备和存储介质
WO2023173545A1 (zh) 超声图像的基准图像的生成方法、装置、设备及存储介质
CN119251501B (zh) 基于2d预训练分割模型监督的3d高斯全景分割方法、计算机设备和程序产品
CN111062860A (zh) 基于场景的图像色彩调整方法、装置和计算机设备
CN112614199A (zh) 语义分割图像转换方法、装置、计算机设备和存储介质
CN110309855A (zh) 图像分割的训练方法、计算机设备及存储介质
CN119693748A (zh) 重量确定方法、装置、电子设备及可读存储介质
CN112183324A (zh) 一种屏下指纹图像的生成方法和生成装置
CN117953582A (zh) 一种姿态估计方法、设备及存储介质
CN113724384B (zh) 机器人拓扑地图生成系统、方法、计算机设备及存储介质
CN111814898B (zh) 图像分割方法、装置、计算机设备和存储介质
CN116777988A (zh) 手部关键点的三维坐标构建方法、装置、设备及存储介质
CN114973382A (zh) 基于人工智能的人脸替换方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22931572

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 08.01.2025)

122 Ep: pct application non-entry in european phase

Ref document number: 22931572

Country of ref document: EP

Kind code of ref document: A1