WO2025016321A1 - 图像处理方法、电子设备和计算机可读存储介质 - Google Patents

图像处理方法、电子设备和计算机可读存储介质 Download PDF

Info

Publication number
WO2025016321A1
WO2025016321A1 PCT/CN2024/105246 CN2024105246W WO2025016321A1 WO 2025016321 A1 WO2025016321 A1 WO 2025016321A1 CN 2024105246 W CN2024105246 W CN 2024105246W WO 2025016321 A1 WO2025016321 A1 WO 2025016321A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
texture
images
texture features
monitoring area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/105246
Other languages
English (en)
French (fr)
Inventor
姚佳文
董和鑫
唐禹行
袁铭泽
夏英达
周靖人
吕乐
石喻
张灵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Publication of WO2025016321A1 publication Critical patent/WO2025016321A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/54—Extraction of image or video features relating to texture
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Definitions

  • the present application relates to the field of image processing, and in particular, to an image processing method, an electronic device, and a computer-readable storage medium.
  • deep learning prediction models can usually be used to predict the risks of the monitoring areas of the target parts of the monitored objects.
  • the accuracy of risk prediction for the monitoring areas using this method is low, resulting in the inability to accurately determine whether there are risks in the monitoring areas.
  • the embodiments of the present application provide an image processing method, an electronic device, and a computer-readable storage medium to at least solve the technical problem of low accuracy in determining whether there is a risk in a monitoring area in the related art.
  • an image processing method comprising: acquiring multi-phase images acquired at different times, wherein the display content of the multi-phase images at least includes a monitoring area of a target part of an object to be monitored; performing texture feature extraction on the multi-phase images to obtain target texture features of the multi-phase images; determining a target nerve distance of the multi-phase images based on a positional relationship between the monitoring area and other areas in the multi-phase images, wherein the other areas are used to characterize areas of the target part other than the monitoring area; generating a risk index for the monitoring area based on the target texture features and the target nerve distance, wherein the risk index is used to characterize the probability of the existence of a risk in the monitoring area.
  • an image processing method including: responding to an input instruction acting on an operation interface, displaying multiple images acquired at different times on the operation interface, wherein the display content of the multiple images at least includes a monitoring area of a target part of the object to be monitored; responding to an image processing instruction acting on the operation interface, displaying a risk indicator of the monitoring area on the operation interface, wherein the risk indicator is generated based on target texture features and target nerve distances of the multiple images, wherein the target texture features are obtained by extracting texture features from the multiple images, and the target nerve distance is determined based on the positional relationship between the monitoring area and other areas in the multiple images.
  • an image processing method including: displaying multiple images acquired at different times on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the display content of the multiple images at least includes a monitoring area of a target part of the object to be monitored; extracting texture features from the multiple images to obtain target texture features of the multiple images; determining a target nerve distance of the multiple images based on a positional relationship between the monitoring area and other areas in the multiple images, wherein the other areas are used to characterize areas of the target part other than the monitoring area; and extracting texture features from the multiple images to obtain target texture features of the multiple images.
  • a risk index of the monitoring area is generated, wherein the risk index is used to characterize the probability of risk in the monitoring area; and a VR device or an AR device is driven to display the risk index.
  • an image processing method including: acquiring multi-phase images collected at different times by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the multi-phase images, and the display content of the multi-phase images at least includes the monitoring area of the target part of the object to be monitored; performing texture feature extraction on the multi-phase images to obtain target texture features of the multi-phase images; determining the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images, wherein the other areas are used to characterize the area of the target part other than the monitoring area; generating a risk indicator of the monitoring area based on the target texture features and the target nerve distance, wherein the risk indicator is used to characterize the probability of the existence of risk in the monitoring area; outputting the risk indicator by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the risk indicator.
  • a computer-assisted cancer prognosis method comprising:
  • a risk index of the monitoring area is generated, wherein the risk index is used to characterize the probability that the cancer area has a risk.
  • a computer-assisted pancreatic cancer prognosis method comprising:
  • a risk index for the monitoring area is generated based on the target texture feature and the target nerve distance, wherein the risk index is used to characterize the probability that the pancreatic cancer area has a risk.
  • a computer-assisted cancer prognosis system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to perform a computer-assisted cancer prognosis method, the method comprising:
  • the target nerve distance of the medical images of multiple phases is determined, wherein the other regions are used to characterize the target part other than the cancer region. area;
  • a risk index of the monitoring area is generated, wherein the risk index is used to characterize the probability that the cancer area has a risk.
  • an electronic device including: a memory storing an executable program; and a processor for running the program, wherein any one of the above methods is executed when the program is running.
  • a computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described above in the claims.
  • a computer program product comprising computer executable instructions, which can implement the steps of any one of the above methods when executed by a processor.
  • a computer program is further provided, which implements the steps of any one of the above methods when executed by a processor.
  • a method in which a multi-phase image acquired at different times is acquired; texture features are extracted from the multi-phase image to obtain target texture features of the multi-phase image; based on the positional relationship between the monitoring area and other areas in the multi-phase image, the target nerve distance of the multi-phase image is determined; based on the target texture features and the target nerve distance, a risk index of the monitoring area is generated.
  • the risk index of the monitoring area is generated according to the target texture features of the monitoring area in the multi-phase image acquired at different times, and the target nerve distance between the monitoring area and other areas.
  • the contact situation between the monitoring area and other areas is fully considered in the risk prediction process of the monitoring area, thereby improving the accuracy of the determined risk index, thereby solving the technical problem of low accuracy in determining whether there is a risk in the monitoring area in the related art.
  • FIG1 is a schematic diagram of a hardware environment of a virtual reality device according to an image processing method according to an embodiment of the present application
  • FIG2 is a structural block diagram of a computing environment of an image processing method according to an embodiment of the present application.
  • FIG3 is a flow chart of an image processing method according to Embodiment 1 of the present application.
  • FIG4 is a schematic diagram of an image processing process according to Embodiment 1 of the present application.
  • FIG5 is a schematic diagram of an image processing result according to Example 1 of the present application.
  • FIG6 is a flow chart of an image processing method according to Embodiment 2 of the present application.
  • FIG7 is a flow chart of an image processing method according to Embodiment 3 of the present application.
  • FIG8 is a flow chart of an image processing method according to Embodiment 4 of the present application.
  • FIG9 is a structural block diagram of an image processing device according to Embodiment 5 of the present application.
  • FIG10 is a structural block diagram of an image processing device according to Embodiment 6 of the present application.
  • FIG11 is a structural block diagram of an image processing device according to Embodiment 7 of the present application.
  • FIG12 is a structural block diagram of an image processing device according to Embodiment 8 of the present application.
  • Figure 13 is a structural block diagram of an electronic device according to embodiment 12 of the present application.
  • Multi-phase CT Multi-phase Computed Tomography, used to obtain multi-phase images covering the monitoring area.
  • PDAC Pancreatic ductal adenocarcinoma, pancreatic ductal adenocarcinoma, abbreviated as pancreatic cancer.
  • CNN Convolutional Neural Network, convolutional neural network.
  • Neural distance refers to the distance between a point on the boundary of the monitoring area and a point on the boundary of other areas near the monitoring area, which can be used to evaluate the relationship between the monitoring area and other areas.
  • Texture perception model It can be used to combine local information and global information in multi-phase images to improve the extraction of texture features in multi-phase images.
  • Cancer prognosis The prediction of the future course of a cancer patient’s condition. Prognosis can include predictions of the patient’s survival time, the likelihood of disease recurrence, treatment efficacy, and quality of life.
  • an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • Fig. 1 is a schematic diagram of the hardware environment of a virtual reality device according to an image processing method of an embodiment of the present application.
  • a virtual reality device 104 is connected to a terminal 106, and the terminal 106 is connected to a server 102 through a network.
  • the virtual reality device 104 is not limited to: a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc.
  • the terminal 104 is not limited to a PC, a mobile phone, a tablet computer, etc.
  • the server 102 can be a server corresponding to a media file operator, and the network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.
  • the virtual reality device 104 of this embodiment includes: a memory, a processor, and a transmission device.
  • the memory is used to store an application program, which can be used to execute: obtaining multiple images collected at different times, wherein the display content of the multiple images at least includes the monitoring area of the target part of the object to be monitored; extracting texture features from the multiple images to obtain target texture features of the multiple images; determining the target nerve distance of the multiple images based on the positional relationship between the monitoring area and other areas in the multiple images, wherein the other areas are used to characterize the area of the target part other than the monitoring area; generating a risk index of the monitoring area based on the target texture features and the target nerve distance, wherein the risk index is used to characterize the probability of the monitoring area having a risk, thereby solving the technical problem of low accuracy in determining whether the monitoring area has a risk in the related art, and achieving the purpose of improving the accuracy of monitoring whether the monitoring area has a risk.
  • the terminal of this embodiment can be used to execute the display of multiple images of the monitoring area collected at different times, target texture features of the multiple images, the positional relationship between the monitoring area and other areas in the multiple images, risk indicators of the monitoring area, etc. on the presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, and send a display instruction to the virtual reality device 104. After receiving the display instruction, the virtual reality device 104 displays it at the target delivery position.
  • VR virtual reality
  • AR augmented reality
  • the virtual reality device 104 of this embodiment has an eye-tracking HMD (Head Mount Display) head display and an eye-tracking module that have the same functions as those in the above embodiment, that is, the screen in the HMD head display is used to display real-time images, and the eye-tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs.
  • the terminal of this embodiment obtains the user's position information and movement information in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space.
  • FIG2 shows an embodiment of using the AR/VR device (or mobile device) shown in FIG1 as a computing node in the computing environment 201 in a block diagram.
  • FIG2 is a structural block diagram of a computing environment of an image processing method according to an embodiment of the present application. As shown in FIG2, the computing environment 201 includes multiple (210-1, 210-2, ..., shown in the figure) computing nodes (such as servers) running on a distributed network.
  • Different computing nodes contain local processing and memory resources, and the terminal user 202 can remotely run applications or store data in the computing environment 201.
  • the application can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in the computing environment 201, representing services "A”, “D”, “E” and "H” respectively.
  • the end user 202 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 202's provision and/or request can be provided to the entry gateway 230.
  • the entry gateway 230 can include a corresponding agent to handle the provision and/or request for the service (one or more services provided in the computing environment 201).
  • Services are provided or deployed based on various virtualization technologies supported by computing environment 201.
  • services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and/or similar methods.
  • Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine to execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS). So that multiple workloads can run on a single operating system instance.
  • OS operating system
  • a Pod e.g., a Kubernetes Pod
  • service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods).
  • the Pod may include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers).
  • One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc.
  • Other services may also be equipped with similar Pods.
  • executing a user request from the end user 202 may require invoking one or more services in the computing environment 201, and executing one or more functions of one service requires invoking one or more functions of another service.
  • service “A” 220-1 receives a user request from the end user 202 from the ingress gateway 230, service “A” 220-1 may call service “D” 220-2, and service “D” 220-2 may request service “E” 220-3 to execute one or more functions.
  • the computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers.
  • the computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.
  • FIG3 is a flow chart of an image processing method according to Embodiment 1 of the present application. As shown in FIG3, the method may include the following steps:
  • Step S302 acquiring multiple images collected at different times.
  • the display content of the multi-phase images at least includes the monitoring area of the target part of the object to be monitored.
  • the above-mentioned monitoring object may refer to an animal whose tissue or organ may have lesions in the body, such as humans, cats, hamsters and other animals.
  • the above-mentioned monitoring area of the target part may refer to an area on the tissue or organ that may have lesions, such as a tumor area on the pancreas.
  • the above-mentioned multi-phase image may refer to image acquisition of the monitoring area at different times. For example, the monitoring area at different times can be scanned and imaged by tomography or magnetic resonance imaging to acquire multi-phase images of the monitoring area at different times.
  • a single or multiple images of the monitoring area acquired in a short period of time may not be able to accurately determine whether a lesion has occurred in the monitoring area, as well as the corresponding lesion conditions, such as whether a tumor has occurred, the trend of tumor lesions, etc.
  • the image processing system may first acquire the above-mentioned multi-phase images including the monitoring area acquired at different times, thereby avoiding the limitation of the lesion conditions of the monitoring area reflected by a single-phase image, resulting in poor accuracy in determining whether there are risks in the monitoring area.
  • the multi-phase images include at least: images of the non-contrast phase, the early development phase, and the current phase.
  • the multi-phase images include at least: images of the non-contrast phase images of the pancreatic phase and images of the portal vein phase.
  • Step S304 extracting texture features of the multi-period images to obtain target texture features of the multi-period images.
  • the above-mentioned target texture features can be texture features obtained by fusing the global texture features and local texture features presented in the monitoring area, wherein the local texture features can refer to the texture features of the monitoring area, which may include but are not limited to: color, texture, shape, edge, corner point and other features of the monitoring area, and the global texture features can refer to the texture features of the entire image in multiple phases of images, such as the texture features of the target part, the texture features of the area outside the target part, etc.
  • the corresponding other areas may be the areas where the blood vessels around the pancreas are located
  • the corresponding local texture features may refer to the color, texture, shape, edge, corner points and other features of the PDAC area
  • the corresponding global texture features may refer to the color, texture, shape, edge, corner points and other features of the pancreas and the blood vessels around the pancreas.
  • the image processing system can, after acquiring multiple images at different times, perform texture feature extraction on the multiple images to determine the target texture features of the above-mentioned multiple images.
  • the PDAC area when extracting texture features from multi-phase images to obtain target texture features, it is considered that there may be more important tissues and organs in areas other than the target part and target part of the monitoring area, for example, there may be more important blood vessels on and around the pancreas except the PDAC area. These important tissues and organs may have a certain impact on the monitoring of the monitoring area. For example, if the color of the blood vessels is close to the color of the PDAC area, there may be a misjudgment of PDAC diffusion. At the same time, it is considered that the monitoring area may also have an impact on these important tissues and organs when it is diseased. For example, the PDAC area may invade the blood vessels around the pancreas.
  • the image processing system can also monitor the monitoring area based on the texture presented by other areas, that is, extracting global texture features, and then fusing the local texture features with the global texture features to obtain the above-mentioned target texture features, thereby ensuring the accuracy of determining whether there is a risk in the monitoring area based on the target texture features.
  • Step S306 determining the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images.
  • the target nerve distance may refer to the distance between the surface of the monitoring area and the surface of other areas. For example, if the monitoring area is located in the PDAC area, the target nerve distance may refer to the distance between the edge of the monitoring area and the blood vessels around the pancreas.
  • the target nerve distances corresponding to different images can be determined based on the positional relationship between the monitoring area and other areas in multiple images, such as the relative position, structural connection and other information between the aforementioned monitoring area and other areas.
  • Step S308 generating a risk index for the monitoring area based on the target texture features and the target nerve distance.
  • the risk index is used to characterize the probability of risk existing in the monitoring area.
  • the above-mentioned risk index may refer to whether there is a risk in the monitoring area, and the probability of the risk existing. Generally, the range of the above-mentioned risk index may be determined as 0-1. The larger the risk index is, the more likely the risk is to occur in the monitoring area, and the greater the negative impact of the lesions in the monitoring area on the monitored object.
  • the risk index of the monitoring area can be determined based on the target texture features and target nerve distances determined above. For example, if the target texture features indicate that the lesion in the monitoring area is more serious and the target nerve distance is shorter, the corresponding risk index is higher.
  • the image processing system can quickly determine the risk index of the monitoring area without the need for the staff to repeatedly compare the texture presented by the monitoring area in the multi-phase images. At the same time, it avoids the situation where the staff cannot accurately observe the changes in the monitoring area due to the small changes in the texture features presented by the monitoring area at different times, thereby reducing the occurrence of misjudgments and reducing the work pressure of the staff.
  • a method in which a multi-phase image acquired at different times is acquired; texture features are extracted from the multi-phase image to obtain target texture features of the multi-phase image; based on the positional relationship between the monitoring area and other areas in the multi-phase image, the target nerve distance of the multi-phase image is determined; based on the target texture features and the target nerve distance, a risk index of the monitoring area is generated.
  • the risk index of the monitoring area is generated according to the target texture features of the monitoring area in the multi-phase image acquired at different times, and the target nerve distance between the monitoring area and other areas.
  • the contact situation between the monitoring area and other areas is fully considered in the risk prediction process of the monitoring area, thereby improving the accuracy of the determined risk index, thereby solving the technical problem of low accuracy in determining whether there is a risk in the monitoring area in the related art.
  • texture features are extracted from multiple images to obtain target texture features of the multiple images, including: extracting features from images to obtain initial texture features of the images; and using at least one texture perception model to fuse global information and local information in the initial texture features to obtain target texture features.
  • the above-mentioned initial texture features may refer to the texture features of the monitoring area and other areas obtained by extracting features from the image, and may be the global texture features of the image.
  • the above-mentioned global information may refer to the information between the aforementioned monitoring area and other areas, such as the texture information between the pancreas and the blood vessels around the pancreas.
  • the above-mentioned local information may refer to the information of the aforementioned monitoring area itself, such as the texture information of the PDAC area.
  • the above-mentioned texture perception model may be a mode for downsampling the initial texture features and processing the downsampling results using an attention mechanism.
  • the texture perception model includes at least a CNN convolutional neural network and a Transformer network.
  • the number of the texture perception models is at least 1, and the texture perception model The greater the number, the higher the accuracy of the target texture features obtained, and the corresponding accuracy in determining whether there is a risk in the monitoring area is higher, but the efficiency is lower. Therefore, considering the efficiency of determining whether there is a risk in the monitoring area, the number of the above texture perception models can be set to 3.
  • the image processing system may first use a preset feature extraction module to perform feature extraction on the multi-period image to determine the initial texture features of the monitoring area in the image, and then use at least one of the above-mentioned texture perception models to perform downsampling processing on the initial texture features to obtain the above-mentioned global information and local information in the initial texture features, and then use the attention mechanism to process the global information and local information to obtain the above-mentioned target texture features.
  • the process of obtaining the target texture features is a process performed by a single texture perception model.
  • the number of texture perception models is set to 3
  • the target texture features corresponding to the previous texture perception model need to be used as the initial texture features of the next texture perception model, and the process of obtaining the target texture features is executed repeatedly, thereby improving the accuracy of the target texture features finally obtained.
  • the feature extraction module for extracting initial texture features includes at least: a 5 ⁇ 5 ⁇ 5 convolution layer and a 3 ⁇ 3 ⁇ 3 convolution layer connected in sequence.
  • At least one texture perception model block is used to fuse the global information and local information in the initial texture features to obtain the target texture features, including: using the convolutional neural network module in the texture perception model to downsample the initial texture features to obtain the local texture features of the image; using the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image.
  • the above-mentioned convolutional neural network module can be a module for capturing local information from initial texture features, such as local texture features of the monitoring area in multiple phase images.
  • the above-mentioned self-attention module can be a module for capturing the dependencies between different local texture features and obtaining target texture features, which can achieve the purpose of fusing local information and global information.
  • the image processing system may first use the convolutional neural network module in the texture perception model to downsample the aforementioned extracted initial texture features to obtain the local texture features of the image, and then use the self-attention module in the texture perception module to perform self-attention processing on the local texture features to obtain the above-mentioned target texture features.
  • a convolutional neural network module in a texture perception model is used to downsample the initial texture features
  • the local texture features of the image include: using the first convolutional layer in the convolutional neural network module to capture the first local information in the initial texture features; using the second convolutional layer in the convolutional neural network module to map the first local information to a preset space to obtain a first mapped texture feature; using the third convolutional layer in the convolutional neural network module to restore the mapped texture features to the original space corresponding to the initial texture features to obtain a restored texture feature; using the normalization layer in the convolutional neural network module to normalize the restored texture features to obtain local texture features.
  • the dimension of the preset space is greater than the dimension of the original space.
  • the first convolution layer in the convolutional neural network module may refer to a convolution layer for capturing the first local information of the monitoring area and other areas from the initial texture features, such as the texture information of the monitoring area and other areas, which may be a 3 ⁇ 3 ⁇ 3 structure.
  • the second convolution layer may refer to a convolution layer for mapping the captured first local information to a high-dimensional preset space to obtain a hidden feature that can be encoded.
  • the convolution layer may be a 1 ⁇ 1 ⁇ 1 structure
  • the third convolution layer may be a convolution layer used to restore the encoded hidden features to obtain restored texture features
  • the normalization layer may be a convolution layer used to normalize the restored texture features to obtain local texture features with higher accuracy.
  • the image processing system in order to ensure the accuracy of the obtained local texture features, can first use the first convolution layer in the convolutional neural network module to capture the local information of the above-mentioned monitoring area and other areas from the initial texture features, that is, the above-mentioned first local information.
  • the initial texture features of the initial multi-period images can be divided into multiple image blocks, and then the corresponding first local information can be captured from the corresponding initial texture features based on the multiple image blocks.
  • the image processing system can use the second convolution layer in the convolutional neural network module to map and encode the first local information into a high-dimensional preset space to obtain the corresponding hidden feature, that is, the first mapped texture feature mentioned above.
  • the first mapped texture feature after mapping can be H is the height of the image block, W is the width of the image block, D is the depth of the image block, and C is the dimension of the image block, C l >C.
  • the image processing system can reuse the third convolution layer in the convolutional neural network module to restore the first mapped texture feature to the restoration space where the first local information is located to obtain the restored texture feature, that is, the above-mentioned restored texture feature, and finally use the normalization layer in the convolutional neural network module to normalize the restored texture feature to obtain the above-mentioned local texture feature with higher accuracy.
  • a self-attention module in a texture perception model is used to perform attention processing on local texture features to obtain target texture features of an image, including: using a first convolutional layer in the self-attention module to capture second local information in the local texture features; using a second convolutional layer in the self-attention module to map the second local information to a preset space to obtain a second mapped texture feature; using a self-attention layer in the self-attention module to perform self-attention processing on the second mapped texture feature to obtain a self-attention texture feature; using a feedforward layer in the self-attention module to perform feature interaction on the self-attention texture features to obtain an interactive texture feature; using a removal layer in the self-attention module to perform feature fusion on the interactive texture features to obtain a target texture feature.
  • the first convolution layer in the self-attention module may be a convolution layer for capturing second local information from local texture features, such as texture information of other regions, and may be a 3 ⁇ 3 ⁇ 3 structure.
  • the second local information may be local information of the initial texture feature
  • the second convolution layer may be a convolution layer for mapping the second local information to a high-dimensional preset space to obtain a hidden feature that can be encoded, and may be a 1 ⁇ 1 ⁇ 1 structure.
  • the self-attention layer may be a convolution layer for performing self-attention processing on hidden features to obtain self-attention texture features that can express the dependency relationship between different hidden features
  • the feedforward layer may be a convolution layer for performing interactive processing on self-attention texture features to obtain visual and interpretable interactive texture features
  • the removal layer may be a convolution layer for deleting and fusing interactive texture features to obtain target texture features with higher accuracy.
  • the local texture features are processed by the self-attention module to obtain
  • the image processing system can first use the first convolution layer in the above-mentioned self-attention module to capture the local information of the monitoring area and other areas from the above-mentioned normalized local texture features, that is, the above-mentioned second local information, and then use the second convolution layer in the self-attention module to encode and map the second local information to a high-dimensional preset space to obtain the corresponding hidden features, that is, the above-mentioned second mapped texture features, and then use the self-attention layer in the self-attention module to perform self-attention processing on the second mapped texture features to obtain the corresponding self-attention texture features, and finally use the feedforward layer in the self-attention module to interactively process the self-attention texture features, so as to obtain interactive texture features that can intuitively express and facilitate the understanding of the self-attention texture features, and use the removal
  • At least one texture perception model is used to fuse the global information and local information in the initial texture features to obtain the target texture features, including: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain the output texture features of the image; performing cross-attention processing on the output texture features of multiple periods of images to obtain cross-attention features; splicing the output texture features and cross-attention features of multiple periods of images to obtain the target texture features.
  • the output texture feature may refer to the hidden feature in the aforementioned preset space, i.e., the second mapping texture feature, which is divided, and the output feature after the hidden feature after division is processed by the aforementioned self-attention layer and feedforward layer, i.e., the interactive texture feature.
  • the second mapping texture feature may be divided into a plurality of non-overlapping 3D (3-dimensional) feature blocks according to a preset height, width, and depth, thereby obtaining the output texture feature.
  • the divided multiple non-overlapping 3D feature blocks can be the same as the aforementioned image blocks, and the 3D feature blocks can be expressed in the form of Fu ⁇ R V ⁇ N ⁇ C , where V represents the volume of the 3D feature block, N represents the number of 3D feature blocks, and C represents the dimension of the second mapping texture feature.
  • the corresponding output texture feature can be expressed in the form of F O ⁇ R D ⁇ nC , where D represents the depth of the 3D feature block and n represents the number of texture perception models.
  • the cross-attention feature may refer to an attention feature obtained by extracting cross-modal information from the output texture features of each of the multiple images and processing the cross-modal information using an additional mask, wherein the additional mask may be expressed in the form of M ⁇ 0,- ⁇ nC ⁇ nC , wherein n represents the number of texture perception models, and C represents the dimension of the output texture feature.
  • the cross-attention feature may be expressed in the form of:
  • F cross represents the output texture feature
  • Q, K, and T are the query matrix, key matrix, and value matrix obtained by linearly projecting the aforementioned output texture feature, respectively.
  • the image processing system may also use the above-mentioned at least one texture perception model to fuse the global information and local information in the initial texture feature, so as to obtain the target texture feature.
  • the above-mentioned output texture features are obtained, and then the cross-modal information of multiple images is extracted from the output texture features of each of the multiple images, and the cross-modal information is cross-attention processed to obtain the above-mentioned cross-attention features.
  • the output texture features and the cross-attention features are spliced. For example, after the output texture features and the cross-attention features are connected in series, the concatenated features are spliced using a preset evaluation aggregation layer to obtain the above-mentioned target texture features.
  • cross-attention processing is performed on the output texture features of multiple periods of images to obtain cross-attention features, including: using a cross-attention module to perform cross-attention processing on the output texture features of images from different periods to obtain cross-attention features.
  • a preset cross-attention module can be used to extract respective cross-modal information from the output texture features of images of different periods, and the output texture features can be cross-attention processed using the cross-modal information to obtain the above-mentioned cross-attention features.
  • the target nerve distance of the multi-phase images is determined, including: performing image segmentation on the image to obtain the segmentation result of the image, wherein the segmentation result at least includes: the monitoring area and other areas; based on the segmentation result, determining a first point set located at the boundary of the monitoring area, and a second point set located at the boundary of other areas; determining a first subset of the first point set that meets a preset condition, and a second subset of the second point set that meets the preset condition; performing cross-attention processing on the first subset and the second subset to obtain the target nerve distance.
  • the first point set may refer to a point set obtained by sampling the boundary of the monitoring area
  • the second point set may refer to a point set obtained by sampling the boundary of other areas.
  • the image processing system can first use a preset image segmentation module, such as the nnUet model (an image segmentation model), to perform image segmentation processing on the image, so as to obtain the respective boundaries of the monitoring area and other areas in the image, wherein the other areas may include a variety of different types of blood vessels, such as the portal vein and splenic vein, the superior mesenteric artery, the superior mesenteric vein, and the celiac duct.
  • a preset image segmentation module such as the nnUet model (an image segmentation model)
  • the other areas may include a variety of different types of blood vessels, such as the portal vein and splenic vein, the superior mesenteric artery, the superior mesenteric vein, and the celiac duct.
  • the image processing system can determine the above-mentioned first point set from the boundary of the monitoring area in the segmentation result, and determine the above-mentioned second point set from the boundary of other areas, such as the above-mentioned blood vessels.
  • X represents a point on the boundary of the monitoring area
  • Y represents the boundary of other areas, such as the point on the blood vessel mentioned above.
  • the corresponding distance from the boundary of the monitoring area to the boundary of other areas can be expressed as:
  • the image processing system determines a first subset from the first point set and determines a second subset from the second point set according to a preset condition, wherein the preset condition may refer to determining a preset number of multiple distance points from the point set.
  • the point closest to the corresponding boundary for example, from the above distance
  • the smallest 20 points are selected to construct the corresponding subset.
  • the other boundary can refer to the boundary of another area where the point is located. For example, if the point is in the monitoring area, the other boundary can refer to the boundary corresponding to other areas.
  • cross-attention processing is performed on multiple points in the first subset and the second subset to obtain the above target neural distance.
  • determining a first subset that satisfies a preset condition in a first point set and a second subset that satisfies a preset condition in a second point set includes: sampling the first point set to obtain a first sampling point set, and sampling the second point set to obtain a second sampling point set; determining a preset number of first sampling points that are closest to the second sampling point set in the first sampling point set to obtain the first subset; and determining a preset number of second sampling points that are closest to the first sampling point set in the second sampling point set to obtain the second subset.
  • the first point set in the process of determining the first subset and the second subset, may be further sampled to obtain a corresponding first sampling point set, and the second point set may be further sampled to obtain a corresponding second sampling point set, and then a preset number of multiple first sampling points closest to the second sampling point set may be determined from the first sampling point set to construct the above-mentioned first subset, and at the same time, a preset number of multiple second sampling points closest to the first sampling point set may be determined from the second sampling point set to construct the above-mentioned second subset.
  • K represents the number of points in the first subset.
  • the second subset can be expressed as:
  • the target neural distance can be expressed as:
  • a risk index for a monitoring area is generated based on target texture features and target nerve distances, including: extracting the structural relationship between the monitoring area and other areas in multiple phase images to obtain structural features of the multiple phase images; splicing the target texture features, target nerve distances and structural features to obtain spliced features; and generating a risk index based on the spliced features.
  • the image processing system may first use a preset structural analysis model, such as a 3D-CNN model, to extract the structural relationship between the monitoring area and other areas in the multi-phase images, thereby obtaining the structural features of the multi-phase images, such as the connection relationship between the blood vessels in the monitoring area and other areas, and then splice the structural features with the aforementioned target texture features and target nerve distances to obtain the above-mentioned splicing features, and finally use the fully connected layer to generate the above-mentioned risk index according to the splicing features.
  • a preset structural analysis model such as a 3D-CNN model
  • the above risk index is generated by using the fully connected layer and the splicing feature prediction
  • the risk loss can be used as the risk loss through likelihood estimation, thereby improving the accuracy of the final output risk index.
  • Figure 4 is a schematic diagram of an image processing process according to Example 1 of the present application, wherein 1 represents multi-phase images collected at different times, taking pancreatic cancer as an example, 11 represents images of the non-contrast stage, 12 represents images of the pancreatic stage, 13 represents images of the portal vein stage, 2 represents the process of extracting features from multi-phase images to obtain target texture features, 21 represents a feature extraction module for extracting initial texture features, 22 represents a texture perception model for obtaining target texture features, 221 represents a convolutional neural network module, 222 represents a self-attention module, 23 represents an output layer feature, 24 represents a cross-attention feature, Block 1, 2, 3 represents a texture perception model, Block4 represents a cross-processing module, Block7 represents a self-attention layer, Block8 represents a feedforward layer, Block9 represents a removal layer, 3 represents a distance determination module for determining the target neural distance, 31 represents
  • a method for predicting the prognosis of pancreatic cancer comprising: acquiring multi-phase medical images acquired at different times, wherein the display content of the multi-phase medical images at least includes tumors and blood vessels of pancreatic cancer; performing texture feature extraction on the multi-phase medical images to obtain target texture features of the multi-phase medical images; determining the target nerve distance of the multi-phase medical images based on the positional relationship between the tumors and blood vessels in the multi-phase medical images; and determining the survival outcome of pancreatic cancer based on the target texture features and the target nerve distance.
  • the above-mentioned pancreatic cancer is a cancer with a relatively high mortality rate at present.
  • the above-mentioned multi-phase medical images may refer to images containing pancreatic cancer lesion areas, that is, images containing pancreatic cancer tumors and blood vessel areas, which are collected at different times.
  • the above-mentioned pancreatic cancer tumor area corresponds to the above-mentioned monitoring area
  • the above-mentioned blood vessel area corresponds to the above-mentioned other areas.
  • the prognosis prediction system needs to first obtain multi-phase medical images collected at different times, including tumor and blood vessel areas of pancreatic cancer, and then perform texture feature extraction on the multi-phase medical images to obtain target texture features of the multi-phase medical images, and at the same time, determine the target nerve distance of each of the multi-phase medical images based on the positional relationship between the tumor and the blood vessel in the multi-phase medical images, and finally determine the survival outcome of pancreatic cancer based on the target texture features and the target nerve distance.
  • 1070 PDAC patients were selected and divided into two groups.
  • the first group included 892 patients.
  • the second largest group included 178 patients, and an additional 3 centers, Center B, Center C, and Center D, were used for independent testing.
  • the multi-phase medical images used in the study were CT images of the non-contrast phase, pancreatic phase, and portal vein phase.
  • Center A in order to improve the comparison effect, 340 patients were selected from 892 patients, and several radiologists with 18 years of experience in pancreatic cancer diagnosis manually labeled the PDAC masks of the multi-phase medical images of these 178 patients to obtain comparison images.
  • a model with self-learning ability was used to predict the PDAC masks of the multi-phase medical images of the remaining patients, and the same annotators, that is, most of the above-mentioned radiologists, checked and adjusted the self-learning model to ensure the accuracy of the prediction results of the self-learning model.
  • the same model such as the semi-supervised generation of other container mask segmentation models, was also used to separately segment the PDAC masks of the multi-phase medical images of other patients.
  • the batch size was set to 16
  • the maximum iteration was set to 1000 epochs
  • the model with better performance was selected according to the training results on the validation set.
  • the performance of the constructed texture perception model is evaluated by comparing it with the ResNet18CNN backbone and the ViT transformer backbone.
  • the evaluation result shows that the texture perception model has better processing effect when processing local information and global information of the pancreas.
  • the performance of the cross-attention module is evaluated by comparing it with common methods such as LSTM and early fusion.
  • the evaluation result shows that the processing results of the cross-attention module for the in-phase feature part and the cross-phase feature part are more reasonable than the processing results of other methods.
  • the various parts of the image processing method shown in Figure 4 are evaluated, and the evaluation results can be shown in Table 1 below.
  • the use of texture perception model and structure extraction model can improve the risk index from 0.630 to 0.648, which is consistent with the fact that tumor invasion seriously affects the survival rate of PDAC patients.
  • the oblique distance using only the tumor and four blood vessels that is, the target nerve distance
  • a 4-variable regression model for prognosis prediction was used to verify the relationship between nerve distance and survival rate. This relationship can be confirmed by the C index of 0.611 in the third to last row in Table 1, that is, the nerve distance has a certain correlation with the risk index, which is consistent with clinical findings. This means that adding the step of measuring nerve distance can effectively improve the accuracy of the predicted risk index.
  • the nerve distance metric proposed in this application performs traditional surface distance metrics, such as bevel distance, which can be effectively applied to distinguish the severity of PDAC, that is, determine the risk index.
  • this application modified the baseline deep learning model and used a single pancreatic stage or all three stages mentioned above as input to build the network architecture.
  • DeepCT-PDAC is the latest method using 3DCNN to consider tumor-related and tumor-vascular relationships.
  • the use of texture-aware models and structure extraction models proposed in this application to capture tumor augmentation patterns and tumor vascular involvement has proven to be effective in nested 5-fold cross-validation and multi-entry independent test sets with better performance.
  • Table 3 uses univariate and multivariate Cox proportional hazards models to evaluate the target texture features and other The effectiveness of clinical pathological factors in the independent test group. As shown in Table 3, risk stratification and other factors, such as pathological TNM stage, are important influencing factors affecting the accuracy of prognostic prediction.
  • risk stratification and other factors such as pathological TNM stage, are important influencing factors affecting the accuracy of prognostic prediction.
  • FIG5 is a schematic diagram of survival curves under different groupings according to Example 1 of the present application, as shown in FIG5 , wherein FIG5 shows the survival curves of patients under different groupings, and the specific groups are: A is a risk indicator curve without grouping, B is grouped according to tumor size, C is grouped according to CA19-9 values, D is grouped according to both tumor size and CA19-9 values, and E and F are grouped according to patient age. It can be concluded from FIG5 that patients with high risks and more serious conditions need to use the target texture features and target nerve distances of the present application as neoadjuvant therapy.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the technical solution of the present application, or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.
  • a storage medium such as ROM/RAM, magnetic disk, optical disk
  • a terminal device which can be a mobile phone, computer, server, or network device, etc.
  • an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • FIG6 is a flow chart of an image processing method according to Embodiment 2 of the present application. As shown in FIG6 , the method may include the following steps:
  • Step S602 in response to an input command on the operation interface, displaying the data collected at different times on the operation interface Multi-issue images.
  • the display content of the multi-phase images at least includes the monitoring area of the target part of the object to be monitored.
  • the above-mentioned input instruction may refer to an instruction for displaying a multi-phase image in a preset operation interface
  • the above-mentioned monitoring object may refer to an animal whose tissue or organ may have lesions in the body, such as humans, cats, hamsters and other animals.
  • the monitoring area of the above-mentioned target part may refer to the area corresponding to the tissue or organ that may have lesions, such as a tumor area.
  • the above-mentioned multi-phase image may refer to image acquisition of the monitoring area at different times, for example, the monitoring area at different times may be scanned and imaged by tomography or magnetic resonance imaging to acquire multi-phase images of the monitoring area at different times.
  • the image processing unit can display multiple images acquired at different times in a preset operation interface.
  • Step S604 in response to the image processing instruction acting on the operation interface, the risk index of the monitoring area is displayed on the operation interface.
  • the risk index is generated based on the target texture features and target neural distance of the multi-phase images.
  • the target texture features are obtained by extracting the texture features of the multi-phase images, and the target neural distance is determined based on the positional relationship between the monitoring area and other areas in the multi-phase images.
  • the target nerve distance may refer to the distance between the surface of the monitoring area and the surfaces of other areas, for example, the distance between the monitoring area and the surfaces of blood vessels in other areas.
  • the image processing system can perform texture feature extraction on the multi-phase images to obtain the target texture features of the multi-phase images, and determine the target nerve distance between the monitoring area and the other area based on the positional relationship between the monitoring area and the other area in the multi-phase images, and then determine the risk index of the monitoring area based on the target texture features and the target nerve distance to determine whether there is a risk in the monitoring area for multiple periods.
  • an image processing method that can be applied to virtual reality scenarios such as virtual reality VR devices and augmented reality AR devices.
  • steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • FIG7 is a flow chart of an image processing method according to Embodiment 3 of the present application. As shown in FIG7 , the method may include the following steps:
  • Step S702 Display multiple images collected at different times on a presentation screen of a virtual reality VR device or an augmented reality AR device.
  • the display content of the multi-phase images at least includes the monitoring area of the target part of the object to be monitored.
  • the above monitoring objects may refer to animals whose tissues or organs may be diseased, such as humans, cats,
  • the monitoring area of the target part may refer to the area corresponding to the tissue or organ where the lesion may occur, such as a tumor area.
  • the multi-phase image may refer to the image acquisition of the monitoring area at different times, for example, the monitoring area at different times may be scanned and imaged by tomography or magnetic resonance imaging to acquire multi-phase images of the monitoring area at different times.
  • the above-mentioned multiple images including the monitoring area collected at different times may be acquired through a virtual reality VR device or an augmented reality AR device.
  • Step S704 extracting texture features of the multi-period images to obtain target texture features of the multi-period images.
  • the above-mentioned target texture features may be texture features obtained by fusing the global texture features and local texture features presented in the monitoring area, wherein the local texture features may refer to the texture features of the monitoring area, which may include but are not limited to: color, texture, shape, edge, corner point and other features of the monitoring area, and the global texture features may refer to the texture features of other areas outside the overall area of the multi-phase image, such as the texture features of the target part, the texture features of the area outside the target part, etc.
  • the image processing system can, after acquiring multiple images at different times, perform texture feature extraction on the multiple images to determine the target texture features of the above-mentioned multiple images.
  • the PDAC area when extracting texture features from multi-phase images to obtain target texture features, it is considered that there may be more important tissues and organs in areas other than the target part and target part of the monitoring area, for example, there may be more important blood vessels on and around the pancreas except the PDAC area. These important tissues and organs may have a certain impact on the monitoring of the monitoring area. For example, if the color of the blood vessels is close to the color of the PDAC area, there may be a misjudgment of PDAC diffusion. At the same time, it is considered that the monitoring area may also have an impact on these important tissues and organs when it is diseased. For example, the PDAC area may invade the blood vessels around the pancreas.
  • the image processing system can also monitor the monitoring area based on the texture presented by other areas, that is, extracting global texture features, and then fusing the local texture features with the global texture features to obtain the above-mentioned target texture features, thereby ensuring the accuracy of determining whether there is a risk in the monitoring area based on the target texture features.
  • Step S706 determining the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images.
  • the distance between the monitoring area and other areas in multiple phase images can be further combined to determine whether there is a risk in the monitoring area, thereby improving the accuracy of the determination result.
  • the undetectable region can be determined based on the positional relationship between the monitoring region and other regions in the multi-phase images, such as the relative position, structural connection, etc. between the monitoring region and other regions. The distance of the target nerve corresponding to the image.
  • Step S708 generating a risk index for the monitoring area based on the target texture features and the target nerve distance.
  • the risk index is used to characterize the probability of risk existing in the monitoring area.
  • the distance between the monitoring region and other regions in the multi-phase images can be further combined to determine whether the monitoring region has risks, thereby improving the accuracy of the determination result.
  • the target nerve distances corresponding to different images can be determined based on the positional relationship between the monitoring area and other areas in multiple images, such as the relative position, structural connection and other information between the aforementioned monitoring area and other areas.
  • Step S710 driving the VR device or AR device to display the risk indicator.
  • a VR device or an AR device can be driven to display the determined risk indicators to the user, so that the staff can more intuitively monitor the risk indicators corresponding to the area.
  • the above image processing method can be applied to a hardware environment composed of a server and a virtual reality device.
  • Multi-phase images, target texture features of multi-phase images, target neural distances and risk indicators are displayed on a presentation screen of a virtual reality VR device or an augmented reality AR device.
  • the server can be a server corresponding to a media file operator.
  • the above network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.
  • the above virtual reality device is not limited to: a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc.
  • the virtual reality device includes: a memory, a processor, and a transmission device.
  • the memory is used to store an application program, which can be used to execute: extracting texture features from multiple images to obtain target texture features of the multiple images, including: extracting features from the images to obtain initial texture features of the images; and fusing global information and local information in the initial texture features using at least one texture perception model to obtain target texture features.
  • the application is also used to execute: using at least one texture perception model block to fuse the global information and local information in the initial texture features to obtain target texture features, including: using the convolutional neural network module in the texture perception model to downsample the initial texture features to obtain the local texture features of the image; using the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image.
  • the application is also used to execute: using the convolutional neural network module in the texture perception model to downsample the initial texture features and the local texture features of the image, including: using the first convolutional layer in the convolutional neural network module to capture the first local information in the initial texture features; using the second convolutional layer in the convolutional neural network module to map the first local information to a preset space to obtain a first mapped texture feature; using the third convolutional layer in the convolutional neural network module to restore the mapped texture features to the original space corresponding to the initial texture features to obtain a restored texture feature; using the normalization layer in the convolutional neural network module to normalize the restored texture features to obtain a local texture feature.
  • the application is also used to execute: using the self-attention module in the texture perception model to perform attention processing on local texture features to obtain target texture features of the image, including: using the first convolution layer in the self-attention module to capture second local information in the local texture features; using the second convolution layer in the self-attention module to map the second local information to a preset space to obtain a second mapped texture feature; using the self-attention layer in the self-attention module to perform self-attention processing on the second mapped texture features to obtain self-attention texture features; using the feedforward layer in the self-attention module to perform feature interaction on the self-attention texture features to obtain interactive texture features; using the removal layer in the self-attention module to perform feature fusion on the interactive texture features to obtain target texture features.
  • the application is also used to execute: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain target texture features, including: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain output texture features of the image; performing cross-attention processing on the output texture features of multiple period images to obtain cross-attention features; splicing the output texture features and cross-attention features of multiple period images to obtain target texture features.
  • the application is also used to execute: performing cross-attention processing on the output texture features of multiple period images to obtain cross-attention features, including: using a cross-attention module to perform cross-attention processing on the output texture features of images from different periods to obtain cross-attention features.
  • the application is also used to execute: determining the target nerve distance of multiple images based on the positional relationship between the monitoring area and other areas in the multiple images, including: performing image segmentation on the image to obtain the segmentation result of the image, wherein the segmentation result includes at least: the monitoring area and other areas; based on the segmentation result, determining a first point set located at the boundary of the monitoring area, and a second point set located at the boundary of other areas; determining a first subset of the first point set that meets a preset condition, and a second subset of the second point set that meets the preset condition; performing cross-attention processing on the first subset and the second subset to obtain the target nerve distance.
  • the application is also used to execute: determining a first subset of the first point set that meets a preset condition, and a second subset of the second point set that meets a preset condition, including: sampling the first point set to obtain a first sampling point set, and sampling the second point set to obtain a second sampling point set; determining a preset number of first sampling points in the first sampling point set that are closest to the second sampling point set to obtain a first subset; determining a preset number of second sampling points in the second sampling point set that are closest to the first sampling point set to obtain a second subset.
  • the application is also used to perform: generating a risk index for the monitoring area based on target texture features and target nerve distances, including: extracting the structural relationship between the monitoring area and other areas in multiple phase images to obtain the structural features of the multiple phase images; splicing the target texture features, target nerve distances and structural features to obtain spliced features; generating a risk index based on the spliced features.
  • the above-mentioned image processing method applied in a VR device or an AR device of this embodiment may include the method of the embodiment shown in Figure 3, so as to achieve the purpose of driving the VR device or the AR device to display multiple images, target texture features, target data distance and risk indicators.
  • the processor of this embodiment can call the application stored in the memory to execute the above steps through the transmission device.
  • the transmission device can receive the media file sent by the server through the network, and can also be used for the processor and Data transfer between memories.
  • a head-mounted display with eye tracking is provided, wherein the screen in the HMD is used to display the displayed video images, the eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyes, the tracking system is used to track the user's position information and motion information in the real three-dimensional space, and the computing processing unit is used to obtain the user's real-time position and motion information from the tracking system, and calculate the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space, etc.
  • a virtual reality device may be connected to a terminal, and the terminal and the server are connected through a network.
  • the virtual reality device is not limited to: a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc.
  • the terminal is not limited to a PC, a mobile phone, a tablet computer, etc.
  • the server may be a server corresponding to a media file operator, and the network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.
  • an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • FIG8 is a flow chart of an image processing method according to Embodiment 4 of the present application. As shown in FIG8 , the method may include the following steps:
  • Step S802 acquiring multiple images collected at different times by calling the first interface.
  • the first interface includes a first parameter, a parameter value of the first parameter is a multi-phase image, and the display content of the multi-phase image at least includes a monitoring area of a target part of the object to be monitored.
  • the above-mentioned monitoring object may refer to an animal whose tissue or organ may have lesions in the body, such as humans, cats, hamsters and other animals.
  • the above-mentioned monitoring area of the target part may refer to the area corresponding to the tissue or organ that may have lesions, such as a tumor area.
  • the above-mentioned multi-phase image may refer to the image acquisition of the monitoring area at different times. For example, the monitoring area at different times can be scanned and imaged by tomography or magnetic resonance imaging to acquire multi-phase images of the monitoring area at different times.
  • the image processing system may acquire the above-mentioned multiple-phase images including the monitoring area collected at different times by calling the first interface.
  • Step S804 extracting texture features of the multi-period images to obtain target texture features of the multi-period images.
  • Step S806 determining the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images.
  • the distance between the monitoring area and other areas in multiple phase images can be further combined to determine whether there is a risk in the monitoring area, thereby improving the accuracy of the determination result.
  • the risk index is used to characterize the probability of risk existing in the monitoring area.
  • the distance between the monitoring region and other regions in the multi-phase images can be further combined to determine whether the monitoring region has risks, thereby improving the accuracy of the determination result.
  • the determined risk index can be output by calling the second interface, so that the staff can more intuitively monitor the risk index corresponding to the area.
  • FIG9 is a structural block diagram of an image processing device according to embodiment 5 of the present application.
  • the device 900 includes: an acquisition module 902, an extraction module 904, a determination module 906 and a generation module 908.
  • the acquisition module 902 is used to acquire multi-phase images collected at different times, wherein the display content of the multi-phase images at least includes the monitoring area of the target part of the object to be monitored; the extraction module 904 is used to extract texture features of the multi-phase images to obtain target texture features of the multi-phase images; the determination module 906 is used to determine the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images, wherein the other areas are used to characterize the areas of the target part other than the monitoring area; the generation module 908 is used to generate a risk indicator of the monitoring area based on the target texture features and the target nerve distance, wherein the risk indicator is used to characterize the probability of the existence of risk in the monitoring area.
  • the fusion unit is also used to: use the convolutional neural network module in the texture perception model to downsample the initial texture features to obtain the local texture features of the image; use the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image.
  • the fusion unit is further used to: use the first convolution layer in the convolutional neural network module to capture the first local information in the initial texture feature; use the second convolution layer in the convolutional neural network module to map the first local information to a preset space to obtain a first mapped texture feature; use the third convolution layer in the convolutional neural network module to map the first local information to a preset space to obtain a first mapped texture feature.
  • the mapped texture features are restored to the original space corresponding to the initial texture features to obtain the restored texture features; the restored texture features are normalized using the normalization layer in the convolutional neural network module to obtain the local texture features.
  • the fusion unit is also used to: utilize the first convolution layer in the self-attention module to capture the second local information in the local texture feature; utilize the second convolution layer in the self-attention module to map the second local information to a preset space to obtain a second mapped texture feature; utilize the self-attention layer in the self-attention module to perform self-attention processing on the second mapped texture feature to obtain a self-attention texture feature; utilize the feedforward layer in the self-attention module to perform feature interaction on the self-attention texture feature to obtain an interactive texture feature; utilize the removal layer in the self-attention module to perform feature fusion on the interactive texture feature to obtain a target texture feature.
  • the fusion unit is also used to: use at least one texture perception model to fuse the global information and local information in the initial texture features to obtain the output texture features of the image; perform cross-attention processing on the output texture features of multiple period images to obtain cross-attention features; and splice the output texture features and cross-attention features of multiple period images to obtain target texture features.
  • the fusion unit is also used to: use a cross-attention module to perform cross-attention processing on the output texture features of images in different periods to obtain cross-attention features.
  • the determination module 906 includes: a segmentation unit, which is used to perform image segmentation on the image to obtain a segmentation result of the image, wherein the segmentation result includes at least: a monitoring area and other areas; a first determination unit, which is used to determine a first point set located at the boundary of the monitoring area and a second point set located at the boundary of other areas based on the segmentation result; a second determination unit, which is used to determine a first subset of the first point set that meets a preset condition and a second subset of the second point set that meets the preset condition; a distance determination unit, which is used to perform cross-attention processing on the first subset and the second subset to obtain a target neural distance.
  • a segmentation unit which is used to perform image segmentation on the image to obtain a segmentation result of the image, wherein the segmentation result includes at least: a monitoring area and other areas
  • a first determination unit which is used to determine a first point set located at the boundary of the monitoring area and a second point set
  • the second determination unit is further used to: sample the first point set to obtain a first sampling point set, and sample the second point set to obtain a second sampling point set; determine a preset number of first sampling points in the first sampling point set that are closest to the second sampling point set to obtain a first subset; determine a preset number of second sampling points in the second sampling point set that are closest to the first sampling point set to obtain a second subset.
  • the generation module 908 includes: a structural relationship extraction unit, which is used to extract the structural relationship between the monitoring area and other areas in the multi-phase image to obtain the structural features of the multi-phase image; a feature splicing unit, which is used to splice the target texture features, target neural distance and structural features to obtain splicing features; an indication generation unit, which is used to generate risk indicators based on the splicing features.
  • the acquisition module 902, extraction module 904, determination module 906 and generation module 908 described above correspond to steps S302 to S308 in Example 1, and the four modules and corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in Example 1.
  • the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above modules may also be run in the AR/VR device provided in Example 1 as part of the device.
  • FIG10 is a structural block diagram of an image processing device according to Embodiment 6 of the present application.
  • the device 1000 includes: a first display module 1002 and a second display module 1004.
  • the first display module 1002 is used to respond to input instructions acting on the operation interface, and display multiple images collected at different times on the operation interface, wherein the display content of the multiple images at least includes the monitoring area of the target part of the object to be monitored.
  • the second display module 1004 is used to respond to image processing instructions acting on the operation interface, and display the risk index of the monitoring area on the operation interface, wherein the risk index is generated based on the target texture features and target nerve distance of the multiple images, the target texture features are obtained by extracting texture features from the multiple images, and the target nerve distance is determined based on the positional relationship between the monitoring area and other areas in the multiple images.
  • first display module 1002 and the second display module 1004 described above correspond to steps S602 to S604 in Example 2, and the two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in Example 1.
  • the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above modules may also be run in the AR/VR device provided in Example 1 as part of the device.
  • FIG11 is a structural block diagram of an image processing device according to Embodiment 7 of the present application.
  • the device 1100 includes: a first display module 1102, an extraction module 1104, a determination module 1106, a generation module 1108, and a second display module 1110.
  • the first display module 1102 is used to display multiple images collected at different times on the presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the display content of the multiple images at least includes the monitoring area of the target part of the object to be monitored, the extraction module 1104 is used to extract texture features of the multiple images to obtain target texture features of the multiple images, the determination module 1106 is used to determine the target nerve distance of the multiple images based on the positional relationship between the monitoring area and other areas in the multiple images, wherein the other areas are used to characterize the areas of the target part other than the monitoring area, the generation module 1108 is used to generate a risk indicator of the monitoring area based on the target texture features and the target nerve distance, wherein the risk indicator is used to characterize the probability of the existence of risk in the monitoring area, and the second display module 1110 is used to drive the VR device or AR device to display the risk indicator.
  • the first display module 1102, the extraction module 1104, the determination module 1106, the generation module 1108 and the second display module 1110 described above correspond to steps S702 to S710 in Example 3.
  • the five modules and the corresponding steps have the same examples and application scenarios, but are not limited to the contents disclosed in Example 1.
  • the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors.
  • the above modules may also be part of a device and may be run on the AR/VR device provided in Example 1. middle.
  • FIG12 is a structural block diagram of an image processing device according to Embodiment 8 of the present application.
  • the device 1200 includes: an acquisition module 1202, an extraction module 1204, a determination module 1206, a generation module 1208, and an output module 1210.
  • the acquisition module 1202 is used to acquire multi-phase images collected at different times by calling the first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is the multi-phase image, and the display content of the multi-phase image at least includes the monitoring area of the target part of the monitored object.
  • the extraction module 1204 is used to extract texture features of the multi-phase images to obtain target texture features of the multi-phase images.
  • the determination module 1206 is used to determine the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images, wherein the other area is used to characterize the area of the target part other than the monitoring area.
  • the generation module 1208 is used to generate a risk indicator of the monitoring area based on the target texture feature and the target nerve distance, wherein the risk indicator is used to characterize the probability of risk in the monitoring area.
  • the output module 1210 is used to output the risk indicator by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the risk indicator.
  • the acquisition module 1202, extraction module 1204, determination module 1206, generation module 1208 and output module 1210 described above correspond to steps S802 to S810 in Example 4, and the five modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1.
  • the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of a device and run in the AR/VR device provided in Example 1.
  • a computer-assisted cancer prognosis method comprising:
  • a risk index of the monitoring area is generated, wherein the risk index is used to characterize the probability that a cancer area has a risk.
  • a computer-assisted pancreatic cancer prognosis method comprising:
  • a risk index of the monitoring area is generated, wherein the risk index is used to characterize the probability of the existence of risk in the pancreatic cancer area.
  • a computer-assisted cancer prognosis system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to perform a computer-assisted cancer prognosis method, the method comprising:
  • a risk index of the monitoring area is generated, wherein the risk index is used to characterize the probability that a cancer area has a risk.
  • the embodiment of the present application may provide an electronic device, which may be any electronic device in a group of electronic devices.
  • the electronic device may also be replaced by a terminal device such as a mobile terminal.
  • the electronic device may be located in at least one grid device among a plurality of grid devices of a computer grid.
  • the above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: obtaining multi-phase images collected at different times, wherein the display content of the multi-phase images at least includes the monitoring area of the target part of the object to be monitored; performing texture feature extraction on the multi-phase images to obtain the target texture features of the multi-phase images; determining the target nerve distance of the multi-phase images based on the positional relationship between the monitoring area and other areas in the multi-phase images, wherein the other areas are used to characterize the areas of the target part other than the monitoring area; generating a risk index for the monitoring area based on the target texture features and the target nerve distance, wherein the risk index is used to characterize the probability of the existence of risk in the monitoring area.
  • Figure 13 is a structural block diagram of an electronic device according to Embodiment 12 of the present application.
  • the electronic device A may include: a processor 1302 and a memory 1304, wherein an executable program is stored; the processor is used to run the program, wherein the image processing method shown in Embodiment 1 is executed when the program is run.
  • the electronic device A may further include: a storage controller and a peripheral interface, wherein the peripheral interface is connected to the radio frequency module, the audio module and the display.
  • the memory can be used to store software programs and modules, such as program instructions/modules corresponding to the image processing method and device in the embodiment of the present application.
  • the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image segmentation method.
  • the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a grid. Examples of the above-mentioned grid include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
  • the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: extract features from the image to obtain initial texture features of the image; use at least one texture perception model to fuse the global information and local information in the initial texture features to obtain target texture features.
  • the processor can call the information and application stored in the memory through the transmission device to perform the following steps: using at least one texture perception model block to fuse the global information and local information in the initial texture features to obtain the target texture features, including: using the convolutional neural network module in the texture perception model to downsample the initial texture features to obtain the local texture features of the image; using the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image.
  • the processor can call the information and application stored in the memory through the transmission device to perform the following steps: using the convolutional neural network module in the texture perception model to downsample the initial texture features and the local texture features of the image, including: using the first convolutional layer in the convolutional neural network module to capture the first local information in the initial texture features; using the second convolutional layer in the convolutional neural network module to map the first local information to a preset space to obtain a first mapped texture feature; using the third convolutional layer in the convolutional neural network module to restore the mapped texture features to the original space corresponding to the initial texture features to obtain restored texture features; using the normalization layer in the convolutional neural network module to normalize the restored texture features to obtain local texture features.
  • the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: using the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image, including: using the first convolution layer in the self-attention module to capture the second local information in the local texture features; using the second convolution layer in the self-attention module to map the second local information to a preset space to obtain a second mapped texture feature; using the self-attention layer in the self-attention module to perform self-attention processing on the second mapped texture features to obtain a self-attention texture feature; using the feedforward layer in the self-attention module to perform feature interaction on the self-attention texture features to obtain an interactive texture feature; using the removal layer in the self-attention module to perform feature fusion on the interactive texture features to obtain a target texture feature.
  • the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain the target texture features, including: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain the output texture features of the image; performing cross-attention processing on the output texture features of multiple-period images to obtain cross-attention features; splicing the output texture features and cross-attention features of multiple-period images to obtain the target texture features.
  • the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: perform cross-attention processing on the output texture features of multiple period images to obtain cross-attention features, including: use the cross-attention module to perform cross-attention processing on the output texture features of images from different periods to obtain cross-attention features.
  • the processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: based on the positional relationship between the monitoring area and other areas in the multi-phase images, determine the target nerve distance of the multi-phase images, including: performing image segmentation on the image to obtain the image segmentation result, wherein the segmentation result at least includes: the monitoring area and other areas; based on the segmentation result, determine a first point set located at the boundary of the monitoring area, and a second point set located at the boundary of other areas; determine a first subset of the first point set that meets the preset conditions, and a second subset of the second point set that meets the preset conditions; perform cross-attention processing on the first subset and the second subset to obtain the target nerve distance.
  • the processor may call the information and application program stored in the memory through the transmission device to perform the following steps: determining a first subset of the first point set that meets a preset condition, and a second subset of the second point set that meets a preset condition, including: sampling the first point set to obtain a first sampling point set, and sampling the second point set to obtain a second sampling point set; determining a preset number of first sampling points in the first sampling point set that are closest to the second sampling point set to obtain the first subset; and determining a preset number of second sampling points in the second sampling point set that are closest to the first sampling point set to obtain the second subset.
  • the processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: based on the target texture features and the target nerve distance, generate a risk index for the monitoring area, including: extracting the structural relationship between the monitoring area and other areas in the multi-phase images to obtain the structural features of the multi-phase images; splicing the target texture features, the target nerve distance and the structural features to obtain the spliced features; and generating the risk index based on the spliced features.
  • a method in which a multi-phase image acquired at different times is acquired; texture features are extracted from the multi-phase image to obtain target texture features of the multi-phase image; based on the positional relationship between the monitoring area and other areas in the multi-phase image, the target nerve distance of the multi-phase image is determined; based on the target texture features and the target nerve distance, a risk index of the monitoring area is generated.
  • the risk index of the monitoring area is generated according to the target texture features of the monitoring area in the multi-phase image acquired at different times, and the target nerve distance between the monitoring area and other areas.
  • the contact situation between the monitoring area and other areas is fully considered in the risk prediction process of the monitoring area, thereby improving the accuracy of the determined risk index, thereby solving the technical problem of low accuracy in determining whether there is a risk in the monitoring area in the related art.
  • the structure shown in FIG. 13 is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, or a mobile Internet device (Mobile FIG13 does not limit the structure of the above electronic devices.
  • the computer terminal A may also include more or fewer components (such as a grid interface, a display device, etc.) than those shown in FIG13, or have a different configuration from that shown in FIG13.
  • a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
  • the embodiment of the present application further provides a computer-readable storage medium.
  • the storage medium can be used to store the program code executed by the image segmentation method provided in the above embodiment 1.
  • the storage medium may be located in any computer terminal in a computer terminal group in a computer grid, or in any mobile terminal in a mobile terminal group.
  • the storage medium is configured to store program codes for executing the following steps: acquiring multi-phase images acquired at different times, wherein the display content of the multi-phase images at least includes a monitoring area of a target part of the object to be monitored; performing texture feature extraction on the multi-phase images to obtain target texture features of the multi-phase images; determining a target nerve distance of the multi-phase images based on a positional relationship between the monitoring area and other areas in the multi-phase images, wherein the other areas are used to characterize areas of the target part other than the monitoring area; generating a risk indicator for the monitoring area based on the target texture features and the target nerve distance, wherein the risk indicator is used to characterize the probability that there is a risk in the monitoring area.
  • the storage medium is configured to store program codes for executing the following steps: extracting features from an image to obtain initial texture features of the image; and fusing global information and local information in the initial texture features using at least one texture perception model to obtain target texture features.
  • the storage medium is configured to store program codes for executing the following steps: using at least one texture perception model block to fuse the global information and local information in the initial texture features to obtain target texture features, including: using the convolutional neural network module in the texture perception model to downsample the initial texture features to obtain local texture features of the image; using the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image.
  • a storage medium is configured to store a program code for executing the following steps: using a convolutional neural network module in a texture perception model to downsample the initial texture features and obtain local texture features of an image, including: using a first convolutional layer in the convolutional neural network module to capture first local information in the initial texture features; using a second convolutional layer in the convolutional neural network module to map the first local information to a preset space to obtain a first mapped texture feature; using a third convolutional layer in the convolutional neural network module to restore the mapped texture features to the original space corresponding to the initial texture features to obtain a restored texture feature; using a normalization layer in the convolutional neural network module to normalize the restored texture features to obtain local texture features.
  • the storage medium is configured to store a program code for executing the following steps: using the self-attention module in the texture perception model to perform attention processing on the local texture features to obtain the target texture features of the image,
  • the method includes: using the first convolution layer in the self-attention module to capture the second local information in the local texture feature; using the second convolution layer in the self-attention module to map the second local information to a preset space to obtain a second mapped texture feature; using the self-attention layer in the self-attention module to perform self-attention processing on the second mapped texture feature to obtain a self-attention texture feature; using the feedforward layer in the self-attention module to perform feature interaction on the self-attention texture feature to obtain an interactive texture feature; using the removal layer in the self-attention module to perform feature fusion on the interactive texture feature to obtain a target texture feature.
  • the storage medium is configured to store program codes for executing the following steps: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain the target texture features, including: using at least one texture perception model to fuse the global information and local information in the initial texture features to obtain the output texture features of the image; performing cross-attention processing on the output texture features of multiple period images to obtain cross-attention features; splicing the output texture features and cross-attention features of multiple period images to obtain the target texture features.
  • the storage medium is configured to store program codes for executing the following steps: performing cross-attention processing on output texture features of images of multiple periods to obtain cross-attention features, including: using a cross-attention module to perform cross-attention processing on output texture features of images of different periods to obtain cross-attention features.
  • the storage medium is configured to store program codes for executing the following steps: determining the target nerve distance of multiple images based on the positional relationship between the monitoring area and other areas in the multiple images, including: performing image segmentation on the image to obtain image segmentation results, wherein the segmentation results at least include: the monitoring area and other areas; based on the segmentation results, determining a first point set located at the boundary of the monitoring area, and a second point set located at the boundary of other areas; determining a first subset of the first point set that meets a preset condition, and a second subset of the second point set that meets the preset condition; performing cross-attention processing on the first subset and the second subset to obtain the target nerve distance.
  • a storage medium is configured to store a program code for executing the following steps: determining a first subset of a first point set that satisfies a preset condition, and a second subset of a second point set that satisfies a preset condition, including: sampling the first point set to obtain a first sampling point set, and sampling the second point set to obtain a second sampling point set; determining a preset number of first sampling points in the first sampling point set that are closest to the second sampling point set to obtain the first subset; determining a preset number of second sampling points in the second sampling point set that are closest to the first sampling point set to obtain the second subset.
  • the storage medium is configured to store program codes for executing the following steps: generating a risk indicator for the monitoring area based on target texture features and target nerve distances, including: extracting the structural relationship between the monitoring area and other areas in multiple phase images to obtain structural features of the multiple phase images; splicing the target texture features, target nerve distances and structural features to obtain spliced features; and generating a risk indicator based on the spliced features.
  • a method is adopted in which a multi-phase image acquired at different times is adopted; texture features are extracted from the multi-phase image to obtain target texture features of the multi-phase image; a target nerve distance of the multi-phase image is determined based on a positional relationship between the monitoring area and other areas in the multi-phase image; and a risk index of the monitoring area is generated based on the target texture features and the target nerve distance.
  • the risk index of the monitoring area is generated according to the target texture features of the monitoring area in the multi-phase image acquired at different times and the target nerve distance between the monitoring area and other areas.
  • the contact between the monitoring area and other areas is fully considered, thereby improving the accuracy of the determined risk indicators, and further solving the technical problem of low accuracy in determining whether there is a risk in the monitoring area in related technologies.
  • An embodiment of the present application further provides a computer program product, including a computer program, which, when executed in a computer, enables the computer to execute the method provided in the embodiment of the present application.
  • the embodiments of the present application also provide a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the method provided by the embodiments of the present application.
  • the disclosed technical content can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Image Processing (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

本申请公开了一种图像处理方法、电子设备和计算机可读存储介质。其中，该方法包括：获取不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率。本申请解决了相关技术中确定监测区域是否存在风险的准确度低的技术问题。

Description

图像处理方法、电子设备和计算机可读存储介质
本申请要求于2023年07月20日提交中国专利局、申请号为202310896158.0、申请名称为“图像处理方法、电子设备和计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理领域，具体而言，涉及一种图像处理方法、电子设备和计算机可读存储介质。
背景技术
目前，通常可以利用深度学习的预测模型，对待监测对象的目标部位的监测区域进行风险预测，但是，由于监测区域和其他区域距离较近，利用这种方法对监测区域进行风险预测的准确度低，导致无法准确的确定出监测区域是否存在风险。
针对上述的问题，目前尚未提出有效的解决方案。
发明内容
本申请实施例提供了一种图像处理方法、电子设备和计算机可读存储介质，以至少解决相关技术中确定监测区域是否存在风险的准确度低的技术问题。
根据本申请实施例的一个方面，提供了一种图像处理方法，包括：获取不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率。
根据本申请实施例的另一方面，还提供了一种图像处理方法，包括：响应作用于操作界面上的输入指令，在操作界面上显示不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；响应作用于操作界面上的图像处理指令，在操作界面上显示监测区域的风险指标，其中，风险指标是基于多期图像的目标纹理特征和目标神经距离生成的，目标纹理特征是对多期图像进行纹理特征提取得到的，目标神经距离是基于多期图像中的监测区域和其他区域的位置关系确定的。
根据本申请实施例的另一方面，还提供了一种图像处理方法，包括：在虚拟现实VR设备或增强现实AR设备的呈现画面上展示不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基 于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率；驱动VR设备或AR设备展示风险指标。
根据本申请实施例的另一方面，还提供了一种图像处理方法，包括：通过调用第一接口获取不同时间采集到的多期图像，其中，第一接口包括第一参数，第一参数的参数值为多期图像，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率；通过调用第二接口输出风险指标，其中，第二接口包括第二参数，第二参数的参数值为风险指标。
根据本申请实施例的另一方面，还提供了一种计算机辅助癌症预后方法，包括：
获取不同时间采集到的多期医学图像，其中，多期所述医学图像的显示内容至少包含待监测对象的目标部位的癌症区域；
对多期所述医学图像进行纹理特征提取，得到多期所述医学图像的目标纹理特征；
基于多期所述医学图像中的所述癌症区域和其他区域的位置关系，确定多期所述医学图像的目标神经距离，其中，所述其他区域用于表征所述目标部位除所述癌症区域之外的区域；
基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述癌症区域存在风险的概率。
根据本申请实施例的另一方面，还提供了一种计算机辅助胰腺癌预后方法，包括：
获取不同时间采集到的多期医学图像，其中，多期所述医学图像的显示内容至少包含待监测对象的胰腺癌区域；
对多期所述医学图像进行纹理特征提取，得到多期所述医学图像的目标纹理特征；
基于多期所述医学图像中的所述胰腺癌区域和其他区域的位置关系，确定多期所述医学图像的目标神经距离，其中，所述其他区域用于表征所述胰腺癌区域之外的区域；
基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述胰腺癌区域存在风险的概率。
根据本申请实施例的另一方面，还提供了一种计算机辅助癌症预后系统，包括存储器、处理器以及存储在所述存储器上并在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序可用于执行一种计算机辅助癌症预后方法，所述方法包括：
获取不同时间采集到的多期医学图像，其中，多期所述医学图像的显示内容至少包含待监测对象的目标部位的癌症区域；
对多期所述医学图像进行纹理特征提取，得到多期所述医学图像的目标纹理特征；
基于多期所述医学图像中的所述癌症区域和其他区域的位置关系，确定多期所述医学图像的目标神经距离，其中，所述其他区域用于表征所述目标部位除所述癌症区域之外的 区域；
基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述癌症区域存在风险的概率。
根据本申请实施例的另一方面，还提供了一种电子设备，包括：存储器，存储有可执行程序；处理器，用于运行程序，其中，程序运行时执行上述任意一项的方法。
根据本申请实施例的另一方面，还提供了一种计算机可读存储介质，计算机可读存储介质包括存储的可执行程序，其中，在可执行程序运行时控制计算机可读存储介质所在设备执行权利要求上述任意一项的方法。
根据本申请实施例的另一方面，还提供了一种计算机程序产品，包括计算机可执行指令，该计算机可执行指令被处理器执行时实现上述任意一项方法的步骤。
根据本申请实施例的另一方面，还提供了一种计算机程序，该计算机程序被处理器执行时实现上述任意一项方法的步骤。
在本申请实施例中，采用获取不同时间采集到的多期图像；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离；基于目标纹理特征和目标神经距离，生成监测区域的风险指标的方式，通过根据在不同时间采集到的多期图像中监测区域的目标纹理特征，以及监测区域与其他区域之间的目标神经距离，来生成监测区域的风险指标，使得监测区域的风险预测过程中，充分考虑了监测区域与其他区域之间的接触情况，从而提高了确定出的风险指标的准确度，进而解决了相关技术中确定监测区域是否存在风险的准确度低的技术问题。
容易注意到的是，上面的通用描述和后面的详细描述仅仅是为了对本申请进行举例和解释，并不构成对本申请的限定。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1是根据本申请实施例的一种图像处理方法的虚拟现实设备的硬件环境的示意图；
图2是根据本申请实施例的一种图像处理方法的计算环境的结构框图；
图3是根据本申请实施例1的一种图像处理方法的流程图；
图4是根据本申请实施例1的一种图像处理过程的示意图；
图5是根据本申请实施例1的一种图像处理结果的示意图；
图6是根据本申请实施例2的一种图像处理方法的流程图；
图7是根据本申请实施例3的图像处理方法的流程图；
图8是根据本申请实施例4的图像处理方法的流程图；
图9是根据本申请实施例5的一种图像处理装置的结构框图；
图10是根据本申请实施例6的一种图像处理装置的结构框图；
图11是根据本申请实施例7的一种图像处理装置的结构框图；
图12是根据本申请实施例8的一种图像处理装置的结构框图；
图13是根据本申请实施例12的一种电子设备的结构框图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
首先，在对本申请实施例进行描述的过程中出现的部分名词或术语适用于如下解释：
Multi-phase CT：Multi-phase Computed Tomography，多期电脑断层扫描，用于获取包含监测区域的多期图像。
PDAC：Pancreatic ductal adenocarcinoma，胰腺导管腺癌，简称胰腺癌。
CNN：Convolutional Neural Network，卷积神经网络。
神经距离：是指监测区域边界上的点，与监测区域附近的其他区域边界上的点之间的距离，可以用来评估监测区域与其他区域之间的关系。
纹理感知模型：可以用来将多期图像中的局部信息和全局信息相结合，改进多期图像中的纹理特征的提取。
癌症预后：对癌症患者未来病情发展的预测。预后可以包括对患者生存时间、疾病复发的可能性、治疗效果以及生活质量等方面的预测。
实施例1
根据本申请实施例，提供了一种图像处理方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图1是根据本申请实施例的一种图像处理方法的虚拟现实设备的硬件环境的示意图。如图1所示，虚拟现实设备104与终端106相连接，终端106与服务器102通过网络进行连接，上述虚拟现实设备104并不限定于：虚拟现实头盔、虚拟现实眼镜、虚拟现实一体机等，上述终端104并不限定于PC、手机、平板电脑等，服务器102可以为媒体文件运营商对应的服务器，上述网络包括但不限于：广域网、城域网或局域网。
可选地，该实施例的虚拟现实设备104包括：存储器、处理器和传输装置。存储器用于存储应用程序，该应用程序可以用于执行：获取不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率，从而解决了相关技术中确定监测区域是否存在风险的准确度低的技术问题，达到了提高对监测区域是否存在风险进行监测时的准确度的目的。
该实施例的终端可以用于执行在虚拟现实(Virtual Reality，简称为VR)设备或增强现实(Augmented Reality，简称为AR)设备的呈现画面上展示在不同时间采集到的包含监测区域的多期图像、多期图像的目标纹理特征、多期图像中监测区域和其他区域的位置关系、监测区域的风险指标等信息，并向虚拟现实设备104发送显示指令，虚拟现实设备104在接收到显示指令之后在目标投放位置显示出来。
可选地，该实施例的虚拟现实设备104带有的眼球追踪的HMD(Head Mount Display，头戴式显示器)头显与眼球追踪模块与上述实施例中的作用相同，也即，HMD头显中的屏幕，用于显示实时的画面，HMD中的眼球追踪模块，用于获取用户眼球的实时运动轨迹。该实施例的终端通过跟踪系统获取用户在真实三维空间的位置信息与运动信息，并计算出用户头部在虚拟三维空间中的三维坐标，以及用户在虚拟三维空间中的视野朝向。
图1示出的硬件结构框图，不仅可以作为上述AR/VR设备(或移动设备)的示例性框图，还可以作为上述服务器的示例性框图，一种可选实施例中，图2以框图示出了使用上述图1所示的AR/VR设备(或移动设备)作为计算环境201中计算节点的一种实施例。图2是根据本申请实施例的一种图像处理方法的计算环境的结构框图，如图2所示，计算环境201包括运行在分布式网络上的多个(图中采用210-1，210-2，…,来示出)计算节点(如服务器)。不同计算节点都包含本地处理和内存资源，终端用户202可以在计算环境201中远程运行应用程序或存储数据。应用程序可以作为计算环境201中的多个服务220-1，220-2，220-3和220-4进行提供，分别代表服务“A”，“D”，“E”和“H”。
终端用户202可以通过客户端上的web浏览器或其他软件应用程序提供和访问服务，在一些实施例中，可以将终端用户202的供应和/或请求提供给入口网关230。入口网关230可以包括一个相应的代理来处理针对服务(计算环境201中提供的一个或多个服务)的供应和/或请求。
服务是根据计算环境201支持的各种虚拟化技术来提供或部署的。在一些实施例中，可以根据基于虚拟机(Virtual Machine，VM)的虚拟化、基于容器的虚拟化和/或类似的方式提供服务。基于虚拟机的虚拟化可以是通过初始化虚拟机来模拟真实的计算机，在不直接接触任何实际硬件资源的情况下执行程序和应用程序。在虚拟机虚拟化机器的同时，根据基于容器的虚拟化，可以启动容器来虚拟化整个操作系统(Operating System，OS)， 以便多个工作负载可以在单个操作系统实例上运行。
在基于容器虚拟化的一个实施例中，服务的若干容器可以被组装成一个Pod(例如，Kubernetes Pod)。举例来说，如图2所示，服务220-2可以配备一个或多个Pod 240-1，240-2，…，240-N(统称为Pod)。Pod可以包括代理245和一个或多个容器242-1，242-2，…，242-M(统称为容器)。Pod中一个或多个容器处理与服务的一个或多个相应功能相关的请求，代理245通常控制与服务相关的网络功能，如路由、负载均衡等。其他服务也可以配备类似的Pod。
在操作过程中，执行来自终端用户202的用户请求可能需要调用计算环境201中的一个或多个服务，执行一个服务的一个或多个功能需要调用另一个服务的一个或多个功能。如图2所示，服务“A”220-1从入口网关230接收终端用户202的用户请求，服务“A”220-1可以调用服务“D”220-2，服务“D”220-2可以请求服务“E”220-3执行一个或多个功能。
上述的计算环境可以是云计算环境，资源的分配由云服务提供上管理，允许功能的开发无需考虑实现、调整或扩展服务器。该计算环境允许开发人员在不构建或维护复杂基础设施的情况下执行响应事件的代码。服务可以被分割完成一组可以自动独立伸缩的功能，而不是扩展单个硬件设备来处理潜在的负载。
在上述运行环境下，本申请提供了如图3所示的图像处理方法。需要说明的是，该实施例的图像处理方法可以由图1所示实施例的移动终端执行。图3是根据本申请实施例1的一种图像处理方法的流程图。如图3所示，该方法可以包括如下步骤：
步骤S302，获取不同时间采集到的多期图像。
其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域。
上述监测对象可以是指体内的组织或者器官可能会出现病变的动物，例如人、猫、地鼠等动物。上述目标部位的监测区域可以是指该组织或器官上可能会发生病变的区域，例如胰腺上的肿瘤区域。上述多期图像可以是指对不同时间的监测区域进行图像采集，例如可以通过断层扫描或者磁共振成像的方式，对不同时间的监测区域进行扫描成像，以采集得到监测区域在不同时间的多期图像。
考虑到目标部位在发生病变时可能会经历较长的时间，而通过短时间内获取到的关于监测区域的单张或多张图像，可能无法准确的确定出该监测区域是否发生了病变，以及对应的病变情况，例如是否出现肿瘤，肿瘤病变趋势等情况，因此，在本实施例的一种可选方案中，为了能够准确的确定出目标部位的监测区域是否发生了病变，是否存在风险，例如出现肿瘤严重程度恶化、肿瘤区域扩散等情况，图像处理系统可以首先获取在不同时间采集到的，包含监测区域的上述多期图像，从而避免因单期图像所体现出的监测区域病变情况的局限性，导致出现确定出监测区域是否存在风险的准确度差的情况。
在本实施例的一种可选方案中，上述多期图像至少包括：非对比阶段的图像、早期发展阶段和当前阶段。以监测区域为胰腺癌区域为例，上述多期图像至少包括：非对比阶段 的图像，胰腺阶段的图像和门静脉阶段的图像。
步骤S304，对多期图像进行纹理特征提取，得到多期图像的目标纹理特征。
上述目标纹理特征可以是对监测区域所呈现出的全局纹理特征和局部纹理特征进行融合，得到的纹理特征，其中，局部纹理特征可以是指监测区域的纹理特征，可以包括但不限于：监测区域的颜色、纹理、形状、边缘、角点等特征，全局纹理特征可以是指多期图像中整个图像的纹理特征，例如目标部位的纹理特征，目标部位外的区域的纹理特征等。
举例来说，若监测区域是PDAC区域，则对应的其他区域可以是胰腺周围的血管所处的区域，对应的局部纹理特征可以是指PDAC区域的颜色、纹理、形状、边缘、角点等特征，对应的全局纹理特征可以是指胰腺以及胰腺周围血管的颜色、纹理、形状、边缘、角点等特征。
在本实施例的一种可选方案中，考虑到根据监测区域所呈现出的图像效果，例如图像纹理，来判断监测区域是否存在风险的主要途径之一，因此，为了提高确定监测区域是否存在风险的准确度，图像处理系统可以在获取到不同时间的多期图像之后，可以对该多期图像进行纹理特征提取，确定出上述多期图像的目标纹理特征。
在本实施例的一种可选方案中，在对多期图像进行纹理特征提取，得到目标纹理特征时，考虑到在除了监测区域的目标部位和目标部位外的区域中，可能会存在较多的重要组织器官，例如在除PDAC区域的胰腺上和胰腺周围会存在较多重要的血管。这些重要组织器官可能会对监测区域的监测产生一定的影响，例如若血管的颜色与PDAC区域的颜色相接近，则可能会出现误判PDAC扩散的情况，同时考虑到监测区域在病变时也可能会对这些重要组织器官产生影响，例如PDAC区域可能会侵犯胰腺周围的血管，因此，为了保证确定监测区域是否存在风险的准确度，在对多期图像进行纹理特征提取时，除了仅根据监测区域所呈现出的纹理特征来对监测区域进行监测外，即提取监测区域的局部纹理特征，图像处理系统还可以对其他区域所呈现出的纹理来对监测区域进行监测，即提取全局纹理特征，然后再对局部纹理特征和全局纹理特征进行融合处理，以得到上述的目标纹理特征，从而保证根据目标纹理特征确定监测区域是否存在风险的准确度。
步骤S306，基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离。
其中，其他区域用于表征目标部位除监测区域之外的区域。
上述目标神经距离可以是指监测区域的表面与其他区域的表面之间的距离，例如若监测区域位PDAC区域时，则上述目标神经距离可以是指监测区域的边缘与胰腺周围血管之间的距离。
在本实施例的一种可选方案中，考虑到其他区域可能会对监测区域的预后预测的准确度产生影响，例如在胰腺周围会存在较多重要的血管，这些血管与PDAC区域较近，在切除PDAC区域时，可能会出现误切除，进而出现对患者产生负面影响的情况，因此，为了能够准确的对监测区域进行预后预测，除了直接利用前述目标纹理特征来确定监测区域是 否存在风险外，还可以进一步的结合多期图像中监测区域和其他区域之间的距离，即上述目标神经距离，来确定监测区域是否存在风险，从而提高确定结果的准确度。
在本实施例的一种可选方案中，可以通过根据多期图像中监测区域和其他区域之间的位置关系，例如前述的监测区域和其他区域之间的相对位置、结构连接等信息，来确定不同图像对应的目标神经距离。
步骤S308，基于目标纹理特征和目标神经距离，生成监测区域的风险指标。
其中，风险指标用于表征监测区域存在风险的概率。
上述风险指标可以是指监测区域是否存在风险，以及存在风险的概率，一般的，可以将上述风险指标的范围确定为0-1，风险指标越大，代表监测区域越可能出现风险，监测区域发生的病变对待监测对象的负面影响越大。
在本实施例的一种可选方案中，可以根据上述确定出的目标纹理特征，以及目标神经距离，来确定上述监测区域的风险指标，例如若目标纹理特征表示监测区域的病变情况越严重，目标神经距离越短，则对应的风险指标越高。通过利用上述多期图像中监测区域对应的目标纹理特征和目标神经距离，图像处理系统可以快速的确定出监测区域的风险指标，无需再由工作人员对多期图像中监测区域所呈现出的纹理进行反复比对，同时避免了因监测区域在不同时间所呈现出的纹理特征的变化较小，而使得工作人员无法准确观察出监测区域的变化情况，进而减小了出现误判的情况，减小了工作人员的工作压力。
在本申请实施例中，采用获取不同时间采集到的多期图像；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离；基于目标纹理特征和目标神经距离，生成监测区域的风险指标的方式，通过根据在不同时间采集到的多期图像中监测区域的目标纹理特征，以及监测区域与其他区域之间的目标神经距离，来生成监测区域的风险指标，使得监测区域的风险预测过程中，充分考虑了监测区域与其他区域之间的接触情况，从而提高了确定出的风险指标的准确度，进而解决了相关技术中确定监测区域是否存在风险的准确度低的技术问题。
在本申请实施例中，对多期图像进行纹理特征提取，得到多期图像的目标纹理特征，包括：对图像进行特征提取，得到图像的初始纹理特征；利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征。
上述初始纹理特征可以是指对图像进行特征提取得到的，监测区域和其他区域各自的纹理特征，可以是图像的全局纹理特征，上述全局信息可以是指前述提到的监测区域和其他区域之间的信息，例如胰腺和胰腺周围血管之间的纹理信息，上述局部信息可以是指前述提到的监测区域自身的信息，例如PDAC区域的纹理信息。上述纹理感知模型可以是用于对初始纹理特征进行下采样，并利用注意力机制对下采样结果进行处理的模式，纹理感知模型至少包含CNN卷积神经网络和Transformer网络。
在本实施例的一种可选方案中，上述纹理感知模型的数量至少为1个，纹理感知模型 的数量越多，得到的目标纹理特征的准确度越高，对应的确定监测区域是否存在风险的准确度越高，但是效率越低，因此，考虑到确定监测区域是否存在风险的效率，上述纹理感知模型的数量可以设置为3。
在本实施例的一种可选方案中，图像处理系统可以首先利用预设的特征提取模块，对多期图进行特征提取，确定出图像中监测区域的初始纹理特征，然后再利用至少一个上述的纹理感知模型，对该初始纹理特征进行下采样处理，得到初始纹理特征中上述的全局信息和局部信息，然后再利用注意力机制对该全局信息和局部信息进行处理，得到上述的目标纹理特征。需要说明的是，该得到目标纹理特征的过程是单个纹理感知模型所执行的过程，若在存在多个纹理感知模型，例如上述的纹理感知模型的数量设置位3，则需要将上一个纹理感知模型对应的目标纹理特征，作为下一个纹理感知模型的初始纹理特征，并多次循环执行该得到目标纹理特征的过程，从而提高最终得到的目标纹理特征的准确度。
在本实施例的一种可选方案中，用于提取初始纹理特征的特征提取模块至少包括：依次连接的5×5×5的卷积层和3×3×3的卷积层。
在本申请实施例中，利用至少一个纹理感知模型块对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征；利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征。
上述卷积神经网络模块可以是用于从初始纹理特征中捕获得到局部信息，例如多期图像中监测区域的局部纹理特征的模块，上述自注意力模块可以是捕捉不同局部纹理特征之间的依赖性，得到目标纹理特征的模块，可以实现将局部信息和全局信息进行融合的目的。
在本实施例的一种可选方案中，在利用至少一个纹理感知模块对初始纹理特征进行处理，得到目标纹理特征的过程中，图像处理系统可以首先利用纹理感知模型中的卷积神经网络模块，对前述提取到的初始纹理特征进行下采样，得到图像的局部纹理特征，然后再利用纹理感知模块中的自注意力模块，对该局部纹理特征进行自注意力处理，从而得到上述的目标纹理特征。
在本申请实施例中，利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征，包括：利用卷积神经网络模块中的第一卷积层，捕获初始纹理特征中的第一局部信息；利用卷积神经网络模块中的第二卷积层，将第一局部信息映射至预设空间，得到第一映射纹理特征；利用卷积神经网络模块中的第三卷积层，将映射纹理特征还原至初始纹理特征对应的原始空间，得到还原纹理特征；利用卷积神经网络模块中的归一化层，对还原纹理特征进行归一化处理，得到局部纹理特征。
上述预设空间的维度大于上述原始空间的维度。上述卷积神经网络模块中的第一卷积层可以是指用于从初始纹理特征中捕获监测区域和其他区域各自的第一局部信息，例如监测区域和其他区域各自的纹理信息的卷积层，可以是3×3×3的结构，第二卷积层可以是指用于将捕捉得到的第一局部信息映射至高维的预设空间中，以得到能够编码的隐藏特征 的卷积层，可以是1×1×1的结构，第三卷积层可以是指用于将编码后的隐藏特征进行还原，得到还原后的纹理特征的卷积层，归一化层可以是用于对还原后的纹理特征进行归一化处理，得到精度较高的局部纹理特征的卷积层。
在本实施例的一种可选方案中，在利用卷积神经网络对初始纹理特征进行下采样，得到局部纹理特征的过程中，为了保证得到的局部纹理特征的精度，图像处理系统可以首先利用卷积神经网络模块中的第一卷积层，从初始纹理特征中捕获得到上述的监测区域和其他区域各自的局部信息，即上述第一局部信息，为了提高捕获第一局部信息的效率，在捕获第一局部信息时，可以将初始多期图像的初始纹理特征划分为多个图像块，然后再根据多个图像块，从对应的初始纹理特征中捕获对应的第一局部信息。
在捕获到第一局部信息之后，图像处理系统可以再利用卷积神经网络模块中的第二卷积层，将该第一局部信息映射编码至高维的预设空间中，得到对应的隐藏特征，即上述的第一映射纹理特征，以映射前的第一局部信息为Fi∈RH×W×D×C为例，映射后的第一映射纹理特征可以是H为图像块的高、W为图像块的宽，D为图像块的深度，C为图像块的维度，Cl＞C。
在得到第一映射纹理特征之后，图像处理系统可以再利用卷积神经网络模块中的第三卷积层，将该第一映射纹理特征还原至第一局部信息所处的还原空间中，以得到还原后的纹理特征，即上述的还原纹理特征，最后再利用卷积神经网络模块中的归一化层，对该还原纹理特征进行归一化处理，从而得到上述的精度较高的局部纹理特征。
在本申请实施例中，利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征，包括：利用自注意力模块中的第一卷积层，捕获局部纹理特征中的第二局部信息；利用自注意力模块中的第二卷积层，将第二局部信息映射至预设空间，得到第二映射纹理特征；利用自注意力模块中的自注意力层，对第二映射纹理特征进行自注意力处理，得到自注意力纹理特征；利用自注意力模块中的前馈层，对自注意力纹理特征进行特征交互，得到交互纹理特征；利用自注意力模块中的移除层，对交互纹理特征进行特征融合，得到目标纹理特征。
与前述卷积神经网络模块中的第一卷积层和第二卷积层相类似，自注意力模块中的第一卷积层可以是用于从局部纹理特征中捕获得到第二局部信息，例如其他区域的纹理信息的卷积层，可以是3×3×3的结构，该第二局部信息可以是初始纹理特征的局部信息，第二卷积层可以是用于将第二局部信息映射至高维的预设空间中，以得到能够编码的隐藏特征的卷积层，可以是1×1×1的结构。上述自注意力层可以是用于对隐藏特征进行自注意力处理，以得到能够表现不同隐藏特征之间的依赖关系的自注意力纹理特征，上述前馈层可以是用于对自注意力纹理特征进行交互处理，得到可视化、可解释的交互纹理特征，上述移除层可以是用于对交互纹理特征进行删除、融合，以得到精度更高的目标纹理特征的卷积层。
在本实施例的一种可选方案中，在利用自注意力模块对局部纹理特征进行处理，得到 目标纹理特征的过程中，为了提高得到的目标纹理特征的精度，图像处理系统可以首先利用上述自注意力模块中的第一卷积层，从前述经过归一化处理的局部纹理特征中，捕获得到监测区域和其他区域各自的局部信息，即上述第二局部信息，接着再利用自注意力模块中的第二卷积层，将该第二局部信息编码映射至高维的预设空间中，得到对应的隐藏特征，即上述的第二映射纹理特征，然后再利用自注意力模块中的自注意力层，对该第二映射纹理特征进行自注意力处理，从而得到对应的自注意力纹理特征，最后再利用自注意力模块中的前馈层，对自注意力纹理特征进行交互处理，从而得到能够直观的表现出方便对自注意力纹理特征进行理解的交互纹理特征，并利用自注意力模块中的移除层对该交互纹理特征进行进一步的删除和融合的操作，从而得到精度更高的目标纹理特征。
在本申请实施例中，利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到图像的输出纹理特征；对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征；将多期图像的输出纹理特征和交叉注意力特征进行拼接，得到目标纹理特征。
上述输出纹理特征可以是指对前述的预设空间中的隐藏特征，即第二映射纹理特征进行划分，并经过前述的自注意力层和前馈层对划分后的隐藏特征进行处理后输出的特征，即交互纹理特征。在本实施例的一种可选方案中，可以按照预设的高度、宽度和深度，将第二映射纹理特征划分为多个非重叠的3D(3-dimensional，三维)特征块，从而得到该输出纹理特征。
在本实施例的一种可选方案中，该划分出的多个非重叠的3D特征块，可以与前述的图像块相同，3D特征块的表现形式可以是Fu∈RV×N×C，其中，V代表的是3D特征块的体积，N代表的是3D特征块的数量，C代表的是第二映射纹理特征的维度。对应的输出纹理特征的表现形式可以是FO∈RD×nC，其中，D代表的是3D特征块的深度，n代表的是纹理感知模型的数量。
上述交叉注意力特征可以是指从多期图像各自的输出纹理特征中提取出交叉模态信息，并利用额外掩码对该交叉模态信息进行处理，得到的注意力特征，其中，额外掩码的表现形式可以是M∈{0，-∞}nC×nC，其中，n代表的是纹理感知模型的数量，C代表的是输出纹理特征的维度，交叉注意力特征的表现形式可以是：
Fcross＝Softmax(QKT+M)V；
其中，Fcross代表的是输出纹理特征，Q、K、T分别是对前述输出纹理特征进行线性投影得到的查询矩阵、键矩阵，以及值矩阵。
在本实施例的一种可选方案中，在确定目标纹理特征时，图像处理系统还可以利用上述的至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息，进行融合处理，从 而得到上述的输出纹理特征，然后从多期图像各自的输出纹理特征中，提取出多个图像各自的交叉模态信息，并对该交叉模态信息进行交叉注意力处理，从而得到上述的交叉注意力特征，最后再将该输出纹理特征和交叉注意力特征进行拼接，例如在将输出纹理特征和交叉注意力特征进行串联后，利用预设的评价汇集层对串联后的特征进行拼接处理，从而得到上述的目标纹理特征。
在本申请实施例中，对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征，包括：利用交叉注意力模块对不同期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征。
在本实施例的一种可选方案中，可以利用预设的交叉注意力模块从不同期的图像的输出纹理特征中，提取出各自的交叉模态信息，并利用该交叉模态信息对输出纹理特征进行交叉注意力处理，以得到上述的交叉注意力特征。
在本申请实施例中，基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，包括：对图像进行图像分割，得到图像的分割结果，其中，分割结果至少包括：监测区域和其他区域；基于分割结果，确定位于监测区域边界的第一点集合，以及位于其他区域边界的第二点集合；确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合；对第一子集合和第二子集合进行交叉注意力处理，得到目标神经距离。
上述第一点集合可以是指对监测区域的边界进行采样得到的点集合，上述第二点集合可以是指对其他区域的边界进行采样得到的点集合。
在本实施例的一种可选方案中，在确定监测区域和其他区域之间的目标神经距离时，图像处理系统可以首先利用预设的图像分割模块，例如nnUet模型(一种图像分割模型)，对图像进行图像分割处理，从而得到图像中的监测区域和其他区域各自的边界，其中，其他区域中可以包括多种不同类型的血管，例如门静脉和脾静脉、肠系膜上动脉、肠系膜上静脉、以及腹腔真管等血管。在得到分割结果之后，图像处理系统可以从分割结果中的监测区域的边界上，确定出上述的第一点集合，并从其他区域的边界，例如上述血管上，确定出上述的第二点集合。
以X表示监测区域的边界上的点，Y表示其他区域的边界，例如上述血管上的点，对应的从监测区域的边界到其他区域的边界的距离可以表示为：
以表示从监测区域边界上的点中确定的第一点集合中的点，表示从其他区域的边界上的点中确定的第二集合中的点，对应的第一集合中的点到其他区域边界的距离，以及第二集合中的点到监测区域边界的距离可以表示为：
然后，图像处理系统按照预设条件，从第一点集合中确定出第一子集合，并从第二点集合中确定出第二子集合，其中，预设条件可以是指从点集合中确定出预设数量的多个距 离对应对方边界最近的点，例如从上述距离中，选取最小的20个点，来构建对应的子集合，对方边界可以是指该点所处区域的另外一个区域的边界，例如，若该点在监测区域中，则对方边界可以是指其他区域对应的边界。最后，对第一子集合和第二子集合中的多个点进行交叉注意力处理，从而得到上述的目标神经距离。
在本申请实施例中，确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合，包括：对第一点集合进行采样，得到第一采样点集合，并对第二点集合进行采样，得到第二采样点集合；确定第一采样点集合中，距离第二采样点集合最近的预设数量的第一采样点，得到第一子集合；确定第二采样点集合中，距离第一采样点集合最近的预设数量的第二采样点，得到第二子集合。
在本实施例的一种可选方案中，在确定第一子集合和第二子集合的过程中，可以首先对第一点集合进行进一步的采样，得到对应的第一采样点集合，并对第二点集合进行进一步的采样，得到对应的第二采样点集合，然后从第一采样点集合中，确定出预设数量的距离第二采样点集合最近的多个第一采样点，从而构建出上述的第一子集合，同时从第二采样点集合中，确定出预设数量的距离第一采样点集合最近的多个第二采样点，从而构建出上述的第二子集合。
以表示第一子集合中的点，以表示第二子集合中的点，对应的第一子集合可以表示为：
其中，K代表的是第一子集合中点的个数。
第二子集合可以表示为：
根据第一子集合中的采样点，以及第二子集合中的采样点，得到的目标神经距离可以表示为：
在本申请实施例中，基于目标纹理特征和目标神经距离，生成监测区域的风险指标，包括：对多期图像中监测区域和其他区域的结构关系进行提取，得到多期图像的结构特征；将目标纹理特征、目标神经距离和结构特征进行拼接，得到拼接特征；基于拼接特征，生成风险指标。
在本实施例的一种可选方案中，在利用目标纹理特征和目标神经距离，生成监测区域的风险指标的过程中，为了保证生成的风险指标的准确度，图像处理系统可以首先利用预设的结构分析模型，例如3D-CNN模型，对多期图像中监测区域和其他区域之间的结构关系进行提取，从而得到多期图像的结构特征，例如监测区域与其他区域中血管之间的连接关系，然后再将该结构特征和前述得到的目标纹理特征、目标神经距离进行拼接处理，得到上述的拼接特征，最后再利用全连接层，根据该拼接特征生成上述的风险指标。
在本实施例的一种可选方案中，在利用全连接层和拼接特征预测生成上述的风险指标 时，可以通过似然估计的方式，作为风险损失，来对风险指标进行调整，从而提高最终输出的风险指标的准确度。
为了方便理解上述对多期图像进行图像处理，以得到对应的风险指标的过程，图4是根据本申请实施例1的一种图像处理过程的示意图，其中，1代表的是不同时间采集到的多期图像，以胰腺癌为例，11代表的是非对比阶段的图像，12代表的是胰腺阶段的图像，13代表的是门静脉阶段的图像，2代表的是对多期图像进行特征提取，得到目标纹理特征的过程，21代表的是用于提取初始纹理特征的特征提取模块，22代表的是用于得到目标纹理特征的纹理感知模型，221代表的是卷积神经网络模块，222代表的是自注意力模块，23代表的是输出层特征，24代表的是交叉注意力特征，Block1、2、3代表的是纹理感知模型，Block4代表的是交叉处理模块，Block7代表的是自注意力层，Block8代表的是前馈层，Block9代表的是移除层，3代表的是用于确定目标神经距离的距离确定模块，31代表的是对多期图像进行图像分割的图像分割结果，32代表的是多期图像的结构特征，33代表的是用于获取第一子集合和第二子集合的子集合确定模块，34代表的是目标神经距离，Block5代表的是结构提取模块，Block6代表的是目标神经距离确定模块，4代表的是根据目标神经距离、目标纹理和特征和结构特征进行拼接得到的拼接特征，5代表的是根据拼接特征预测得到的风险指标。按照图4给出的箭头所示的处理方向，对多期图像进行处理，能够提高预测生成的风险指标的准确度。
在本申请实施例中，还提供了一种胰腺癌的预后预测方法，该方法包括：获取不同时间采集到的多期医学图像，其中，多期医学图像的显示内容至少包含胰腺癌的肿瘤和血管；对多期医学图像进行纹理特征提取，得到多期医学图像的目标纹理特征；基于多期医学图像中的肿瘤和血管的位置关系，确定多期医学图像的目标神经距离；基于目标纹理特征和目标神经距离，确定胰腺癌的存活结果。
上述胰腺癌是当前致死率较高的癌症，上述多期医学图像可以是指包含胰腺癌病变区域，即包含胰腺癌肿瘤和血管区域的，在不同时间采集到的图像，上述胰腺癌的肿瘤区域与前述的监测区域对应，上述血管区域与前述的其他区域对应。
在本实施例的一种可选方案中，由于胰腺癌的致死率较高，能够准确的预测出胰腺癌的存活结果，对于患者或者医护人员来说均是十分重要的治疗信息，因此，为了提高预测出的胰腺癌的存活结果的准确度，预后预测系统需要首先获取不同时间采集到的，包含胰腺癌的肿瘤和血管区域的多期医学图像，然后再对多期医学图像进行纹理特征提取，从而得到多期医学图像的目标纹理特征，同时根据多期医学图像中肿瘤和血管之间的位置关系，来确定出多期医学图像各自的目标神经距离，最后再根据该目标纹理特征和目标神经距离，来确定出胰腺癌的存活结果。
为了证明上述过程的可行性，在本申请中对多为PDAC患者进行了多中心研究，一下为研究过程和研究结果：
具体的，选取了1070位PDAC患者，将这些患者分为两大组，第一大组包括892位患 者，在中心A中进行患病情况比对的研究，第二大组包括178位患者，使用额外的3个中心，中心B、中心C和中心D，进行独立测试研究，在研究过程中使用到的多期医学图像分别是非对比阶段、胰腺阶段和门静脉阶段的CT图像。在中心A中，为了提高比对效果，从892位患者中选取340位患者，并由多位具有18年胰腺癌诊断经验的放射科医生，对这178位患者的多期医学图像的PDAC掩模进行手动标记，得到对比图像，然后使用具有自学习能力的模型对其余患者的多期医学图像的PDAC掩模进行预测，并由相同的注释员，即上述多为放射科医生，对该自学习模型进行检查和调整，以保证自学习模型的预测结果的准确度。对于其他中心的独立测试研究，也采用相同的模型，例如半监督产生其他容器掩模分割模型，来单独的对其他患者的多期医学图像的PDAC掩模进行分割，
除此以外，在实施研究的过程中，为了提高研究结果的准确度，还使用了嵌套式5倍交叉验证，并通过在轴向方向上旋转体积肿瘤和随机选择具有随机位移的裁剪区域来增强训练数据，并且为了提高预测的效率，对于纹理感知模型，将输出特征尺寸设置Ct＝64，对于结构提取模型，将输出特征尺寸设置为Cs＝64，对于神经距离，将输出特征尺寸设置为K＝32，将批次大小设置为16，将最大迭代被设置为1000个时期，并且在用于测试的训练期间，根据对验证集合的训练结果，选择具有较佳性能的模型。选取环境选取PyTorch1.11并在单个NVIDIA32G-V100GPU上训练模型。
研究过程：首先，通过将纹理感知模型(TAT)与ResNet18CNN主干和ViT变压器主干比较，来评估构建出的纹理感知模型(TAT)的性能，评估结果为纹理感知模型在对胰腺的局部信息和全局信息进行处理时，处理效果更好。同时通过将交叉注意力模块和常见的方法，例如LSTM、早期融合等方法进行比较，来评估交叉注意力模块的性能，评估结果为交叉注意力模块对同相特征部和交叉相特征部的处理结果，比其他方法的处理结果更合理。其次，对图4示出的图像处理方法的各部分进行评估，评估结果可以如下表1所示。
表1
表1是上述多个部分对应的评估结果，其中，C指数代表的是风险指标。
从表1中可以看出，利用纹理感知模型和结构提取模型，能够将风险指标从0.630改进至0.648，与肿瘤侵入会严重影响PDAC患者的存活率相符。此外，为了验证临床发现的风险指标与神经距离有关，还使用了仅使用肿瘤和四个血管的斜切距离，即目标神经距离 用于预后预测的4-变量回归模型，来验证神经距离与存活率之间的关系，该关系可以由表1中倒数第三行的C指数0.611得到证实，即神经距离与风险指标具有一定的相关性，其与临床发现一致，这意味着添加测量神经距离这一步，能够有效的改善预测得到的风险指标的准确度，本申请提出的神经距离度量执行传统的表面距离度量，如斜切距离，能够有效的适用于区分PDAC的严重性，即确定风险指标。
比较：为了进一步评估构建的模型的性能，可以将构建的模型与最新的深度预测方法进行比较，具体的比较结果如下表2所示。
表2
从表2可以看出，与现有的多种预测方法，例如3DCNN-P模型、早期融合方法、DeepCT-PDAC模型相比，本申请的C指数和AUC指数(Area Under Curve,ROC曲线线下面积)的预测结果与患病情况更贴近。
为了提高模型的性能，本申请修改了基线深度学习模型，并采用单个胰腺阶段或前述的所有三个阶段作为输入，来构建网络架构。目前，DeepCT-PDAC是使用3DCNN考虑肿瘤相关和肿瘤-血管关系的最新方法，与该方法相比，本申请提出的使用纹理感知模型和结构提取模型来捕获肿瘤增江模式和肿瘤血管参与，证明了在嵌套式5倍交叉验证和多入口独立测试集中，具有更好的性能的有效性，
表3
表3是使用单变量和多变量Cox比例风险模型，来评估本申请的目标纹理特征和其他 独立测试组中的临床病理学因子的有效性。如表3所示，风险分层和其他因素，例如病理性TNM阶段，是影响预后预测准确度的重要影响因素。在单变量分析中，选择显著变量(p＜0.05)之后，在调整重要的预后标志物如pT(HR＝2.438，p＜0.0001)和切除边缘(HR＝1.681，p＝0.091)之后，本申请提出的分期在多变量分析(HR＝1.847，p＝0.027)中保持强。值得注意的是，本申请提出的标记在所有手术前标记中保持最强，如肿瘤尺寸和CA19-9。
为了证明选取目标纹理特征和目标神经距离作为手术前的新辅助治疗的工具的有效性，图5是根据本申请实施例1的一种不同分组下生存曲线的示意图，如图5所示，其中，图5展示了在不同分组下患者的生存曲线，具体分组分别是：A是不分组情况下的风险指标曲线，B为根据肿瘤大小进行分组、C为根据CA19-9的值进行分组、D为同时根据肿瘤大小和CA19-9的值进行分组、E和F为根据患者年龄进行分组。通过图5可以得出，具有高风险、病情较严重的患者，更需要利用本申请的目标纹理特征和目标神经距离作为新辅助治疗的工作。
需要说明的是，本申请所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请并不受所描述的动作顺序的限制，因为依据本申请，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本申请所必须的。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例的方法。
实施例2
根据本申请实施例，还提供了一种图像处理方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图6是根据本申请实施例2的一种图像处理方法的流程图，如图6所示，该方法可以包括如下步骤：
步骤S602，响应作用于操作界面上的输入指令，在操作界面上显示不同时间采集到的 多期图像。
其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域。
上述输入指令可以是指在预设的操作界面中显示多期图像的指令，上述监测对象可以是指体内的组织或者器官可能会出现病变的动物，例如人、猫、地鼠等动物。上述目标部位的监测区域可以是指该可能会发生病变的组织或器官对应的区域，例如肿瘤区域。上述多期图像可以是指对不同时间的监测区域进行图像采集，例如可以通过断层扫描或者磁共振成像的方式，对不同时间的监测区域进行扫描成像，以采集得到监测区域在不同时间的多期图像。
在本实施例的一种可选方案中，在接收到输入指令之后，图像处理相同可以在预设的操作界面中，显示在不同时间采集到的多期图像。
步骤S604，响应作用于操作界面上的图像处理指令，在操作界面上显示监测区域的风险指标。
其中，风险指标是基于多期图像的目标纹理特征和目标神经距离生成的，目标纹理特征是对多期图像进行纹理特征提取得到的，目标神经距离是基于多期图像中的监测区域和其他区域的位置关系确定的。
上述目标神经距离可以是指监测区域的表面与其他区域的表面之间的距离，例如监测区域与其他区域中血管表面的距离。
在本实施例的一种可选方案中，在接收到图像处理指令之后，图像处理系统可以对多期图像进行纹理特征提取，以获得多期图像的目标纹理特征，并根据多期图像中监测区域和其他区域的位置关系，确定出监测区域和其他区域之间的目标神经距离，然后再根据该目标纹理特征和目标神经距离，确定出监测区域的风险指标，确定监测区域多期是否存在风险。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例3
根据本申请实施例，还提供了一种可以应用于虚拟现实VR设备、增强现实AR设备等虚拟现实场景下的图像处理方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图7是根据本申请实施例3的图像处理方法的流程图。如图7所示，该方法可以包括如下步骤：
步骤S702，在虚拟现实VR设备或增强现实AR设备的呈现画面上展示不同时间采集到的多期图像。
其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域。
上述监测对象可以是指体内的组织或者器官可能会出现病变的动物，例如人、猫、地 鼠等动物。上述目标部位的监测区域可以是指该可能会发生病变的组织或器官对应的区域，例如肿瘤区域。上述多期图像可以是指对不同时间的监测区域进行图像采集，例如可以通过断层扫描或者磁共振成像的方式，对不同时间的监测区域进行扫描成像，以采集得到监测区域在不同时间的多期图像。
在本实施例的一种可选方案中，为了使确定出的风险指标更准确，可以通过虚拟现实VR设备或增强现实AR设备，获取在不同时间采集到的，包含监测区域的上述多期图像。
步骤S704，对多期图像进行纹理特征提取，得到多期图像的目标纹理特征。
上述目标纹理特征可以是对监测区域所呈现出的全局纹理特征和局部纹理特征进行融合，得到的纹理特征，其中，局部纹理特征可以是指监测区域的纹理特征，可以包括但不限于：监测区域的颜色、纹理、形状、边缘、角点等特征，全局纹理特征可以是指多期图像整体区域以外的其他区域的纹理特征，例如目标部位的纹理特征，目标部位外的区域的纹理特征等。
在本实施例的一种可选方案中，考虑到根据监测区域所呈现出的图像效果，例如图像纹理，来判断监测区域是否存在风险的主要途径之一，因此，为了提高确定监测区域是否存在风险的准确度，图像处理系统可以在获取到不同时间的多期图像之后，可以对该多期图像进行纹理特征提取，确定出上述多期图像的目标纹理特征。
在本实施例的一种可选方案中，在对多期图像进行纹理特征提取，得到目标纹理特征时，考虑到在除了监测区域的目标部位和目标部位外的区域中，可能会存在较多的重要组织器官，例如在除PDAC区域的胰腺上和胰腺周围会存在较多重要的血管。这些重要组织器官可能会对监测区域的监测产生一定的影响，例如若血管的颜色与PDAC区域的颜色相接近，则可能会出现误判PDAC扩散的情况，同时考虑到监测区域在病变时也可能会对这些重要组织器官产生影响，例如PDAC区域可能会侵犯胰腺周围的血管，因此，为了保证确定监测区域是否存在风险的准确度，在对多期图像进行纹理特征提取时，除了仅根据监测区域所呈现出的纹理特征来对监测区域进行监测外，即提取监测区域的局部纹理特征，图像处理系统还可以对其他区域所呈现出的纹理来对监测区域进行监测，即提取全局纹理特征，然后再对局部纹理特征和全局纹理特征进行融合处理，以得到上述的目标纹理特征，从而保证根据目标纹理特征确定监测区域是否存在风险的准确度。
步骤S706，基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离。
其中，其他区域用于表征目标部位除监测区域之外的区域。
在本实施例的一种可选方案中，除了直接利用前述目标纹理特征来确定监测区域是否存在风险外，还可以进一步的结合多期图像中监测区域和其他区域之间的距离，即上述目标神经距离，来确定监测区域是否存在风险，从而提高确定结果的准确度。
在本实施例的一种可选方案中，可以通过根据多期图像中监测区域和其他区域之间的位置关系，例如前述的监测区域和其他区域之间的相对位置、结构连接等信息，来确定不 同图像对应的目标神经距离。
步骤S708，基于目标纹理特征和目标神经距离，生成监测区域的风险指标。
其中，风险指标用于表征监测区域存在风险的概率。
在本实施例的一种可选方案中，考虑到其他区域可能会对监测区域的预后预测的准确度产生影响，例如在胰腺周围会存在较多重要的血管，这些血管与PDAC区域较近，在切除PDAC区域时，可能会出现误切除，进而出现对患者产生负面影响的情况，因此，为了能够准确的对监测区域进行预后预测，除了直接利用前述目标纹理特征来确定监测区域是否存在风险外，还可以进一步的结合多期图像中监测区域和其他区域之间的距离，即上述目标神经距离，来确定监测区域是否存在风险，从而提高确定结果的准确度。
在本实施例的一种可选方案中，可以通过根据多期图像中监测区域和其他区域之间的位置关系，例如前述的监测区域和其他区域之间的相对位置、结构连接等信息，来确定不同图像对应的目标神经距离。
步骤S710，驱动VR设备或AR设备展示风险指标。
在本实施例的一种可选方案中，可以驱动VR设备或AR设备，向用户展示确定出的风险指标，以使工作人员能够更直观的监测区域对应的风险指标。
可选地，在本实施例中，上述图像处理方法可以应用于由服务器、虚拟现实设备所构成的硬件环境中。在虚拟现实VR设备或增强现实AR设备的呈现画面上展示多期图像、多期图像的目标纹理特征、目标神经距离和风险指标，服务器可以为媒体文件运营商对应的服务器，上述网络包括但不限于：广域网、城域网或局域网，上述虚拟现实设备并不限定于：虚拟现实头盔、虚拟现实眼镜、虚拟现实一体机等。
可选地，虚拟现实设备包括：存储器、处理器和传输装置。存储器用于存储应用程序，该应用程序可以用于执行：对多期图像进行纹理特征提取，得到多期图像的目标纹理特征，包括：对图像进行特征提取，得到图像的初始纹理特征；利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征。
在本申请实施例中，该应用程序还用于执行：利用至少一个纹理感知模型块对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征；利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征。
在本申请实施例中，该应用程序还用于执行：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征，包括：利用卷积神经网络模块中的第一卷积层，捕获初始纹理特征中的第一局部信息；利用卷积神经网络模块中的第二卷积层，将第一局部信息映射至预设空间，得到第一映射纹理特征；利用卷积神经网络模块中的第三卷积层，将映射纹理特征还原至初始纹理特征对应的原始空间，得到还原纹理特征；利用卷积神经网络模块中的归一化层，对还原纹理特征进行归一化处理，得到局部纹理特征。
在本申请实施例中，该应用程序还用于执行：利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征，包括：利用自注意力模块中的第一卷积层，捕获局部纹理特征中的第二局部信息；利用自注意力模块中的第二卷积层，将第二局部信息映射至预设空间，得到第二映射纹理特征；利用自注意力模块中的自注意力层，对第二映射纹理特征进行自注意力处理，得到自注意力纹理特征；利用自注意力模块中的前馈层，对自注意力纹理特征进行特征交互，得到交互纹理特征；利用自注意力模块中的移除层，对交互纹理特征进行特征融合，得到目标纹理特征。
在本申请实施例中，该应用程序还用于执行：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到图像的输出纹理特征；对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征；将多期图像的输出纹理特征和交叉注意力特征进行拼接，得到目标纹理特征。
在本申请实施例中，该应用程序还用于执行：对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征，包括：利用交叉注意力模块对不同期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征。
在本申请实施例中，该应用程序还用于执行：基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，包括：对图像进行图像分割，得到图像的分割结果，其中，分割结果至少包括：监测区域和其他区域；基于分割结果，确定位于监测区域边界的第一点集合，以及位于其他区域边界的第二点集合；确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合；对第一子集合和第二子集合进行交叉注意力处理，得到目标神经距离。
在本申请实施例中，该应用程序还用于执行：确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合，包括：对第一点集合进行采样，得到第一采样点集合，并对第二点集合进行采样，得到第二采样点集合；确定第一采样点集合中，距离第二采样点集合最近的预设数量的第一采样点，得到第一子集合；确定第二采样点集合中，距离第一采样点集合最近的预设数量的第二采样点，得到第二子集合。
在本申请实施例中，该应用程序还用于执行：基于目标纹理特征和目标神经距离，生成监测区域的风险指标，包括：对多期图像中监测区域和其他区域的结构关系进行提取，得到多期图像的结构特征；将目标纹理特征、目标神经距离和结构特征进行拼接，得到拼接特征；基于拼接特征，生成风险指标。
需要说明的是，该实施例的上述应用在VR设备或AR设备中的图像处理方法可以包括图3所示实施例的方法，以实现驱动VR设备或AR设备展示多期图像、目标纹理特征、目标数据距离和风险指标的目的。
可选地，该实施例的处理器可以通过传输装置调用上述存储器存储的应用程序以执行上述步骤。传输装置可以通过网络接收服务器发送的媒体文件，也可以用于上述处理器与 存储器之间的数据传输。
可选地，在虚拟现实设备中，带有眼球追踪的头戴式显示器，该HMD头显中的屏幕，用于显示展示的视频画面，HMD中的眼球追踪模块，用于获取用户眼球的实时运动轨迹，跟踪系统，用于追踪用户在真实三维空间的位置信息与运动信息，计算处理单元，用于从跟踪系统中获取用户的实时位置与运动信息，并计算出用户头部在虚拟三维空间中的三维坐标，以及用户在虚拟三维空间中的视野朝向等。
在本申请实施例中，虚拟现实设备可以与终端相连接，终端与服务器通过网络进行连接，上述虚拟现实设备并不限定于：虚拟现实头盔、虚拟现实眼镜、虚拟现实一体机等，上述终端并不限定于PC、手机、平板电脑等，服务器可以为媒体文件运营商对应的服务器，上述网络包括但不限于：广域网、城域网或局域网。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例4
根据本申请实施例，还提供了一种图像处理方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图8是根据本申请实施例4的图像处理方法的流程图，如图8所示，该方法可以包括如下步骤：
步骤S802，通过调用第一接口获取不同时间采集到的多期图像。
其中，第一接口包括第一参数，第一参数的参数值为多期图像，多期图像的显示内容至少包含待监测对象的目标部位的监测区域。
上述监测对象可以是指体内的组织或者器官可能会出现病变的动物，例如人、猫、地鼠等动物。上述目标部位的监测区域可以是指该可能会发生病变的组织或器官对应的区域，例如肿瘤区域。上述多期图像可以是指对不同时间的监测区域进行图像采集，例如可以通过断层扫描或者磁共振成像的方式，对不同时间的监测区域进行扫描成像，以采集得到监测区域在不同时间的多期图像。
在本实施例的一种可选方案中，为了使确定出的风险指标更准确，图像处理系统可以通过调用第一接口，获取在不同时间采集到的，包含监测区域的上述多期图像。
步骤S804，对多期图像进行纹理特征提取，得到多期图像的目标纹理特征。
在本实施例的一种可选方案中，考虑到根据监测区域所呈现出的图像效果，例如图像纹理，来判断监测区域是否存在风险的主要途径之一，因此，为了提高确定监测区域是否存在风险的准确度，图像处理系统可以在获取到不同时间的多期图像之后，可以对该多期图像进行纹理特征提取，确定出上述多期图像的目标纹理特征。
步骤S806，基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离。
其中，其他区域用于表征目标部位除监测区域之外的区域。
在本实施例的一种可选方案中，除了直接利用前述目标纹理特征来确定监测区域是否存在风险外，还可以进一步的结合多期图像中监测区域和其他区域之间的距离，即上述目标神经距离，来确定监测区域是否存在风险，从而提高确定结果的准确度。
步骤S808，基于目标纹理特征和目标神经距离，生成监测区域的风险指标。
其中，风险指标用于表征监测区域存在风险的概率。
在本实施例的一种可选方案中，考虑到其他区域可能会对监测区域的预后预测的准确度产生影响，例如在胰腺周围会存在较多重要的血管，这些血管与PDAC区域较近，在切除PDAC区域时，可能会出现误切除，进而出现对患者产生负面影响的情况，因此，为了能够准确的对监测区域进行预后预测，除了直接利用前述目标纹理特征来确定监测区域是否存在风险外，还可以进一步的结合多期图像中监测区域和其他区域之间的距离，即上述目标神经距离，来确定监测区域是否存在风险，从而提高确定结果的准确度。
步骤S810，通过调用第二接口输出风险指标，其中，第二接口包括第二参数，第二参数的参数值为风险指标。
在本实施例的一种可选方案中，可以通过调用第二接口输出确定出的风险指标，以使工作人员能够更直观的监测区域对应的风险指标。
实施例5
根据本申请实施例，还提供了一种用于实施上述图像处理方法的装置，该装置可以部署在目标客户端中。图9是根据本申请实施例5的一种图像处理装置的结构框图，如图9所示，该装置900包括：获取模块902，提取模块904，确定模块906和生成模块908。
其中，获取模块902用于获取不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；提取模块904用于对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；确定模块906用于基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；生成模块908用于基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率。
在本申请上述实施例中，提取模块904包括：提取单元，用于对图像进行特征提取，得到图像的初始纹理特征；融合单元，用于利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征。
在本申请实施例中，融合单元还用于：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征；利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征。
在本申请实施例中，融合单元还用于：利用卷积神经网络模块中的第一卷积层，捕获初始纹理特征中的第一局部信息；利用卷积神经网络模块中的第二卷积层，将第一局部信息映射至预设空间，得到第一映射纹理特征；利用卷积神经网络模块中的第三卷积层，将 映射纹理特征还原至初始纹理特征对应的原始空间，得到还原纹理特征；利用卷积神经网络模块中的归一化层，对还原纹理特征进行归一化处理，得到局部纹理特征。
在本申请实施例中，融合单元还用于：利用自注意力模块中的第一卷积层，捕获局部纹理特征中的第二局部信息；利用自注意力模块中的第二卷积层，将第二局部信息映射至预设空间，得到第二映射纹理特征；利用自注意力模块中的自注意力层，对第二映射纹理特征进行自注意力处理，得到自注意力纹理特征；利用自注意力模块中的前馈层，对自注意力纹理特征进行特征交互，得到交互纹理特征；利用自注意力模块中的移除层，对交互纹理特征进行特征融合，得到目标纹理特征。
在本申请实施例中，融合单元还用于：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到图像的输出纹理特征；对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征；将多期图像的输出纹理特征和交叉注意力特征进行拼接，得到目标纹理特征。
在本申请实施例中，融合单元还用于：用交叉注意力模块对不同期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征。
在本申请实施例中，确定模块906包括：分割单元，用于对图像进行图像分割，得到图像的分割结果，其中，分割结果至少包括：监测区域和其他区域；第一确定单元，用于基于分割结果，确定位于监测区域边界的第一点集合，以及位于其他区域边界的第二点集合；第二确定单元，用于确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合；距离确定单元，用于对第一子集合和第二子集合进行交叉注意力处理，得到目标神经距离。
在本申请实施例中，第二确定单元还用于：对第一点集合进行采样，得到第一采样点集合，并对第二点集合进行采样，得到第二采样点集合；确定第一采样点集合中，距离第二采样点集合最近的预设数量的第一采样点，得到第一子集合；确定第二采样点集合中，距离第一采样点集合最近的预设数量的第二采样点，得到第二子集合。
在本申请实施例中，生成模块908包括：结构关系提取单元，用于对多期图像中监测区域和其他区域的结构关系进行提取，得到多期图像的结构特征；特征拼接单元，用于将目标纹理特征、目标神经距离和结构特征进行拼接，得到拼接特征；指示生成单元，用于基于拼接特征，生成风险指标。
此处需要说明的是，上述的获取模块902，提取模块904，确定模块906和生成模块908，对应于实施例1中的步骤S302至步骤S308，四个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的AR/VR设备中。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例6
根据本申请实施例，还提供了一种用于实施上述图像处理方法的装置，该装置可以部署在目标客户端中。图10是根据本申请实施例6的一种图像处理装置的结构框图，如图10所示，该装置1000包括：第一显示模块1002和第二显示模块1004。
其中，第一显示模块1002用于响应作用于操作界面上的输入指令，在操作界面上显示不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域，第二显示模块1004用于响应作用于操作界面上的图像处理指令，在操作界面上显示监测区域的风险指标，其中，风险指标是基于多期图像的目标纹理特征和目标神经距离生成的，目标纹理特征是对多期图像进行纹理特征提取得到的，目标神经距离是基于多期图像中的监测区域和其他区域的位置关系确定的。
此处需要说明的是，上述的第一显示模块1002和第二显示模块1004，对应于实施例2中的步骤S602至步骤S604，两个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的AR/VR设备中。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例7
根据本申请实施例，还提供了一种用于实施上述图像处理方法的装置，该装置可以部署在目标客户端中。图11是根据本申请实施例7的一种图像处理装置的结构框图，如图11所示，该装置1100包括：第一展示模块1102、提取模块1104，确定模块1106、生成模块1108和第二展示模块1110。
其中，第一展示模块1102用于在虚拟现实VR设备或增强现实AR设备的呈现画面上展示不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域，提取模块1104用于对多期图像进行纹理特征提取，得到多期图像的目标纹理特征，确定模块1106用于基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域，生成模块1108用于基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率，第二展示模块1110用于驱动VR设备或AR设备展示风险指标。
此处需要说明的是，上述的第一展示模块1102、提取模块1104，确定模块1106、生成模块1108和第二展示模块1110，对应于实施例3中的步骤S702至步骤S710，五个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的AR/VR设备 中。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例8
根据本申请实施例，还提供了一种用于实施上述图像处理方法的装置，该装置可以部署在目标客户端中。图12是根据本申请实施例8的一种图像处理装置的结构框图，如图12所示，该装置1200包括：获取模块1202，提取模块1204，确定模块1206、生成模块1208和输出模块1210。
其中，获取模块1202用于通过调用第一接口获取不同时间采集到的多期图像，其中，第一接口包括第一参数，第一参数的参数值为多期图像，多期图像的显示内容至少包含待监测对象的目标部位的监测区域，提取模块1204用于对多期图像进行纹理特征提取，得到多期图像的目标纹理特征，确定模块1206用于基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域、生成模块1208用于基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率、输出模块1210用于通过调用第二接口输出风险指标，其中，第二接口包括第二参数，第二参数的参数值为风险指标。
此处需要说明的是，上述的获取模块1202，提取模块1204，确定模块1206、生成模块1208和输出模块1210，对应于实施例4中的步骤S802至步骤S810，五个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的AR/VR设备中。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例9
根据本申请实施例，还提供了一种计算机辅助癌症预后方法，包括：
获取不同时间采集到的多期医学图像，其中，多期医学图像的显示内容至少包含待监测对象的目标部位的癌症区域；
对多期医学图像进行纹理特征提取，得到多期医学图像的目标纹理特征；
基于多期医学图像中的癌症区域和其他区域的位置关系，确定多期医学图像的目标神经距离，其中，其他区域用于表征目标部位除癌症区域之外的区域；
基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征癌症区域存在风险的概率。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例10
根据本申请实施例，还提供了一种计算机辅助胰腺癌预后方法，包括：
获取不同时间采集到的多期医学图像，其中，多期医学图像的显示内容至少包含待监测对象的胰腺癌区域；
对多期医学图像进行纹理特征提取，得到多期医学图像的目标纹理特征；
基于多期医学图像中的胰腺癌区域和其他区域的位置关系，确定多期医学图像的目标神经距离，其中，其他区域用于表征胰腺癌区域之外的区域；
基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征胰腺癌区域存在风险的概率。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例11
根据本申请实施例，还提供了一种计算机辅助癌症预后系统，包括存储器、处理器以及存储在存储器上并在处理器上运行的计算机程序，处理器执行计算机程序可用于执行一种计算机辅助癌症预后方法，方法包括：
获取不同时间采集到的多期医学图像，其中，多期医学图像的显示内容至少包含待监测对象的目标部位的癌症区域；
对多期医学图像进行纹理特征提取，得到多期医学图像的目标纹理特征；
基于多期医学图像中的癌症区域和其他区域的位置关系，确定多期医学图像的目标神经距离，其中，其他区域用于表征目标部位除癌症区域之外的区域；
基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征癌症区域存在风险的概率。
需要说明的是，本申请上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例12
本申请的实施例可以提供一种电子设备，该电子设备可以是电子设备群中的任意一个电子设备。可选地，在本实施例中，上述电子设备也可以替换为移动终端等终端设备。
可选地，在本实施例中，上述电子设备可以位于计算机网格的多个网格设备中的至少一个网格设备。
在本实施例中，上述电子设备可以执行图像分割方法中以下步骤的程序代码：获取不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率。
可选地，图13是根据本申请实施例12的一种电子设备的结构框图。如图所示，该电子设备A可以包括：处理器1302和存储器1304，其中，存储有可执行程序；处理器，用于运行程序，其中，程序运行时执行实施例1中所示的图像处理方法。
可选地，如图13所示，在电子设备A中还可以包括：存储控制器和外设接口，其中，外设接口与射频模块、音频模块和显示器连接。
其中，存储器可用于存储软件程序以及模块，如本申请实施例中的图像处理方法和装置对应的程序指令/模块，处理器通过运行存储在存储器内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的图像分割方法。存储器可包括高速随机存储器，还可以包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器可进一步包括相对于处理器远程设置的存储器，这些远程存储器可以通过网格连接至终端A。上述网格的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：对图像进行特征提取，得到图像的初始纹理特征；利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：利用至少一个纹理感知模型块对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征；利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征，包括：利用卷积神经网络模块中的第一卷积层，捕获初始纹理特征中的第一局部信息；利用卷积神经网络模块中的第二卷积层，将第一局部信息映射至预设空间，得到第一映射纹理特征；利用卷积神经网络模块中的第三卷积层，将映射纹理特征还原至初始纹理特征对应的原始空间，得到还原纹理特征；利用卷积神经网络模块中的归一化层，对还原纹理特征进行归一化处理，得到局部纹理特征。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征，包括：利用自注意力模块中的第一卷积层，捕获局部纹理特征中的第二局部信息；利用自注意力模块中的第二卷积层，将第二局部信息映射至预设空间，得到第二映射纹理特征；利用自注意力模块中的自注意力层，对第二映射纹理特征进行自注意力处理，得到自注意力纹理特征；利用自注意力模块中的前馈层，对自注意力纹理特征进行特征交互，得到交互纹理特征；利用自注意力模块中的移除层，对交互纹理特征进行特征融合，得到目标纹理特征。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到图像的输出纹理特征；对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征；将多期图像的输出纹理特征和交叉注意力特征进行拼接，得到目标纹理特征。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征，包括：利用交叉注意力模块对不同期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，包括：对图像进行图像分割，得到图像的分割结果，其中，分割结果至少包括：监测区域和其他区域；基于分割结果，确定位于监测区域边界的第一点集合，以及位于其他区域边界的第二点集合；确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合；对第一子集合和第二子集合进行交叉注意力处理，得到目标神经距离。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合，包括：对第一点集合进行采样，得到第一采样点集合，并对第二点集合进行采样，得到第二采样点集合；确定第一采样点集合中，距离第二采样点集合最近的预设数量的第一采样点，得到第一子集合；确定第二采样点集合中，距离第一采样点集合最近的预设数量的第二采样点，得到第二子集合。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：基于目标纹理特征和目标神经距离，生成监测区域的风险指标，包括：对多期图像中监测区域和其他区域的结构关系进行提取，得到多期图像的结构特征；将目标纹理特征、目标神经距离和结构特征进行拼接，得到拼接特征；基于拼接特征，生成风险指标。
在本申请实施例中，采用获取不同时间采集到的多期图像；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离；基于目标纹理特征和目标神经距离，生成监测区域的风险指标的方式，通过根据在不同时间采集到的多期图像中监测区域的目标纹理特征，以及监测区域与其他区域之间的目标神经距离，来生成监测区域的风险指标，使得监测区域的风险预测过程中，充分考虑了监测区域与其他区域之间的接触情况，从而提高了确定出的风险指标的准确度，进而解决了相关技术中确定监测区域是否存在风险的准确度低的技术问题。
本领域普通技术人员可以理解，图13所示的结构仅为示意，计算机终端也可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌上电脑以及移动互联网设备(Mobile  Internet Devices，MID)、PAD等终端设备。图13其并不对上述电子装置的结构造成限定。例如，计算机终端A还可包括比图13中所示更多或者更少的组件(如网格接口、显示装置等)，或者具有与图13所示不同的配置。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。
实施例13
本申请的实施例还提供了一种计算机可读存储介质。可选地，在本实施例中，上述存储介质可以用于保存上述实施例1所提供的图像分割方法所执行的程序代码。
在本实施例中，上述存储介质可以位于计算机网格中计算机终端群中的任意一个计算机终端中，或者位于移动终端群中的任意一个移动终端中。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取不同时间采集到的多期图像，其中，多期图像的显示内容至少包含待监测对象的目标部位的监测区域；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，其中，其他区域用于表征目标部位除监测区域之外的区域；基于目标纹理特征和目标神经距离，生成监测区域的风险指标，其中，风险指标用于表征监测区域存在风险的概率。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：对图像进行特征提取，得到图像的初始纹理特征；利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：利用至少一个纹理感知模型块对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征；利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：利用纹理感知模型中的卷积神经网络模块，对初始纹理特征进行下采样，图像的局部纹理特征，包括：利用卷积神经网络模块中的第一卷积层，捕获初始纹理特征中的第一局部信息；利用卷积神经网络模块中的第二卷积层，将第一局部信息映射至预设空间，得到第一映射纹理特征；利用卷积神经网络模块中的第三卷积层，将映射纹理特征还原至初始纹理特征对应的原始空间，得到还原纹理特征；利用卷积神经网络模块中的归一化层，对还原纹理特征进行归一化处理，得到局部纹理特征。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：利用纹理感知模型中的自注意力模块，对局部纹理特征进行注意力处理，得到图像的目标纹理特征， 包括：利用自注意力模块中的第一卷积层，捕获局部纹理特征中的第二局部信息；利用自注意力模块中的第二卷积层，将第二局部信息映射至预设空间，得到第二映射纹理特征；利用自注意力模块中的自注意力层，对第二映射纹理特征进行自注意力处理，得到自注意力纹理特征；利用自注意力模块中的前馈层，对自注意力纹理特征进行特征交互，得到交互纹理特征；利用自注意力模块中的移除层，对交互纹理特征进行特征融合，得到目标纹理特征。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到目标纹理特征，包括：利用至少一个纹理感知模型对初始纹理特征中的全局信息和局部信息进行融合，得到图像的输出纹理特征；对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征；将多期图像的输出纹理特征和交叉注意力特征进行拼接，得到目标纹理特征。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：对多期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征，包括：利用交叉注意力模块对不同期图像的输出纹理特征进行交叉注意力处理，得到交叉注意力特征。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离，包括：对图像进行图像分割，得到图像的分割结果，其中，分割结果至少包括：监测区域和其他区域；基于分割结果，确定位于监测区域边界的第一点集合，以及位于其他区域边界的第二点集合；确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合；对第一子集合和第二子集合进行交叉注意力处理，得到目标神经距离。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：确定第一点集合中满足预设条件的第一子集合，以及第二点集合中满足预设条件的第二子集合，包括：对第一点集合进行采样，得到第一采样点集合，并对第二点集合进行采样，得到第二采样点集合；确定第一采样点集合中，距离第二采样点集合最近的预设数量的第一采样点，得到第一子集合；确定第二采样点集合中，距离第一采样点集合最近的预设数量的第二采样点，得到第二子集合。
在本申请实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：基于目标纹理特征和目标神经距离，生成监测区域的风险指标，包括：对多期图像中监测区域和其他区域的结构关系进行提取，得到多期图像的结构特征；将目标纹理特征、目标神经距离和结构特征进行拼接，得到拼接特征；基于拼接特征，生成风险指标。
在本申请实施例中，采用获取不同时间采集到的多期图像；对多期图像进行纹理特征提取，得到多期图像的目标纹理特征；基于多期图像中的监测区域和其他区域的位置关系，确定多期图像的目标神经距离；基于目标纹理特征和目标神经距离，生成监测区域的风险指标的方式，通过根据在不同时间采集到的多期图像中监测区域的目标纹理特征，以及监测区域与其他区域之间的目标神经距离，来生成监测区域的风险指标，使得监测区域的风 险预测过程中，充分考虑了监测区域与其他区域之间的接触情况，从而提高了确定出的风险指标的准确度，进而解决了相关技术中确定监测区域是否存在风险的准确度低的技术问题。
实施例14
本申请的实施例还提供了一种计算机程序产品，包括计算机程序，所述计算机程序在计算机中执行时，令计算机执行本申请的实施例提供的方法。
实施例15
本申请的实施例还提供了一种计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行本申请的实施例提供的方法。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
在本申请的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上仅是本申请的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (19)

  1. 一种图像处理方法，其特征在于，包括：
    获取不同时间采集到的多期图像，其中，多期所述图像的显示内容至少包含待监测对象的目标部位的监测区域；
    对多期所述图像进行纹理特征提取，得到多期所述图像的目标纹理特征；
    基于多期所述图像中的所述监测区域和其他区域的位置关系，确定多期所述图像的目标神经距离，其中，所述其他区域用于表征所述目标部位除所述监测区域之外的区域；
    基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述监测区域存在风险的概率。
  2. 根据权利要求1所述的方法，其特征在于，对多期所述图像进行纹理特征提取，得到多期所述图像的目标纹理特征，包括：
    对所述图像进行特征提取，得到所述图像的初始纹理特征；
    利用至少一个纹理感知模型对所述初始纹理特征中的全局信息和局部信息进行融合，得到所述目标纹理特征。
  3. 根据权利要求2所述的方法，其特征在于，利用至少一个纹理感知模型块对所述初始纹理特征中的全局信息和局部信息进行融合，得到所述目标纹理特征，包括：
    利用所述纹理感知模型中的卷积神经网络模块，对所述初始纹理特征进行下采样，所述图像的局部纹理特征；
    利用所述纹理感知模型中的自注意力模块，对所述局部纹理特征进行注意力处理，得到所述图像的目标纹理特征。
  4. 根据权利要求3所述的方法，其特征在于，利用所述纹理感知模型中的卷积神经网络模块，对所述初始纹理特征进行下采样，所述图像的局部纹理特征，包括：
    利用所述卷积神经网络模块中的第一卷积层，捕获所述初始纹理特征中的第一局部信息；
    利用所述卷积神经网络模块中的第二卷积层，将所述第一局部信息映射至预设空间，得到第一映射纹理特征；
    利用所述卷积神经网络模块中的第三卷积层，将所述映射纹理特征还原至所述初始纹理特征对应的原始空间，得到还原纹理特征；
    利用所述卷积神经网络模块中的归一化层，对所述还原纹理特征进行归一化处理，得到所述局部纹理特征。
  5. 根据权利要求3所述的方法，其特征在于，利用所述纹理感知模型中的自注意力模块，对所述局部纹理特征进行注意力处理，得到所述图像的目标纹理特征，包括：
    利用所述自注意力模块中的第一卷积层，捕获所述局部纹理特征中的第二局部信息；
    利用所述自注意力模块中的第二卷积层，将所述第二局部信息映射至预设空间，得到第二映射纹理特征；
    利用所述自注意力模块中的自注意力层，对所述第二映射纹理特征进行自注意力处理，得到自注意力纹理特征；
    利用所述自注意力模块中的前馈层，对所述自注意力纹理特征进行特征交互，得到交互纹理特征；
    利用所述自注意力模块中的移除层，对所述交互纹理特征进行特征融合，得到所述目标纹理特征。
  6. 根据权利要求2所述的方法，其特征在于，利用至少一个纹理感知模型对所述初始纹理特征中的全局信息和局部信息进行融合，得到所述目标纹理特征，包括：
    利用至少一个纹理感知模型对所述初始纹理特征中的全局信息和局部信息进行融合，得到所述图像的输出纹理特征；
    对多期所述图像的所述输出纹理特征进行交叉注意力处理，得到交叉注意力特征；
    将多期所述图像的所述输出纹理特征和所述交叉注意力特征进行拼接，得到所述目标纹理特征。
  7. 根据权利要求6所述的方法，其特征在于，对多期所述图像的所述输出纹理特征进行交叉注意力处理，得到交叉注意力特征，包括：
    利用交叉注意力模块对不同期所述图像的所述输出纹理特征进行交叉注意力处理，得到所述交叉注意力特征。
  8. 根据权利要求1至7任意一项所述的方法，其特征在于，基于多期所述图像中的所述监测区域和其他区域的位置关系，确定多期所述图像的目标神经距离，包括：
    对所述图像进行图像分割，得到所述图像的分割结果，其中，所述分割结果至少包括：所述监测区域和所述其他区域；
    基于所述分割结果，确定位于所述监测区域边界的第一点集合，以及位于所述其他区域边界的第二点集合；
    确定所述第一点集合中满足预设条件的第一子集合，以及所述第二点集合中满足所述预设条件的第二子集合；
    对所述第一子集合和所述第二子集合进行交叉注意力处理，得到所述目标神经距离。
  9. 根据权利要求8所述的方法，其特征在于，确定所述第一点集合中满足预设条件的第一子集合，以及所述第二点集合中满足所述预设条件的第二子集合，包括：
    对所述第一点集合进行采样，得到第一采样点集合，并对所述第二点集合进行采样，得到第二采样点集合；
    确定所述第一采样点集合中，距离所述第二采样点集合最近的预设数量的第一 采样点，得到所述第一子集合；
    确定所述第二采样点集合中，距离所述第一采样点集合最近的所述预设数量的第二采样点，得到所述第二子集合。
  10. 根据权利要求1至9任意一项所述的方法，其特征在于，基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，包括：
    对多期所述图像中所述监测区域和所述其他区域的结构关系进行提取，得到多期所述图像的结构特征；
    将所述目标纹理特征、所述目标神经距离和所述结构特征进行拼接，得到拼接特征；
    基于所述拼接特征，生成所述风险指标。
  11. 一种图像处理方法，其特征在于，包括：
    响应作用于操作界面上的输入指令，在所述操作界面上显示不同时间采集到的多期图像，其中，多期所述图像的显示内容至少包含待监测对象的目标部位的监测区域；
    响应作用于所述操作界面上的图像处理指令，在所述操作界面上显示所述监测区域的风险指标，其中，所述风险指标是基于多期所述图像的目标纹理特征和目标神经距离生成的，所述目标纹理特征是对多期所述图像进行纹理特征提取得到的，所述目标神经距离是基于多期所述图像中的所述监测区域和其他区域的位置关系确定的。
  12. 一种图像处理方法，其特征在于，包括：
    通过调用第一接口获取不同时间采集到的多期图像，其中，所述第一接口包括第一参数，所述第一参数的参数值为多期所述图像，多期所述图像的显示内容至少包含待监测对象的目标部位的监测区域；
    对多期所述图像进行纹理特征提取，得到多期所述图像的目标纹理特征；
    基于多期所述图像中的所述监测区域和其他区域的位置关系，确定多期所述图像的目标神经距离，其中，所述其他区域用于表征所述目标部位除所述监测区域之外的区域；
    基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述监测区域存在风险的概率；
    通过调用第二接口输出所述风险指标，其中，所述第二接口包括第二参数，所述第二参数的参数值为所述风险指标。
  13. 一种计算机辅助癌症预后方法，其特征在于，包括：
    获取不同时间采集到的多期医学图像，其中，多期所述医学图像的显示内容至少包含待监测对象的目标部位的癌症区域；
    对多期所述医学图像进行纹理特征提取，得到多期所述医学图像的目标纹理特 征；
    基于多期所述医学图像中的所述癌症区域和其他区域的位置关系，确定多期所述医学图像的目标神经距离，其中，所述其他区域用于表征所述目标部位除所述癌症区域之外的区域；
    基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述癌症区域存在风险的概率。
  14. 一种计算机辅助胰腺癌预后方法，其特征在于，包括：
    获取不同时间采集到的多期医学图像，其中，多期所述医学图像的显示内容至少包含待监测对象的胰腺癌区域；
    对多期所述医学图像进行纹理特征提取，得到多期所述医学图像的目标纹理特征；
    基于多期所述医学图像中的所述胰腺癌区域和其他区域的位置关系，确定多期所述医学图像的目标神经距离，其中，所述其他区域用于表征所述胰腺癌区域之外的区域；
    基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述胰腺癌区域存在风险的概率。
  15. 一种计算机辅助癌症预后系统，其特征在于，包括存储器、处理器以及存储在所述存储器上并在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序可用于执行一种计算机辅助癌症预后方法，所述方法包括：
    获取不同时间采集到的多期医学图像，其中，多期所述医学图像的显示内容至少包含待监测对象的目标部位的癌症区域；
    对多期所述医学图像进行纹理特征提取，得到多期所述医学图像的目标纹理特征；
    基于多期所述医学图像中的所述癌症区域和其他区域的位置关系，确定多期所述医学图像的目标神经距离，其中，所述其他区域用于表征所述目标部位除所述癌症区域之外的区域；
    基于所述目标纹理特征和所述目标神经距离，生成所述监测区域的风险指标，其中，所述风险指标用于表征所述癌症区域存在风险的概率。
  16. 一种电子设备，其特征在于，包括：
    存储器，存储有可执行程序；
    处理器，用于运行所述程序，其中，所述程序运行时执行权利要求1至14中任意一项所述的方法。
  17. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括存储的可执行程序，其中，在所述可执行程序运行时控制所述计算机可读存储介质所在设备执行权利要求1至14中任意一项所述的方法。
  18. 一种计算机程序产品，其特征在于，包括计算机可执行指令，所述计算机可执行指令被处理器执行时实现权利要求1至14中任意一项所述方法的步骤。
  19. 一种计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至14中任意一项所述方法的步骤。
PCT/CN2024/105246 2023-07-20 2024-07-12 图像处理方法、电子设备和计算机可读存储介质 Pending WO2025016321A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310896158.0A CN117152674A (zh) 2023-07-20 2023-07-20 图像处理方法、电子设备和计算机可读存储介质
CN202310896158.0 2023-07-20

Publications (1)

Publication Number Publication Date
WO2025016321A1 true WO2025016321A1 (zh) 2025-01-23

Family

ID=88885746

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/105246 Pending WO2025016321A1 (zh) 2023-07-20 2024-07-12 图像处理方法、电子设备和计算机可读存储介质

Country Status (3)

Country Link
CN (1) CN117152674A (zh)
TW (1) TW202505482A (zh)
WO (1) WO2025016321A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117152674A (zh) * 2023-07-20 2023-12-01 阿里巴巴达摩院(杭州)科技有限公司 图像处理方法、电子设备和计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110026798A1 (en) * 2008-02-19 2011-02-03 The Trustees Of The University Of Pennyslvania System and method for automated segmentation, characterization, and classification of possibly malignant lesions and stratification of malignant tumors
CN105872454A (zh) * 2015-02-05 2016-08-17 富士通株式会社 图像显示设备和图像显示方法
CN113160205A (zh) * 2021-05-04 2021-07-23 曲阜师范大学 一种基于距离场的精确定义目标肿瘤安全消融边界的方法
CN114708971A (zh) * 2022-04-20 2022-07-05 推想医疗科技股份有限公司 一种风险评估方法及装置、存储介质及电子设备
CN115526843A (zh) * 2022-09-14 2022-12-27 阿里巴巴（中国）有限公司 医学图像的处理方法和系统
CN117152674A (zh) * 2023-07-20 2023-12-01 阿里巴巴达摩院(杭州)科技有限公司 图像处理方法、电子设备和计算机可读存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130080134A1 (en) * 2008-07-25 2013-03-28 Fundação D. Anna Sommer Champalimaud e Dr. Carlos Montez Champalimaud Systems and methods for predicting favorable-risk disease for patients enrolled in active surveillance
CN110021430A (zh) * 2019-04-09 2019-07-16 科大讯飞股份有限公司 一种病灶的属性信息预测方法及装置
CN113077434B (zh) * 2021-03-30 2023-01-24 零氪智慧医疗科技(天津)有限公司 基于多模态信息的肺癌识别方法、装置及存储介质
CN114241538B (zh) * 2022-02-21 2022-05-06 深圳大学 一种基于库欣综合征的图像特征识别方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110026798A1 (en) * 2008-02-19 2011-02-03 The Trustees Of The University Of Pennyslvania System and method for automated segmentation, characterization, and classification of possibly malignant lesions and stratification of malignant tumors
CN105872454A (zh) * 2015-02-05 2016-08-17 富士通株式会社 图像显示设备和图像显示方法
CN113160205A (zh) * 2021-05-04 2021-07-23 曲阜师范大学 一种基于距离场的精确定义目标肿瘤安全消融边界的方法
CN114708971A (zh) * 2022-04-20 2022-07-05 推想医疗科技股份有限公司 一种风险评估方法及装置、存储介质及电子设备
CN115526843A (zh) * 2022-09-14 2022-12-27 阿里巴巴（中国）有限公司 医学图像的处理方法和系统
CN117152674A (zh) * 2023-07-20 2023-12-01 阿里巴巴达摩院(杭州)科技有限公司 图像处理方法、电子设备和计算机可读存储介质

Also Published As

Publication number Publication date
CN117152674A (zh) 2023-12-01
TW202505482A (zh) 2025-02-01

Similar Documents

Publication Publication Date Title
CN111801708B (zh) 使用光线投射和实时深度进行遮挡渲染的方法
US8004516B2 (en) Image processing system for volume rendering
JP2022533207A (ja) 稠密３ｄ再構築データのキャッシュおよび更新
WO2024141092A1 (zh) 图像处理方法、计算机可读存储介质以及计算机终端
CA2975089A1 (en) Touchless advanced image processing and visualization
US20230245317A1 (en) Low latency interactive segmentation of medical images within a web-based deployment architecture
WO2025016321A1 (zh) 图像处理方法、电子设备和计算机可读存储介质
US11366983B2 (en) Study-level multi-view processing system
WO2024146649A1 (zh) 图像处理方法、存储介质及计算机终端
WO2025007940A1 (zh) 图像处理方法、电子设备及存储介质
CN107194163A (zh) 一种显示方法和系统
WO2024244916A1 (zh) 图像分割方法、电子设备和存储介质
WO2021081771A1 (zh) 基于vrds ai医学影像的心脏冠脉的分析方法和相关装置
US20220157450A1 (en) Capturing user constructed map of bodily region of interest for remote telemedicine navigation
CN114927229B (zh) 手术仿真方法、装置、电子设备及存储介质
CN117671052A (zh) 一种基于数据驱动的乳腺图像处理和重建的方法及系统
WO2020173055A1 (zh) VRDS 4D医学影像多设备Ai联动显示方法及产品
CN116597159A (zh) 特征提取方法、生物对象部位的状态识别方法及电子设备
US11636628B2 (en) Composite imagery rendering in diminished reality environment for medical diagnosis
CN117058100B (zh) 图像识别方法、电子设备和计算机可读存储介质
US20250195142A1 (en) Tool navigation using a customized medical simulation
CN111128350B (zh) 在线编辑方法及装置
KR102807689B1 (ko) 시공간 특징 퓨전 모델을 통해 수술 중 출혈을 검출하는 방법 및 장치
EP4534033A1 (en) Method, device, and program for providing patient-specific 3d surgical simulation
JP5570768B2 (ja) 医用画像取得システムおよび医用画像取得方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24842314

Country of ref document: EP

Kind code of ref document: A1