WO2021179189A1 - 大脑成瘾性状评估的可视化方法、装置及介质 - Google Patents

大脑成瘾性状评估的可视化方法、装置及介质 Download PDF

Info

Publication number
WO2021179189A1
WO2021179189A1 PCT/CN2020/078694 CN2020078694W WO2021179189A1 WO 2021179189 A1 WO2021179189 A1 WO 2021179189A1 CN 2020078694 W CN2020078694 W CN 2020078694W WO 2021179189 A1 WO2021179189 A1 WO 2021179189A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
loss function
processing
sample image
visualization
Prior art date
Application number
PCT/CN2020/078694
Other languages
English (en)
French (fr)
Inventor
王书强
余雯
肖晨晨
胡圣烨
Original Assignee
深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳先进技术研究院 filed Critical 深圳先进技术研究院
Priority to EP20924905.1A priority Critical patent/EP3971773A4/en
Priority to PCT/CN2020/078694 priority patent/WO2021179189A1/zh
Publication of WO2021179189A1 publication Critical patent/WO2021179189A1/zh
Priority to US17/549,258 priority patent/US20220101527A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/32—Normalisation of the pattern dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • This application relates to the field of big data technology, and in particular to a visualization method, device and medium for evaluating brain addiction traits.
  • Functional magnetic resonance imaging is a neuroimaging method that can accurately locate specific cerebral cortical areas and capture changes in blood oxygen that can reflect neuronal activity.
  • Combining FMRI and deep learning technology can extract complex features from the original data, but the feature extraction method is poor in interpretability and requires a large number of FMRI images as a basis. Due to the complexity of the FMRI image acquisition process and the high experimental cost, it is difficult to obtain FMRI images, which in turn limits the research of deep learning methods in the field of FMRI image evaluation and visualization.
  • the embodiments of the application provide a visualization method, device and computer-readable storage medium for evaluating brain addiction traits. While reducing the number of FMRI image samples required, it can more intuitively and accurately locate the brain area of nicotine addiction, and achieve Visualization of evaluation results.
  • the embodiments of the present application provide a visualization method for evaluating brain addiction traits, including:
  • the visualization processing request includes a to-be-processed image
  • the visualization processing request is used to request to obtain a visualization evaluation result of the to-be-processed image
  • Masking processing is performed on the image to be processed to obtain a perturbation image after the mask is masked
  • the visual processing model is called to classify the perturbation image to obtain the classification result, and the classification result is calculated to obtain the evaluation value of the perturbation image, and the evaluation value of the perturbation image is less than that of the unmasked image Masked evaluation value of the image to be processed;
  • the client sends a visualization processing request including the image to be processed to the server, so that the server can mask the image to be processed to obtain the masked perturbation image, and pass the trained
  • the visualization processing model classifies the perturbation image, obtains the classification result, and performs weighting calculation on the classification result to obtain the evaluation value of the perturbation image.
  • the evaluation value of the processed image is used to determine whether the mask area is a key area that affects the classification result, and then the visual evaluation result is determined according to the evaluation value of the perturbation image.
  • the visual evaluation result is the key area that affects the evaluation value.
  • an embodiment of the present application provides a visual processing device for evaluating addiction traits of the brain, including:
  • a transceiving unit configured to receive a visualization processing request from a client, where the visualization processing request includes a to-be-processed image, and the visualization processing request is used to request to obtain a visualization evaluation result of the to-be-processed image;
  • the processing unit is configured to perform mask masking processing on the image to be processed to obtain a perturbation image after the mask is masked; call a visualization processing model to classify the perturbation image to obtain a classification result, and classify the perturbation image Calculate the result to determine the visual evaluation result;
  • the transceiver unit is further configured to send the visual evaluation result to the client.
  • an embodiment of the present application provides a visual processing device for evaluating addiction traits of the brain, including a processor, a memory, and a communication interface.
  • the processor, the memory, and the communication interface are connected to each other, wherein the The memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method described in the first aspect.
  • the memory is used to store a computer program
  • the computer program includes program instructions
  • the processor is configured to call the program instructions to execute the method described in the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more first instructions, and the one or more first instructions are suitable for The processor loads and executes the method as described in the first aspect.
  • the client sends a visualization processing request to the server.
  • the visualization processing request includes the image to be processed.
  • the server performs masking processing on the image to be processed according to the visualization processing request to obtain the masked perturbation image.
  • Masking processing of the image to be processed can compare different areas to obtain the key areas that can affect the classification result; classify the perturbation image through the trained visualization processing model, and obtain the classification result, and the classification
  • the result is weighted and calculated to obtain the evaluation value of the perturbation image.
  • the evaluation value can be used to determine whether the mask area is a key area that affects the classification result.
  • the visual evaluation result is determined according to the evaluation value of the perturbation image.
  • the visual evaluation result is the key area that affects the evaluation value, and the visual evaluation result is sent to the client, where the training method of the visualization processing model To: Iteratively train at least a set of input sample images through the semi-supervised ternary generation confrontation network with independent classifiers, so that the generator generates images closer to the real FMRI images, so that the classifier extraction is related to nicotine addiction traits
  • the more discriminative features Through the method of this embodiment, random noise vectors can be converted into accurate FMRI images without a large number of FMRI images as basic samples, which solves the problem of difficulty in obtaining FMRI images, saves experimental costs, and promotes classification through model training
  • the device extracts more discriminative features related to nicotine addiction traits, and obtains more accurate classification results. It can locate the nicotine addiction brain area more intuitively and accurately, and realize the visualization of the evaluation results.
  • FIG. 1 is an architecture diagram of a visualization system for evaluating addiction traits of the brain provided by an embodiment of the present application
  • Fig. 2 is a flowchart of a visualization method for evaluating addiction traits of the brain provided by an embodiment of the present application;
  • FIG. 3 is a flowchart of another visualization method for evaluating addiction traits of the brain provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a network layer tensor decomposition provided by an embodiment of the present application.
  • Fig. 5 is a framework diagram of a visual processing model provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a classifier network provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a second-level pooling module provided by an embodiment of the present application.
  • Fig. 8 is a schematic structural diagram of a visual processing device for evaluating addiction traits of the brain provided by an embodiment of the present application.
  • Fig. 9 is a schematic structural diagram of another visual processing device for evaluating addiction traits of the brain provided by an embodiment of the present application.
  • FMRI can accurately locate specific cortical areas of the brain and capture changes in blood oxygen that reflect neuronal activity.
  • the combination of FMRI technology and machine learning technology has a broader application prospect in the field of biomedicine.
  • This application takes the evaluation of nicotine addiction traits in the brain of rats as an example.
  • the use of machine learning to study the relevant characteristics of brain nicotine addiction requires a large number of FMRI images as the basis for model training.
  • FMRI images can be seen as a time series composed of hundreds of three-dimensional brain anatomical structure images, that is, fourth-order images. , Including more than 100,000 different voxels (Voxel).
  • the embodiment of the present application provides a visualization method for evaluating the addiction traits of the brain.
  • the image processing method designs a semi-supervised ternary generative confrontation network with an independent classifier, and the ternary generative confrontation network includes a generator network , Discriminator network and classifier network.
  • the trait evaluation model of rat brain nicotine addiction built by the ternary generation confrontation network can generate realistic FMRI images from random noise, and generate visual evaluations with the help of mask processing methods. result. While reducing the number of FMRI image samples required, it can more intuitively and accurately locate the brain area of nicotine addiction.
  • the real FMRI image or the FMRI image generated by the generator can be masked.
  • the masking process includes: masking a random or designated area on the image with a mask so that the masked area does not participate in the calculation process , Use the classifier in the visualized processing model that has been trained to classify the real FMRI image processed by the mask or the FMRI image generated by the generator, and calculate the classification result, and determine whether the masked area is based on the calculation result It will affect the classification results. If yes, it is considered that the masked area is the brain area activated by nicotine addiction; if not, it is considered that the masked area is not the brain area activated by nicotine addiction.
  • this embodiment can also be applied to other fields, such as: visualization tasks for assisted diagnosis of other diseases based on medical images, visualization of critical lesion areas that have a great impact on the results of disease diagnosis, and so on. There is no restriction here.
  • the above-mentioned visualization method for evaluating brain addiction traits can be applied to the visualization system for evaluating brain addiction traits as shown in FIG. 1, and the visualization system for evaluating brain addiction traits may include a client 101 and a server 102.
  • the shape and quantity of the client 101 are used for example, and do not constitute a limitation to the embodiment of the present application. For example, two clients 101 may be included.
  • the client 101 may be a client that sends a visualization processing request to the server 102, or it may be used to provide the server 102 with the first sample image, the second sample data annotation pair, the noise vector, and the image processing model during the training of the image processing model.
  • the vector-labeled client can also be a client connected to the FMRI device.
  • the client can be any of the following: terminal, independent application, application programming interface (API) or software development tool Package (Software Development Kit, SDK).
  • the terminal may include, but is not limited to: smart phones (such as Android phones, IOS phones, etc.), tablet computers, portable personal computers, mobile Internet devices (Mobile Internet Devices, MID) and other devices, which are not limited in the embodiment of the present application.
  • the server 102 may include, but is not limited to, a cluster server.
  • the client 101 sends a visualization processing request to the server 102, and the server 102 obtains the visualization evaluation result of the to-be-processed image according to the to-be-processed image contained in the visualization processing request.
  • the to-be-processed image is The image is masked and processed to obtain the perturbation image after the mask is masked.
  • the perturbation image is classified through the pre-trained visualization processing model to obtain the classification result, and the classification result is calculated to determine the visualization evaluation result
  • Fig. 2 is a schematic flowchart of a visualization method for evaluating addiction traits of the brain provided by an embodiment of the present application.
  • the image processing method may include parts 201 to 206, in which:
  • the client 101 sends a visualization processing request to the server 102.
  • the client 101 sends a visualization processing request to the server 102.
  • the server 102 receives a visualization processing request from the client 101.
  • the visualization processing request includes the image to be processed, and the visualization processing request is used to request to obtain the image to be processed.
  • Visual evaluation results where the image to be processed is an FMRI image of the rat brain injected with different concentrations of nicotine. Specifically, it can be a real FMRI image, optionally, it can also be generated by a generator that has completed optimization training FMRI image. Further, if it is a real FMRI image, the server 102 may perform normalization preprocessing on the image to be processed, and the range of voxel values of the image to be processed after the normalization process may be [-1, 1].
  • the server 102 performs masking processing on the image to be processed to obtain a perturbation image after the mask is masked.
  • the server 102 performs masking processing on a designated or random area in the image to be processed to obtain a masked perturbation image.
  • the mask masking process can be understood as masking certain designated or random areas on the image with a mask, so that it does not participate in the processing or the calculation of processing parameters.
  • a set R of different mask areas can be determined, the set of mask areas includes at least one mask area, and the set of mask areas is a set of areas in the image to be processed that need to be masked.
  • mask masking is performed on each mask area in the mask area set R to obtain a perturbation image, that is, a scalar m(u) associated with each voxel u ⁇ in the FMRI image is masked.
  • the mask masking processing method may include, but is not limited to: replacing the mask area set R with a constant, adding noise to the mask area set R, and blurring the mask area set R, for different masks
  • the perturbation image obtained by the shielding method can be expressed as:
  • m: ⁇ [0,1] represents the mask
  • u 0 is the average voxel value
  • ⁇ (u) is the Gaussian noise sample of each voxel value
  • ⁇ 0 is the maximum isotropy of the Gaussian blur kernel g ⁇ standard deviation.
  • ⁇ 0 is usually 10
  • a relatively fuzzy mask can be obtained.
  • masking processing can be performed on different regions, so as to obtain key regions that can affect the classification result.
  • the server 102 calls the visualization processing model to classify the perturbation image, and obtains the classification result.
  • the classifier in the visualization processing model is called to classify the perturbation image, and the classification result is obtained.
  • the visual processing model is a model constructed by repeated iterative training of the generator network, the discriminator network, and the classifier network using the first sample image, the second sample image annotation pair, the noise vector, and the vector annotation.
  • the classifier can classify the brain anatomical structure features extracted from the FMRI image. For example: FMRI images can be classified into three categories: injection of high-concentration nicotine at 0.12 mg/kg, injection of low-concentration nicotine at 0.03 mg/kg, and injection of saline. Then after the perturbation image is input into the classifier, the possible classification result is 60% probability of 0.12mg/kg high concentration nicotine, 30% probability is 0.03mg/kg low concentration nicotine, 10% probability The probability is normal saline.
  • the server 102 calculates the classification result to obtain the evaluation value of the perturbation image.
  • the classification result is calculated.
  • the classifier outputs a weighted vector m* that belongs to the classification results of different concentrations of nicotine addiction traits.
  • the weighted vector may be output in the form of a normalized exponential function (softmax) probability for the last layer of the classifier network.
  • softmax normalized exponential function
  • the evaluation value of the perturbation image is obtained The evaluation value may be calculated by substituting the weight vector into a preset evaluation standard function to obtain the evaluation value.
  • the evaluation value of the perturbation image is smaller than the evaluation value of the image to be processed that has not been masked.
  • the image to be processed without masking processing can also be calculated by the above evaluation standard function to obtain the evaluation value f c (x 0 ) of the image to be processed without mask processing, where x 0 can represent the real FMRI image .
  • the visual evaluation result is determined, that is, the key area that affects the classification result is determined. Further, it can be determined whether the mask area is a key area that affects the classification result according to the obtained evaluation value. like It is considered that the masked area of the mask is a key area that affects the classification result; optionally, you can set the evaluation score difference threshold, that is, the evaluation score obtained after masking the mask area set R The difference between the evaluation score f c (x 0 ) of the original image to be processed without the mask is greater than the threshold, and the mask shielded area set R is considered to be a brain area activated by nicotine addiction.
  • the key area corresponding to the visual evaluation result is the learning objective function, which can be expressed as:
  • ⁇ means to encourage as many masks as possible to be closed, that is, the mask area to be masked is as accurate as possible to the key area instead of the entire FMRI image
  • c is the classification label, that is, the category of nicotine addiction traits in rats.
  • the classification result of the image to be processed can also be obtained, and the classification result can be weighted.
  • the classification result can be weighted.
  • the server 102 sends the visual evaluation result to the client 101.
  • the evaluation result based on the evaluation score may be sent to the client 101.
  • the client 101 receives the visual evaluation result.
  • the evaluation score and the corresponding mask area set R may be sent to the client 101, so that the operating user 103 of the client 101 determines the mask area set R based on the evaluation score and the corresponding mask area set R Whether to activate the brain area for nicotine addiction.
  • the server 102 performs mask masking processing on the image to be processed in the visualization processing request to obtain the masked perturbation image, here
  • the masking process of the image to be processed can compare different regions, so as to obtain the key regions that can affect the classification result.
  • the perturbation image is classified by the trained visualization processing model, the classification result is obtained, and the classification result is weighted to determine the visualization evaluation result, and the visualization evaluation result is sent to the client 101.
  • FIG. 3 is a schematic flowchart of a visualization method for evaluating addiction traits of the brain provided by an embodiment of the present application.
  • the visualization method for evaluating addiction traits of the brain may include parts 301 to 305. in:
  • the server 102 obtains a noise vector and a vector label.
  • the server 102 may obtain a noise vector from the client 101 or other data storage platforms, and label the vector matching the noise vector.
  • the noise vector is a one-dimensional random noise vector with Gaussian distribution, and the noise vector is used for input to the generator, so that the generator network generates a corresponding FMRI image according to the noise vector.
  • the vector is labeled as the classification label corresponding to the noise vector, for example: 0.12mg/kg of high-concentration nicotine, 0.03mg/kg of low-concentration nicotine, physiological saline, etc.
  • the vector is labeled with one-hot code (one The form of -hot) is input to the generator network along with the corresponding noise vector.
  • the server 102 processes the noise vector and the vector label through a deconvolution network to obtain a target image label pair.
  • the server 102 when the server 102 obtains the noise vector and the corresponding vector label, it inputs the noise vector and the corresponding vector label to the generator network composed of the quantized deconvolution layer, so that the generator network generates Corresponding target image annotation pair, the target image annotation pair includes the target generated image and the target generated image annotation, where the target generated image is the FMRI image generated by the generator, and the target generated image annotation can be understood as the one-hot encoding of the above vector annotation (one-hot) mode.
  • the generator network adopts a deep deconvolution neural network, including multiple quantized deconvolution layers.
  • the generated brain anatomical feature map is enlarged layer by layer to generate and real FMRI images of the same size.
  • each deconvolution layer except the last layer includes a deconvolution layer, a normalization layer (Batch Normalization) and an activation function layer (Leaky ReLU), and the last deconvolution layer includes a deconvolution layer And activation function layer (tanh).
  • the deconvolution layer undergoes parameter compression through a tensor decomposition (Tensor-Train) method, where the deconvolution layer
  • the convolution kernel tensor of can be expressed as the corresponding tensor decomposition (Tensor-Train) form, as shown in Figure 4, which is a schematic diagram of tensor decomposition (Tensor-Train)
  • the deconvolution layer can be performed according to the following formula break down:
  • the server 102 obtains a label pair of the first sample image and the second sample image.
  • the server 102 may obtain the first sample image and the second sample image annotation pair from the client 101 or other data storage platforms.
  • the second sample image annotation pair includes the second sample image and the sample image annotation.
  • Both the sample image and the second sample image are real FMRI images, and the sample image is annotated as a classification annotation corresponding to the second sample image.
  • the sample image annotation and the vector annotation in step 301 belong to one type of annotation.
  • the first sample image is used for input into the classifier network, so that the classifier network performs classification prediction on the first sample image to obtain the prediction label of the first sample image.
  • the second sample image annotation pair is used for the target image annotation pair generated by the generator, and the first sample image and the predicted annotation of the first sample image are input into the discriminator model to train the visualization processing model according to the discrimination result, or input Perform supervised training in the classifier to obtain cross entropy.
  • the server 102 trains a visualization processing model according to the target image annotation pair, the first sample image and the second sample image annotation pair, to obtain a model loss function.
  • the server 102 trains the visualization processing according to the target image tag pair, the first sample image and the second sample image tag pair Model to obtain a model loss function, so that a visual processing model can be further constructed based on the model loss function, that is, step 305 is executed.
  • the framework diagram of the visual processing model can be seen in FIG. 5.
  • the model is mainly based on a ternary generative confrontation network, and the ternary generative confrontation network includes a generator, a classifier, and a discriminator.
  • the training process mainly includes: inputting noise vectors and vector annotations into the generator to obtain FMRI image annotation pairs generated by the generator, which can also be described as target image annotation pairs in this application.
  • the real unlabeled FMRI image can also be described as the first sample image.
  • the real labeled FMRI image is obtained and the annotation pair is
  • the real FMRI image of the FMRI image is normalized and preprocessed.
  • the FMRI image annotation pair that is actually annotated can also be described as the second sample image annotation pair.
  • the server 102 can train the visualization processing model according to the target image tagging pair, the first sample image and the second sample image tagging pair.
  • the target image tagging pair generated by the generator is input into the discriminator to obtain the discrimination result, and the first sample image and second sample image annotation pair A discrimination result, and at the same time, based on the reconstruction loss between the first sample image, the second sample image and the target generated image input to the classifier, together constitute the loss function of the generator, and through the backpropagation algorithm, according to the generated
  • the generator's loss function gradient descent updates the generator network layer tensor decomposition kernel matrix parameters; input the first sample image into the classifier to obtain the predicted label, and input the first sample image and the predicted label into the discriminator Perform the discrimination and obtain the second discrimination result.
  • the loss function of the classifier uses the backpropagation algorithm to update the classifier network layer tensor decomposition kernel matrix parameters according to the gradient descent of the generated classifier’s loss function; the first sample image and the prediction of the first sample image are labeled, The second sample image annotation pair and the target image annotation generated by the generator network are input to the discriminator to determine the loss function of the discriminator, and the back propagation algorithm is used to update the discrimination according to the gradient descent of the loss function of the generated discriminator
  • the parameters of the kernel matrix of the tensor decomposition of the processor network layer uses the backpropagation algorithm to update the classifier network layer tensor decomposition kernel matrix parameters according to the gradient descent of the generated classifier’s loss function; the first sample image and the prediction of the first sample image are labeled, The second sample image annotation pair and the target image annotation generated by the generator network are input to the discriminator to determine the loss function of the discriminator, and the back propagation algorithm is used to update the discrimination according to the gradient descent of the loss function
  • the aforementioned model loss function includes a generation loss function, and the generation loss function is the loss function of the generator. Then, the visual processing model is trained according to the target image label pair, the first sample image and the second sample image label pair, and the model loss function is obtained. The target image label pair can be discriminated and the first discrimination result is generated. The target image label Annotate the target generated image and the target generated image. According to the target generated image and the second sample image, the reconstruction loss is determined, and the generation loss function of the generator is determined according to the first discrimination result and the reconstruction loss.
  • the loss function of the generator includes two parts: one part is to label the generated target image and input it to the discriminator to perform discrimination processing, so that the discrimination result tends to be true loss; the other part generates the image and The reconstruction loss between real FMRI images, where the real FMRI images are the first sample image and the second sample image. Then the loss function of the generator can be expressed as:
  • the loss function of the generator can be determined from two aspects, so that the constructed visualization model is more accurate, and by constructing the generator model, the random noise vector can be converted into an accurate FMRI image, which solves the problem of FMRI image acquisition. Difficult problems save the cost of the experiment.
  • the aforementioned model loss function includes a classification loss function
  • the classification loss function is the loss function of the classifier.
  • the visual processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain the model loss function, which can be used to classify the first sample image to obtain the predicted label of the first sample image , And discriminate the first sample image and the predicted label, generate the second judgment result, perform supervisory training on the target image label pair and the second sample image label pair, and obtain the first cross entropy and the second sample label of the target image respectively
  • the second cross entropy of the image annotation pair determines the classification loss function of the classifier according to the second discrimination result, the first cross entropy and the second cross entropy.
  • the loss function of the classifier includes two parts: one part is the cross-entropy loss function obtained by supervising the target image annotation pair and the second sample image annotation pair; the other part is the first sample image and its The predicted label obtained after classification processing is input to the discriminator for discrimination processing, so that the discrimination result tends to be true unsupervised loss.
  • the loss function of the classifier can be expressed as:
  • the first cross entropy is R p
  • the second cross entropy is R L
  • L supervised is the supervision loss
  • L unsupervised is the semi-supervised loss
  • the second cross entropy RL is equivalent to calculating the relative entropy (KL divergence) between the distribution P c (x, y) learned by the classifier and the real data distribution P real (x, y).
  • Minimizing R p is equivalent to minimizing relative entropy (KL divergence) D KL (P g (x,y)
  • This classifier model achieves the goal of minimizing the relative entropy (KL divergence) D KL (P g (x,y)
  • the classifier network includes a quantized convolutional layer, an average pooling layer, a quantized dense connection block, a second-order pooling module, and a transition layer.
  • the FMRI image is composed of
  • the relevant schematic diagram of the tensor decomposition can be found in As shown in FIG. 4 in step 302, the step of decomposing the tensor of the convolutional layer is the same as the step of decomposing the tensor of the deconvolutional layer. You can refer to the related description in step 302, which will not be repeated here.
  • the weight tensor W of the fully connected layer can also be decomposed according to the following formula:
  • the second-order pooling module is deployed after the Zhang quantization densely connected block.
  • the schematic diagram of the module is shown in Figure 7.
  • the module includes two parts: a compression module and a calibration module.
  • the second-order pooling reduces the dimensionality of the input 4-dimensional feature map through 1x1x1 convolution, calculates the covariance information between different channels in the 4-dimensional feature map after dimensionality reduction, and obtains the covariance matrix.
  • group convolution Convolve with 1x1x1 to obtain a weight vector with the same number of channels as the 4-dimensional feature map, and calculate the inner product of the weight vector and the input feature map to obtain the weighted output feature map.
  • the backpropagation algorithm is used to make the important channels of the feature map have a large weight, and the unimportant channels have a small weight, so as to extract a more representative global high-order feature map and improve the brain nicotine addiction. Accuracy rate of character evaluation.
  • the generator and the classifier promote each other and jointly learn the potential high-dimensional probability distribution of the FMRI image.
  • the classifier model is constructed based on the second-order pooling module, which can extract more discriminative features related to nicotine addiction traits in the brain through the dependence of different areas of the FMRI image and the correlation information between high-order features and different channels. Improve the accuracy of the evaluation of brain nicotine addiction traits, so that the classifier model can be applied to the mask-based visual evaluation, that is, steps 201-206.
  • the aforementioned model loss function includes a discriminant loss function, and the discriminant loss function is the loss function of the discriminator.
  • the visual processing model is trained according to the target image label pair, the first sample image and the second sample image label pair, and the model loss function is obtained.
  • the target image label pair can be discriminated and the third discrimination result is generated.
  • This image and the predicted label are subjected to discrimination processing to generate the fourth discrimination result
  • the second sample image label pair is subjected to discrimination processing to generate the fifth discrimination result.
  • the discrimination is determined
  • the discriminative loss function of the device Among them, the first sample image and the second sample image are input in the form of a fourth-order tensor, and the prediction annotation and the image annotation of the second sample image are input in the form of one-hot encoding.
  • the loss function of the discriminator includes three parts: the first part is to perform the discrimination processing on the target image labeling pair, and obtain the loss that makes the discrimination result tend to be false; the second part is the prediction of the first sample image and its corresponding The label performs the discrimination process, and the loss that makes the discrimination result tends to be false is obtained; the third part is the discrimination process for the second sample image labeling pair, and the loss that makes the discrimination result tend to be true is obtained.
  • the loss function of the discriminator can be expressed as:
  • the discriminator network uses a dense deep neural network for feature extraction.
  • the number of layers of the dense deep neural network can be 30 layers, consisting of a quantized convolutional layer, a quantized dense connection block, and a quantized transition.
  • Layer and Zhang Quanquan fully connected layer composition are used to determine the above-mentioned target image annotation pair, The true or false of the first sample image and its corresponding prediction annotation, and the second sample image annotation pair.
  • Both the convolution kernel tensor of the above-mentioned quantized convolution layer and the weight matrix of the fully connected layer can be expressed in the form of corresponding tensor decomposition (Tensor-Train).
  • the relevant schematic diagram of the tensor decomposition (Tensor-Train) can be seen above Step 302 is shown in FIG. 4.
  • the relevant description of the tensor decomposition of the fully connected layer and the convolutional layer please refer to the corresponding part of the above step 302 and the description of the classifier network, which will not be repeated here.
  • the target image annotation pair, the first sample image and its corresponding prediction annotation, and the second sample image annotation pair are input to the discriminator and undergo feature extraction of each module to obtain a feature map of the rat brain that retains spatial information and time series information , And the last layer of quantization fully connected layer judges the authenticity of each group of image annotation pairs, and outputs the corresponding judgment results.
  • the discriminator can discriminate the data output by the generator and the classifier.
  • the generator, classifier, and discriminator jointly form a ternary generation confrontation network, so that the generator can generate an image closer to the real FMRI image.
  • the classifier can extract more discriminative features related to nicotine addiction traits to obtain more accurate classification results.
  • the server 102 constructs a visual processing model according to the model loss function.
  • a visualization processing model is constructed according to the model loss function.
  • the parameters of the generator can be updated according to the loss function of the generator through the backpropagation algorithm
  • the parameters of the classifier can be updated according to the loss function of the classifier
  • the discriminator can be updated according to the loss function of the discriminator. If the parameters of is updated, the visual processing model can be constructed according to the parameters of the generator, the parameters of the classifier, and the parameters of the discriminator.
  • the generator network layer tensor decomposition kernel matrix G k [i k ,j k ] parameters can be updated according to the loss function gradient descent of the generator, and the classifier network layer tensor decomposition kernel can be updated according to the loss function gradient descent of the classifier.
  • matrix G k [i k, j k ] parameters based on the loss of function of the gradient discriminator discriminating layer decreases updated network tensor exploded nuclear matrix G k [i k, j k ] parameters.
  • backpropagation is used to solve the gradient of the loss function to the kernel matrix G k [i k ,j k ].
  • the generator, classifier and discriminator are continuously optimized. Make the target generated image generated by the generator more in line with the distribution of real FMRI image data.
  • the classifier can more accurately distinguish the boundaries between different analogies in the real distribution, and feed back the FMRI image labeling pair to the discriminator, so that the discriminatory performance of the discriminator can be further improved.
  • the entire ternary generative confrontation network model reaches the Nash equilibrium, and an optimized visualization processing model is obtained.
  • the process of training the visualization processing model can be divided into three processes: training, verification, and testing.
  • the sample image data can be divided according to a certain proportion to obtain training set samples, validation set samples, and test set samples of different proportions. For example: divide the sample at a ratio of 80%:10%:10%.
  • the training process can refer to the implementation in steps 301-305 above.
  • the validation set samples are used to verify the trained visual evaluation model, and based on the verification results, the optimal visual evaluation model is selected to obtain the optimal visual evaluation model.
  • the test set is used for input to the classifier of the optimized visual evaluation model, and the mask-based method obtains the evaluation results of nicotine addiction traits, namely steps 201-206, thereby realizing the visualization of the brain regions activated by nicotine addiction.
  • the server 102 obtains the noise vector and the vector label, and processes the noise vector and the vector label through the deconvolution network to obtain the target image label pair. Then the visualization processing model can be trained according to the target image label pair and the obtained first sample image and second sample image label pair, and the model loss function including the generation loss function, the classification loss function and the discriminative loss function can be obtained, and according to the model Loss function, build a visual processing model.
  • the random noise vector can be converted into an accurate FMRI image, which solves the problem of difficulty in obtaining FMRI images and saves the cost of the experiment.
  • the classifier can also promote the classifier to extract more discriminative features related to nicotine addiction traits, and obtain more accurate classification results, so that the trained and optimized classifier can be used to obtain and mask different mask regions in the FMRI image. Later, the changes in the evaluation results of nicotine addiction characteristics caused by nicotine addiction can more intuitively and accurately locate the brain areas of nicotine addiction, and realize the visualization of the evaluation results.
  • the embodiment of the present application also proposes a visual processing device for evaluating brain addiction traits.
  • the visual processing device for evaluating brain addiction traits can be a computer program (including program code) running in a processing device; please refer to Figure 8, the visual processing device for evaluating brain addiction traits can run the following units:
  • the transceiver unit 801 is configured to receive a visualization processing request from a client, where the visualization processing request includes an image to be processed, and the visualization processing request is used to request to obtain a visualization evaluation result of the image to be processed;
  • the processing unit 802 is configured to perform mask masking processing on the image to be processed to obtain a masked perturbation image; call a visualization processing model to classify the perturbation image to obtain a classification result, and perform a classification process on the perturbation image.
  • the classification result is calculated to obtain the evaluation value of the perturbation image, and the evaluation value of the perturbation image is smaller than the evaluation value of the image to be processed that has not been masked; according to the evaluation value of the perturbation image, Determine the visual evaluation result;
  • the transceiver unit 801 is further configured to send the visual evaluation result to the client.
  • the masking processing includes obfuscation processing
  • the processing unit 802 may also be used to determine a set of mask areas, where the set of mask areas includes at least one mask area;
  • the processing unit 802 may also be used to obtain a noise vector and vector label, and pass the noise vector and the vector label Deconvolution network processing to obtain a target image annotation pair, where the target image annotation pair includes a target generated image and a target generated image annotation;
  • the visualization processing model is constructed.
  • the model loss function includes a generation loss function, and the generation loss function is a loss function of a generator;
  • the visualization processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain a model loss function, and the processing unit 802 may also be used to perform an analysis on the target image Performing discrimination processing on an annotation pair to generate a first discrimination result, where the target image annotation pair includes the target generated image and the target generated image annotation;
  • the generation loss function of the generator is determined.
  • the model loss function includes a classification loss function, and the classification loss function is a loss function of a classifier;
  • the visualization processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain the model loss function, and the processing unit 802 can also be used to Performing classification processing on the sample image to obtain the prediction label of the first sample image, and performing discrimination processing on the first sample image and the prediction label to generate a second discrimination result;
  • the model loss function includes a discriminant loss function, and the discriminant loss function is a loss function of a discriminator;
  • the visualization processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain a model loss function, and the processing unit 802 may also be used to perform an analysis on the target image
  • the label pair is subjected to discrimination processing, and the third discrimination result is generated;
  • the model loss function includes the generation loss function, the classification loss function, and the discriminative loss function
  • the processing unit 802 may also be used for
  • the visualization processing model is constructed according to the parameters of the generator, the parameters of the classifier, and the parameters of the discriminator.
  • part of the steps involved in the visualization method for evaluating brain addiction traits shown in FIG. 2 and FIG. 3 can be executed by the processing unit in the visualization processing device for evaluating brain addiction traits.
  • steps 201 and 206 shown in FIG. 2 may be executed by the transceiver unit 801; for another example, step 202 shown in FIG. 2 may be executed by the processing unit 802.
  • each unit in the visual processing device for evaluating addiction traits of the brain can be separately or completely combined into one or several other units to form, or some of the units can be further It is divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • FIG. 9 is a schematic structural diagram of a visual processing device for evaluating addiction traits of the brain provided by an embodiment of the present application.
  • the visual processing device for evaluating addiction traits of the brain includes a processor 901, a memory 902, and a communication interface 903.
  • the processor 901, the memory 902, and the communication interface 903 are connected by at least one communication bus, and the processor 901 is configured to support the processing device to perform the corresponding functions of the processing device in the methods of FIG. 2 and FIG. 3.
  • the memory 902 is configured to store at least one instruction suitable for being loaded and executed by the processor, and these instructions may be one or more computer programs (including program codes).
  • the communication interface 903 is used for receiving data and for sending data.
  • the communication interface 903 is used to send visualization processing requests and the like.
  • the processor 901 may call the program code stored in the memory 902 to perform the following operations:
  • the visualization processing request includes a to-be-processed image
  • the visualization processing request is used to request to obtain a visualization evaluation result of the to-be-processed image
  • Masking processing is performed on the image to be processed to obtain a perturbation image after the mask is masked
  • the visual processing model is called to classify the perturbation image to obtain the classification result, and the classification result is calculated to obtain the evaluation value of the perturbation image, and the evaluation value of the perturbation image is less than that of the unmasked image Masked evaluation value of the image to be processed;
  • the masking processing includes fuzzification processing
  • Said masking processing is performed on the image to be processed to obtain a perturbed image after the mask is masked, and the processor 901 may call the program code stored in the memory 902 to perform the following operations:
  • the processor 901 may invoke the program code stored in the memory 902 to perform the following operations:
  • the visualization processing model is constructed.
  • the model loss function includes a generation loss function, and the generation loss function is a loss function of a generator;
  • the visualization processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain a model loss function, and the processor 901 may call a program stored in the memory 902 Code to do the following:
  • the target image tag pair includes the target generated image and the target generated image tag
  • the generation loss function of the generator is determined.
  • the model loss function includes a classification loss function, and the classification loss function is a loss function of a classifier;
  • the visualization processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain a model loss function, and the processor 901 may call a program stored in the memory 902 Code to do the following:
  • the model loss function includes a discriminant loss function, and the discriminant loss function is a loss function of the discriminator;
  • the visualization processing model is trained according to the target image annotation pair, the first sample image and the second sample image annotation pair to obtain a model loss function, and the processor 901 can call a program stored in the memory 902 Code to do the following:
  • the model loss function includes the generation loss function, the classification loss function, and the discrimination loss function
  • the visual processing model is constructed according to the model loss function, and the processor 901 can call the program code stored in the memory 902 to perform the following operations:
  • the visualization processing model is constructed according to the parameters of the generator, the parameters of the classifier, and the parameters of the discriminator.
  • the embodiment of the present application also provides a computer-readable storage medium (Memory), which can be used to store the computer software instructions used by the processing device in the embodiment shown in FIG. 2 and FIG.
  • these instructions may be one or more computer programs (including program codes).
  • the above-mentioned computer-readable storage medium includes, but is not limited to, flash memory, hard disk, and solid-state hard disk.
  • the above-mentioned embodiments it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions can be stored in a computer-readable storage medium or transmitted through a computer-readable storage medium.
  • Computer instructions can be sent from one website site, computer, server, or data center to another website site, computer via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) , Server or data center for transmission.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Probability & Statistics with Applications (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

一种大脑成瘾性状评估的可视化方法、装置及介质。该可视化方法包括：接收客户端的可视化处理请求（201），可视化处理请求包括待处理图像；对待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像（202）；调用可视化处理模型对微扰图像进行分类处理，得到分类结果（203）；对分类结果进行计算，得到微扰图像的评估数值（204），微扰图像的评估数值小于未经过掩膜屏蔽处理的待处理图像的评估数值；根据微扰图像的评估数值，确定可视化评估结果（205）；发送可视化评估结果至客户端。采用该可视化方法，在降低FMRI图像样本需求数量的同时，可以较为直观、准确的定位尼古丁成瘾脑区，实现评估结果的可视化。

Description

大脑成瘾性状评估的可视化方法、装置及介质 技术领域
本申请涉及大数据技术领域，尤其涉及一种大脑成瘾性状评估的可视化方法、装置及介质。
背景技术
功能性磁共振成像(functional magnetic resonance imaging，FMRI)是一种神经影像学方式，能够对特定的大脑活动皮层区域进行准确定位，并捕获能够反映神经元活动的血氧变化。将FMRI与深度学习技术相结合可以从原始数据中提取出复杂的特征，但是，该特征提取方式的解释性差，且需要大量的FMRI图像作为基础。由于FMRI图像采集过程复杂、实验成本高，导致了FMRI图像获取困难，进而限制了深度学习方法对FMRI图像评估和可视化领域的研究。
发明内容
本申请实施例提供一种大脑成瘾性状评估的可视化方法、装置及计算机可读存储介质，在降低了FMRI图像样本需求数量的同时，可以更直观、准确的定位尼古丁成瘾脑区，实现了评估结果的可视化。
第一方面，本申请实施例提供一种大脑成瘾性状评估的可视化方法，包括：
接收客户端的可视化处理请求，所述可视化处理请求包括待处理图像，所述可视化处理请求用于请求获取所述待处理图像的可视化评估结果；
对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像；
调用可视化处理模型对所述微扰图像进行分类处理，得到分类结果，并对所述分类结果进行计算，得到所述微扰图像的评估数值，所述微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值；
根据所述微扰图像的评估数值，确定所述可视化评估结果；
发送所述可视化评估结果至所述客户端。
在该技术方案中，客户端发送包括待处理图像的可视化处理请求后至服务器，以使服务器对该待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像，通过已训练好的可视化处理模型对微扰图像进行分类，得到的分类结果，并对该分类结果进行加权计算，得到微扰图像的评估数值，该微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值，该评估数值用于 确定掩膜区域是否为影响分类结果的关键区域，则根据微扰图像的评估数值，确定可视化评估结果，该可视化评估结果为影响评估数值的关键区域，将该可视化评估结果发送至客户端。通过这种方法，无需大量的FMRI图像作为基础样本，仍然可以准确定位尼古丁成瘾激活区域，从而实现评估结果的可视化。
第二方面，本申请实施例提供一种大脑成瘾性状评估的可视化处理装置，包括：
收发单元，用于接收客户端的可视化处理请求，所述可视化处理请求包括待处理图像，所述可视化处理请求用于请求获取所述待处理图像的可视化评估结果；
处理单元，用于对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像；调用可视化处理模型对所述微扰图像进行分类处理，得到分类结果，并对所述分类结果进行计算，确定所述可视化评估结果；
所述收发单元，还用于发送所述可视化评估结果至所述客户端。
第三方面，本申请实施例提供一种大脑成瘾性状评估的可视化处理装置，包括处理器、存储器和通信接口，所述处理器、所述存储器和所述通信接口相互连接，其中，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行如第一方面所描述的方法。该处理设备解决问题的实施方式以及有益效果可以参见上述第一方面所描述的方法以及有益效果，重复之处不再赘述。
第四方面，本申请实施例提供一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有一条或多条第一指令，所述一条或多条第一指令适于由处理器加载并执行如第一方面所描述的方法。
本申请实施例中，客户端发送可视化处理请求至服务器，该可视化处理请求包括待处理图像，服务器根据该可视化处理请求对待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像，此处对待处理图像进行掩膜屏蔽处理可以对不同区域进行对比，以便得到可以影响分类结果的关键区域；通过已训练好的可视化处理模型对微扰图像进行分类，得到的分类结果，并对该分类结果进行加权计算，得到微扰图像的评估数值，该评估数值可以用于确定掩膜区域是否为影响分类结果的关键区域，微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值，则根据微扰图像的评估数值，确定可视化评估结果，该可视化评估结果为影响评估数值的关键区域，并将该可视化评估结果 发送至客户端，其中，可视化处理模型的训练方法为：通过带有独立分类器的半监督三元生成对抗网络对输入的至少一组样本图像进行迭代训练，使得生成器生成更接近真实FMRI图像的图像，使得分类器提取与尼古丁成瘾性状相关的更有判别性的特征。通过本实施例的方法，可以将随机噪声向量转化为准确的FMRI图像，无需大量的FMRI图像作为基础样本，解决了FMRI图像获取困难问题，节约了实验的成本，并且，通过模型训练可以促进分类器提取出与尼古丁成瘾性状相关的更有判别性的特征，得到更准确的分类结果，可以更直观、准确的定位尼古丁成瘾脑区，实现了评估结果的可视化。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种大脑成瘾性状评估的可视化系统的架构图；
图2是本申请实施例提供的一种大脑成瘾性状评估的可视化方法的流程图；
图3是本申请实施例提供的另一种大脑成瘾性状评估的可视化方法的流程图；
图4是本申请实施例提供的一种网络层张量分解的示意图；
图5是本申请实施例提供的一种可视化处理模型的框架图；
图6是本申请实施例提供的一种分类器网络的结构示意图；
图7是本申请实施例提供的一种二阶池化模块的结构示意图；
图8是本申请实施例提供的一种大脑成瘾性状评估的可视化处理装置的结构示意图；
图9是本申请实施例提供的另一种大脑成瘾性状评估的可视化处理装置的结构示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施例的技术方案进行描述。
为了清楚地描述本申请实施例的方案，下面将结合本申请实施例中的附图， 对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”等是用于区别不同对象，而非用于描述特定顺序。此外，术语“包括”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或模块的过程、方法、系统、产品或装置没有限定于已列出的步骤或模块，而是可选地还包括没有列出的步骤或模块，或可选地还包括对于这些过程、方法、产品或装置固有的其它步骤或模块。
FMRI作为一种非介入的技术，能够对特定的大脑活动皮层区域进行准确的定位，且能够捕获反映神经元活动的血氧变化。将FMRI技术与机器学习技术相结合，在生物医学领域的应用前景更为广泛，本申请以对大鼠脑部尼古丁成瘾性状的评估为例。目前，利用机器学习研究大脑尼古丁成瘾的相关特性，需要以大量的FMRI图像作为模型训练基础，其中，FMRI图像可以看成上百个三维脑部解剖结构图像组成的时间序列，即四阶图像，包含100000以上不同的体素(Voxel)。但是，FMRI图像采集过程复杂，仪器价格昂贵，实验成本高，需要较长的获取时间，导致FMRI图像的获取比较困难，实验基础样本数据不足。另一方面，机器学习的过程及结果的可解释性差，不能直观、准确、可视化的呈现对大脑尼古丁成瘾性状评估的结果。
为解决上述问题，本申请实施例提供一种大脑成瘾性状评估的可视化方法，该图像处理方法设计带有独立分类器的半监督三元生成对抗网络，该三元生成对抗网络包括生成器网络、判别器网络及分类器网络，通过该三元生成对抗网络构建的大鼠脑部尼古丁成瘾的性状评估模型，可以从随机噪声生成逼真的FMRI图像，并借助掩膜处理方法，生成可视化评估结果。在降低了FMRI图像样本需求数量的同时，可以更直观、准确的定位尼古丁成瘾脑区。
具体的，可以对真实的FMRI图像或生成器生成的FMRI图像经过掩膜处理，其中，掩膜处理过程包括：用掩膜对图像上随机或指定区域进行屏蔽，使该屏蔽区域不参与计算处理，利用已训练完成的可视化处理模型中的分类器对通过掩膜处理后的真实的FMRI图像或生成器生成的FMRI图像进行分类，并对分类结果进行计算，根据计算结果，判定屏蔽的区域是否对分类结果产生影响，若 是，则认为屏蔽的区域为尼古丁成瘾激活脑区；若否，则认为屏蔽的区域非尼古丁成瘾激活脑区。
可选的，本实施方式也可以应用于其他领域，例如：基于医学影像的其它疾病辅助诊断的可视化任务，对疾病诊断结果影响很大的关键性病变区域进行可视化处理，等等。此处不做限制。
上述提及的大脑成瘾性状评估的可视化方法可应用于如图1所示的大脑成瘾性状评估的可视化系统中，该大脑成瘾性状评估的可视化系统可包括客户端101及服务器102。该客户端101的形态和数量用于举例，并不构成对本申请实施例的限定。例如，可以包括两个客户端101。
其中，客户端101可以为向服务器102发送可视化处理请求的客户端，也可以为在图像处理模型训练时，用于为服务器102提供第一样本图像、第二样本数据标注对、噪声向量及向量标注的客户端，也可以为与FMRI设备相连接的客户端，该客户端可以为以下任一种：终端、独立的应用程序、应用程序编程接口(Application Programming Interface，API)或者软件开发工具包(Software Development Kit，SDK)。其中，终端可以包括但不限于：智能手机(如Android手机、IOS手机等)、平板电脑、便携式个人计算机、移动互联网设备(Mobile Internet Devices，MID)等设备，本申请实施例不做限定。服务器102可以包括但不限于集群服务器。
在本申请的实施例中，客户端101向服务器102发送可视化处理请求，服务器102根据该可视化处理请求所包含的待处理图像，获取该待处理图像的可视化评估结果，具体的，对该待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像，通过预先训练好的可视化处理模型对该微扰图像进行分类处理，得到分类结果，并对分类结果进行计算，以确定可视化评估结果，将该可视化评估结果发送至客户端101，以使客户端101的操作用户103可以根据该可视化评估结果对尼古丁成瘾激活脑区进行准确的定位。
请参见图2，图2是本申请实施例提供的一种大脑成瘾性状评估的可视化方法的流程示意图，如图2所示，该图像处理方法可以包括201～206部分，其中：
201、客户端101发送可视化处理请求至服务器102。
具体的，客户端101发送可视化处理请求至服务器102，相应的，服务器102 接收来自客户端101的可视化处理请求，该可视化处理请求包括待处理图像，该可视化处理请求用于请求获取待处理图像的可视化评估结果，其中，该待处理图像为注射了不同浓度尼古丁的大鼠脑部的FMRI图像，具体的，可以为真实的FMRI图像，可选的，也可以为完成优化训练的生成器生成的FMRI图像。进一步的，若为真实的FMRI图像，则服务器102可以对该待处理图像进行归一化预处理，归一化处理后的待处理图像的体素值的范围可以为[-1,1]。
202、服务器102对待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像。
具体的，服务器102对待处理图像中指定或随机区域进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像。该掩膜屏蔽处理可以理解为用掩模对图像上某些指定或随机区域进行屏蔽,使其不参加处理或不参加处理参数的计算。
进一步的，可以确定不同掩膜区域的集合R，该掩膜区域集合包括至少一个掩膜区域，该掩膜区域集合为待处理图像中的需要进行掩膜屏蔽处理的各个区域的集合，从待处理图像中对掩膜区域集合R中各个掩膜区域进行掩膜屏蔽处理，得到微扰图像，即对FMRI图像中每个体素u∈∧关联的一个标量m(u)进行掩膜屏蔽操作。可选的，该掩膜屏蔽处理方法可以包括但不限于：将掩膜区域集合R替换为常数，在掩膜区域集合R中加入噪声以及将掩膜区域集合R模糊化，针对不同的掩膜屏蔽方法得到的微扰图像可以表示为：
Figure PCTCN2020078694-appb-000001
其中，m:∧→[0,1]表示掩膜，u 0为平均体素值，η(u)为每个体素值的高斯噪声样本，σ 0为高斯模糊核g σ的最大等向性标准偏差。可选的，通常σ 0取10时，可以得到较为模糊的掩膜。
通过执行本实施方式，可以对不同区域进行掩膜屏蔽处理，以便得到可以影响分类结果的关键区域。
203、服务器102调用可视化处理模型对微扰图像进行分类处理，得到分类结果。
具体的，在得到微扰图像的情况下，调用可视化处理模型中的分类器对微扰图像进行分类处理，得到分类结果。该可视化处理模型为生成器网络、判别 器网络及分类器网络利用第一样本图像、第二样本图像标注对、噪声向量及向量标注进行反复迭代训练构建的模型。其中，分类器可以对从FMRI图像中提取到的脑部解剖结构特征进行分类。例如：可以将FMRI图像分类三类：注射0.12mg/kg的高浓度尼古丁、注射0.03mg/kg的低浓度尼古丁及注射生理盐水。则在将微扰图像输入至分类器中后，可能得到的分类结果为60％的概率为0.12mg/kg的高浓度尼古丁，30％的概率为0.03mg/kg的低浓度尼古丁，10％的概率为生理盐水。
204、服务器102对分类结果进行计算，得到微扰图像的评估数值。
具体的，在得到分类结果的情况下，对分类结果进行计算。分类器输出属于不同浓度尼古丁成瘾性状的分类结果的加权向量m *，该加权向量可以为分类器网络中最后一层网络以归一化指数函数(softmax)的概率形式输出。通过该加权向量计算结果，得到所述微扰图像的评估数值
Figure PCTCN2020078694-appb-000002
该评估数值可以为将加权向量代入预设的评估标准函数中计算，得到评估数值，该微扰图像的评估数值小于未经过掩膜屏蔽处理的待处理图像的评估数值。则未经过掩膜屏蔽处理的待处理图像也可以通过上述评估标准函数的计算，得到未经过掩膜处理的待处理图像的评估数值f c(x 0)，其中x 0可以表示真实的FMRI图像。
205、根据微扰图像的评估数值，确定可视化评估结果。
具体的，在获取到微扰图像的评估数值后，确定可视化评估结果，即确定影响分类结果的关键区域。进一步的，可以根据获取到的评估数值，确定掩膜区域是否为影响分类结果的关键区域。若
Figure PCTCN2020078694-appb-000003
则认为该掩膜屏蔽区域是影响分类结果的关键区域；可选的，可以设置评估分数相差阈值，即屏蔽掩膜区域集合R后得到的评估分数
Figure PCTCN2020078694-appb-000004
与没有加入掩膜的原始待处理图像的评估分数f c(x 0)的差值大于阈值，则认为该掩膜屏蔽区域集合R是尼古丁成瘾激活脑区。其中，可视化评估结果所对应的关键区域即为学习目标函数，该目标函数可以表示为：
Figure PCTCN2020078694-appb-000005
其中，λ表示鼓励尽可能多的掩膜处于关闭状态，即将屏蔽的掩膜区域尽量精确到关键性的区域，而非整个FMRI图像，c是分类标注，即大鼠尼古丁成瘾性状的类别。
可选的，还可以获取待处理图像的分类结果，并对该分类结果进行加权计 算。通过该可选的实施方式，便于将微扰图像的分类及计算结果与待处理图像的分类及计算结果进行比较，以定位对分类结果产生明显影响的掩膜屏蔽区域，则该区域即为影响分类结果的关键区域。
206、服务器102发送可视化评估结果至客户端101。
具体的，在得到上述的评估分数的情况下，可以将基于该评估分数的评估结果发送至客户端101。相应的，客户端101接收该可视化评估结果。可选的，可以将评估分数及相应的掩膜区域集合R发送至客户端101，以使客户端101的操作用户103基于该评估分数及相应的掩膜区域集合R确定该掩膜区域集合R是否为尼古丁成瘾激活脑区。
可见，通过实施图2所描述的方法，客户端101在发送可视化处理请求后，服务器102对可视化处理请求中的待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像，此处对待处理图像进行掩膜屏蔽处理可以对不同区域进行对比，以便得到可以影响分类结果的关键区域。通过已训练好的可视化处理模型对微扰图像进行分类，得到的分类结果，并对该分类结果进行加权计算，以确定可视化评估结果，并将该可视化评估结果发送至客户端101。通过本实施例的方法，无需大量的FMRI图像作为基础样本，仍然可以准确定位对大鼠脑部成瘾性状评估结果影响最大的尼古丁成瘾激活区域，从而实现大鼠尼古丁成瘾脑区的可视化。
请参见图3，图3是本申请实施例提供的一种大脑成瘾性状评估的可视化方法的流程示意图，如图3所示，该大脑成瘾性状评估的可视化方法可以包括301～305部分，其中：
301、服务器102获取噪声向量及向量标注。
具体的，服务器102可以从客户端101或其他数据储存平台获取噪声向量，及与该噪声向量相匹配的向量标注。其中，噪声向量为具有高斯分布的一维随机噪声向量，该噪声向量用于输入到生成器中，以使生成器网络根据该噪声向量生成相应的FMRI图像。相应的，向量标注为该噪声向量所对应的分类标注，例如：0.12mg/kg的高浓度尼古丁、0.03mg/kg的低浓度尼古丁、生理盐水，等等，该向量标注以独热编码(one-hot)的形式随着相应的噪声向量输入至生成器网络中。
302、服务器102将噪声向量及向量标注通过反卷积网络处理，得到目标图 像标注对。
具体的，服务器102在获取到噪声向量及相应的向量标注的情况下，将该噪声向量及相应的向量标注输入至由张量化反卷积层组成的生成器网络中，以使生成器网络生成相应的目标图像标注对，该目标图像标注对包括目标生成图像及目标生成图像标注，其中，目标生成图像为由生成器生成的FMRI图像，目标生成图像标注可以理解为上述向量标注的独热编码(one-hot)模式。
具体的，该生成器网络采用深度反卷积神经网络，包括多个张量化反卷积层，上述噪声向量经过多层反卷积后将生成的脑部解剖特征图逐层放大，生成与真实FMRI图像尺寸大小相同的图像。其中，除最后一层外的每个反卷积层均包括反卷积层、归一化层(Batch Normalization)及激活函数层(Leaky ReLU)，最后一层反卷积层包括反卷积层及激活函数层(tanh)。
进一步的，本申请对上述深度反卷积神经网络中的反卷积层进行了张量化改进，该反卷积层经过张量分解(Tensor-Train)方法进行参数压缩，其中，反卷积层的卷积核张量可以表示为对应的张量分解(Tensor-Train)形式，如图4所示，图4为张量分解(Tensor-Train)示意图，则反卷积层可以按照如下公式进行分解：
W(i 1,j 1),…,(i d,j d))＝G 1[i 1,j 1]G 2[i 2,j 2]…G d[i d,j d]
则反卷积层的张量分解(Tensor-Train)步骤如下：
网络输入层：
Figure PCTCN2020078694-appb-000006
网络输出层：
Figure PCTCN2020078694-appb-000007
卷积核：
Figure PCTCN2020078694-appb-000008
卷积层张量化后得到：
Figure PCTCN2020078694-appb-000009
其中，
Figure PCTCN2020078694-appb-000010
303、服务器102获取第一样本图像及第二样本图像标注对。
具体的，服务器102可以从客户端101或其他数据存储平台获取第一样本图 像及第二样本图像标注对，该第二样本图像标注对包括第二样本图像及样本图像标注，其中，第一样本图像及第二样本图像均为真实的FMRI图像，样本图像标注为第二样本图像对应的分类标注，此处样本图像标注与上述步骤301中的向量标注属于一类标注。进一步的，第一样本图像用于输入至分类器网络中，以使分类器网络对该第一样本图像进行分类预测，得到第一样本图像的预测标注。第二样本图像标注对用于与生成器生成的目标图像标注对、第一样本图像及第一样本图像的预测标注输入到判别器模型中，以根据判别结果训练可视化处理模型，或输入至分类器中进行监督训练，得到交叉熵。
304、服务器102根据目标图像标注对、第一样本图像及第二样本图像标注对训练可视化处理模型，得到模型损失函数。
具体的，在获取到目标图像标注对，第一样本图像及第二样本图像标注对的情况下，服务器102根据目标图像标注对，第一样本图像及第二样本图像标注对训练可视化处理模型，得到模型损失函数，以使可以进一步根据该模型损失函数，构建可视化处理模型，即执行步骤305。
具体的，该可视化处理模型的框架图可参见图5所示，该模型主要基于三元生成对抗网络，该三元生成对抗网络包括生成器、分类器及判别器。如图5所示，其训练过程主要包括：将噪声向量及向量标注输入至生成器中，得到生成器生成的FMRI图像标注对，在本申请中也可以描述为目标图像标注对。获取真实无标注的FMRI图像并进行归一化预处理，在本申请中真实无标注的FMRI图像也可以描述为第一样本图像，同时，获取真实有标注的FMRI图像标注对，并对其中的真实FMRI图像进行归一化预处理，在本申请中真实有标注的FMRI图像标注对也可以描述为第二样本图像标注对。则服务器102可以根据目标图像标注对，第一样本图像及第二样本图像标注对训练可视化处理模型，具体的，将生成器生成的目标图像标注对输入至判别器中得到判别结果，得到第一判别结果，同时，基于输入至分类器中的第一样本图像、第二样本图像及目标生成图像之间的重建损失，共同构成生成器的损失函数，通过反向传播算法，根据生成的生成器的损失函数梯度下降更新生成器网络层张量分解核矩阵参数；将第一样本图像输入至分类器中，得到预测标注，并将第一样本图像及预测标注输入至判别器中进行判别，得到第二判别结果，同时，基于输入至分类器中的第一样本图像、第二样本图像标注对及生成器网络生成的目标图像标注对之间的交叉熵损失函数，共同构成分类器的损失函数，通过反向传播算法，根据生成的分类 器的损失函数梯度下降更新分类器网络层张量分解核矩阵参数；将第一样本图像、第一样本图像的预测标注、第二样本图像标注对及生成器网络生成的目标图像标注对输入至判别器进行判别，以构建判别器的损失函数，并通过反向传播算法，根据生成的判别器的损失函数梯度下降更新判别器网络层张量分解核矩阵参数。
进一步的，上述模型损失函数包括生成损失函数，该生成损失函数即为生成器的损失函数。则根据目标图像标注对、第一样本图像及第二样本图像标注对训练可视化处理模型，得到模型损失函数，可以为对目标图像标注对进行判别处理，生成第一判别结果，该目标图像标注对包括目标生成图像及目标生成图像标注。根据目标生成图像及第二样本图像，确定重建损失，并根据第一判别结果及重建损失，确定生成器的生成损失函数。
具体的，生成器的损失函数包括两个部分：一部分为将生成的目标图像标注对输入至判别器进行判别处理，使判别结果趋向为真的损失；另一部分根据生成器生成的目标生成图像与真实的FMRI图像之间的重建损失，其中，真实的FMRI图像即为第一样本图像及第二样本图像。则该生成器的损失函数可以表示为：
Figure PCTCN2020078694-appb-000011
其中，
Figure PCTCN2020078694-appb-000012
表示对目标图像标的注判别结果趋向为真的损失；
Figure PCTCN2020078694-appb-000013
表示生成器的目标生成图像与真实的FMRI图像之间的重建损失。
基于该实施方式，可以从两方面确定生成器的损失函数，使构建的可视化模型更为准确，并且，通过构建生成器模型，可以将随机噪声向量转化为准确的FMRI图像，解决了FMRI图像获取困难问题，节约了实验的成本。
进一步的，上述模型损失函数包括分类损失函数，该分类损失函数即为分类器的损失函数。则根据目标图像标注对、第一样本图像及第二样本图像标注对训练可视化处理模型，得到模型损失函数，可以为对第一样本图像进行分类 处理，得到第一样本图像的预测标注，并对第一样本图像及预测标注进行判别，生成第二判别结果，对目标图像标注对及第二样本图像标注对进行监督训练，分别获取目标图像标注的第一交叉熵及第二样本图像标注对的第二交叉熵，根据第二判别结果、第一交叉熵及第二交叉熵，确定分类器的分类损失函数。
具体的，分类器的损失函数包括两个部分：一部分为对目标图像标注对及第二样本图像标注对进行监督训练，得到的交叉熵损失函数；另一部分为将第一样本图像及对其进行分类处理后得到的预测标注输入至判别器中进行判别处理，使判别结果趋向为真的无监督损失。则该分类器的损失函数可以表示为：
Figure PCTCN2020078694-appb-000014
L supervised＝R L+α pR p
Figure PCTCN2020078694-appb-000015
Figure PCTCN2020078694-appb-000016
Figure PCTCN2020078694-appb-000017
其中，第一交叉熵为R p，第二交叉熵为R L，L supervised为监督损失，L unsupervised为半监督损失。具体的，第二交叉熵R L等价于计算分类器学到的分布P c(x,y)与真实数据分布P real(x,y)之间的相对熵(KL散度)。而通过引入第一交叉熵R p计算目标生成图像标注对的交叉熵，可以使生成器生成接近于真实分布的FMRI图像标注对。从而提高分类器的分类性能。最小化R p等价于最小化相对熵(KL散度)D KL(P g(x,y)||P c(x,y))。由于P g(x,y)/P c(x,y)无法直接计算，故KL散度D KL(P g(x,y)||P c(x,y))也无法直接计算。本分类器模型通过间接最小化R p达到实现最小化相对熵(KL散度)D KL(P g(x,y)||P c(x,y))的目的。
具体的，分类器网络的结构示意图如图6所示，该分类器网络包括张量化卷积层、平均池化层、张量化密集连接块、二阶池化模块及过渡层组成，FMRI图像在输入至该分类器网络后，经过上述各个组成单元的处理，可以得到不同浓度尼古丁成瘾性状的评估结果，该评估结果可以理解为分类结果，关于分类结果的相关描述可以参见步骤203，此处不赘述。其中，张量化卷积层的卷积核张量及全连接层的权重矩阵均可以表示为对应的张量分解(Tensor-Train)形式，该张量分解(Tensor-Train)的相关示意图可以参见步骤302中的图4所示，其中，卷积层张量分解步骤与反卷积层张量分解的步骤相同，可以参见步骤302中的相关描述，此处不赘述。全连接层的权重张量W也可以按照如下公式进行张量分 解：
W((i 1,j 1),…,(i d,j d))＝G 1[i 1,j 1]G 2[i 2,j 2]…G d[i d,j d]
则全连接层的张量化表示如下：
Figure PCTCN2020078694-appb-000018
二阶池化模块部署在张量化密集连接块之后，该模块的结构示意图如图7所示，该模块包括压缩模块及校准模块两个部分，在分类器处理FMRI图像的过程中，二阶池化模块通过1x1x1卷积对输入的4维特征图进行通道降维，计算降维后的4维特征图中不同通道之间的协方差信息，得到协方差矩阵，根据协方差矩阵通过分组卷积和1x1x1卷积得到与4维特征图通道数相同的权重向量，并计算权重向量和输入特征图的内积，得到加权后的输出特征图。最后，在自注意力机制的作用下，通过反向传播算法使特征图重要的通道权重大，不重要的通道权重小，从而提取更有代表性的全局高阶特征图，提高大脑尼古丁成瘾性状评估的准确率。
基于该实施方式，生成器与分类器相互促进、共同学习FMRI图像潜在的高维概率分布。并且，基于二阶池化模块构建分类器模型，可以通过FMRI图像不同区域的依赖关系和高阶特征不同通道间的相关性信息，提取与大脑尼古丁成瘾性状相关的更有判别性的特征，提高大脑尼古丁成瘾性状评估的准确率,以使可以将该分类器模型应用到基于掩膜的可视化评估中，即步骤201-206。
进一步的，上述模型损失函数包括判别损失函数，该判别损失函数即为判别器的损失函数。则根据目标图像标注对、第一样本图像及第二样本图像标注对训练可视化处理模型，得到模型损失函数，可以为对目标图像标注对进行判别处理，生成第三判别结果，对第一样本图像及预测标注进行判别处理，生成第四判别结果，对第二样本图像标注对进行判别处理，生成第五判别结果，根据第三判别结果、第四判别结果及第五判别结果，确定判别器的判别损失函数。其中，第一样本图像及第二样本图像以四阶张量的形式作为输入，预测标注及第二样本图像的图像标注以独热编码(one-hot)的形式作为输入。
具体的，判别器的损失函数包括三个部分：第一部分为对目标图像标注对进行判别处理，得到使判别结果趋向为假的损失；第二部分为对第一样本图像及其对应的预测标注进行判别处理，得到使判别结果趋向为假的损失；第三部分为对第二样本图像标注对进行判别处理，得到使判别结果趋向为真的损失。 则该判别器的损失函数可以表示为：
Figure PCTCN2020078694-appb-000019
其中，
Figure PCTCN2020078694-appb-000020
表示对目标图像标注对的判别结果趋向为假的损失，
Figure PCTCN2020078694-appb-000021
表示对第一样本图像及其对应的预测标注的判别结果趋向为假的损失，
Figure PCTCN2020078694-appb-000022
表示对第二样本图像标注对的判别结果趋向为真的损失。
具体的，判别器网络采用密集型深度神经网络进行特征提取，可选的，该密集型深度神经网络的层数可以为30层，由张量化卷积层、张量化密集连接块、张量化过渡层和张量化全连接层组成。其中，张量化卷积层包括卷积层、归一化层(Batch Normalization)及激活函数层(Leaky ReLU)，张量化全连接层中的函数(sigmoid)则用于判断上述目标图像标注对、第一样本图像及其对应的预测标注、第二样本图像标注对的真假。上述张量化卷积层的卷积核张量及全连接层的权重矩阵均可以表示为对应的张量分解(Tensor-Train)形式，该张量分解(Tensor-Train)的相关示意图可以参见上述步骤302中的图4所示。并且，关于全连接层及卷积层张量分解的相关描述，可以参见上述步骤302及分类器网络描述的相应部分，此处不赘述。目标图像标注对、第一样本图像及其对应的预测标注、第二样本图像标注对输入至判别器后经过各个模块的特征提取，获得保留空间信息和时间序列信息的大鼠脑区特征图，并由最后一层张量化全连接层判断各组图像标注对的真假，并输出相应的判别结果。
基于该实施方式，判别器可以对生成器及分类器输出的数据进行判别，通过生成器、分类器及判别器共同构成的三元生成对抗网络，使得生成器生成更接近真实FMRI图像的图像，并且，使得分类器提取与尼古丁成瘾性状相关的更有判别性的特征，得到更准确的分类结果。
305、服务器102根据模型损失函数，构建可视化处理模型。
具体的，在获取到包括生成损失函数、判别损失函数及分类损失函数的模型损失函数后，根据该模型损失函数，构建可视化处理模型。
进一步的，可以通过反向传播算法，根据生成器的损失函数，对生成器的参数进行更新，根据分类器的损失函数，对分类器的参数进行更新，根据判别器的损失函数，对判别器的参数进行更新，则可以根据生成器的参数、分类器的参数及判别器的参数，构建该可视化处理模型。
具体的，可以根据生成器的损失函数梯度下降更新生成器网络层张量分解核矩阵G k[i k,j k]参数，根据分类器的损失函数梯度下降更新分类器网络层张量分解核矩阵G k[i k,j k]参数，根据判别器的损失函数梯度下降更新判别器网络层张量分解核矩阵G k[i k,j k]参数。则在损失函数优化过程中，反向传播求解损失函数对核矩阵G k[i k,j k]的梯度。经过生成器网络、分类器网络及判别器网络的迭代协同训练，不断优化生成器，分类器和判别器。使生成器生成的目标生成图像更符合真实FMRI图像数据的分布。且分类器也可以更精准区分真实分布的不同类比之间的边界，并反馈FMRI图像标注对给判别器，以使判别器的判别性能得到进一步的提升。最终使整个三元生成对抗网络模型达到纳什均衡，得到优化后的可视化处理模型。
可选的，在对可视化处理模型训练的过程中，可以分为训练、验证及测试三个过程。则在获取第一样本图像、第二样本图像等样本图像数据时，可以按照一定比例对样本图像数据进行划分，得到不同比例的训练集样本、验证集样本及测试集样本。例如：以80％：10％：10％比例划分样本。则训练过程可以参见上述步骤301-305中的实施方式。在每次训练迭代过程中，验证集样本用于对训练的可视化评估模型进行验证，基于验证的结果选择最优的可视化评估模型，得到最优的可视化评估模型。测试集用于输入至完成优化的可视化评估模型的分类器中，基于掩膜的方法得到尼古丁成瘾性状评估结果，即步骤201-206，从而实现了尼古丁成瘾激活脑区的可视化。
可见，通过实施图3所描述的方法，服务器102在获取到噪声向量及向量标注，对噪声向量及向量标注通过反卷积网络处理，得到目标图像标注对。则可以根据目标图像标注对及获取到的第一样本图像、第二样本图像标注对训练可视化处理模型，得到包括生成损失函数、分类损失函数及判别损失函数的模型损失函数，并根据该模型损失函数，构建可视化处理模型。通过执行本实施方式，可以将随机噪声向量转化为准确的FMRI图像，解决了FMRI图像获取困难问题，节约了实验的成本。并且，还可以促进分类器提取出与尼古丁成瘾性状相关的更有判别性的特征，得到更准确的分类结果，从而将完成训练优化的分 类器用于获取对FMRI图像中不同掩膜区域进行屏蔽后，引起的尼古丁成瘾性状评估结果的变化，可以更直观、准确的定位尼古丁成瘾脑区，实现了评估结果的可视化，
基于上述方法实施例的描述，本申请实施例还提出一种大脑成瘾性状评估的可视化处理装置。该大脑成瘾性状评估的可视化处理装置可以是运行于处理设备中的计算机程序(包括程序代码)；请参见图8所示，该大脑成瘾性状评估的可视化处理装置可以运行如下单元：
收发单元801，用于接收客户端的可视化处理请求，所述可视化处理请求包括待处理图像，所述可视化处理请求用于请求获取所述待处理图像的可视化评估结果；
处理单元802，用于对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像；调用可视化处理模型对所述微扰图像进行分类处理，得到分类结果，并对所述分类结果进行计算，得到所述微扰图像的评估数值，所述微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值；根据所述微扰图像的评估数值，确定所述可视化评估结果；
所述收发单元801，还用于发送所述可视化评估结果至所述客户端。
在一种实施方式中，所述屏蔽处理包括模糊化处理；
所述对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后微扰图像，处理单元802，还可用于确定掩膜区域集合，所述掩膜区域集合包括至少一个掩膜区域；
从所述待处理图像中对所述掩膜区域集合中各个掩膜区域进行模糊化处理，得到所述微扰图像，所述待处理图像中包含所述掩膜区域集合中各个掩膜区域。
再一种实施方式中，所述调用可视化处理模型对所述微扰图像进行分类处理之前，处理单元802，还可用于获取噪声向量及向量标注，并将所述噪声向量及所述向量标注通过反卷积网络处理，得到目标图像标注对，所述目标图像标注对包括目标生成图像及目标生成图像标注；
获取第一样本图像及第二样本图像标注对，所述第二样本图像标注对包括第二样本图像及样本图像标注；
根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数；
根据所述模型损失函数，构建所述可视化处理模型。
再一种实施方式中，所述模型损失函数包括生成损失函数，所述生成损失函数为生成器的损失函数；
所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，处理单元802，还可用于对所述目标图像标注对进行判别处理，生成第一判别结果，所述目标图像标注对包括所述目标生成图像及所述目标生成图像标注；
根据所述目标生成图像、所述第一样本图像及所述第二样本图像，确定重建损失；
根据所述第一判别结果及所述重建损失，确定所述生成器的生成损失函数。
再一种实施方式中，所述模型损失函数包括分类损失函数，所述分类损失函数为分类器的损失函数；
所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，处理单元802，还可用于对所述第一样本图像进行分类处理，得到所述第一样本图像的预测标注，并将所述第一样本图像及所述预测标注进行判别处理，生成第二判别结果；
对所述目标图像标注对及所述第二样本图像标注对进行监督训练，分别获取所述目标图像标注的第一交叉熵及所述第二样本图像标注对的第二交叉熵；
根据所述第二判别结果、所述第一交叉熵及所述第二交叉熵，确定所述分类器的分类损失函数。
再一种实施方式中，所述模型损失函数包括判别损失函数，所述判别损失函数为判别器的损失函数；
所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，处理单元802，还可用于对所述目标图像标注对进行判别处理，生成第三判别结果；
对所述第一样本图像及所述预测标注进行判别处理，生成第四判别结果；
对所述第二样本图像标注对进行判别处理，生成第五判别结果；
根据所述第三判别结果、所述第四判别结果及所述第五判别结果，确定所述判别器的判别损失函数。
再一种实施方式中，所述模型损失函数包括所述生成损失函数、所述分类损失函数及所述判别损失函数；
所述根据所述模型损失函数，构建所述可视化处理模型，处理单元802，还可用于
通过反向传播算法，根据所述生成器的损失函数，对所述生成器的参数进行更新；
通过反向传播算法，根据所述分类器的损失函数，对所述分类器的参数进行更新；
通过反向传播算法，根据所述判别器的损失函数，对所述判别器的参数进行更新；
根据所述生成器的参数、所述分类器的参数及所述判别器的参数，构建所述可视化处理模型。
根据本申请的一个实施例，图2及图3所示的大脑成瘾性状评估的可视化方法所涉及的部分步骤可由大脑成瘾性状评估的可视化处理装置中的处理单元来执行。例如，图2中所示的步骤201和206可由收发单元801执行；又如，图2所示的步骤202可由处理单元802执行。根据本申请的另一个实施例，大脑成瘾性状评估的可视化处理装置中的各个单元可以分别或全部合并为一个或若干个另外的单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个单元来构成，这可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。
请参见图9，是本申请实施例提供的一种大脑成瘾性状评估的可视化处理装置的结构示意图，该大脑成瘾性状评估的可视化处理装置包括处理器901、存储器902及通信接口903，处理器901、存储器902及通信接口903通过至少一条通信总线连接，处理器901被配置为支持处理设备执行图2及图3方法中处理设备相应的功能。
存储器902用于存放有适于被处理器加载并执行的至少一条指令，这些指令可以是一个或一个以上的计算机程序(包括程序代码)。
通信接口903用于接收数据和用于发送数据。例如，通信接口903用于发送可视化处理请求等。
在本申请实施例中，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
通过通信接口903接收客户端的可视化处理请求，所述可视化处理请求包括 待处理图像，所述可视化处理请求用于请求获取所述待处理图像的可视化评估结果；
对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像；
调用可视化处理模型对所述微扰图像进行分类处理，得到分类结果，并对所述分类结果进行计算，得到所述微扰图像的评估数值，所述微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值；
根据所述微扰图像的评估数值，确定所述可视化评估结果；
通过通信接口903发送所述可视化评估结果至所述客户端。
作为一种可选的实施方式，所述屏蔽处理包括模糊化处理；
所述对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后微扰图像，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
确定掩膜区域集合，所述掩膜区域集合包括至少一个掩膜区域；
从所述待处理图像中对所述掩膜区域集合中各个掩膜区域进行模糊化处理，得到所述微扰图像，所述待处理图像中包含所述掩膜区域集合中各个掩膜区域。
作为一种可选的实施方式，所述调用可视化处理模型对所述微扰图像进行分类处理之前，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
获取噪声向量及向量标注，并将所述噪声向量及所述向量标注通过反卷积网络处理，得到目标图像标注对，所述目标图像标注对包括目标生成图像及目标生成图像标注；
获取第一样本图像及第二样本图像标注对，所述第二样本图像标注对包括第二样本图像及样本图像标注；
根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数；
根据所述模型损失函数，构建所述可视化处理模型。
作为一种可选的实施方式，所述模型损失函数包括生成损失函数，所述生成损失函数为生成器的损失函数；
所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
对所述目标图像标注对进行判别处理，生成第一判别结果，所述目标图像 标注对包括所述目标生成图像及所述目标生成图像标注；
根据所述目标生成图像、所述第一样本图像及所述第二样本图像，确定重建损失；
根据所述第一判别结果及所述重建损失，确定所述生成器的生成损失函数。
作为一种可选的实施方式，所述模型损失函数包括分类损失函数，所述分类损失函数为分类器的损失函数；
所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
对所述第一样本图像进行分类处理，得到所述第一样本图像的预测标注，并将所述第一样本图像及所述预测标注进行判别处理，生成第二判别结果；
对所述目标图像标注对及所述第二样本图像标注对进行监督训练，分别获取所述目标图像标注的第一交叉熵及所述第二样本图像标注对的第二交叉熵；
根据所述第二判别结果、所述第一交叉熵及所述第二交叉熵，确定所述分类器的分类损失函数。
作为一种可选的实施方式，所述模型损失函数包括判别损失函数，所述判别损失函数为判别器的损失函数；
所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
对所述目标图像标注对进行判别处理，生成第三判别结果；
对所述第一样本图像及所述预测标注进行判别处理，生成第四判别结果；
对所述第二样本图像标注对进行判别处理，生成第五判别结果；
根据所述第三判别结果、所述第四判别结果及所述第五判别结果，确定所述判别器的判别损失函数。
作为一种可选的实施方式，所述模型损失函数包括所述生成损失函数、所述分类损失函数及所述判别损失函数；
所述根据所述模型损失函数，构建所述可视化处理模型，该处理器901可以调用存储器902中存储的程序代码以执行以下操作：
通过反向传播算法，根据所述生成器的损失函数，对所述生成器的参数进行更新；
通过反向传播算法，根据所述分类器的损失函数，对所述分类器的参数进行更新；
通过反向传播算法，根据所述判别器的损失函数，对所述判别器的参数进行更新；
根据所述生成器的参数、所述分类器的参数及所述判别器的参数，构建所述可视化处理模型。
本申请实施例还提供了一种计算机可读存储介质(Memory)，可以用于存储图2及图3中所示实施例中处理设备所用的计算机软件指令，在该存储空间中还存放了适于被处理器加载并执行的至少一条指令，这些指令可以是一个或一个以上的计算机程序(包括程序代码)。
上述计算机可读存储介质包括但不限于快闪存储器、硬盘、固态硬盘。
本领域普通技术人员可以意识到，结合本申请中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者通过计算机可读存储介质进行传输。计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。
以上所述的具体实施方式，对本申请的目的、技术方案和有益效果进行了 进一步详细说明，所应理解的是，以上所述仅为本申请的具体实施方式而已，并不用于限定本申请的保护范围，凡在本申请的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本申请的保护范围之内。

Claims (10)

  1. 一种大脑成瘾性状评估的可视化方法，其特征在于，所述方法包括：
    接收客户端的可视化处理请求，所述可视化处理请求包括待处理图像，所述可视化处理请求用于请求获取所述待处理图像的可视化评估结果；
    对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像；
    调用可视化处理模型对所述微扰图像进行分类处理，得到分类结果，并对所述分类结果进行计算，得到所述微扰图像的评估数值，所述微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值；
    根据所述微扰图像的评估数值，确定所述可视化评估结果；
    发送所述可视化评估结果至所述客户端。
  2. 根据权利要求1所述的方法，其特征在于，所述屏蔽处理包括模糊化处理；所述对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后微扰图像，包括：确定掩膜区域集合，所述掩膜区域集合包括至少一个掩膜区域；
    从所述待处理图像中对所述掩膜区域集合中各个掩膜区域进行模糊化处理，得到所述微扰图像，所述待处理图像中包含所述掩膜区域集合中各个掩膜区域。
  3. 根据权利要求1所述的方法，其特征在于，所述调用可视化处理模型对所述微扰图像进行分类处理之前，所述方法还包括：
    获取噪声向量及向量标注，并将所述噪声向量及所述向量标注通过反卷积网络处理，得到目标图像标注对，所述目标图像标注对包括目标生成图像及目标生成图像标注；
    获取第一样本图像及第二样本图像标注对，所述第二样本图像标注对包括第二样本图像及样本图像标注；
    根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数；
    根据所述模型损失函数，构建所述可视化处理模型。
  4. 根据权利要求3所述的方法，其特征在于，所述模型损失函数包括生成损失函数，所述生成损失函数为生成器的损失函数；
    所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，包括：
    对所述目标图像标注对进行判别处理，生成第一判别结果，所述目标图像标注对包括所述目标生成图像及所述目标生成图像标注；
    根据所述目标生成图像、所述第一样本图像及所述第二样本图像，确定重建损失；
    根据所述第一判别结果及所述重建损失，确定所述生成器的生成损失函数。
  5. 根据权利要求3所述的方法，其特征在于，所述模型损失函数包括分类损失函数，所述分类损失函数为分类器的损失函数；
    所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，包括：
    对所述第一样本图像进行分类处理，得到所述第一样本图像的预测标注，并将所述第一样本图像及所述预测标注进行判别处理，生成第二判别结果；
    对所述目标图像标注对及所述第二样本图像标注对进行监督训练，分别获取所述目标图像标注的第一交叉熵及所述第二样本图像标注对的第二交叉熵；
    根据所述第二判别结果、所述第一交叉熵及所述第二交叉熵，确定所述分类器的分类损失函数。
  6. 根据权利要求3所述的方法，其特征在于，所述模型损失函数包括判别损失函数，所述判别损失函数为判别器的损失函数；
    所述根据所述目标图像标注对、所述第一样本图像及所述第二样本图像标注对训练所述可视化处理模型，得到模型损失函数，包括：
    对所述目标图像标注对进行判别处理，生成第三判别结果；
    对所述第一样本图像及所述预测标注进行判别处理，生成第四判别结果；
    对所述第二样本图像标注对进行判别处理，生成第五判别结果；
    根据所述第三判别结果、所述第四判别结果及所述第五判别结果，确定所述判别器的判别损失函数。
  7. 根据权利要求3所述的方法，其特征在于，所述模型损失函数包括所述生成损失函数、所述分类损失函数及所述判别损失函数；
    所述根据所述模型损失函数，构建所述可视化处理模型，包括：
    通过反向传播算法，根据所述生成器的损失函数，对所述生成器的参数进行更新；
    通过反向传播算法，根据所述分类器的损失函数，对所述分类器的参数进行更新；
    通过反向传播算法，根据所述判别器的损失函数，对所述判别器的参数进行更新；
    根据所述生成器的参数、所述分类器的参数及所述判别器的参数，构建所述可视化处理模型。
  8. 一种大脑成瘾性状评估的可视化处理装置，其特征在于，包括：
    收发单元，用于接收客户端的可视化处理请求，所述可视化处理请求包括待处理图像；
    处理单元，用于对所述待处理图像进行掩膜屏蔽处理，得到屏蔽掩膜后的微扰图像；调用可视化处理模型对所述微扰图像进行分类处理，得到分类结果，并对所述分类结果进行计算，得到所述微扰图像的评估数值，所述微扰图像的评估数值小于未经过掩膜屏蔽处理的所述待处理图像的评估数值；根据所述微扰图像的评估数值，确定所述可视化评估结果；
    所述收发单元，还用于发送所述可视化评估结果至所述客户端。
  9. 一种大脑成瘾性状评估的可视化处理装置，其特征在于，包括处理器、存储器和通信接口，所述处理器、所述存储器和所述通信接口相互连接，其中，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行如权利要求1-7中任一项所述的方法。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有一条或多条指令，所述一条或多条指令适于由处理器加载并执行如权利要求1-7任一项所述的方法。
PCT/CN2020/078694 2020-03-11 2020-03-11 大脑成瘾性状评估的可视化方法、装置及介质 WO2021179189A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP20924905.1A EP3971773A4 (en) 2020-03-11 2020-03-11 METHOD AND VISUALIZATION DEVICE FOR ASSESSING BRAIN ADDICTION CHARACTERISTICS, AND MEDIUM
PCT/CN2020/078694 WO2021179189A1 (zh) 2020-03-11 2020-03-11 大脑成瘾性状评估的可视化方法、装置及介质
US17/549,258 US20220101527A1 (en) 2020-03-11 2021-12-13 Visualization method for evaluating brain addiction traits, apparatus, and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/078694 WO2021179189A1 (zh) 2020-03-11 2020-03-11 大脑成瘾性状评估的可视化方法、装置及介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/549,258 Continuation US20220101527A1 (en) 2020-03-11 2021-12-13 Visualization method for evaluating brain addiction traits, apparatus, and medium

Publications (1)

Publication Number Publication Date
WO2021179189A1 true WO2021179189A1 (zh) 2021-09-16

Family

ID=77671110

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/078694 WO2021179189A1 (zh) 2020-03-11 2020-03-11 大脑成瘾性状评估的可视化方法、装置及介质

Country Status (3)

Country Link
US (1) US20220101527A1 (zh)
EP (1) EP3971773A4 (zh)
WO (1) WO2021179189A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019004350A1 (ja) 2017-06-29 2019-01-03 株式会社 Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ データ識別器訓練方法、データ識別器訓練装置、プログラム及び訓練方法
US11663840B2 (en) * 2020-03-26 2023-05-30 Bloomberg Finance L.P. Method and system for removing noise in documents for image processing

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106250707A (zh) * 2016-08-12 2016-12-21 王双坤 一种基于深度学习算法处理头部结构像数据的方法
US20190046068A1 (en) * 2017-08-10 2019-02-14 Siemens Healthcare Gmbh Protocol independent image processing with adversarial networks
CN110443798A (zh) * 2018-12-25 2019-11-12 电子科技大学 一种基于磁共振图像的自闭症检测方法、装置及系统
CN110580695A (zh) * 2019-08-07 2019-12-17 深圳先进技术研究院 一种多模态三维医学影像融合方法、系统及电子设备
CN110610488A (zh) * 2019-08-29 2019-12-24 上海杏脉信息科技有限公司 分类训练和检测的方法与装置
CN111383217A (zh) * 2020-03-11 2020-07-07 深圳先进技术研究院 大脑成瘾性状评估的可视化方法、装置及介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106250707A (zh) * 2016-08-12 2016-12-21 王双坤 一种基于深度学习算法处理头部结构像数据的方法
US20190046068A1 (en) * 2017-08-10 2019-02-14 Siemens Healthcare Gmbh Protocol independent image processing with adversarial networks
CN110443798A (zh) * 2018-12-25 2019-11-12 电子科技大学 一种基于磁共振图像的自闭症检测方法、装置及系统
CN110580695A (zh) * 2019-08-07 2019-12-17 深圳先进技术研究院 一种多模态三维医学影像融合方法、系统及电子设备
CN110610488A (zh) * 2019-08-29 2019-12-24 上海杏脉信息科技有限公司 分类训练和检测的方法与装置
CN111383217A (zh) * 2020-03-11 2020-07-07 深圳先进技术研究院 大脑成瘾性状评估的可视化方法、装置及介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3971773A4 *

Also Published As

Publication number Publication date
EP3971773A4 (en) 2022-07-20
US20220101527A1 (en) 2022-03-31
EP3971773A1 (en) 2022-03-23

Similar Documents

Publication Publication Date Title
Al-Haija et al. Breast cancer diagnosis in histopathological images using ResNet-50 convolutional neural network
Zeebaree et al. Gene selection and classification of microarray data using convolutional neural network
Polsinelli et al. A light CNN for detecting COVID-19 from CT scans of the chest
CN110993094B (zh) 一种基于医学图像的智能辅助诊断方法及终端
Pattanaik et al. Unsupervised deep learning cad scheme for the detection of malaria in blood smear microscopic images
WO2021097675A1 (zh) 一种基于医学图像的智能辅助诊断方法及终端
EP3916597B1 (en) Detecting malware with deep generative models
WO2021179189A1 (zh) 大脑成瘾性状评估的可视化方法、装置及介质
Abd Elaziz et al. An efficient artificial rabbits optimization based on mutation strategy for skin cancer prediction
WO2023108873A1 (zh) 一种脑网络和脑成瘾连接计算方法及装置
CN111383217B (zh) 大脑成瘾性状评估的可视化方法、装置及介质
CN112052874A (zh) 一种基于生成对抗网络的生理数据分类方法及系统
US11972604B2 (en) Image feature visualization method, image feature visualization apparatus, and electronic device
CN113887559A (zh) 一种脑不在环路应用的脑机信息融合分类方法及系统
Pal et al. Advanced convolutional neural network model to identify melanoma skin cancer
Ahmadieh et al. Visual image reconstruction based on EEG signals using a generative adversarial and deep fuzzy neural network
Cinar et al. Development of early stage diabetes prediction model based on stacking approach
Ekong et al. Bayesian depthwise convolutional neural network design for brain tumor MRI classification. Diagnostics. 2022; 12: 1657
Li et al. CAGAN: Classifier‐augmented generative adversarial networks for weakly‐supervised COVID‐19 lung lesion localisation
Cao et al. Alzheimer’s Disease Stage Detection Method Based on Convolutional Neural Network
Hamran et al. Brain Tumor Detection using Convolutional Neural Networks with Skip Connections
US20240169187A1 (en) Systems and Methods for Supplementing Data With Generative Models
Alhamdi et al. Brain Tumor Detection using Machine Learning and Deep Learning
Swetha et al. Breast Cancer Detection Mammogram Imagesusing Convolution Neural Network
Selitskiy et al. Loss Function with Memory for Trustworthiness Threshold Learning: Case of Face and Facial Expression Recognition

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20924905

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020924905

Country of ref document: EP

Effective date: 20211213

NENP Non-entry into the national phase

Ref country code: DE