CN109166087A - Style conversion method, device, medical supply, image system and the storage medium of medical image - Google Patents
Style conversion method, device, medical supply, image system and the storage medium of medical image Download PDFInfo
- Publication number
- CN109166087A CN109166087A CN201811150272.4A CN201811150272A CN109166087A CN 109166087 A CN109166087 A CN 109166087A CN 201811150272 A CN201811150272 A CN 201811150272A CN 109166087 A CN109166087 A CN 109166087A
- Authority
- CN
- China
- Prior art keywords
- medical image
- image
- style
- medical
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000003062 neural network model Methods 0.000 claims abstract description 44
- 239000003814 drug Substances 0.000 claims description 49
- 210000005036 nerve Anatomy 0.000 claims description 35
- 230000006870 function Effects 0.000 claims description 31
- 238000012549 training Methods 0.000 claims description 17
- 238000003384 imaging method Methods 0.000 claims description 14
- 238000002059 diagnostic imaging Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000002604 ultrasonography Methods 0.000 claims description 5
- 238000003325 tomography Methods 0.000 claims 1
- 238000013528 artificial neural network Methods 0.000 abstract description 24
- 238000012546 transfer Methods 0.000 abstract description 13
- 238000004422 calculation algorithm Methods 0.000 abstract description 8
- 230000008859 change Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 10
- 238000002591 computed tomography Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000013139 quantization Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000004218 nerve net Anatomy 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Epidemiology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Primary Health Care (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Radiology & Medical Imaging (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
This application involves style conversion method, device, medical supply, image system and the storage medium of a kind of medical image, the above method includes: acquisition neural network model；Obtain original medical image；Using original medical image as the input of neural network model, to obtain target medical image using neural network model；Wherein, for the difference of the content characteristic of the content characteristic of original medical image and target medical image in preset threshold value, the style and features of original medical image are different from the style and features of target medical image.The above method, pass through the neural network based on Style Transfer algorithm, the different target medical image of style and features is converted by original medical image, it can be suitably used for carrying out style conversion between the medical image of various mode, and the change for being able to maintain the content characteristic of image does not easily cause distortion in preset threshold value, facilitates doctor that the medical image of the target medical image being converted to and other mode is compared observation.
Description
Technical field
The present invention relates to the field of medical imaging, more particularly to style conversion method, device, the medicine of a kind of medical image
Equipment, image system and storage medium.
Background technique
Medical diagnosis and treatment in, it is often necessary to the same area carry out such as computed tomography, magnetic resonance with
And the shooting of the different modalities medical image such as X-ray, it is observed with comparing, but the different modalities medical image due to obtaining
Style and features differ greatly, and may result in the content between picture and are difficult to correspond to each other, bring inconvenience to diagnosing and treating.
Between traditional different modalities medical image style and features conversion generally by greyscale transformation method or
Mapping method based on texture.But the tonal range of medical image is unstable, and mapping mode is different, passes through greyscale transformation
Mode carry out the conversions of style and features and be easy to cause image fault；And since the information of medical image cannot use simple line
Reason description, so poor by the conversion adaptability that the method for texture mapping carries out style and features.
Summary of the invention
Based on this, it is necessary in view of the above technical problems, provide style conversion method, the device, doctor of a kind of medical image
Equipment, image system and storage medium are learned, the style and features conversion for rapidly carrying out different modalities medical image is can be convenient, fits
Answering property is extensively and distortion is lower.
A kind of style conversion method of medical image, comprising:
Obtain neural network model；
Obtain original medical image；
Using the original medical image as the input of the neural network model, to be obtained using the neural network model
To target medical image；
Wherein, the content characteristic of the original medical image is with the difference of the content characteristic of the target medical image pre-
If threshold value in, the style and features of the original medical image are different from the style and features of the target medical image.
The style conversion method of above-mentioned medical image, by the neural network based on Style Transfer algorithm, by primitive medicine
Image is converted into the different target medical image of style and features, can be suitable for carrying out image between the medical image of various mode
Style conversion, and the change for being able to maintain the content characteristic of image does not easily cause distortion in preset threshold value, facilitates doctor will
The medical image of the target medical image and other mode that are converted to compares observation.
The acquisition neural network model includes: in one of the embodiments,
Create first nerves network；
Obtain the first medical image based on the first image mode and based on being different from the second of first image mode
Second medical image of image mode；
Using first medical image as the input of the first nerves network, the first nerves network output is obtained
Tertiary medicine image；
Based on first medical image, second medical image and the tertiary medicine image, training described the
One neural network；
The neural network model after being trained；
Wherein, the content characteristic of second medical image is with the difference of the content characteristic of first medical image pre-
If threshold value in, the style and features of second medical image are different from the style and features of first medical image.
The image mode of the medical image includes that computed tomography images, magnetic are total in one of the embodiments,
Shake at least one of image, radioscopic image, ultrasound image and molecule image.
It is described based on first medical image, second medical image and described in one of the embodiments,
Tertiary medicine image, the training first nerves network include:
Obtain nervus opticus network；
Using first medical image, second medical image and the tertiary medicine image as second mind
Input through network, to calculate loss function；
The parameter of the first nerves network is updated by the loss function.
It is described by first medical image, second medical image and described in one of the embodiments,
Input of three medical images as the nervus opticus network include: to calculate loss function
Compare the content characteristic of first medical image and the tertiary medicine image, to obtain content deltas data；
Compare the style and features of second medical image and the tertiary medicine image, to obtain stylistic differences data；
The content deltas data and the stylistic differences data weighting are summed, to obtain the loss function.
Second medical image and the tertiary medicine image in one of the embodiments, to obtain
Stylistic differences data include:
Quantify the style and features difference of second medical image and the tertiary medicine image by gram matrix, with
Obtain the stylistic differences data.
The parameter packet that the first nerves network is updated by the loss function in one of the embodiments,
It includes:
The ginseng of the first nerves network is updated by the first medical image described in multiple groups and second medical image
Number；
In the case that the loss function reaches the preset condition of convergence, the parameter of the first nerves network is determined.
In one of the embodiments, the first nerves network include LeNet, GoogLeNet, VGGNet, ResNet,
At least one of DenseNet, VNet and UNet.
In one of the embodiments, the nervus opticus network include LeNet, GoogLeNet, VGGNet, ResNet,
At least one of DenseNet, VNet and UNet.
A kind of style conversion equipment of medical image, comprising:
Network obtains module, for obtaining neural network model；
Image collection module, for obtaining original medical image；
Image conversion module, for using the original medical image as the input of the neural network model, to utilize
The neural network model obtains target style medical image；
Wherein, the difference of the content characteristic of the content characteristic of the original medical image and the target style medical image
In preset threshold value, the style and features of the style and features of the original medical image and the target style medical image are not
Together.
The style conversion equipment of above-mentioned medical image, by the neural network based on Style Transfer algorithm, by primitive medicine
Image is converted into the different target medical image of style and features, can be suitable for carrying out image between the medical image of various mode
Style conversion, and the change for being able to maintain the content characteristic of image does not easily cause distortion in preset threshold value, facilitates doctor will
The medical image of the target medical image and other mode that are converted to compares observation.
A kind of medical supply including memory, processor and stores the meter that can be run on a memory and on a processor
The step of calculation machine program, the processor realizes the above method when executing described program.
A kind of computer readable storage medium, is stored thereon with computer program, realization when which is executed by processor
The step of above method.
A kind of Medical Imaging System, comprising:
Imaging device, for shooting the original medical image；And
Above-mentioned medical supply, for the original medical image to be converted to the target medical image.
Above-mentioned medical image system is converted original medical image to by the neural network based on Style Transfer algorithm
The different target medical image of style and features can be suitable for carrying out the conversion of image style between the medical image of various mode,
And the change for being able to maintain the content characteristic of image do not easily cause distortion in preset threshold value, facilitates doctor that will be converted to
Target medical image and the medical image of other mode compare observation.
Detailed description of the invention
Fig. 1 is the flow diagram of the style conversion method of one embodiment traditional Chinese medicine image；
Fig. 2 is the flow diagram of the style conversion method step S120 of one embodiment traditional Chinese medicine image；
Fig. 3 is the flow diagram of the style conversion method step S127 of one embodiment traditional Chinese medicine image；
Fig. 4 is the structural schematic diagram of the style conversion equipment of one embodiment traditional Chinese medicine image；
Fig. 5 is the structural schematic diagram of one embodiment traditional Chinese medicine picture system.
Specific embodiment
It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood
The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, and
It is not used in restriction the application.
Fig. 1 is the flow diagram of the style conversion method of one embodiment traditional Chinese medicine image, as shown in Figure 1, a kind of doctor
Learn the style conversion method of image, comprising the following steps:
Step S120: neural network model is obtained.
Step S140: original medical image is obtained.
Step S160: using original medical image as the input of neural network model, to be obtained using neural network model
Target medical image；
Wherein, the difference of the content characteristic of the content characteristic of original medical image and target medical image is in preset threshold value
Interior, the style and features of original medical image are different from the style and features of target medical image.
Specifically, when carrying out medical diagnosis or treatment, doctor is frequently necessary to carry out different modalities to region of interest
The shooting of medical image, and the medical image by obtaining compares observation.It obtains first for carrying out image style conversion
Neural network model, the basic model of the neural network model can be Style Transfer model, which can be with
Based on convolutional neural networks, it is trained to obtain by the medical image of multiple groups different modalities.After obtaining Style Transfer model, obtain
The original medical image of style conversion to be carried out is taken, original medical image can generally be obtained by corresponding medical imaging devices,
Original medical image can be computed tomography (Computed Tomography, abbreviation CT) image, magnetic resonance
(Magnetic Resonance, abbreviation MR) image, radioscopic image, ultrasound image (Ultrasonic, abbreviation US) and molecule
The medical image of the mode such as (Molecular Imaging abbreviation MI) image, the medical image of every kind of mode have it corresponding
Style and features.
Using obtained original medical image as the input of above-mentioned Style Transfer model, Style Transfer model is to primitive medicine
The characteristics of image of image is extracted and is converted, and keeps the variation of the content characteristic of original medical image in preset threshold value,
And convert the style and features of original medical image to the style and features of the medical image for the different modalities to be compared, thus
Style Transfer model is set to export target medical image.It is understood that the particular content about above-mentioned Style Transfer model, it can
With " the A Neural Algorithm of Artistic Style " delivered with reference to Leon A.Gatys et al. in September, 2015
Document, these documents can in a manner of citation include in this application.
Further, during carrying out the conversion of image style, the variation of the content characteristic of original medical image is kept
In preset threshold value, wherein content characteristic include anatomical structure in medical image and between correlation, such as have
Body may include the features such as the shape, size of histoorgan and position, Ke Yitong in the areas imaging and image of image
Cross the content characteristic for for example calculating the quantization of pixel shared by scanned position original medical image and target medical image in medical image
The threshold value of difference, difference can be determined according to situations such as actual imaging accuracy requirement.Target medical image and original medical image
The difference of content characteristic be maintained in preset threshold value and can cause image fault to avoid in style is converted so that mesh
Mark medical image is easy to correspond to each other with the medical image for needing to compare observation.
Above-mentioned style and features include at least one of color, gray scale, texture of medical image.Style and features be every kind at
The imaging characteristics as possessed by the medical image of mode, computed tomography images, magnetic resonance image, radioscopic image, ultrasound
The medical image of the various image modes such as image has its specific style and features, and image style can generally pass through texture, face
The features such as color, gray scale are embodied.Above-mentioned style conversion method can be by neural network model to different modalities medical image
Style and features extract and convert, so that original medical image be made to be converted into the style and features of target.For neural network
The specific style and features of the target medical image of output can determine that above-mentioned neural network model can be set according to actual needs
It, can also be with for the style and features for converting the style and features of certain mode medical image to specific another mode medical image
It is to be mutually converted between the style and features of the medical image of a variety of image modes, to realize wider applicability.
The style conversion method of above-mentioned medical image, by the neural network based on Style Transfer algorithm, by primitive medicine
Image is converted into the different target medical image of style and features, can be suitable for carrying out image between the medical image of various mode
Style conversion, and the change for being able to maintain the content characteristic of image does not easily cause distortion in preset threshold value, facilitates doctor will
The medical image of the target medical image and other mode that are converted to compares observation.
Fig. 2 is the flow diagram of the style conversion method step S120 of above-mentioned medical image in one embodiment, such as Fig. 2
It is shown, in one embodiment, step S120 specifically includes the following steps:
Step S121: creation first nerves network.
Step S123: the first medical image based on the first image mode is obtained and based on being different from the first image mode
Second medical image of the second image mode.
Step S125: using the first medical image as the input of first nerves network, the output of first nerves network is obtained
Tertiary medicine image.
Step S127: the first medical image, the second medical image and tertiary medicine image, training first nerves net are based on
Network.
Step S129: the neural network model after being trained.
Wherein, the difference of the content characteristic of the content characteristic of the second medical image and the first medical image is in preset threshold value
Interior, the style and features of the second medical image are different from the style and features of the first medical image.
Specifically, obtaining for neural network model can be by the training method of backpropagation training neural network reality
It is existing, first nerves network is initially set up, generally can carry out the conversion of image style using convolutional neural networks.First nerves network
May include LeNet, Google's network (GoogLeNet), VGGNet, residual error network (ResNet), DenseNet, VNet and
At least one of UNet, it is to be understood that in addition to above-mentioned neural network, first nerves network can also be used and can be applied
In the other kinds of neural network of image style conversion.
The first medical image and the second medical image are obtained as training data, the first medical image after establishing neural network
It is to need to carry out the medical image of the image mode of style conversion, and the second medical image is then target style and features after conversion
The medical image of image mode, above-mentioned image mode include computed tomography images, magnetic resonance image, radioscopic image, surpass
At least one of acoustic image and molecule image.Wherein.The difference of the content characteristic of first medical image and the second medical image
It is different in preset threshold value, general can be respectively is scanned same target area by corresponding medical imaging devices
Shooting is to obtain the first medical image and the second medical image, using the first medical image as the input of neural network, to obtain
The tertiary medicine image of neural network output.
Second medical image schedules to last image to be obtained, and tertiary medicine image is then the image of neural network reality output,
Backpropagation training is carried out to neural network based on the first medical image, the second medical image and tertiary medicine image, update changes
For the parameter of neural network, when the tertiary medicine image of neural network output meets demand, such as when tertiary medicine image
Style and features reach preset threshold value sign or identical, Yi Ji with the style and features of the second medical image in some aspects
The content characteristic of the content characteristic of three medical images and the first medical image reaches preset threshold value or complete in some aspects
When identical, it is determined that the parameter of neural network, or nerve net is determined when the number of neural network iteration reaches predetermined quantity
The parameter of network, parameter obtain neural network model for carrying out image style conversion after determining.It is understood that can pass through
Multiple groups training data is trained neural network, such as by corresponding medical imaging devices respectively to multiple and different targets
Region is scanned shooting, obtains the first medical image of multiple groups and the second medical image, utilizes multiple groups medical image training nerve
Network model can effectively improve the accuracy that neural network model carries out style conversion.
Fig. 3 is the flow diagram of the style conversion method step S127 of above-mentioned medical image in one embodiment, such as Fig. 3
It is shown, above-mentioned steps S127 specifically includes the following steps:
Step S1271: nervus opticus network is obtained.
Step S1273: using the first medical image, the second medical image and tertiary medicine image as nervus opticus network
Input, to calculate loss function.
Step S1275: the parameter of first nerves network is updated by loss function.
Specifically, the training of above-mentioned first nerves network can be realized by way of calculating loss function, passes through
Two neural networks obtain loss function, and nervus opticus network can be preparatory trained convolutional neural networks, nervus opticus net
Network may include LeNet, Google's network (GoogLeNet), VGGNet, residual error network (ResNet), DenseNet, VNet and
At least one of UNet.It can be applied to calculate loss function it is understood that nervus opticus network can also use
Other kinds of neural network.Using the first medical image, the second medical image and tertiary medicine image as nervus opticus network
Input, the parameter of nervus opticus network is fixed in training process, obtains loss function, loss function by nervus opticus network
In may include tertiary medicine image relative to the first medical image content characteristic loss and tertiary medicine image relative to
The style and features of second medical image lose, and the parameter of first nerves network is updated according to obtained loss function, until loss
Function meets the demand of image style conversion, determines the parameter of first nerves network.
In one embodiment, above-mentioned steps S1273 includes:
Compare the content characteristic of the first medical image and tertiary medicine image, to obtain content deltas data.
Compare the style and features of the second medical image and tertiary medicine image, to obtain stylistic differences data.
Content deltas data and stylistic differences data weighting are summed, to calculate loss function.
Specifically, the construction of above-mentioned loss function includes content characteristic error and style and features error, by comparing first
The content characteristic of medical image and tertiary medicine image gets content characteristic error, and compares the second medical image and
The style and features of three medical images obtain style and features error, content deltas data and stylistic differences data are weighted respectively
Part is lost to the content characteristic loss part of loss function and style and features, then two parts are summed, to be calculated
Loss function.
Further, pass through the style of such as gram (Gram) matrix quantization the second medical image and tertiary medicine image
Feature difference, to obtain stylistic differences data.For the content characteristic difference between tertiary medicine image and the first medical image,
Illustratively, it can be obtained by calculating the pixel value difference of the first medical image and tertiary medicine image one by one, and for second
Style and features difference between medical image and tertiary medicine image then can carry out quantization meter by such as gram matrix
It calculates.Gram matrix can calculate the correlation between image style and features, embody what each style and features occurred in the picture
Amount, to obtain the stylistic differences data between the second medical image and tertiary medicine image.It is understood that above content
Variance data and stylistic differences data can also can quantify the meter of content deltas and stylistic differences between picture by other
Calculation mode obtains.
In one embodiment, above-mentioned steps S1275 includes:
The parameter of first nerves network is updated by the first medical image of multiple groups and the second medical image.
In the case that loss function reaches the preset condition of convergence, the parameter of first nerves network is determined.
Specifically, in order to improve the accuracy that neural network model carries out the conversion of image style, multiple groups first can be passed through
Medical image and the second medical image are as training data training first nerves network.By multiple groups training data (including first
Medical image, the second medical image and tertiary medicine image) training, when the loss function obtained by nervus opticus network reaches
When to the preset condition of convergence, which can be applied in first nerves network.The specific condition of convergence can root
It determines, such as can be judged by the slope of loss function convergence curve according to the demand converted to style, work as convergence curve
When reaching steady, corresponding network parameter is determined.In this way, the first medical image and can be passed through in first nerves network
Two medical images are trained the model, and advanced optimize, i.e., the loss function can be by the first medical image and
Two medical images are advanced optimized.Here, the loss function that nervus opticus network above-mentioned obtains can be considered as model
Initial abstraction function.
Fig. 4 is the structural schematic diagram of the style conversion equipment of one embodiment traditional Chinese medicine image, as shown in figure 4, at one
In embodiment, the style conversion equipment 300 of medical image includes: that network obtains module 320, for obtaining neural network model；
Image collection module 340, for obtaining original medical image；Image conversion module 360, for using original medical image as mind
Input through network model, to obtain target medical image using neural network model；Wherein, the content of original medical image is special
The difference of the content characteristic of sign and target style medical image is in preset threshold value, the style and features and mesh of original medical image
The style and features for marking style medical image are different.
Specifically, network obtains module 320 and obtains neural network model, which is that preparatory train can
The neural network model of image style conversion is carried out, which can store in the storage mediums such as hard disk or flash memory or online
It is stored on cloud, image collection module 340 obtains original medical image, and original medical image is the figure for needing to carry out style conversion
Picture is generally scanned shooting by corresponding medical imaging devices and obtains, and image conversion module 360 is by image collection module 340
The original medical image of acquisition is input to network and obtains in the neural network model that module 320 obtains, and obtains neural network model
The target medical image of output, the difference of the content characteristic of the content characteristic and original medical image of target medical image is default
Threshold value in, and style and features are converted to the style and features of needs.
The style conversion equipment 300 of above-mentioned medical image, by the neural network based on Style Transfer algorithm, by original doctor
It learns image and is converted into the different target medical image of style and features, can be suitable for carrying out figure between the medical image of various mode
As style conversion, and the change for being able to maintain the content characteristic of image does not easily cause distortion in preset threshold value, facilitates doctor
The medical image of the target medical image being converted to and other mode is compared into observation.
In one embodiment, a kind of medical supply is provided, including memory, processor and storage are on a memory and can
The computer program run on a processor, processor execute following steps when executing the program: obtaining neural network model；It obtains
Take original medical image；Using original medical image as the input of neural network model, to obtain mesh using neural network model
Mark medical image；Wherein, the content characteristic of original medical image is with the difference of the content characteristic of target medical image preset
In threshold value, the style and features of original medical image are different from the style and features of target medical image.
In one embodiment, a kind of computer readable storage medium is provided, is deposited on the computer readable storage medium
Computer program is contained, which may make processor to execute following steps when being executed by processor: obtaining neural network model；
Obtain original medical image；Using original medical image as the input of neural network model, to be obtained using neural network model
Target medical image；Wherein, the difference of the content characteristic of the content characteristic of original medical image and target medical image is default
Threshold value in, the style and features of original medical image are different from the style and features of target medical image.
Fig. 5 is the structural schematic diagram of one embodiment traditional Chinese medicine picture system, as shown in figure 5, in one embodiment, doctor
Learning image system 500 includes: imaging device 520, for shooting original medical image；And above-mentioned medical supply 540, it is used for
Original medical image is converted into target medical image.
It specifically, include the medical supply 540 and imaging device for being in communication with each other connection in medical image system 500
520.Imaging device 520 can be the equipment for obtaining medical image such as CT or MR, in imaging device 520 to area-of-interest
It is scanned after shooting obtains original medical image, original medical image is sent to medical supply 540,540 benefit of medical supply
Original medical image be converted to target medical image with neural network model, the content characteristic of target medical image with
The difference of the content characteristic of original medical image is in preset threshold value, and style and features are converted into the image style of needs, just
Yu doctor is applied to diagnosing and treating.Such as when the operation such as being punctured or being melted, imaging device 520 can be passed through in the preoperative
CT image is shot to operative site, then preoperative CT image is converted to by ultrasonic style image by medical supply 540, is being carried out
When operation, so that it may the real-time ultrasonic image of the paired observation ultrasound style and ultrasonic probe acquisition, to be carried out to operation
Convenient accurately guidance.
Above-mentioned medical image system 500 is shot imaging device 520 by the neural network based on Style Transfer algorithm
Original medical image the different target medical image of style and features is converted by medical supply 540, can be suitable for various
Different types of imaging device, and the change for being able to maintain the content characteristic of image does not easily cause distortion in preset threshold value,
Facilitate doctor that the medical image of the target medical image being converted to and other mode is compared observation.
It is above-mentioned that the computer-readable restriction for depositing storage medium and computer equipment may refer to above for method
Specific restriction, details are not described herein.
It should be noted that those of ordinary skill in the art will appreciate that realizing all or part of stream in the above method
Journey is relevant hardware can be instructed to complete by computer program, which can be stored in one and computer-readable deposit
In storage media；Above-mentioned program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, above-mentioned storage is situated between
Matter can be magnetic disk, CD, read-only memory (Read-Only Memory, abbreviation ROM) or random access memory
(Random Access Memory, abbreviation RAM) etc..
Each technical characteristic of embodiment described above can be combined arbitrarily, for simplicity of description, not to above-mentioned reality
It applies all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not deposited
In contradiction, all should be considered as described in this specification.
The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously
It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art
It says, without departing from the concept of this application, various modifications and improvements can be made, these belong to the protection of the application
Range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.
Claims (13)
1. a kind of style conversion method of medical image characterized by comprising
Obtain neural network model；
Obtain original medical image；
Using the original medical image as the input of the neural network model, to obtain mesh using the neural network model
Mark medical image；
Wherein, the content characteristic of the original medical image is with the difference of the content characteristic of the target medical image preset
In threshold value, the style and features of the original medical image are different from the style and features of the target medical image.
2. the method according to claim 1, wherein the acquisition neural network model includes:
Create first nerves network；
Obtain the first medical image based on the first image mode and based on the second imaging for being different from first image mode
Second medical image of mode；
Using first medical image as the input of the first nerves network, the of the first nerves network output is obtained
Three medical images；
Based on first medical image, second medical image and the tertiary medicine image, training first mind
Through network；
The neural network model after being trained；
Wherein, the content characteristic of second medical image is with the difference of the content characteristic of first medical image preset
In threshold value, the style and features of second medical image are different from the style and features of first medical image.
3. according to the method described in claim 2, it is characterized in that, the image mode of the medical image includes computerized tomography
At least one of scan image, magnetic resonance image, radioscopic image, ultrasound image and molecule image.
4. according to the method described in claim 2, it is characterized in that, described be based on first medical image, second doctor
Image and the tertiary medicine image are learned, the training first nerves network includes:
Obtain nervus opticus network；
Using first medical image, second medical image and the tertiary medicine image as the nervus opticus net
The input of network, to calculate loss function；
The parameter of the first nerves network is updated by the loss function.
5. according to the method described in claim 4, it is characterized in that, described by first medical image, second medicine
The input of image and the tertiary medicine image as the nervus opticus network includes: to calculate loss function
Compare the content characteristic of first medical image and the tertiary medicine image, to obtain content deltas data；
Compare the style and features of second medical image and the tertiary medicine image, to obtain stylistic differences data；
The content deltas data and the stylistic differences data weighting are summed, to obtain the loss function.
6. according to the method described in claim 5, it is characterized in that, second medical image and third doctor
Image is learned, includes: to obtain stylistic differences data
Quantify the style and features difference of second medical image and the tertiary medicine image, by gram matrix to obtain
The stylistic differences data.
7. according to the method described in claim 4, it is characterized in that, described update the first nerves by the loss function
The parameter of network includes:
The parameter of the first nerves network is updated by the first medical image described in multiple groups and second medical image；
In the case that the loss function reaches the preset condition of convergence, the parameter of the first nerves network is determined.
8. method as claimed in any of claims 1 to 7, which is characterized in that the first nerves network includes
At least one of LeNet, GoogLeNet, VGGNet, ResNet, DenseNet, VNet and UNet.
9. method according to any one of claims 4 to 7, which is characterized in that the nervus opticus network includes
At least one of LeNet, GoogLeNet, VGGNet, ResNet, DenseNet, VNet and UNet.
10. a kind of style conversion equipment of medical image characterized by comprising
Network obtains module, for obtaining neural network model；
Image collection module, for obtaining original medical image；
Image conversion module, for using the original medical image as the input of the neural network model, described in utilizing
Neural network model obtains target style medical image；
Wherein, the content characteristic of the original medical image is with the difference of the content characteristic of the target style medical image pre-
If threshold value in, the style and features of the original medical image are different from the style and features of the target style medical image.
11. a kind of medical supply including memory, processor and stores the calculating that can be run on a memory and on a processor
Machine program, which is characterized in that the processor realizes any one of claim 1 to 9 the method when executing described program
The step of.
12. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The step of any one of claim 1 to 9 the method is realized when execution.
13. a kind of Medical Imaging System characterized by comprising
Imaging device, for shooting the original medical image；And
Medical supply described in claim 11, for the original medical image to be converted to the target medical image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811150272.4A CN109166087A (en) | 2018-09-29 | 2018-09-29 | Style conversion method, device, medical supply, image system and the storage medium of medical image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811150272.4A CN109166087A (en) | 2018-09-29 | 2018-09-29 | Style conversion method, device, medical supply, image system and the storage medium of medical image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109166087A true CN109166087A (en) | 2019-01-08 |
Family
ID=64892997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811150272.4A Pending CN109166087A (en) | 2018-09-29 | 2018-09-29 | Style conversion method, device, medical supply, image system and the storage medium of medical image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109166087A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109745062A (en) * | 2019-01-30 | 2019-05-14 | 腾讯科技（深圳）有限公司 | Generation method, device, equipment and the storage medium of CT image |
| CN109977905A (en) * | 2019-04-04 | 2019-07-05 | 北京百度网讯科技有限公司 | Method and apparatus for handling eye fundus image |
| CN110399924A (en) * | 2019-07-26 | 2019-11-01 | 北京小米移动软件有限公司 | A kind of image processing method, device and medium |
| CN110782410A (en) * | 2019-10-24 | 2020-02-11 | 中国人民解放军陆军军医大学第一附属医院 | Ultrasonic imaging method based on CT image |
| CN110910351A (en) * | 2019-10-31 | 2020-03-24 | 上海交通大学 | Ultrasound image modality migration and classification method and terminal based on generation countermeasure network |
| CN111127309A (en) * | 2019-12-12 | 2020-05-08 | 杭州格像科技有限公司 | Portrait style transfer model training method, portrait style transfer method and device |
| CN111161132A (en) * | 2019-11-15 | 2020-05-15 | 上海联影智能医疗科技有限公司 | System and method for image style conversion |
| CN111724450A (en) * | 2019-03-20 | 2020-09-29 | 上海科技大学 | Medical image reconstruction system, method, terminal and medium based on deep learning |
| CN111723840A (en) * | 2020-05-08 | 2020-09-29 | 天津大学 | Clustering and style migration method for ultrasonic images |
| CN111815506A (en) * | 2020-07-17 | 2020-10-23 | 上海眼控科技股份有限公司 | Image generation method and device, computer equipment and storage medium |
| CN112541856A (en) * | 2020-12-07 | 2021-03-23 | 重庆邮电大学 | Medical image style migration method combining Markov field and Graham matrix characteristics |
| CN112561814A (en) * | 2020-06-16 | 2021-03-26 | 上海联影智能医疗科技有限公司 | Image conversion apparatus and method |
| CN112700365A (en) * | 2019-10-22 | 2021-04-23 | 财团法人工业技术研究院 | Image conversion method and image conversion network |
| CN113327673A (en) * | 2020-02-28 | 2021-08-31 | 株式会社日立制作所 | Medical imaging apparatus, medical image processing apparatus, and image processing program |
| JP2021530780A (en) * | 2019-05-05 | 2021-11-11 | 平安科技(深▲せん▼)有限公司Ping An Technology (Shenzhen) Co., Ltd. | Focal determination method, device, computer equipment and storage medium based on image conversion |
| CN114611568A (en) * | 2022-01-18 | 2022-06-10 | 北京邮电大学 | Cross-equipment generalization detection method of medical image diagnosis and treatment product and related equipment |
| WO2023060735A1 (en) * | 2021-10-11 | 2023-04-20 | 中国科学院深圳先进技术研究院 | Image generation model training and image generation methods, apparatus, device, and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107103331A (en) * | 2017-04-01 | 2017-08-29 | 中北大学 | A kind of image interfusion method based on deep learning |
| US20180144509A1 (en) * | 2016-09-02 | 2018-05-24 | Artomatix Ltd. | Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures |
| CN108537720A (en) * | 2017-03-01 | 2018-09-14 | 杭州九言科技股份有限公司 | A kind of image processing method and device |
| CN108537776A (en) * | 2018-03-12 | 2018-09-14 | 维沃移动通信有限公司 | A kind of image Style Transfer model generating method and mobile terminal |
| CN108596830A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of image Style Transfer model training method and image Style Transfer method |
-
2018
- 2018-09-29 CN CN201811150272.4A patent/CN109166087A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180144509A1 (en) * | 2016-09-02 | 2018-05-24 | Artomatix Ltd. | Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures |
| CN108537720A (en) * | 2017-03-01 | 2018-09-14 | 杭州九言科技股份有限公司 | A kind of image processing method and device |
| CN107103331A (en) * | 2017-04-01 | 2017-08-29 | 中北大学 | A kind of image interfusion method based on deep learning |
| CN108537776A (en) * | 2018-03-12 | 2018-09-14 | 维沃移动通信有限公司 | A kind of image Style Transfer model generating method and mobile terminal |
| CN108596830A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of image Style Transfer model training method and image Style Transfer method |
Non-Patent Citations (1)
| Title |
|---|
| 谢璐: "数字人脸图像皮肤美化及整体风格化关键技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109745062A (en) * | 2019-01-30 | 2019-05-14 | 腾讯科技（深圳）有限公司 | Generation method, device, equipment and the storage medium of CT image |
| US12016717B2 (en) | 2019-01-30 | 2024-06-25 | Tencent Technology (Shenzhen) Company Limited | CT image generation method and apparatus, computer device, and computer-readable storage medium |
| WO2020156195A1 (en) * | 2019-01-30 | 2020-08-06 | 腾讯科技（深圳）有限公司 | Ct image generation method and apparatus, computer device and computer-readable storage medium |
| CN111724450A (en) * | 2019-03-20 | 2020-09-29 | 上海科技大学 | Medical image reconstruction system, method, terminal and medium based on deep learning |
| CN109977905A (en) * | 2019-04-04 | 2019-07-05 | 北京百度网讯科技有限公司 | Method and apparatus for handling eye fundus image |
| CN109977905B (en) * | 2019-04-04 | 2021-08-06 | 北京百度网讯科技有限公司 | Method and apparatus for processing fundus images |
| JP7064050B2 (en) | 2019-05-05 | 2022-05-09 | 平安科技（深▲せん▼）有限公司 | Focal determination method, device, computer equipment and storage medium based on image conversion |
| JP2021530780A (en) * | 2019-05-05 | 2021-11-11 | 平安科技(深▲せん▼)有限公司Ping An Technology (Shenzhen) Co., Ltd. | Focal determination method, device, computer equipment and storage medium based on image conversion |
| CN110399924A (en) * | 2019-07-26 | 2019-11-01 | 北京小米移动软件有限公司 | A kind of image processing method, device and medium |
| US11120604B2 (en) | 2019-07-26 | 2021-09-14 | Beijing Xiaomi Mobile Software Co., Ltd. | Image processing method, apparatus, and storage medium |
| CN110399924B (en) * | 2019-07-26 | 2021-09-07 | 北京小米移动软件有限公司 | Image processing method, device and medium |
| TWI730467B (en) * | 2019-10-22 | 2021-06-11 | 財團法人工業技術研究院 | Method of transforming image and network for transforming image |
| CN112700365A (en) * | 2019-10-22 | 2021-04-23 | 财团法人工业技术研究院 | Image conversion method and image conversion network |
| US11238624B2 (en) | 2019-10-22 | 2022-02-01 | Industrial Technology Research Institute | Image transform method and image transform network |
| CN110782410A (en) * | 2019-10-24 | 2020-02-11 | 中国人民解放军陆军军医大学第一附属医院 | Ultrasonic imaging method based on CT image |
| CN110782410B (en) * | 2019-10-24 | 2022-03-18 | 中国人民解放军陆军军医大学第一附属医院 | Ultrasonic imaging method based on CT image |
| CN110910351A (en) * | 2019-10-31 | 2020-03-24 | 上海交通大学 | Ultrasound image modality migration and classification method and terminal based on generation countermeasure network |
| CN110910351B (en) * | 2019-10-31 | 2022-07-29 | 上海交通大学 | Ultrasound image modality migration and classification method and terminal based on generation countermeasure network |
| CN111161132B (en) * | 2019-11-15 | 2024-03-05 | 上海联影智能医疗科技有限公司 | System and method for image style conversion |
| CN111161132A (en) * | 2019-11-15 | 2020-05-15 | 上海联影智能医疗科技有限公司 | System and method for image style conversion |
| CN111127309B (en) * | 2019-12-12 | 2023-08-11 | 杭州格像科技有限公司 | Portrait style migration model training method, portrait style migration method and device |
| CN111127309A (en) * | 2019-12-12 | 2020-05-08 | 杭州格像科技有限公司 | Portrait style transfer model training method, portrait style transfer method and device |
| CN113327673A (en) * | 2020-02-28 | 2021-08-31 | 株式会社日立制作所 | Medical imaging apparatus, medical image processing apparatus, and image processing program |
| CN111723840A (en) * | 2020-05-08 | 2020-09-29 | 天津大学 | Clustering and style migration method for ultrasonic images |
| CN112561814A (en) * | 2020-06-16 | 2021-03-26 | 上海联影智能医疗科技有限公司 | Image conversion apparatus and method |
| CN112561814B (en) * | 2020-06-16 | 2024-06-07 | 上海联影智能医疗科技有限公司 | Image conversion apparatus and method |
| CN111815506A (en) * | 2020-07-17 | 2020-10-23 | 上海眼控科技股份有限公司 | Image generation method and device, computer equipment and storage medium |
| CN112541856B (en) * | 2020-12-07 | 2022-05-03 | 重庆邮电大学 | Medical image style migration method combining Markov field and Graham matrix characteristics |
| CN112541856A (en) * | 2020-12-07 | 2021-03-23 | 重庆邮电大学 | Medical image style migration method combining Markov field and Graham matrix characteristics |
| WO2023060735A1 (en) * | 2021-10-11 | 2023-04-20 | 中国科学院深圳先进技术研究院 | Image generation model training and image generation methods, apparatus, device, and medium |
| CN114611568A (en) * | 2022-01-18 | 2022-06-10 | 北京邮电大学 | Cross-equipment generalization detection method of medical image diagnosis and treatment product and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109166087A (en) | Style conversion method, device, medical supply, image system and the storage medium of medical image | |
| US11534136B2 (en) | Three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging | |
| CN107203989A (en) | End-to-end chest CT image dividing method based on full convolutional neural networks | |
| CN108537794A (en) | Medical image processing method, device and computer readable storage medium | |
| CN106651750A (en) | Method and system used for 2D/3D image registration based on convolutional neural network regression | |
| CN110503652B (en) | Method and device for determining relationship between mandible wisdom tooth and adjacent teeth and mandible tube, storage medium and terminal | |
| CN108171272A (en) | A kind of evaluation method and device of Medical Imaging Technology | |
| CN111260748B (en) | Digital synthesis X-ray tomography method based on neural network | |
| CN107909622A (en) | Model generating method, the scanning planing method of medical imaging and medical image system | |
| Mahapatra et al. | Pathological retinal region segmentation from oct images using geometric relation based augmentation | |
| CN103295234B (en) | Based on the medical image segmentation system and method for deformation surface model | |
| CN103069456A (en) | Dual modality imaging including quality metrics | |
| CN110766694B (en) | Interactive segmentation method of three-dimensional medical image | |
| CN111091616A (en) | Method and device for reconstructing three-dimensional ultrasonic image | |
| CN110246580A (en) | Cranium silhouette analysis method and system based on neural network and random forest | |
| CN111340209A (en) | Network model training method, image segmentation method and focus positioning method | |
| CN111814891A (en) | Medical image synthesis method, device and storage medium | |
| CN108198235A (en) | A kind of three dimentional reconstruction method, apparatus, equipment and storage medium | |
| CN113159040A (en) | Method, device and system for generating medical image segmentation model | |
| CN112598649A (en) | 2D/3D spine CT non-rigid registration method based on generation of countermeasure network | |
| CN111080676B (en) | Method for tracking endoscope image sequence feature points through online classification | |
| CN114792326A (en) | Surgical navigation point cloud segmentation and registration method based on structured light | |
| CN101390132A (en) | Automated robust learning of geometries for mr-examinations | |
| CN109767429B (en) | Image screening method and device | |
| Shi et al. | A Virtual Monochromatic Imaging Method for Spectral CT Based on Wasserstein Generative Adversarial Network With a Hybrid Loss. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 201800 No. 2258 Chengbei Road, Jiading District, Shanghai Applicant after: Shanghai Lianying Medical Technology Co., Ltd Address before: 201800 No. 2258 Chengbei Road, Jiading District, Shanghai Applicant before: SHANGHAI UNITED IMAGING HEALTHCARE Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190108 |