CN110288568A - Method for processing fundus images, device, equipment and storage medium - Google Patents

Method for processing fundus images, device, equipment and storage medium Download PDF

Info

Publication number
CN110288568A
CN110288568A CN201910443966.5A CN201910443966A CN110288568A CN 110288568 A CN110288568 A CN 110288568A CN 201910443966 A CN201910443966 A CN 201910443966A CN 110288568 A CN110288568 A CN 110288568A
Authority
CN
China
Prior art keywords
matrix
eye fundus
fundus image
image
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910443966.5A
Other languages
Chinese (zh)
Inventor
杨叶辉
杨大陆
许言午
王磊
黄艳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Confucius Health Technology Co.,Ltd.
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201910443966.5A priority Critical patent/CN110288568A/en
Publication of CN110288568A publication Critical patent/CN110288568A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

The application provides a kind of method for processing fundus images, device, equipment and storage medium, wherein, this method comprises: eye fundus image is input in first network model, obtain the first matrix, it include multiple images block in eye fundus image, it is corresponded between the first matrix dot of each of first matrix and each image block, a kind of suspected abnormality type of the numerical representation method of each the first matrix dot；Processing is extended to the first matrix and obtains the second matrix；Second matrix and eye fundus image are combined to obtain weighting lesion figure；Weighting lesion figure is input in the second network model, obtains eye fundus image analysis as a result, eye fundus image analysis result includes the lesion rank of eye fundus image and the lesion type of each image block.Due to having merged the detailed information in the Global Information of eye fundus image and the image block of part, accurately eye fundus image is analyzed and identified, obtains accurate eye fundus image analysis as a result, improving analysis and the identification precision of eye fundus image.

Description

Method for processing fundus images, device, equipment and storage medium
Technical field
The invention relates to image technique field more particularly to a kind of method for processing fundus images, device, equipment and Storage medium.
Background technique
With the development of image technique, image technique starts to be applied to every field, such as is applied to medical field, public Security fields, field of face identification etc..It wherein, in the medical field, can be using the side such as image analysis, processing and identification Formula handles medical image, obtains image analysis result；Then medical staff can tie according to image analysis result The medical practice for closing itself is differentiated, and then medical staff analyzes illness and lesion.Wherein, for eye fundus image, Eye fundus image is analyzed by the way of image analysis, has begun to obtain more application.
In the prior art, eye fundus image can be divided into image block, then using existing image block rank mark Detection mode analyzes image block, obtains eye fundus image analysis as a result, eye fundus image analysis result includes eye fundus image Lesion rank and lesion type；Wherein, lesion rank is, for example, slight, moderate, severe.
However in the prior art, the detection mode of image block rank mark is limited to the analysis to image block, has ignored figure As the global context information in the eye fundus image except block, the analysis of the eye fundus image obtained from result is inaccurate, thus, eye The precision of analysis and the identification of base map picture is lower.
Summary of the invention
The embodiment of the present application provides a kind of method for processing fundus images, device, equipment and storage medium, existing for solving The problems in technology.
The application first aspect provides a kind of method for processing fundus images, comprising:
Eye fundus image to be processed is input in first network model, the first matrix is obtained, wherein the eye fundus image In include multiple images block, first matrix includes multiple first matrix dots, each described first matrix dot and each It is corresponded between described image block, a kind of suspected abnormality type of the numerical representation method of each first matrix dot；
Processing is extended to first matrix, obtains the second matrix, the size of second matrix and the eyeground The size of image is identical；
Second matrix is combined with the eye fundus image, obtains weighting lesion figure；
The weighting lesion figure is input in the second network model, obtains eye fundus image analysis result, wherein the eye Bottom image analysis result includes the lesion rank of the eye fundus image and the lesion type of each described image block.
Optionally, eye fundus image to be processed is input in first network model described, before obtaining the first matrix, Further include:
Pre-training is carried out to initial first network model using preset lesion type data, the after obtaining pre-training One network model；
Pre-training is carried out to initial the second network model using preset lesion rank data, the after obtaining pre-training Two network models；
First network model after the pre-training and the second network model after the pre-training are attached, and adopted With the lesion rank data to after connection first network model and the second network model be trained, the after obtaining training The second network model after one network model and training.
Optionally, processing is extended to first matrix, obtains the second matrix, comprising:
Total head expansion is carried out to the first matrix dot of each of first matrix, obtains second matrix, wherein Second matrix includes multiple matrix-blocks, and one is a pair of between each described matrix-block and each described first matrix dot It answers, includes multiple second matrix dots in each described matrix-block, each second matrix dot in each described matrix-block The numerical value of a kind of suspected abnormality type of numerical representation method, each second matrix dot in each described matrix-block is right for matrix-block The numerical value for the first matrix dot answered.
Optionally, described that eye fundus image to be processed is input in first network model, obtain the first matrix, comprising:
Eye fundus image to be processed is input in first network model, first matrix and third matrix are exported, In, the third matrix includes multiple third matrix dots, each described third matrix dot and each described first matrix dot Between correspond, the first matrix dot corresponding to the numerical representation method of each third matrix dot third matrix dot is doubted Like the probability of lesion type；
Processing is extended to first matrix, obtains the second matrix, comprising:
First matrix and the third matrix are combined, the 4th matrix is obtained, wherein the 4th matrix Size is identical as the size of first matrix, includes multiple 4th matrix dots in the 4th matrix；
Total head expansion is carried out to the 4th matrix dot of each of the 4th matrix, obtains second matrix, wherein Second matrix includes multiple matrix-blocks, and one is a pair of between each described matrix-block and each described 4th matrix dot It answers, includes multiple second matrix dots in each described matrix-block, each second matrix dot in each described matrix-block The numerical value of a kind of suspected abnormality type of numerical representation method, each second matrix dot in each described matrix-block is right for matrix-block The numerical value for the 4th matrix dot answered.
Optionally, the 4th matrix is LP=(L+1) ⊙ P；Wherein, L is first matrix, and P is the third square Battle array.
Optionally, the weighting lesion figure is WM=W ⊙ Iori, wherein IoriFor the eye fundus image, W is described second Matrix.
Optionally, the receptive field of each node of the output layer in the first network model covers only described each The range of image block corresponding to node.
Optionally, eye fundus image to be processed is input in first network model described, before obtaining the first matrix, Further include:
Using preset image preprocessing mode, image preprocessing is carried out to the eye fundus image to be processed, is obtained everywhere Eye fundus image after reason, wherein described image pretreatment mode is for the illumination point in the balanced eye fundus image to be processed Cloth.
Optionally, described image pretreatment mode is Gaussian smoothing filter mode；
It is described that treated that eye fundus image isWherein, IoriFor the eye to be processed Base map picture, G (θ) are preset Gaussian kernel, and θ is preset Gaussian kernel dimensional parameters, and α, β, γ are preset hyper parameter.
Optionally, first matrix is three-dimensional matrice, and second matrix is three-dimensional matrice, and the third matrix is three Matrix is tieed up, the 4th matrix is three-dimensional matrice.
The application second aspect provides a kind of eye fundus image processing unit, comprising:
First processing units obtain the first matrix for eye fundus image to be processed to be input in first network model, It wherein, include multiple images block in the eye fundus image, first matrix includes multiple first matrix dots, each described the It is corresponded between one matrix dot and each described image block, the numerical representation method of each first matrix dot is a kind of to doubt Like lesion type；
Expanding element obtains the second matrix for being extended processing to first matrix, second matrix it is big It is small identical as the size of the eye fundus image；
Combining unit obtains weighting lesion figure for second matrix to be combined with the eye fundus image；
The second processing unit obtains eye fundus image point for the weighting lesion figure to be input in the second network model Analyse result, wherein the eye fundus image analysis result includes the lesion rank and each described image block of the eye fundus image Lesion type.
Optionally, described device, further includes:
First training unit, for eye fundus image to be processed to be input to first network mould in the first processing units In type, before obtaining the first matrix, pre-training is carried out to initial first network model using preset lesion type data, is obtained First network model after to pre-training；
Second training unit, for being instructed in advance using preset lesion rank data to the second initial network model Practice, the second network model after obtaining pre-training；
Third training unit, for by the first network model after the pre-training and the second network after the pre-training Model is attached, and using the lesion rank data to after connection first network model and the second network model instruct Practice, the second network model after first network model and training after obtaining training.
Optionally, the expanding element, is specifically used for:
Total head expansion is carried out to the first matrix dot of each of first matrix, obtains second matrix, wherein Second matrix includes multiple matrix-blocks, and one is a pair of between each described matrix-block and each described first matrix dot It answers, includes multiple second matrix dots in each described matrix-block, each second matrix dot in each described matrix-block The numerical value of a kind of suspected abnormality type of numerical representation method, each second matrix dot in each described matrix-block is right for matrix-block The numerical value for the first matrix dot answered.
Optionally, the first processing units, are specifically used for:
Eye fundus image to be processed is input in first network model, first matrix and third matrix are exported, In, the third matrix includes multiple third matrix dots, each described third matrix dot and each described first matrix dot Between correspond, the first matrix dot corresponding to the numerical representation method of each third matrix dot third matrix dot is doubted Like the probability of lesion type；
The expanding element, is specifically used for:
First matrix and the third matrix are combined, the 4th matrix is obtained, wherein the 4th matrix Size is identical as the size of first matrix, includes multiple 4th matrix dots in the 4th matrix；
Total head expansion is carried out to the 4th matrix dot of each of the 4th matrix, obtains second matrix, wherein Second matrix includes multiple matrix-blocks, and one is a pair of between each described matrix-block and each described 4th matrix dot It answers, includes multiple second matrix dots in each described matrix-block, each second matrix dot in each described matrix-block The numerical value of a kind of suspected abnormality type of numerical representation method, each second matrix dot in each described matrix-block is right for matrix-block The numerical value for the 4th matrix dot answered.
Optionally, the 4th matrix is LP=(L+1) ⊙ P；Wherein, L is first matrix, and P is the third square Battle array.
Optionally, the weighting lesion figure is WM=W ⊙ Iori, wherein IoriFor the eye fundus image, W is described second Matrix.
Optionally, the receptive field of each node of the output layer in the first network model covers only described each The range of image block corresponding to node.
Optionally, described device, further includes:
Third processing unit, for eye fundus image to be processed to be input to first network mould in the first processing units In type, before obtaining the first matrix, using preset image preprocessing mode, image is carried out to the eye fundus image to be processed Pretreatment, the eye fundus image that obtains that treated, wherein described image pretreatment mode is for the balanced eyeground figure to be processed Illumination patterns as in.
Optionally, described image pretreatment mode is Gaussian smoothing filter mode；
It is described that treated that eye fundus image isWherein, IoriFor the eye to be processed Base map picture, G (θ) are preset Gaussian kernel, and θ is preset Gaussian kernel dimensional parameters, and α, β, γ are preset hyper parameter.
Optionally, first matrix is three-dimensional matrice, and second matrix is three-dimensional matrice, and the third matrix is three Matrix is tieed up, the 4th matrix is three-dimensional matrice.
The application third aspect provides a kind of eye fundus image processing equipment, comprising: transmitter, receiver, memory and place Manage device；
The memory is for storing computer instruction；The processor by run memory storage it is described based on The method that any implementation of first aspect provides is realized in the instruction of calculation machine.
The application fourth aspect provides a kind of storage medium, comprising: readable storage medium storing program for executing and computer instruction, the calculating Machine instruction is stored in the readable storage medium storing program for executing；The computer instruction provides for realizing any implementation of first aspect Method.
Method for processing fundus images, appliance arrangement and storage medium provided by the embodiments of the present application, by first to eyeground Each image block of image is analyzed, and the suspected abnormality type of each image block is obtained, and then obtains a three-dimensional square Battle array；Then, the size of three-dimensional matrice is expanded, obtains the second matrix identical with the size of eye fundus image；By the second square The information of battle array and the information of eye fundus image are combined, and are obtained weighting lesion figure, are included eye fundus image in the weighting lesion figure In detailed information and global Global Information；Then, to weighting lesion figure identify, obtain eye fundus image analysis as a result, into And obtain the lesion rank of eye fundus image and the lesion type of each image block.To due to having merged the whole of eye fundus image Detailed information in the image block of body information and part, accurately can be analyzed and be identified to eye fundus image, available Accurate eye fundus image analysis is as a result, improve analysis and the identification precision of eye fundus image.Further, medical staff According to the medical practice of itself, medical judgment further is carried out to eye fundus image and eye fundus image analysis result, and then determine The real cause of disease and lesion out.Also, the image block that the embodiment of the present application uses is analyzed, and does not use the analysis of pixel scale Mode can reduce time and the cost of image analysis, improve the analysis efficiency of eye fundus image；Also, compared to end-to-end Black box disaggregated model, second network model of the embodiment of the present application can directly export the disease of lesion rank He each block of pixels Stove type, the available more output informations of the embodiment of the present application.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this Shen Some embodiments please for those of ordinary skill in the art without any creative labor, can be with Other attached drawings are obtained according to these attached drawings.
Fig. 1 is a kind of flow chart of method for processing fundus images provided by the embodiments of the present application；
Fig. 2 is the flow chart of another method for processing fundus images provided by the embodiments of the present application；
Fig. 3 is image processing flow figure one provided by the present application；
Fig. 4 is the schematic diagram of the first matrix provided by the present application；
Fig. 5 is image processing flow figure two provided by the present application；
Fig. 6 is a kind of structural schematic diagram of eye fundus image processing unit provided by the embodiments of the present application；
Fig. 7 is the structural schematic diagram of another eye fundus image processing unit provided by the embodiments of the present application；
Fig. 8 is a kind of structural schematic diagram of eye fundus image processing equipment provided by the embodiments of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art All other embodiment obtained without creative efforts, shall fall in the protection scope of this application.
With the development of image technique, image technique starts to be applied to every field, such as is applied to medical field, public Security fields, field of face identification etc..It wherein, in the medical field, can be using the side such as image analysis, processing and identification Formula handles medical image, obtains image analysis result, and then medical staff can carry out according to image analysis result Differentiate, and then medical staff analyzes illness and lesion.Wherein, the health of eye is increasingly valued by people, and can be obtained Take eye fundus image；Then for eye fundus image, eye fundus image is analyzed by the way of image analysis, is had begun Obtain more application.
Following methods can be provided to analyze to eye fundus image.
First way, using end-to-end black box disaggregated model, black box disaggregated model uses depth network algorithm, by eye Base map picture is input in black box disaggregated model, exports Analysis of Ocular Fundus result；Analysis of Ocular Fundus result includes the lesion of eye fundus image The lesion type of rank and eye fundus image.
But above-mentioned first way, black box disaggregated model are limited to the characteristic of model itself, it can not be accurately to eyeground Image is analyzed, which position that lesion is located at eye fundus image can not be accurately analyzed；Also, black box disaggregated model relies on Learn automatically in black box deep learning model to the valuable feature of classification, in the limited situation of amount of training data, black box mould Type is often difficult study to the feature with Generalization Capability.
Eye fundus image can be divided into multiple by the second way using the detection algorithm marked based on image block rank Then image block is analyzed image block, eye fundus image analysis result is obtained.
But the above-mentioned second way, due to being division, the image analysis for carrying out image block for each eye fundus image, Confirm the eye fundus image analysis result of each eye fundus image；In image analysis process, it is limited to different lesion appearances Difference, distributional difference, size difference, are similarly limited to data volume, are difficult to detect accurate lesion, that is, obtained eyeground figure As analysis result is inaccurate.Also, the detection mode of image block rank mark is limited to the analysis to image block, has ignored figure As the global context information in the eye fundus image except block, that is, be easy to be interfered by the complex background in eye fundus image, such as blood Pipe, the image organizational that optic disk etc. is constituted.
The third mode can click through each pixel in eye fundus image using the partitioning algorithm marked based on Pixel-level Row analysis obtains eye fundus image analysis result.
But the third above-mentioned mode, it needs to carry out image analysis to each pixel in eye fundus image, causes algorithm Complexity, the process being labeled to each pixel is many and diverse, and the efficiency for obtaining analysis result is lower.
The application provides a kind of method for processing fundus images, appliance arrangement and storage medium, asks for solving above-mentioned technology Topic.
Fig. 1 is a kind of flow chart of method for processing fundus images provided by the embodiments of the present application, as shown in Figure 1, this method, Include:
S101, eye fundus image to be processed is input in first network model, obtains the first matrix, wherein eyeground figure It include multiple images block as in, the first matrix includes multiple first matrix dots, each first matrix dot and each image block Between correspond, a kind of suspected abnormality type of the numerical representation method of each the first matrix dot.
In this step, the present embodiment can be eye fundus image processing unit or eye fundus image processing with executing subject Equipment or other detection devices etc..
Obtain eye fundus image Iori, eye fundus image IoriIt is the color image as obtained from being scanned to eye, wherein Eye fundus image IoriWidth be M, eye fundus image IoriLength be N, that is, eye fundus image IoriWidth x length having a size of M × N.
Then, by eye fundus image IoriIt is input in the first network model of preset maturation, first network model can be right Eye fundus image IoriCarry out image analysis and identification.Specifically, eye fundus image IoriIt can be made of, can use multiple images block Grid is by eye fundus image IoriIt is divided into n × n block, that is, obtained n × n image block, included multiple in each image block Pixel；First network model is a kind of advanced neural network model, it is preferred that first network model is that stove grade attention is raw At network (lesion attention net) model；By eye fundus image IoriIt is input in first network model, passes through the first net Network model is to eye fundus image IoriCarry out whole image analysis and to eye fundus image IoriEach of image block carry out figure As analysis, first matrix dot is exported for each image block, and the first matrix dot has numerical value, the first matrix dot The numerical representation method suspected abnormality type of image block corresponding with the first matrix dot；In turn, by each image block corresponding first Matrix dot constitutes first matrix, and then can export first matrix L.It is found that each first matrix dot with it is each It is that correspondingly, the size of the first matrix L is n × n between a image block；(i, j) a first matrix in first matrix L The numerical value m of point belongs to [0, k], wherein m is integer, and k is the positive integer more than or equal to 1, and i ∈ [1, n], j ∈ [1, n], i, j are Positive integer；The numerical value m of (i, j) a first matrix dot, characterizing (i, j) a first matrix dot is m kind suspected abnormality class Type, that is, characterizing with (i, j) a first matrix dot corresponding (i, j) a image block is m kind suspected abnormality type.
And, it is preferred that due to eye fundus image IoriIt is color image, so that the first matrix is a three-dimensional matrice, that is, First matrix is a three-dimensional data.There are three faces for first matrix tool, can be corresponding in turn to three kinds of different image dimensions, example Such as, it is corresponding in turn in red (Red), green (Green), blue (Blue) three kinds of colors, alternatively, being corresponding in turn in tone (Hue), saturation Spend (Saturation), lightness (Value).For example, it is said from the angle of RGB, there are three faces for the first matrix tool, are corresponding in turn to Yu Hong (Red), green (Green), blue (Blue) three kinds of colors, and the data in every one side are then the intensity of these three colors respectively Value；For example, set the first matrix as X three-dimensional matrice (256,256,3), then X (::, 1) represents red two-dimensional matrix, X (::, 2) represents the two-dimensional matrix of green, and X (::, 3) represents blue two-dimensional matrix, wherein the first peacekeeping the is two-dimensional to be taken Value range is 0-255, and the value range of the third dimension is 1-3.So that the size of first matrix L is n × n × 3 for accurate.
S102, processing is extended to the first matrix, obtains the second matrix, the size of the second matrix and eye fundus image it is big It is small identical.
Optionally, step S102 is specifically included: being carried out total head expansion to the first matrix dot of each of first matrix, is obtained To the second matrix, wherein the second matrix includes multiple matrix-blocks, between each matrix-block and each first matrix dot one by one It is corresponding, it include multiple second matrix dots in each matrix-block, the numerical tabular of each second matrix dot in each matrix-block A kind of suspected abnormality type is levied, the numerical value of each second matrix dot in each matrix-block is the first square corresponding to matrix-block The numerical value of lattice point.
In this step, it for the detailed information and global Global Information in available eye fundus image, needs to generate one A weighting lesion figure.
Firstly, it is necessary to processing will be extended to the first matrix, obtain and eye fundus image IoriIdentical second square of size Battle array.Specifically, due to the eye fundus image I for being M × N by sizeoriIt divides for n × n image block, and acquired first square Each of battle array L is corresponded between the first matrix dot and each image block；For each of first matrix L first First matrix dot is copied as multiple second matrix dots by matrix dot, multiple second matrixes as obtained from first matrix dot Point may be constructed a matrix-block；And then the second matrix is constituted by each matrix-block.It is found that image block and the first matrix dot it Between correspond, corresponded between the first matrix dot and matrix-block, then, between image block and matrix-block be also correspond 's；The number of the second matrix dot in matrix-block, the number of the pixel in image block corresponding with matrix-block, the two is phase Together.Also, since the first matrix dot is copied as multiple second matrix dots, then belong to the of the same matrix-block in the second matrix The numerical value of two matrix dots is identical；Also, for the same matrix-block, the numerical value of the second matrix dot in matrix-block, It is the numerical value of the first matrix dot corresponding with matrix-block.In turn, the numerical representation method of each second matrix dot in each matrix-block A kind of suspected abnormality type.To carry out by the first matrix dot of each of first matrix L in such a way that total head is expanded The number of matrix dot is expanded, and obtains the second matrix W that size is M × N, the second matrix W is a weighting matrix.
For example, the eye fundus image I for being 1024 × 1024 by sizeori, divide for 16 × 16 image blocks；It obtains One 16 × 16 first matrix L；Then, expanding to each of the first matrix L matrix dot is 64 identical matrix dots, That is, expanding is 64 identical second matrix dots, 64 identical second matrix dots constitute a matrix-block；Each matrix-block, The second matrix W is constituted, and then has obtained the second matrix W that size is 1024 × 1024.
Also, the first matrix L is three-dimensional matrice, and for accurate, the second matrix W is also three-dimensional matrice, the second matrix W Size be M × N × 3.
In addition, when being extended processing to the first matrix L, it can also be by the way of the expansion of non-weight.That is, right In each the first matrix dot, the second different matrix dot of multiple numerical value, the number of the second matrix dot are generated according to the first matrix dot Value may belong to [A, B], wherein the numerical value of current first matrix dot of A, B are first matrix adjacent with current first matrix dot The numerical value of point；Multiple second matrix dots as obtained from first matrix dot may be constructed a matrix-block；And then by each Matrix-block constitutes the second matrix；This mode, and obtained the second matrix W that a size is M × N.But this side Formula, whether since the first adjacent matrix dot has corresponded to adjacent image block, it is uncertain for being associated with illness between image block, It can not constitute with the numerical value of adjacent corresponding first matrix dot of image block, the foundation of the value as the second matrix dot；From And obtained final image analysis result, and the obtained image analysis result of mode expanded not as good as total head.
S103, the second matrix and eye fundus image are combined, obtain weighting lesion figure.
Optionally, weighting lesion figure is WM=W ⊙ Iori, wherein IoriFor eye fundus image, W is the second matrix.
It in this step, can will be above-mentioned for the detailed information and global Global Information in available eye fundus image Second matrix W and eye fundus image IoriIt is combined, to obtain a weighting lesion figure.Preferably, by the second matrix W and eyeground Image Iori, point-to-point multiplication is carried out, weighting lesion figure WM=W ⊙ I is obtainedori.To which acquired weighting lesion figure WM is contained Original eye fundus image IoriIn general image information, that is, the environmental information of perilesional；Acquired weighting lesion figure WM is also Contain the information that each image block belongs to m kind suspected abnormality type.
S104, weighting lesion figure is input in the second network model, obtains eye fundus image analysis result, wherein eyeground Image analysis result includes the lesion rank of eye fundus image and the lesion type of each image block.
In this step, by the obtained weighting lesion figure WM of above-mentioned steps, it is input to the second network of preset maturation In model；Preferably, the second network model is sugared net hierarchical network models.Sugared net hierarchical network models can be arbitrary classics Classification pessimistic concurrency control, e.g., Inception-v3 model, residual error neural network (Residual Neural Network, ResNet) model, DenseNet model etc..
Then, the second network model is further analyzed and processed weighting lesion figure WM, and according to each figure As the suspected abnormality type of block, weighting lesion figure WM is further analyzed and is identified.Second network model exports eyeground The lesion rank of image；Lesion rank characterizes the lesion rank of eye fundus image, and lesion rank is, for example, slight, moderate, severe； Alternatively, lesion rank is divided into the presence or absence of lesion, that is, two ranks it have been divided into；Alternatively, lesion rank is divided into five ranks.Second Network model can also export the lesion type of each image block of eye fundus image, for example, the lesion of (i, j) a image block Type is m kind lesion type.
For example, if the eye fundus image to diabetes is analyzed, it includes eyeground figure that eye fundus image, which analyzes result, The lesion type of sugared the net rank and each image block of picture；Sugared net rank is, for example, eye fundus image with the presence or absence of sugared net, alternatively, Sugared net rank is, for example, that the sugared net rank of eye fundus image is slight or moderate or severe；Lesion type is, for example, m kind sugar net Lesion type.If the eye fundus image to macula lutea is analyzed, eye fundus image analysis result includes the macula lutea grade of eye fundus image Other and each image block lesion type；Macula lutea rank is, for example, eye fundus image with the presence or absence of macula lutea, alternatively, macula lutea rank example The macula lutea rank of eye fundus image in this way is slight or moderate or severe；Lesion type is, for example, m kind macula lutea lesion type.
The present embodiment obtains the first matrix by the way that eye fundus image to be processed to be input in first network model, In, include multiple images block in eye fundus image, the first matrix includes multiple first matrix dots, each first matrix dot with it is each It is corresponded between a image block, a kind of suspected abnormality type of the numerical representation method of each the first matrix dot；To the first matrix It is extended processing, obtains the second matrix, the size of the second matrix and the size of eye fundus image are identical；By the second matrix and eyeground Image is combined, and obtains weighting lesion figure；Weighting lesion figure is input in the second network model, eye fundus image analysis is obtained As a result, wherein it includes the lesion rank of eye fundus image and the lesion type of each image block that eye fundus image, which analyzes result,.Pass through Each image block of eye fundus image is analyzed first, obtains the suspected abnormality type of each image block, and then obtain One three-dimensional matrice；Then, the size of three-dimensional matrice is expanded, obtains the second square identical with the size of eye fundus image Battle array；The information of the information of second matrix and eye fundus image is combined, weighting lesion figure is obtained, includes in the weighting lesion figure Detailed information and global Global Information in eye fundus image；Then, weighting lesion figure is identified, obtains eye fundus image point Analysis as a result, obtain the lesion rank of eye fundus image and the lesion type of each image block in turn.To due to having merged eyeground Detailed information in the Global Information of image and the image block of part, accurately can be analyzed and be identified to eye fundus image, Available accurate eye fundus image analysis is as a result, improve analysis and the identification precision of eye fundus image.Further, Medical staff further carries out medical judgment to eye fundus image and eye fundus image analysis result according to itself medical practice, And then determine the real cause of disease and lesion.Also, the image block that the embodiment of the present application uses is analyzed, and Pixel-level is not used Other analysis mode can reduce time and the cost of image analysis, improve the analysis efficiency of eye fundus image；Also, compared to Second network model of end-to-end black box disaggregated model, the embodiment of the present application can directly export lesion rank and each picture The lesion type of plain block, the available more output informations of the embodiment of the present application.
Fig. 2 is the flow chart of another method for processing fundus images provided by the embodiments of the present application, as shown in Fig. 2, the party Method, comprising:
S201, pre-training is carried out to initial first network model using preset lesion type data, obtains pre-training First network model afterwards.
In this step, the present embodiment can be eye fundus image processing unit or eye fundus image processing with executing subject Equipment or other detection devices etc..
Firstly the need of the first network model for obtaining maturation and the second mature network model.
Lesion type data have been got in advance, wherein lesion type data include multiple eye fundus images, each eye Base map picture is made of multiple images block, each image block has corresponded to a kind of lesion type.By lesion type data, it is input to just In the first network model to be trained to begin；In turn, using lesion type data, pre-training is carried out to first network model, After the convergence of first network model, the first network model after obtaining pre-training.Preferably, first network model is lesion grade Attention generates network (lesion attention net) model.
S202, pre-training is carried out to the second initial network model using preset lesion rank data, obtains pre-training The second network model afterwards.
In this step, lesion rank data have been got in advance, wherein include multiple eyeground in lesion rank data Image, each eye fundus image have corresponded to a lesion rank.By lesion rank data, it is input to initial to be trained second In network model；In turn, pre-training is carried out to the second network model using lesion rank data, restrains it in the second network model Afterwards, the second network model after obtaining pre-training.Preferably, the second network model, for sugared net hierarchical network models.
S203, the first network model after pre-training and the second network model after pre-training are attached, and used Lesion rank data to after connection first network model and the second network model be trained, obtain training after first network The second network model after model and training.
In this step, after step S202, it is also necessary to after pre-training first network model and pre-training after Second network model, is further trained, and mature first network model and the second mature network model have been obtained.
Firstly, the first network model after pre-training and the second network model after pre-training are attached, that is, will be pre- The input layer of the output layer of first network model after training and the second network model after pre-training is attached, in turn, will First network model after pre-training and the second network model after pre-training are combined.
Then, using lesion rank data, again to after connection first network model and the second network model instruct Practice.That is, using two network models after the training connection simultaneously of lesion rank data, until convergence, to obtain mature lesion Grade attention generates network model and mature sugared net hierarchical network models.
S204, eye fundus image progress image preprocessing to be processed is obtained everywhere using preset image preprocessing mode Eye fundus image after reason, wherein image preprocessing mode is for the illumination patterns in balanced eye fundus image to be processed.
Optionally, image preprocessing mode is Gaussian smoothing filter mode；Treated, and eye fundus image is Wherein, IoriFor eye fundus image to be processed, G (θ) is preset Gaussian kernel, and θ is preset Gaussian kernel ruler Very little parameter, α, β, γ are preset hyper parameter.
In this step, eye fundus image I is obtainedori, eye fundus image IoriIt is the coloured silk as obtained from being scanned to eye Chromatic graph picture, wherein eye fundus image IoriWidth be M, eye fundus image IoriLength be N, that is, eye fundus image IoriWidth x length Having a size of M × N.
Then, it in order to enhance image readability, alleviate the problem of illumination patterns are brought to image readability, needs to eyeground Image IoriCarry out image preprocessing.It can be using modes such as histogram equalization or Gaussian smoothing filters, to eye fundus image IoriImage preprocessing is carried out, the eye fundus image I that obtains that treated, thus balanced eye fundus image IoriIn illumination patterns, alleviate The unbalanced problem of illumination in eye fundus image.
Wherein, the mode of image preprocessing is not limited to the modes such as histogram equalization, Gaussian smoothing filter；Image is located in advance The mode of reason, as long as can enough enhance image readability, alleviate the problem of illumination patterns are brought to image readability.
It for example, can be by eye fundus image IoriIt is input in Gaussian filter, to eye fundus image IoriCarry out figure As pretreatment, the eye fundus image that obtains that treatedWherein,For convolution operation, G (θ) is Preset Gaussian kernel, θ are preset Gaussian kernel dimensional parameters, and α, β, γ are preset hyper parameter.For example, α=4, β=- 4, γ =128.Eye fundus image I can be enhanced in Gaussian filteroriReadability.
S205, by treated, eye fundus image is input in first network model, obtains the first matrix and third matrix, In, include multiple images block in eye fundus image, the first matrix includes multiple first matrix dots, each first matrix dot with it is each It is corresponded between a image block, a kind of suspected abnormality type of the numerical representation method of each the first matrix dot.Third matrix packet Multiple third matrix dots are included, are corresponded between each third matrix dot and each first matrix dot, each third square The probability of the suspected abnormality type of first matrix dot corresponding to the numerical representation method of lattice point third matrix dot.
Optionally, it is right to cover only each node institute for the receptive field of each node of the output layer in first network model The range for the image block answered.
In this step, after step s 204, treated eye fundus image I is input to mature first network model In, first network model can eye fundus image I carries out image analysis and identification to treated.
Specifically, eye fundus image IoriIt can be made of multiple images block, wherein the pixel value of each image block characterizes Lesion information, i.e., each image block includes lesion information.To eye fundus image IoriAfter carrying out image preprocessing, obtain Treated eye fundus image I and original eye fundus image IoriSize be identical；Also, treated eye fundus image I by Multiple images block is constituted, and grid can be used eye fundus image IoriIt is divided into n × n block, that is, obtained n × n image block, often It include multiple pixels in one image block, the image block of treated eye fundus image I and original eye fundus image IoriFigure As being one-to-one between block.
By first network model, to treated, eye fundus image I carries out whole image analysis and to treated Each of eye fundus image I image block carries out image analysis, obtains first matrix L.Wherein, the first matrix L, Ke Yican The introduction of step S101 as shown in Figure 1, repeats no more.
Also, while obtaining the first matrix L, to treated, eye fundus image I carries out entirety to first network model Image analysis and image analysis is carried out to treated each of eye fundus image I image block, for each image block A third matrix dot can also be exported；, and third matrix dot has numerical value, the numerical representation method of third matrix dot and third The corresponding image block of matrix dot, the probability of belonging suspected abnormality type；In turn, by the corresponding third matrix of each image block Point constitutes a third matrix, and then can export a third matrix P.It is found that each third matrix dot and each figure As being that correspondingly, the size of third matrix P is n × n between block；Also, each third matrix dot and each first It is corresponded between matrix dot.The numerical value of (i, j) a third matrix dot in third matrix P is Pi,j, and Pi,j∈ [0,1], Likewise, i ∈ [1, n], j ∈ [1, n], i, j are positive integer；The numerical value P of (i, j) a third matrix doti,j, characterize (i, J) a image block is the probability of m kind suspected abnormality type, that is, characterizes doubting for the first matrix dot corresponding to third matrix dot Like the probability of lesion type.
Also, the first matrix L is three-dimensional matrice, and third matrix P is also three-dimensional matrice, it is accurate for, third matrix P's is big Small is n × n × 3.
Wherein, first network model has the feature that each for needing to guarantee to arrive output layer (feature map) The receptive field of node only covers the range of image block corresponding with each first matrix dot corresponding to each node；That is, The receptive field of each node of output layer (feature map), only covers the range of image block corresponding to each node, Image block is the image block in original eye fundus image.
In order to guarantee the receptive field of first network model, meets features above, need to be arranged the convolution of first network model The convolution kernel size of layer, padding, has continuous action relation between the number of plies at step-length.
For example, if original eye fundus image IoriLength and width dimensions be 1024 × 1024, thus, image preprocessing The length and width dimensions of eye fundus image I afterwards are 1024 × 1024；Rasterizing parameter n=16, original eye fundus image I are setoriHave 1024 × 1024 image blocks, eye fundus image I have 1024 × 1024 image blocks, and the size of each above-mentioned image block is 64 ×64.At this point, the convolutional layer of first network model, can be configured in the way of table 1.
The convolutional layer of 1 first network model of table
Channel type The size and number of convolution kernel Step-length Padding Output layer (Feature map) size
Input … … … 1024*1024*3
Convolutional layer 1 5*5*32 1 2 1020*1020*32
Convolutional layer 2 2*2*32 2 0 512*512*32
Convolutional layer 3 2*2*64 2 0 256*256*64
Convolutional layer 4 2*2*128 2 0 128*128*128
Convolutional layer 5 2*2*256 2 0 64*64*256
Convolutional layer 6 2*2*512 2 0 32*32*1024
Convolutional layer 7 2*2*1024 2 0 16*16*1024
Convolutional layer 8 1*1*1024 1 0 16*16*1024
In the examples described above, first network model has 8 layers of convolutional layer；In table 1 in " size and number of convolution kernel " before Two be convolution kernel size, last is the number of convolution kernel.In the examples described above, the convolution kernel ruler of each convolutional layer Very little, step-length, padding, there is continuous action relation between the number of plies, entire first network model is whole continuous action relation.Such as table 1 Shown, the receptive field of each node, covers eye in the feature map that the 7th convolutional layer and the 8th convolutional layer export 67 × 67 image blocks in base map picture；The receptive field of adjacent node, corresponding image block have the overlapping of 3 pixels, this In, it is to solve the problems, such as that lesion appears in image block close call.
Such as above-mentioned example it is found that the convolution kernel size of the last one convolutional layer of first network model is 1*1.Due to first The convolution kernel size of the last one convolutional layer of network model is 1*1, so the receptive field of each node of output layer, is only to cover Cover the range of image block corresponding to each node.Guarantee in turn the first matrix dot of each of the first obtained matrix L with It can be corresponded between each image block, and corresponding position will not generate entanglement.
For example, Fig. 3 is image processing flow figure one provided by the present application, as shown in figure 3, by after image preprocessing Image I is input in first network model, obtains the first matrix；Then according to the first matrix, the second matrix is generated；Again by Image I after two matrixes and image preprocessing carries out dot product, obtains weighting lesion figure WM.In above process, image preprocessing Image I afterwards includes multiple images block；According to first network model provided in this embodiment, the first matrix L can be made It can be corresponded between each first matrix dot and each image block, also, the position of each the first matrix dot, be According to determined by the position of image block corresponding with the first matrix dot, the corresponding position between the first matrix dot and image block is not Entanglement can be generated.Fig. 4 is the schematic diagram of the first matrix provided by the present application, as shown in figure 4, giving involved in Fig. 3 the One matrix, first matrix include multiple first matrix dots, and the numerical value m of each matrix dot has symbolized the doubtful disease of m kind Stove type.
S206, the first matrix and third matrix are combined, obtain the 4th matrix, wherein the size of the 4th matrix with The size of first matrix is identical, includes multiple 4th matrix dots in the 4th matrix.
Optionally, the 4th matrix is LP=(L+1) ⊙ P；Wherein, L is the first matrix, and P is third matrix, and ⊙ is matrix dot Multiply operator.
In this step, before obtaining the second matrix, it is necessary first to the first square of suspected abnormality type will be used to indicate The third matrix P of battle array L and the probability for being used to indicate suspected abnormality type are combined, and obtain a 4th matrix L P=(L+1) ⊙P.It is found that the size of the 4th matrix L P is identical with the first matrix L size；The 4th square in 4th matrix L P The number of lattice point and the number of the first matrix dot in the first matrix L are identical.
Also, the first matrix L is three-dimensional matrice, and third matrix P is also three-dimensional matrice, and the 4th matrix L P is also three-dimensional square Battle array, it is accurate for, the first matrix L is that three-dimensional matrice is n × n × 3, then the size of the 4th matrix L P is n × n × 3.
S207, total head expansion is carried out to the 4th matrix dot of each of the 4th matrix, obtains the second matrix, wherein the Two matrixes include multiple matrix-blocks, are corresponded between each matrix-block and each the 4th matrix dot, each matrix-block In include multiple second matrix dots, a kind of suspected abnormality class of the numerical representation method of each second matrix dot in each matrix-block Type, the numerical value of each second matrix dot in each matrix-block are the numerical value of the 4th matrix dot corresponding to matrix-block.
In this step, it needs that processing will be extended to the 4th matrix, obtains identical with the size of eye fundus image I Two matrixes.Specifically, due to dividing the eye fundus image I that size is M × N for n × n image block, and the acquired 4th It is corresponded between the 4th matrix dot of each of matrix L P and each image block；For each of the 4th matrix L P 4th matrix dot is copied as multiple second matrix dots by the 4th matrix dot, multiple second as obtained from the 4th matrix dot Matrix dot may be constructed a matrix-block；And then the second matrix is constituted by each matrix-block.It is found that image block and the 4th matrix It corresponds between point, is corresponded between the 4th matrix dot and matrix-block, then, between image block and matrix-block be also one a pair of It answers；The number of the second matrix dot in matrix-block, the number of the pixel in image block corresponding with matrix-block, the two is phase Together.Also, since the 4th matrix dot is copied as multiple second matrix dots, then belong to the of the same matrix-block in the second matrix The numerical value of two matrix dots is identical；Also, for the same matrix-block, the numerical value of the second matrix dot in matrix-block, It is the numerical value of the 4th matrix dot corresponding with matrix-block.In turn, the numerical representation method of each second matrix dot in each matrix-block A kind of suspected abnormality type, also, the numerical representation method of each second matrix dot in each matrix-block gone out corresponding image The probability for the suspected abnormality type that block belongs to.To which the 4th matrix dot of each of the 4th matrix L P be expanded with total head Mode, the number for carrying out matrix dot are expanded, and the second matrix W that size is M × N is obtained.
The size of second matrix W is identical as the size of eye fundus image I after image preprocessing, also, the ruler of the second matrix W Very little and eye fundus image IoriSize it is identical.
S208, the second matrix and eye fundus image are combined, obtain weighting lesion figure.
Optionally, weighting lesion figure be WM=W ⊙ I, wherein I is treated eye fundus image, and W is the second matrix.
It in this step, can will be above-mentioned for the detailed information and global Global Information in available eye fundus image Eye fundus image I is combined the second matrix W that step S207 is obtained with treated, to obtain a weighting lesion figure.It is preferred that , by the second matrix W and treated eye fundus image I, point-to-point multiplication is carried out, obtains weighting lesion figure WM=W ⊙ I.To, General image information in acquired weighting lesion figure WM contains that treated eye fundus image I, that is, the environment of perilesional is believed Breath；The suspected abnormality type of acquired weighting lesion figure WM further comprises that treated each of eye fundus image I image block Information, that is, contain each image block and belong to m kind suspected abnormality type and belong to the general of m kind suspected abnormality type Rate.
S209, weighting lesion figure is input in the second network model, obtains eye fundus image analysis result, wherein eyeground Image analysis result includes the lesion rank of eye fundus image and the lesion type of each image block.
In this step, by the obtained weighting lesion figure WM of step S208, be input to step S203 second arrived In network model, the second network model is further analyzed and processed weighting lesion figure WM, and according to each image The suspected abnormality type of block is further analyzed and identifies to weighting lesion figure WM.Second network model exports eyeground figure Lesion rank, the lesion type of each image block of eye fundus image of picture.
For example, Fig. 5 is image processing flow figure two provided by the present application, as shown in figure 5, by after image preprocessing Image I is input in first network model and carries out image analysis and identification, obtains the first matrix, wherein the first matrix can join As shown in Figure 4；The second matrix is generated according to the first matrix；According to the image I after the second matrix and image preprocessing, weighted Weighting lesion figure is input in the second network model and identifies by lesion figure, obtains eye fundus image analysis result.
From first network model shown in fig. 5 and the second network model it is found that first network model and the second network model By the joint training in step S203, the second network model can give the back pass of lesion rank to first network model, and then complete At joint training, wherein be closely bound up between lesion type and lesion rank；To by first network model and second The performance enhancement of network model, each network model can be completed to restrain using less training data.Also, to eye During base map picture is analyzed, the second network model can send back pass information, the back pass information to first network model In include the letter of image block and the global information of eye fundus image, wherein each image block that global information refers to belongs to m kind Suspected abnormality type and the probability for belonging to m kind suspected abnormality type.
The present embodiment is analyzed by each image block to eye fundus image, obtains the doubtful of each image block Lesion type, and then obtain a three-dimensional matrice；Then, the size of three-dimensional matrice is expanded, is obtained and eye fundus image Identical second matrix of size；The information of the information of second matrix and eye fundus image is combined, weighting lesion figure is obtained, it should It include the detailed information and global Global Information in eye fundus image in weighting lesion figure；Then, weighting lesion figure is known Not, eye fundus image analysis is obtained as a result, obtaining the lesion rank of eye fundus image and the lesion type of each image block in turn.From And the detailed information in the image block of the Global Information due to having merged eye fundus image and part, it can be accurately to eyeground figure As being analyzed and being identified, available accurate eye fundus image analysis is as a result, improve the analysis and knowledge of eye fundus image Other precision.Further, medical staff further analyzes eye fundus image and eye fundus image according to itself medical practice As a result medical judgment is carried out, and then determines the real cause of disease and lesion.Also, it is every due to the output layer of first network model The receptive field of a node is the range for only covering image block corresponding to each node；Guarantee in the first obtained matrix in turn Each first matrix dot and each image block between can correspond, and corresponding position will not generate entanglement, protect Demonstrate,prove eye fundus image precision of analysis.Used first network model and the second network model pass through joint training, from And each network model can be completed to restrain using less training data；And it analysis for eye fundus image and identified Cheng Zhong, the information of available image block and the global information of eye fundus image, it is contemplated that the eye except image block message, image block Global context information in base map picture, and then accurately determine lesion type and lesion rank.
Fig. 6 is a kind of structural schematic diagram of eye fundus image processing unit provided by the embodiments of the present application, as shown in fig. 6, should Device, comprising:
First processing units 31 obtain the first square for eye fundus image to be processed to be input in first network model Battle array, wherein include multiple images block in eye fundus image, the first matrix includes multiple first matrix dots, each first matrix dot It is corresponded between each image block, a kind of suspected abnormality type of the numerical representation method of each the first matrix dot.
Expanding element 32 obtains the second matrix, the size and eye of the second matrix for being extended processing to the first matrix The size of base map picture is identical.
Combining unit 33 obtains weighting lesion figure for the second matrix to be combined with eye fundus image.
The second processing unit 34 is input in the second network model for that will weight lesion figure, obtains eye fundus image analysis As a result, wherein it includes the lesion rank of eye fundus image and the lesion type of each image block that eye fundus image, which analyzes result,.
Device provided in this embodiment is same as realizing the skill in the method for processing fundus images of aforementioned any embodiment offer Art scheme, it is similar that the realization principle and technical effect are similar, repeats no more.
Fig. 7 is the structural schematic diagram of another eye fundus image processing unit provided by the embodiments of the present application, reality shown in Fig. 6 On the basis of applying example, as shown in fig. 7, the device, further includes:
First training unit 41, for eye fundus image to be processed to be input to first network mould in first processing units 31 In type, before obtaining the first matrix, pre-training is carried out to initial first network model using preset lesion type data, is obtained First network model after to pre-training.
Second training unit 42, for being instructed in advance using preset lesion rank data to the second initial network model Practice, the second network model after obtaining pre-training.
Third training unit 43, for by the first network model after pre-training and the second network model after pre-training into Row connection, and using lesion rank data to after connection first network model and the second network model be trained, obtain instruction The second network model after first network model and training after white silk.
Expanding element 32, is specifically used for: carrying out total head expansion to the first matrix dot of each of first matrix, obtains the Two matrixes, wherein the second matrix includes multiple matrix-blocks, and one is a pair of between each matrix-block and each first matrix dot It answers, includes multiple second matrix dots in each matrix-block, the numerical representation method of each second matrix dot in each matrix-block The numerical value of a kind of suspected abnormality type, each second matrix dot in each matrix-block is the first matrix corresponding to matrix-block The numerical value of point.
Alternatively, first processing units 31, are specifically used for: eye fundus image to be processed is input in first network model, Export the first matrix and third matrix, wherein third matrix includes multiple third matrix dots, each third matrix dot with it is each It corresponds between a first matrix dot, the first square corresponding to the numerical representation method of each third matrix dot third matrix dot The probability of the suspected abnormality type of lattice point.
Expanding element 32, is specifically used for: the first matrix and third matrix being combined, the 4th matrix is obtained, wherein the The size of four matrixes is identical as the size of the first matrix, includes multiple 4th matrix dots in the 4th matrix；To in the 4th matrix Each the 4th matrix dot carries out total head expansion, obtains the second matrix, wherein and the second matrix includes multiple matrix-blocks, each It is corresponded between matrix-block and each the 4th matrix dot, includes multiple second matrix dots in each matrix-block, it is each A kind of suspected abnormality type of the numerical representation method of each second matrix dot in a matrix-block, each second square in each matrix-block The numerical value of lattice point is the numerical value of the 4th matrix dot corresponding to matrix-block.
4th matrix is LP=(L+1) ⊙ P；Wherein, L is the first matrix, and P is third matrix.
Weighting lesion figure is WM=W ⊙ Iori, wherein IoriFor eye fundus image, W is the second matrix.
The receptive field of each node of output layer in first network model, covers only image corresponding to each node The range of block.
Device provided by the embodiments of the present application, further includes:
Third processing unit 44, for eye fundus image to be processed to be input to first network mould in first processing units 31 In type, before obtaining the first matrix, using preset image preprocessing mode, image is carried out to eye fundus image to be processed and is located in advance Reason, the eye fundus image that obtains that treated, wherein image preprocessing mode is for the illumination point in balanced eye fundus image to be processed Cloth.
Image preprocessing mode is Gaussian smoothing filter mode；Treated, and eye fundus image isWherein, IoriFor eye fundus image to be processed, G (θ) is preset Gaussian kernel, and θ is default Gaussian kernel dimensional parameters, α, β, γ be preset hyper parameter.
Optionally, first matrix is three-dimensional matrice, and second matrix is three-dimensional matrice, and the third matrix is three Matrix is tieed up, the 4th matrix is three-dimensional matrice.
Device provided in this embodiment is same as realizing the skill in the method for processing fundus images of aforementioned any embodiment offer Art scheme, it is similar that the realization principle and technical effect are similar, repeats no more.
Fig. 8 is a kind of structural schematic diagram of eye fundus image processing equipment provided by the embodiments of the present application, as shown in figure 8, should Eye fundus image processing equipment, comprising: transmitter 71, receiver 72, memory 73 and processor 74；
Memory 73 is for storing computer instruction；Computer instruction of the processor 74 for run memory 73 to store is real Existing previous embodiment provides the technical solution of the method for processing fundus images of any implementation.
The application also provides a kind of storage medium, comprising: readable storage medium storing program for executing and computer instruction, computer instruction storage In readable storage medium storing program for executing；The method for processing fundus images for any implementation that computer instruction provides for realizing previous example Technical solution.
In above-mentioned eye fundus image processing equipment in the specific implementation, it should be understood that processor 74 can be central processing unit (English: Central Processing Unit, referred to as: CPU), it can also be other general processors, digital signal processor (English: Digital Signal Processor, abbreviation: DSP), specific integrated circuit (English: Application Specific Integrated Circuit, referred to as: ASIC) etc..General processor can be microprocessor or the processor It is also possible to any conventional processor etc..The step of method in conjunction with disclosed in the embodiment of the present application, can be embodied directly in firmly Part processor executes completion, or in processor hardware and software module combination execute completion.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned includes: read-only memory (English Text: read-only memory, abbreviation: ROM), RAM, flash memory, hard disk, solid state hard disk, tape (English: magnetic Tape), floppy disk (English: floppy disk), CD (English: optical disc) and any combination thereof.
Finally, it should be noted that the above various embodiments is only to illustrate the technical solution of the application, rather than its limitations；To the greatest extent Pipe is described in detail the application referring to foregoing embodiments, those skilled in the art should understand that: its according to So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into Row equivalent replacement；And these are modified or replaceed, each embodiment technology of the application that it does not separate the essence of the corresponding technical solution The range of scheme.

Claims (12)

1. a kind of method for processing fundus images characterized by comprising
Eye fundus image to be processed is input in first network model, the first matrix is obtained, wherein is wrapped in the eye fundus image Include multiple images block, first matrix includes multiple first matrix dots, each described first matrix dot and each described in It is corresponded between image block, a kind of suspected abnormality type of the numerical representation method of each first matrix dot；
Processing is extended to first matrix, obtains the second matrix, the size of second matrix and the eye fundus image Size it is identical；
Second matrix is combined with the eye fundus image, obtains weighting lesion figure；
The weighting lesion figure is input in the second network model, obtains eye fundus image analysis result, wherein the eyeground figure As analysis result includes the lesion rank of the eye fundus image and the lesion type of each described image block.
2. the method according to claim 1, wherein eye fundus image to be processed is input to the first net described In network model, before obtaining the first matrix, further includes:
Pre-training is carried out to initial first network model using preset lesion type data, the first net after obtaining pre-training Network model；
Pre-training is carried out to the second initial network model using preset lesion rank data, the second net after obtaining pre-training Network model；
First network model after the pre-training and the second network model after the pre-training are attached, and use institute State lesion rank data to after connection first network model and the second network model be trained, obtain training after the first net The second network model after network model and training.
3. obtaining second the method according to claim 1, wherein being extended processing to first matrix Matrix, comprising:
Total head expansion is carried out to the first matrix dot of each of first matrix, obtains second matrix, wherein described Second matrix includes multiple matrix-blocks, is corresponded between each described matrix-block and each described first matrix dot, often It include multiple second matrix dots in one matrix-block, the numerical tabular of each second matrix dot in each described matrix-block A kind of suspected abnormality type is levied, the numerical value of each second matrix dot in each described matrix-block is the corresponding to matrix-block The numerical value of one matrix dot.
4. the method according to claim 1, wherein described be input to first network for eye fundus image to be processed In model, the first matrix is obtained, comprising:
Eye fundus image to be processed is input in first network model, exports first matrix and third matrix, wherein institute Stating third matrix includes multiple third matrix dots, one between each described third matrix dot and each described first matrix dot One is corresponding, the suspected abnormality of the first matrix dot corresponding to the numerical representation method of each third matrix dot third matrix dot The probability of type；
Processing is extended to first matrix, obtains the second matrix, comprising:
First matrix and the third matrix are combined, the 4th matrix is obtained, wherein the size of the 4th matrix It is identical as the size of first matrix, it include multiple 4th matrix dots in the 4th matrix；
Total head expansion is carried out to the 4th matrix dot of each of the 4th matrix, obtains second matrix, wherein described Second matrix includes multiple matrix-blocks, is corresponded between each described matrix-block and each described 4th matrix dot, often It include multiple second matrix dots in one matrix-block, the numerical tabular of each second matrix dot in each described matrix-block A kind of suspected abnormality type is levied, the numerical value of each second matrix dot in each described matrix-block is the corresponding to matrix-block The numerical value of four matrix dots.
5. according to the method described in claim 4, it is characterized in that, the 4th matrix is LP=(L+1) ⊙ P；Wherein, L is First matrix, P are the third matrix.
6. the method according to claim 1, wherein the weighting lesion figure is WM=W ⊙ Iori, wherein Iori For the eye fundus image, W is second matrix.
7. method according to claim 1-6, which is characterized in that output layer in the first network model The receptive field of each node covers only the range of image block corresponding to each node.
8. method according to claim 1-6, which is characterized in that input eye fundus image to be processed described Into first network model, before obtaining the first matrix, further includes:
Using preset image preprocessing mode, image preprocessing is carried out to the eye fundus image to be processed, after obtaining processing Eye fundus image, wherein described image pretreatment mode is for the illumination patterns in the balanced eye fundus image to be processed.
9. according to the method described in claim 8, it is characterized in that, described image pretreatment mode is Gaussian smoothing filter side Formula；
It is described that treated that eye fundus image isWherein, IoriFor the eyeground figure to be processed Picture, G (θ) are preset Gaussian kernel, and θ is preset Gaussian kernel dimensional parameters, and α, β, γ are preset hyper parameter.
10. a kind of eye fundus image processing unit characterized by comprising
First processing units obtain the first matrix for eye fundus image to be processed to be input in first network model, In, it include multiple images block in the eye fundus image, first matrix includes multiple first matrix dots, each described first It is corresponded between matrix dot and each described image block, the numerical representation method of each first matrix dot is a kind of doubtful Lesion type；
Expanding element obtains the second matrix for being extended processing to first matrix, the size of second matrix with The size of the eye fundus image is identical；
Combining unit obtains weighting lesion figure for second matrix to be combined with the eye fundus image；
The second processing unit obtains eye fundus image analysis knot for the weighting lesion figure to be input in the second network model Fruit, wherein the eye fundus image analysis result includes the lesion rank of the eye fundus image and the disease of each described image block Stove type.
11. a kind of eye fundus image processing equipment characterized by comprising transmitter, receiver, memory and processor；
The memory is for storing computer instruction；The processor is used to run the computer of the memory storage The described in any item methods of claim 1-9 are realized in instruction.
12. a kind of storage medium characterized by comprising readable storage medium storing program for executing and computer instruction, the computer instruction are deposited Storage is in the readable storage medium storing program for executing；The computer instruction is for realizing the described in any item methods of claim 1-9.
CN201910443966.5A 2019-05-27 2019-05-27 Method for processing fundus images, device, equipment and storage medium Pending CN110288568A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910443966.5A CN110288568A (en) 2019-05-27 2019-05-27 Method for processing fundus images, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910443966.5A CN110288568A (en) 2019-05-27 2019-05-27 Method for processing fundus images, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN110288568A true CN110288568A (en) 2019-09-27

Family

ID=68002686

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910443966.5A Pending CN110288568A (en) 2019-05-27 2019-05-27 Method for processing fundus images, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110288568A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402246A (en) * 2020-03-20 2020-07-10 北京工业大学 Eye ground image classification method based on combined network
CN111583184A (en) * 2020-04-14 2020-08-25 上海联影智能医疗科技有限公司 Image analysis method, network, computer device, and storage medium
CN112927228A (en) * 2021-04-14 2021-06-08 依未科技(北京)有限公司 Image evaluation method and device, and model training method and device
CN116894805A (en) * 2022-03-29 2023-10-17 山东第一医科大学附属省立医院(山东省立医院) Lesion characteristic identification system based on wide-angle fundus image

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103006215A (en) * 2012-12-14 2013-04-03 中国科学院自动化研究所 Brain functional region positioning method based on local smoothing regressions
CN107680684A (en) * 2017-10-12 2018-02-09 百度在线网络技术（北京）有限公司 For obtaining the method and device of information
CN108960257A (en) * 2018-07-06 2018-12-07 东北大学 A kind of diabetic retinopathy grade stage division based on deep learning

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103006215A (en) * 2012-12-14 2013-04-03 中国科学院自动化研究所 Brain functional region positioning method based on local smoothing regressions
CN107680684A (en) * 2017-10-12 2018-02-09 百度在线网络技术（北京）有限公司 For obtaining the method and device of information
CN108960257A (en) * 2018-07-06 2018-12-07 东北大学 A kind of diabetic retinopathy grade stage division based on deep learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
YEHUI YANG ET AL.: "Lesion Detection and Grading of Diabetic Retinopathy via Two-Stages Deep Convolutional Neural Networks", 《INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION》 *
卢旭 等: "基于卷积神经网络的糖尿病视网膜眼底图像出血病变检测", 《视频应用与工程》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402246A (en) * 2020-03-20 2020-07-10 北京工业大学 Eye ground image classification method based on combined network
CN111583184A (en) * 2020-04-14 2020-08-25 上海联影智能医疗科技有限公司 Image analysis method, network, computer device, and storage medium
CN112927228A (en) * 2021-04-14 2021-06-08 依未科技(北京)有限公司 Image evaluation method and device, and model training method and device
CN116894805A (en) * 2022-03-29 2023-10-17 山东第一医科大学附属省立医院(山东省立医院) Lesion characteristic identification system based on wide-angle fundus image
CN116894805B (en) * 2022-03-29 2024-03-19 山东第一医科大学附属省立医院(山东省立医院) Lesion characteristic identification system based on wide-angle fundus image

Similar Documents

Publication Publication Date Title
CN110288568A (en) Method for processing fundus images, device, equipment and storage medium
Kobler et al. Variational networks: connecting variational methods and deep learning
Fang et al. Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum
WO2021109876A1 (en) Image processing method, apparatus and device, and storage medium
CN109658344A (en) Image de-noising method, device, equipment and storage medium based on deep learning
CN106204779B (en) Check class attendance method based on plurality of human faces data collection strategy and deep learning
CN104067311B (en) Digital makeup
CN109214990A (en) A kind of depth convolutional neural networks image de-noising method based on Inception model
CN107123089A (en) Remote sensing images super-resolution reconstruction method and system based on depth convolutional network
CN109902798A (en) The training method and device of deep neural network
CN110210486A (en) A kind of generation confrontation transfer learning method based on sketch markup information
CN109685060A (en) Image processing method and device
CN103914699A (en) Automatic lip gloss image enhancement method based on color space
Johnson et al. Darwinism of color image difference models
CN108416327A (en) A kind of object detection method, device, computer equipment and readable storage medium storing program for executing
JP2007503629A (en) Neural network for processing a data array having existing phase information such as an image and method of using the network
CN110232326A (en) A kind of D object recognition method, device and storage medium
CN109871845A (en) Certificate image extracting method and terminal device
CN108647634A (en) Framing mask lookup method, device, computer equipment and storage medium
CN104182771B (en) Based on the graphical analysis method of the time series data with packet loss automatic coding
CN106683110A (en) User terminal and object tracking method and device thereof
CN109741338A (en) A kind of face dividing method, device and equipment
CN109858487A (en) Weakly supervised semantic segmentation method based on watershed algorithm and image category label
CN106778785A (en) Build the method for image characteristics extraction model and method, the device of image recognition
CN110321892A (en) A kind of picture screening technique, device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20200331

Address after: 100000 Room 220, 2nd Floor, Building 4, No. 1, Shangdi East Road, Haidian District, Beijing

Applicant after: Beijing Confucius Health Technology Co.,Ltd.

Address before: 100085 Beijing, Haidian District, No. ten on the ground floor, No. 10 Baidu building, layer 2

Applicant before: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd.