CN113962899A - Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium - Google Patents

Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium Download PDF

Info

Publication number
CN113962899A
CN113962899A CN202111327342.0A CN202111327342A CN113962899A CN 113962899 A CN113962899 A CN 113962899A CN 202111327342 A CN202111327342 A CN 202111327342A CN 113962899 A CN113962899 A CN 113962899A
Authority
CN
China
Prior art keywords
vectors
fundus image
feature
matrix
activation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111327342.0A
Other languages
Chinese (zh)
Inventor
李建强
彭浩然
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN202111327342.0A priority Critical patent/CN113962899A/en
Publication of CN113962899A publication Critical patent/CN113962899A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/15—Correlation function computation including computation of convolution operations
    • G06F17/153—Multidimensional correlation or convolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic

Abstract

The invention provides a fundus image processing method, a fundus image processing apparatus, an electronic device, and a storage medium, wherein the fundus image processing method includes: acquiring an initial characteristic matrix corresponding to the fundus image, and acquiring two transverse and longitudinal characteristic vectors based on the initial characteristic matrix; the two feature vectors are respectively input into two residual shrinkage modules to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer; obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors; and obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix. The invention provides a fundus image processing method, a fundus image processing device, an electronic device and a storage medium, which can realize the function of removing noise in a fundus image.

Description

Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for processing an image of an eye fundus, an electronic device, and a storage medium.
Background
In the process of acquiring the fundus data, due to the exposure degree, the operation difference when the instrument is used and the functions of the instrument and equipment, the fundus images have large difference in color and brightness, in this case, the RGB numerical value difference (noise) which is difficult to distinguish by naked eyes appears on the background of the fundus images, and the method is a great challenge for a deep learning model, so the noise of the fundus images needs to be removed. Currently, there is no technical means capable of removing fundus image noise.
Disclosure of Invention
The invention provides a fundus image processing method, a fundus image processing device, electronic equipment and a storage medium, which are used for solving the defect that a fundus image has noise which is difficult to distinguish by naked eyes in the prior art and realizing the function of removing the noise in the fundus image.
The invention provides a fundus image processing method, which comprises the following steps:
acquiring an initial characteristic matrix corresponding to the fundus image, and acquiring two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
the two feature vectors are respectively input into two residual shrinkage modules to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
According to the fundus image processing method provided by the present invention, the obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors, includes:
performing vector product operation on the first activation vector and the first pooling vector to obtain a first convolution channel threshold;
performing vector product operation on the second activation vector and the second pooling vector to obtain a second convolution channel threshold;
and obtaining two new feature matrixes based on the first convolution channel threshold, the second convolution channel threshold and the two feature vectors.
According to the fundus image processing method provided by the present invention, obtaining two new feature matrices based on the first convolution channel threshold, the second convolution channel threshold, and the two feature vectors includes:
performing soft thresholding operation on the first feature vector based on the first convolution channel threshold to obtain a first new feature matrix;
and performing soft thresholding operation on the second feature vector based on the second convolution channel threshold to obtain a second new feature matrix.
According to the fundus image processing method provided by the present invention, obtaining a denoised fundus image based on the two new feature matrices and the initial feature matrix comprises:
converging the two new feature matrixes to obtain a converged matrix;
and jumping and connecting the initial characteristic matrix and the convergence matrix to obtain the denoised fundus image.
According to the fundus image processing method provided by the invention, the acquiring of the initial feature matrix corresponding to the fundus image and the obtaining of the two feature vectors in the transverse direction and the longitudinal direction based on the initial feature matrix comprise:
acquiring an initial characteristic matrix corresponding to a fundus image, inputting the initial characteristic matrix into a first convolution layer, and performing regularization processing, activation processing and convolution processing on the initial characteristic matrix to obtain a first intermediate characteristic matrix;
inputting the first intermediate feature matrix into a second convolution layer, and performing regularization processing, activation processing and convolution processing on the first intermediate feature matrix to obtain a second intermediate feature matrix;
and performing transverse expansion and longitudinal expansion on the second intermediate feature matrix to obtain two feature vectors.
According to the fundus image processing method provided by the invention, the residual shrinkage module further comprises a first full-connection layer, a regularization layer, a second activation layer and a second full-connection layer which are arranged between the global average pooling layer and the first activation layer.
The present invention also provides a fundus image processing apparatus including:
the acquisition module is used for acquiring an initial characteristic matrix corresponding to the fundus image and acquiring two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
the first processing module is used for inputting the two characteristic vectors into the two residual shrinkage modules respectively to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
the second processing module is used for obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and the third processing module is used for obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the fundus image processing method as described in any one of the above when executing the program.
The present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the fundus image processing method as described in any one of the above.
The present invention also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the fundus image processing method as described in any one of the above.
The invention provides a fundus image processing method, a fundus image processing device, an electronic device and a storage medium, by
After global average pooling and activation processing are carried out on two feature vectors corresponding to the fundus image, pooling vectors and activation vectors are obtained, and then corresponding convolution channel thresholds are determined in a self-adaptive mode through the pooling vectors and the activation vectors, namely, a convolution channel threshold which is adaptive to the current process and reasonable is automatically learned, and manual definition of the convolution channel threshold is not needed.
Because the noise position of the fundus image cannot be accurately observed by the human eyes through the channel, the manually defined convolution channel threshold is inaccurate, the characteristic vector is restored into a new characteristic matrix through the convolution channel threshold determined in a self-adaptive mode, and finally the initial characteristic matrix is combined, so that the effect of removing the noise of the fundus image is better. In addition, the initial characteristic matrix is further expanded into two transverse and longitudinal characteristic vectors, the two characteristic vectors are separately processed and denoised, and finally the two characteristic vectors are combined, so that the denoising effect of the fundus image is improved.
Therefore, the fundus image processing method provided by the invention can overcome the defect that the fundus image in the prior art has noise which is difficult to distinguish by naked eyes, and realizes the function of removing the noise in the fundus image.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is one of the flow diagrams of a fundus image processing method provided by the present invention;
FIG. 2 is a second schematic flow chart of the fundus image processing method according to the present invention;
FIG. 3 is a schematic structural view of a fundus image processing apparatus provided by the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The fundus image processing method, apparatus, electronic device, and storage medium of the present invention will be described below with reference to fig. 1 to 4.
As shown in fig. 1, the present invention provides a fundus image processing method, including:
and 110, acquiring an initial characteristic matrix corresponding to the fundus image, and obtaining two transverse and longitudinal characteristic vectors based on the initial characteristic matrix.
It is understood that the fundus image is an image based on the retinal fundus captured by the fundus camera. And performing primary processing on the fundus image, and converting the fundus image into an initial characteristic matrix.
Based on the initial feature matrix, the companies corresponding to the two feature vectors in the horizontal direction and the vertical direction are obtained as follows:
S1＝ΦL(xi，j，c)
S2＝ΦT(xi，j，c)
wherein x isi，j，cA feature matrix phi representing the length, width and number of channels i, j, c, respectivelyLIndicating a transverse flattening operation, [ phi ]TShowing the longitudinal flattening operation.
Step 120, inputting the two feature vectors into two residual shrinkage modules respectively to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer.
It can be understood that the other two feature vectors are respectively input to the corresponding residual shrinkage modules, the global average pooling layer in the residual shrinkage module processes the feature vectors to obtain pooled vectors, and the pooled vectors are processed by the first activation layer to obtain activation vectors.
The first activation layer may activate the pooling vector based on a Sigmoid function to obtain an activation vector.
And step 130, obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors.
It can be understood that a convolution channel threshold of a feature vector is obtained by performing a vector product operation on an activation vector and a pooling vector, and the feature vector is restored to a feature matrix through the convolution channel threshold.
The convolution channel threshold value can be adjusted in a self-adaptive mode through the activation vector and the pooling vector, and manual setting of the convolution channel threshold value is not needed.
Wherein the convolution channel threshold is a channel threshold of a channel-wise convolution.
And step 140, obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
It will be appreciated that the new feature matrix is derived based on the convolution channel thresholds described above.
In some embodiments, the deriving two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and deriving two new feature matrices based on the two convolution channel thresholds and the two feature vectors, includes:
performing vector product operation on the first activation vector and the first pooling vector to obtain a first convolution channel threshold;
performing vector product operation on the second activation vector and the second pooling vector to obtain a second convolution channel threshold;
and obtaining two new feature matrixes based on the first convolution channel threshold, the second convolution channel threshold and the two feature vectors.
It can be understood that the vector product operation of the first activation vector and the first pooling vector is based on vector special multiplication element-wise multiplication, and the vector product operation of the first activation vector and the first pooling vector is performed to obtain the first convolution channel threshold.
And performing vector product operation on the second activation vector and the second pooling vector, wherein the vector product operation is performed on the second activation vector and the second pooling vector based on vector special multiplication element-wise multiplication to obtain a second convolution channel threshold.
Restoring the first feature vector to a first new feature matrix based on the first convolution channel threshold; based on the second convolution channel threshold, the second eigenvector is restored to a second new eigenvector matrix.
Further, the convolution channel threshold is calculated based on the following formula:
τn＝an*Average|sn|
wherein n is 1 or 2, Average represents global Average pooling, | | | represents absolute value, anRepresents an activation vector, a1Representing a first activation vector, a2Representing a second activation vector, snRepresenting the feature vector before input to the residual shrinking module, i.e. the flattened feature vector, s1Representing the first eigenvector, i.e. the eigenvector of the transverse scan, s2Representing the second feature vector, i.e. the feature vector of the longitudinal scan.
In some embodiments, said deriving two new feature matrices based on said first convolution channel threshold, said second convolution channel threshold, and said two feature vectors comprises:
performing soft thresholding operation on the first feature vector based on the first convolution channel threshold to obtain a first new feature matrix;
and performing soft thresholding operation on the second feature vector based on the second convolution channel threshold to obtain a second new feature matrix.
It will be appreciated that the soft thresholding operation is performed based on the following equation:
Figure BDA0003347666260000071
wherein x is(i，j，c)A feature matrix representing length, width and number of channels i, j, c respectively,
Figure BDA0003347666260000083
denotes x(i，j，c)Corresponding output matrix, snRepresenting a certain feature vector, τ, which is flattenednRepresenting the convolution channel threshold corresponding to the feature vector being flattened, and F representing the soft thresholding algorithm operation.
The soft thresholding algorithm is to set the characteristic in a certain value range [ - τ, τ ] to zero and approach the characteristic value with an absolute value greater than τ to the direction of zero, so that the characteristic value of the input data is shrunk to the direction of zero, which is a more flexible way to delete the redundant information of data, and the value range [ - τ, τ ] can be adaptively adjusted, and the formula is as follows:
Figure BDA0003347666260000081
x denotes the input characteristic, y denotes the output characteristic, and τ denotes the convolution channel threshold, which is self-adjustable in this embodiment. To avoid the situation where the y value is constant at 0 in the soft thresholding algorithm, here the threshold should be a not positive number.
The derivative formula of the soft thresholding function is as follows:
Figure BDA0003347666260000082
as shown in the above equation, the derivative of the function is either zero or one. With similar properties to the ReLU activation function, the soft thresholding function is also useful to prevent "gradient vanishing" and "gradient explosion".
In some embodiments, the deriving the denoised fundus image based on the two new feature matrices and the initial feature matrix comprises:
converging the two new feature matrixes to obtain a converged matrix;
and jumping and connecting the initial characteristic matrix and the convergence matrix to obtain the denoised fundus image.
It can be understood that the two new feature matrices are aggregated together by a summation operation to obtain an aggregation matrix.
And performing jump connection on the initial characteristic matrix and the aggregation matrix, namely performing Identity short operation on the initial characteristic matrix and the aggregation matrix to obtain the denoised fundus image.
In some embodiments, the acquiring an initial feature matrix corresponding to the fundus image and obtaining two feature vectors in a transverse direction and a longitudinal direction based on the initial feature matrix includes:
acquiring an initial characteristic matrix corresponding to a fundus image, inputting the initial characteristic matrix into a first convolution layer, and performing regularization processing, activation processing and convolution processing on the initial characteristic matrix to obtain a first intermediate characteristic matrix;
inputting the first intermediate feature matrix into a second convolution layer, and performing regularization processing, activation processing and convolution processing on the first intermediate feature matrix to obtain a second intermediate feature matrix;
and performing transverse expansion and longitudinal expansion on the second intermediate feature matrix to obtain two feature vectors.
It is understood that the first convolutional layer and the second convolutional layer may both activate the initial feature matrix based on a Linear rectification function (ReLU).
In some embodiments, the residual shrinkage module further comprises a first fully-connected layer, a regularization layer, a second activation layer, and a second fully-connected layer disposed between the global average pooling layer and the first activation layer.
It is to be understood that a first fully-connected layer, a regularization layer, a second activation layer, and a second fully-connected layer are disposed in order between the global average pooling layer and the first activation layer.
And the second activation layer is activated based on a Sigmoid function.
In other embodiments, as shown in fig. 2, the process of the fundus image processing method provided by the present invention is that an initial feature matrix corresponding to a fundus image is sequentially input into a first convolution layer and a second convolution layer, each convolution layer is subjected to regularization processing, linear rectification function activation processing, and convolution processing, and the second convolution layer inputs the processed feature matrix into a matrix expansion layer and expands the processed feature matrix into two feature vectors in the horizontal direction and the vertical direction.
Each feature vector sequentially passes through the global average pooling layer, the first full-connection layer, the regularization layer, the second activation layer and the second full-connection layer to obtain an activation vector; and obtaining a convolution channel threshold value based on the pooling vector and the activation vector output by the global average pooling layer, and obtaining a new feature matrix based on the convolution channel threshold value and the feature vector.
And converging the two new characteristic matrixes to obtain a converged matrix, and then, jumping and connecting the initial characteristic matrix and the converged matrix to obtain the denoised fundus image.
The fundus image processing apparatus provided by the present invention will be described below, and the fundus image processing apparatus described below and the fundus image processing method described above can be referred to in correspondence with each other.
In summary, the fundus image processing method provided by the present invention includes: acquiring an initial characteristic matrix corresponding to the fundus image, and acquiring two transverse and longitudinal characteristic vectors based on the initial characteristic matrix; the two feature vectors are respectively input into two residual shrinkage modules to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer; obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors; and obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
In the fundus image processing method provided by the invention, two characteristic vectors corresponding to a fundus image are subjected to global average pooling processing and activation processing to obtain pooling vectors and activation vectors, and then the corresponding convolution channel threshold is determined in a self-adaptive manner through the pooling vectors and the activation vectors, namely, a convolution channel threshold which is adaptive to the current process and reasonable is automatically learned, and manual definition of the convolution channel threshold is not needed.
Because the noise position of the fundus image cannot be accurately observed by the human eyes through the channel, the manually defined convolution channel threshold is inaccurate, the characteristic vector is restored into a new characteristic matrix through the convolution channel threshold determined in a self-adaptive mode, and finally the initial characteristic matrix is combined, so that the effect of removing the noise of the fundus image is better. In addition, the initial characteristic matrix is further expanded into two transverse and longitudinal characteristic vectors, the two characteristic vectors are separately processed and denoised, and finally the two characteristic vectors are combined, so that the denoising effect of the fundus image is improved.
Therefore, the fundus image processing method provided by the invention can overcome the defect that the fundus image in the prior art has noise which is difficult to distinguish by naked eyes, and realizes the function of removing the noise in the fundus image.
As shown in fig. 3, a fundus image processing apparatus 300 according to the present invention includes: an acquisition module 310, a first processing module 320, a second processing module 330, and a third processing module 340.
The obtaining module 310 is configured to obtain an initial feature matrix corresponding to the fundus image, and obtain two feature vectors in a horizontal direction and a vertical direction based on the initial feature matrix.
The first processing module 320 is configured to input the two feature vectors to the two residual shrinkage modules, respectively, to obtain two pooled vectors and two activated vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer.
The second processing module 330 is configured to obtain two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtain two new feature matrices based on the two convolution channel thresholds and the two feature vectors.
The third processing module 340 is configured to obtain a denoised fundus image based on the two new feature matrices and the initial feature matrix.
In some embodiments, the second processing module 330 includes: the device comprises a first operation unit, a second operation unit and a characteristic matrix construction unit.
The first operation unit is used for carrying out vector product operation on the first activation vector and the first pooling vector to obtain a first convolution channel threshold.
The second operation unit is used for performing vector product operation on the second activation vector and the second pooling vector to obtain a second convolution channel threshold.
The feature matrix construction unit is used for obtaining two new feature matrices based on the first convolution channel threshold, the second convolution channel threshold and the two feature vectors.
In some embodiments, the feature matrix construction unit includes: a first building element and a second building element.
The first construction unit is used for performing soft thresholding operation on the first feature vector based on the first convolution channel threshold value to obtain a first new feature matrix.
And the second construction unit is used for performing soft thresholding operation on the second feature vector based on the second convolution channel threshold value to obtain a second new feature matrix.
In some embodiments, the third processing module 340 includes: a convergence unit and a jump connection unit.
And the convergence unit is used for converging the two new feature matrixes to obtain a convergence matrix.
And the hopping connection unit is used for carrying out hopping connection on the initial characteristic matrix and the convergence matrix to obtain the denoised fundus image.
In some embodiments, the obtaining module 310 includes: an image processing unit, a feature processing unit and a matrix expansion unit.
The image processing unit is used for acquiring an initial characteristic matrix corresponding to the fundus image, inputting the initial characteristic matrix into the first convolution layer, and performing regularization processing, activation processing and convolution processing on the initial characteristic matrix to obtain a first intermediate characteristic matrix.
The feature processing unit is used for inputting the first intermediate feature matrix into a second convolution layer, and performing regularization processing, activation processing and convolution processing on the first intermediate feature matrix to obtain a second intermediate feature matrix.
And the matrix expansion unit is used for transversely expanding and longitudinally expanding the second intermediate characteristic matrix to obtain two characteristic vectors.
In some embodiments, the residual shrinkage module further comprises a first fully-connected layer, a regularization layer, a second activation layer, and a second fully-connected layer disposed between the global average pooling layer and the first activation layer.
The electronic apparatus, the computer program product, and the storage medium provided by the present invention are described below, and the electronic apparatus, the computer program product, and the storage medium described below and the fundus image processing method described above may be referred to in correspondence with each other.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a fundus image processing method, the method comprising:
step 110, acquiring an initial characteristic matrix corresponding to the fundus image, and obtaining two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
step 120, inputting the two feature vectors into two residual shrinkage modules respectively to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
step 130, obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and step 140, obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product including a computer program storable on a non-transitory computer-readable storage medium, the computer program being capable of executing, when executed by a processor, a fundus image processing method provided by the above methods, the method including:
step 110, acquiring an initial characteristic matrix corresponding to the fundus image, and obtaining two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
step 120, inputting the two feature vectors into two residual shrinkage modules respectively to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
step 130, obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and step 140, obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a fundus image processing method provided by the above methods, the method including:
step 110, acquiring an initial characteristic matrix corresponding to the fundus image, and obtaining two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
step 120, inputting the two feature vectors into two residual shrinkage modules respectively to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
step 130, obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and step 140, obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A fundus image processing method, comprising:
acquiring an initial characteristic matrix corresponding to the fundus image, and acquiring two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
the two feature vectors are respectively input into two residual shrinkage modules to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors, and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
2. A fundus image processing method according to claim 1, wherein said deriving two convolution channel thresholds based on said two activation vectors and said two pooling vectors, and deriving two new feature matrices based on said two convolution channel thresholds and said two feature vectors comprises:
performing vector product operation on the first activation vector and the first pooling vector to obtain a first convolution channel threshold;
performing vector product operation on the second activation vector and the second pooling vector to obtain a second convolution channel threshold;
and obtaining two new feature matrixes based on the first convolution channel threshold, the second convolution channel threshold and the two feature vectors.
3. A fundus image processing method according to claim 2, wherein said deriving two new feature matrices based on said first convolution channel threshold, said second convolution channel threshold, and said two feature vectors comprises:
performing soft thresholding operation on the first feature vector based on the first convolution channel threshold to obtain a first new feature matrix;
and performing soft thresholding operation on the second feature vector based on the second convolution channel threshold to obtain a second new feature matrix.
4. A fundus image processing method according to claim 1, wherein said obtaining a denoised fundus image based on said two new feature matrices and said initial feature matrix comprises:
converging the two new feature matrixes to obtain a converged matrix;
and jumping and connecting the initial characteristic matrix and the convergence matrix to obtain the denoised fundus image.
5. A fundus image processing method according to claim 1, wherein said acquiring an initial feature matrix corresponding to a fundus image and deriving two feature vectors in a transverse direction and a longitudinal direction based on said initial feature matrix comprises:
acquiring an initial characteristic matrix corresponding to a fundus image, inputting the initial characteristic matrix into a first convolution layer, and performing regularization processing, activation processing and convolution processing on the initial characteristic matrix to obtain a first intermediate characteristic matrix;
inputting the first intermediate feature matrix into a second convolution layer, and performing regularization processing, activation processing and convolution processing on the first intermediate feature matrix to obtain a second intermediate feature matrix;
and performing transverse expansion and longitudinal expansion on the second intermediate feature matrix to obtain two feature vectors.
6. A fundus image processing method according to any of claims 1 to 5, characterized in that said residual shrinkage module further comprises a first fully connected layer, a regularization layer, a second active layer and a second fully connected layer arranged between said global average pooling layer and said first active layer.
7. An eye fundus image processing apparatus, comprising:
the acquisition module is used for acquiring an initial characteristic matrix corresponding to the fundus image and acquiring two transverse and longitudinal characteristic vectors based on the initial characteristic matrix;
the first processing module is used for inputting the two characteristic vectors into the two residual shrinkage modules respectively to obtain two pooling vectors and two activation vectors; wherein the residual shrinkage module comprises a global average pooling layer and a first activation layer;
the second processing module is used for obtaining two convolution channel thresholds based on the two activation vectors and the two pooling vectors and obtaining two new feature matrices based on the two convolution channel thresholds and the two feature vectors;
and the third processing module is used for obtaining the denoised fundus image based on the two new feature matrixes and the initial feature matrix.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the fundus image processing method according to any one of claims 1 to 6 when executing the program.
9. A non-transitory computer-readable storage medium on which a computer program is stored, the computer program realizing the steps of the fundus image processing method according to any one of claims 1 to 6 when executed by a processor.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the fundus image processing method according to any one of claims 1 to 6 when executed by a processor.
CN202111327342.0A 2021-11-10 2021-11-10 Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium Pending CN113962899A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111327342.0A CN113962899A (en) 2021-11-10 2021-11-10 Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111327342.0A CN113962899A (en) 2021-11-10 2021-11-10 Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
CN113962899A true CN113962899A (en) 2022-01-21

Family

ID=79470104

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111327342.0A Pending CN113962899A (en) 2021-11-10 2021-11-10 Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium

Country Status (1)

Country Link
CN (1) CN113962899A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180349319A1 (en) * 2017-06-02 2018-12-06 Mitsubishi Electric Research Laboratories, Inc. Online convolutional dictionary learning
CN110348543A (en) * 2019-06-10 2019-10-18 腾讯医疗健康(深圳)有限公司 Eye fundus image recognition methods, device, computer equipment and storage medium
CN110516575A (en) * 2019-08-19 2019-11-29 上海交通大学 GAN based on residual error domain richness model generates picture detection method and system
CN112085731A (en) * 2020-09-18 2020-12-15 深圳市易图资讯股份有限公司 Security early warning method, device and equipment based on satellite map and storage medium
CN113255527A (en) * 2021-05-28 2021-08-13 汉谷云智(武汉)科技有限公司 Method and equipment for monitoring operation normative during concrete unloading process
CN113486929A (en) * 2021-06-17 2021-10-08 中国地质大学（武汉） Rock slice image identification method based on residual shrinkage module and attention mechanism

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180349319A1 (en) * 2017-06-02 2018-12-06 Mitsubishi Electric Research Laboratories, Inc. Online convolutional dictionary learning
CN110348543A (en) * 2019-06-10 2019-10-18 腾讯医疗健康(深圳)有限公司 Eye fundus image recognition methods, device, computer equipment and storage medium
CN110516575A (en) * 2019-08-19 2019-11-29 上海交通大学 GAN based on residual error domain richness model generates picture detection method and system
CN112085731A (en) * 2020-09-18 2020-12-15 深圳市易图资讯股份有限公司 Security early warning method, device and equipment based on satellite map and storage medium
CN113255527A (en) * 2021-05-28 2021-08-13 汉谷云智(武汉)科技有限公司 Method and equipment for monitoring operation normative during concrete unloading process
CN113486929A (en) * 2021-06-17 2021-10-08 中国地质大学（武汉） Rock slice image identification method based on residual shrinkage module and attention mechanism

Similar Documents

Publication Publication Date Title
CN109118495B (en) Retinal vessel segmentation method and device
Yang et al. An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise
JP6656111B2 (en) Method and system for removing image noise
WO2018214671A1 (en) Image distortion correction method and device and electronic device
CN111275643B (en) Real noise blind denoising network system and method based on channel and space attention
CN107133923B (en) Fuzzy image non-blind deblurring method based on adaptive gradient sparse model
US20230085605A1 (en) Face image processing method, apparatus, device, and storage medium
CN110189260B (en) Image noise reduction method based on multi-scale parallel gated neural network
CN112419184A (en) Spatial attention map image denoising method integrating local information and global information
KR102095443B1 (en) Method and Apparatus for Enhancing Image using Structural Tensor Based on Deep Learning
CN110674759A (en) Monocular face in-vivo detection method, device and equipment based on depth map
Huang et al. Two-step approach for the restoration of images corrupted by multiplicative noise
CN110809126A (en) Video frame interpolation method and system based on adaptive deformable convolution
CN113191983A (en) Image denoising method and device based on deep learning attention mechanism
US10229479B2 (en) Image signal processing apparatus, image signal processing method and image signal processing program
US11948278B2 (en) Image quality improvement method and image processing apparatus using the same
CN113763268B (en) Blind restoration method and system for face image
CN113658091A (en) Image evaluation method, storage medium and terminal equipment
CN114897711A (en) Method, device and equipment for processing images in video and storage medium
CN117422653A (en) Low-light image enhancement method based on weight sharing and iterative data optimization
CN108898557B (en) Image restoration method and apparatus, electronic device, computer program, and storage medium
CN113962899A (en) Fundus image processing method, fundus image processing apparatus, electronic device, and storage medium
CN116703768A (en) Training method, device, medium and equipment for blind spot denoising network model
JP2021086284A (en) Image processing device, image processing method, and program
CN115937121A (en) Non-reference image quality evaluation method and system based on multi-dimensional feature fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination