CN112101438B - Left-right eye classification method, device, server and storage medium - Google Patents
Left-right eye classification method, device, server and storage medium Download PDFInfo
- Publication number
- CN112101438B CN112101438B CN202010935200.1A CN202010935200A CN112101438B CN 112101438 B CN112101438 B CN 112101438B CN 202010935200 A CN202010935200 A CN 202010935200A CN 112101438 B CN112101438 B CN 112101438B
- Authority
- CN
- China
- Prior art keywords
- classification
- image
- right eye
- classification model
- global
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 210000001508 eye Anatomy 0.000 claims abstract description 120
- 238000013145 classification model Methods 0.000 claims abstract description 68
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 51
- 238000000605 extraction Methods 0.000 claims description 44
- 238000012549 training Methods 0.000 claims description 43
- 238000013528 artificial neural network Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000011176 pooling Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000010276 construction Methods 0.000 description 4
- 238000012014 optical coherence tomography Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 208000022873 Ocular disease Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 210000002159 anterior chamber Anatomy 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 230000004393 visual impairment Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention provides a left-right eye classification method, which comprises the following steps: acquiring an eyeball AS-OCT image of a tested person; inputting the eyeball AS-OCT image into a preset left eye classification model and a preset right eye classification model, and outputting a classification result; based on the classification result, whether the eyeball of the subject is the left eye or the right eye is determined. According to the invention, the AS-OCT image of the eyeball is identified and classified by using the left eye classification model and the right eye classification model, so that whether the AS-OCT image belongs to the left eye or the right eye is rapidly judged, and the accuracy and the efficiency of the left eye and the right eye identification are improved.
Description
Technical Field
The embodiment of the invention relates to the field of computer image processing and neural network learning, in particular to a left-right eye classification method, a left-right eye classification device, a server and a storage medium.
Background
Anterior segment optical coherence tomography (AS-OCT) images are used for early screening of ocular disorders to prevent vision loss, and currently common methods of visualization of anterior segment angular structures are keratoscopy, ultrasound Biological Microscopy (UBM), and Optical Coherence Tomography (OCT). However, only UBM and OCT can obtain objective and repeatable anterior chamber angle size measurements. In ophthalmic diagnosis and treatment, a doctor often needs to compare the current left-eye or right-eye image of the patient with the historical left-eye or right-eye image data of the patient to observe the subtle changes of related diseases.
Because related data lacks high-efficiency classification and arrangement means for left eyes and right eyes, doctors often rely on manual comparison of AS-OCT images, and the task has the defects of large workload, low efficiency, easy error, time consumption and the like.
Disclosure of Invention
The invention provides a left and right eye classification method, a device, a server and a storage medium, which realize the classification of left and right eyes through a neural network model and have the effects of high efficiency, high accuracy and time saving.
In a first aspect, the present invention provides a left-right eye classification method, including:
acquiring an eyeball AS-OCT image of a tested person;
inputting the eyeball AS-OCT image into a preset left eye classification model and a preset right eye classification model, and outputting a classification result;
based on the classification result, whether the eyeball of the subject is the left eye or the right eye is determined.
Further, the left-right eye classification model comprises a convolution feature extraction layer and a full connection layer which are connected in sequence, wherein the convolution feature extraction layer comprises a key region branch and a global region branch, and the global region branch comprises an attention module.
Further, the attention module includes a channel attention extraction layer and a spatial attention extraction layer.
Further, the training method of the left and right eye classification model comprises the following steps:
acquiring a training set, wherein the training set is a pre-stored eyeball AS-OCT image and a corresponding left eye and right eye judgment result;
substituting the training set into a preset first classification model to perform parameter adjustment so as to generate a second classification model;
substituting the training set into the second classification model to obtain a first global image feature;
substituting the first global image feature into the attention module to generate an attention profile;
determining the critical area based on the attention profile;
and constructing the left and right eye classification models based on the key region and the second classification model.
Further, the inputting the eyeball AS-OCT image into a preset left-right eye classification model, outputting a classification result, and the method comprises the following steps:
taking the eyeball AS-OCT image AS a global image, and generating a key area image based on the global image;
inputting the global image into the convolution feature extraction layer to generate a second global image feature;
simultaneously, inputting the key region image into the convolution feature extraction layer to generate key region image features;
and inputting the second global image features and the key region image features into a full-connection layer to generate a classification result.
In a second aspect, the present invention provides a neural network-based left-right eye classification device, comprising:
the acquisition module is used for acquiring an eyeball AS-OCT image of the tested person;
the calculation module is used for inputting the eyeball AS-OCT image into a preset left-right eye classification model and outputting a classification result;
and the judging module is used for judging whether the eyeball of the tested person is the left eye or the right eye based on the classification result.
Further, the left and right eye classification model is a convolution feature extraction layer and a full connection layer which are sequentially connected, the convolution feature extraction layer comprises a key region branch and a global region branch, the global region branch comprises an attention module, the left and right eye classification device based on the neural network further comprises a training module, and the training module comprises:
the acquisition unit is used for acquiring a training set, wherein the training set is a pre-stored eyeball AS-OCT image and a corresponding left-right eye judgment result;
the first model generation unit is used for substituting the training set into a preset first classification model to perform parameter adjustment so as to generate a second classification model;
the first computing unit is used for substituting the training set into the second classification model to obtain a first global image feature;
a second computing unit configured to substitute the first global image feature into the attention module to generate an attention profile;
a key region determining unit configured to determine the key region based on the attention profile;
and the second model generating unit is used for constructing the left-right eye classification model based on the key region and the second classification model.
Further, the computing module includes:
a key region image generation unit configured to generate a key region image based on the global image with the eyeball AS-OCT image AS the global image;
a third calculation unit, configured to input the global image into the convolution feature extraction layer, and generate a second global image feature;
and the fourth calculation unit is used for inputting the key region image into the convolution feature extraction layer at the same time to generate key region image features.
And the classification unit is used for inputting the second global image features and the key region image features into a full-connection layer to generate classification results.
In a third aspect, the present invention provides a server comprising a memory, a processor and a program stored on the memory and executable on the processor, the processor implementing the left-right eye classification method as described in any one of the above when executing the program.
In a fourth aspect, the present invention provides a terminal-readable storage medium having stored thereon a program which, when executed by a processor, is capable of implementing a left-right eye classification method as described in any one of the above.
The invention realizes the classification of the left eye and the right eye through the neural network model, and achieves the effects of high efficiency, high accuracy and time saving.
Drawings
Fig. 1 is a flowchart of a neural network-based left-right eye classification method according to the first embodiment.
Fig. 2 is a schematic diagram of a left-right eye classification model according to the first embodiment.
Fig. 3 is a flowchart of a neural network-based left-right eye classification method according to the second embodiment.
Fig. 4 is a block diagram of a neural network-based left-right eye classification device according to the third embodiment.
Fig. 5 is a block diagram of an alternative embodiment of the third embodiment.
Fig. 6 is a block diagram of a server according to the fourth embodiment.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts steps as a sequential process, many of the steps may be implemented in parallel, concurrently, or with other steps. Furthermore, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Furthermore, the terms "first," "second," and the like, may be used herein to describe various directions, acts, steps, or elements, etc., but these directions, acts, steps, or elements are not limited by these terms. These terms are only used to distinguish one direction, action, step or element from another direction, action, step or element. For example, the first characteristic information may be the second characteristic information or the third characteristic information, and similarly, the second characteristic information and the third characteristic information may be the first characteristic information without departing from the scope of the present application. The first feature information, the second feature information and the third feature information are all feature information of the distributed file system, but are not the same feature information. The terms "first," "second," and the like, are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, "plurality", "batch" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Example 1
AS shown in fig. 1, the present embodiment provides a method for classifying left and right eyes based on a neural network, which receives an input AS-OCT image of an eye of a subject and generates a left and right eye determination result. The method specifically comprises the following steps:
s101, acquiring an eyeball AS-OCT image of a tested person.
S102, inputting the eyeball AS-OCT image into a preset left eye classification model and a preset right eye classification model, and outputting a classification result.
The left and right eye classification model according to the present embodiment is a trained image classification model for classifying an input image into one of a plurality of preset classes, and is typically implemented by using a very large scale convolutional neural network, and may be any one of a CNN model, a VGG model, a rset model, or an AlexNet model, or any combination of the above models.
In an alternative embodiment, the left and right eye classification model comprises an input layer, a convolution feature extraction layer, a full connection layer and an output layer in sequence, wherein the convolution feature extraction layer comprises a key region branch and a global region branch.
In another alternative embodiment, as shown in fig. 2, the left-right eye classification model sequentially includes an input layer 1, a convolution feature extraction layer 2, a full connection layer 3, and an output layer 4, wherein the convolution feature extraction layer 2 includes a critical region branch 210 and a global region branch 220, and the global region branch 220 includes an attention module 221. Wherein optionally the attention module comprises a channel attention extraction layer and a spatial attention extraction layer.
In this embodiment, the convolution feature extraction layer is configured to extract features of the input image, where the global features refer to overall properties of the input image, such as color features, texture features, and/or shape features. Critical area features refer to features extracted from critical areas of an image, such as edges, corners, lines, curves, and/or areas with specific properties. In this embodiment, the AS-OCT image is an eyeball scan image, the convolution feature extraction layer is used to extract global image features and key region features, and the full connection layer is used to connect the input global image features and key region features, and send the output values to the classifier to achieve classification of left and right eyes.
S103, judging whether eyeballs of the tested person are left eyes or right eyes based on the classification result.
This step outputs the classification result as two values based on the present invention using the scene, and the classification result is K, which is 1 or 0, for example. When k=1, the eyeball of the subject is determined to be the left eye, and k=0, the eyeball of the subject is determined to be the right eye.
The embodiment realizes the classification of the left eye and the right eye through the neural network model and has the effects of high efficiency, high accuracy and time saving. The convolution characteristic extraction layer is divided into a key region branch and a global region branch in the classification model to form a multi-stage network architecture, so that the key region information loss is avoided.
Example two
As shown in fig. 3, the present embodiment adds the training method of the left and right eye classification model on the basis of the above embodiment, and the model training process in this step is illustrated by taking a classification network formed by a convolution feature extraction layer and a full connection layer as an example, and includes the following steps:
s2011, acquiring a training set, wherein the training set is a pre-stored eyeball AS-OCT image and a corresponding left and right eye judgment result.
The training set is composed of N images, each of which is labeled with one of K different categories, in this embodiment, K has a value of 2, illustratively, K has a value of 0 or 1, and when k=1, the eyeball of the subject is determined to be the left eye, and k=0, the eyeball of the subject is determined to be the right eye. For example, the number of the training set images is n=n1+n2, including the AS-OCT images of N1 healthy eyeballs, n1=50000, including the AS-OCT images of N2 diseased eyeballs and healthy eyeballs, n2=15000, and the training set further includes the left-right eye classification values corresponding to the N AS-OCT images.
S2012, substituting the training set into a preset first classification model to perform parameter adjustment so as to generate a second classification model.
The first classification model in the step is a pre-training model, and comprises the number of constructed convolution layers, a connection mode and a network structure, and the training process is only used for adjusting parameters. In the pre-training model of the present example, a network of resnet18 trained based on an image-net data set is selected as a convolution feature extraction layer, and the output class of the full-connection layer is modified to be 2 classes. The network structure and parameters are shown in table 1:
TABLE 1
S2013, substituting the training set into the second classification model to acquire a first global image feature.
S2014, substituting the first global image feature into the attention module to generate an attention distribution map.
And loading the second classification model obtained by training, and visualizing the attention distribution map of all AS-OCT images in the training set by using Grad-CAM technology so AS to obtain key areas with important significance for the left and right eye classification.
The formula for calculating the attention profile at this step is as follows:
in the formula (1), y c A score representing the score for each category c,elements representing the ith row and jth column of the k Zhang Tezheng diagram, Z being the total number of pixels, in equation (2), ++>Representing the calculation of the class c attention profile +.>The weight of the kth feature map.
And S2015, determining the key region based on the attention distribution map.
This step obtains the key region located in the above step from the global AS-OCT image, and the input image size is 2000×1750, and the focus distribution map determines that the image blocks with the AS-OCT image center expanded by 320 pixels up, down, left, and right respectively are the key regions.
S2016, constructing the left and right eye classification models based on the key region and the second classification model.
Optionally, after the key area is determined, the constructed left-right eye classification model comprises an input layer, a first convolution layer conv1, a first bn1 layer, a first Relu1 activation layer, a first pooling layer maxpool1, a first block convolution block, a second block convolution block, a third block convolution block, a fourth block convolution block, a second pooling layer avgpool1 and a first full connection layer fc1 which are sequentially connected. Each block convolution block comprises a convolution layer, a bn layer, a relu layer, a convolution layer and a combination of the bn layers connected in sequence.
The first convolution layer conv1 convolution kernel is 7x7, stride is 2, and the number of convolution kernels is 64. The first pooling layer maxpool1 has a core size of 3×3 and stride of 2. The convolution layer kernel size in the first block is 3*3, and the number of convolution kernels is 64. The convolution layer kernel size in the second block is 3*3, and the number of convolution kernels is 128. The convolution layer kernel size in the third block is 3*3, and the number of convolution kernels is 256. The convolution layer in the fourth block has a core size of 3*3 and the number of convolution cores is 512.
In an alternative embodiment of this step, the global region branch further includes an attention module, where the attention module includes a channel attention extraction layer and a spatial attention extraction layer, and the construction process of the attention module is respectively:
channel attention extraction layer construction: and carrying out maximum pooling and average pooling on the input feature map of each channel to obtain two tensors with the size of [ B, C, 1], wherein B represents the batch size, C represents the number of channels, splicing the two tensors in the dimension of the channels, passing through two full-connection layers to finally obtain a channel attention distribution tensor, and carrying out dot multiplication on the attention distribution tensor and the input feature map to obtain F_map_with_c_attention. In one embodiment, the channel attention extraction layer is composed of a channel average pooling layer channel_avg_pool, a general maximum pooling layer channel_max_pool, a full connection layer channel_fc1, a full connection layer channel_fc2, a relu layer, a sigmod layer, which are connected in sequence.
Spatial attention extraction layer construction process: respectively carrying out average pooling and maximum pooling on all input feature images to obtain tensors with two sizes of [ B,1, W and H ], splicing the two tensors along the channel direction, obtaining a spatial_attribute through a convolution layer, and carrying out point multiplication on the spatial_attribute and the F_map_with_c_attribute to obtain a final feature image F_map_with_attribute after attention guidance, wherein the calculation process is as follows:
in the above formulas (3) (4), F represents an input feature map, avgPool and MaxPool represent average pooling and maximum pooling, respectively, MLP represents a fully connected neural network, σ represents a Sigmod function, W 0 And W is 1 Representing the corresponding weight, f 7×7 Representing a convolution operation with a convolution kernel of 7x 7.
In one embodiment, the spatial attention extraction layer is optionally composed of an average pooling layer spatial_mean, a maximum pooling layer spatial_max, a convolution layer conv, a sigmod layer, connected in sequence.
Embedding the attention module constructed in the above steps between all the convolution blocks of the convolution feature extraction layer, taking a pretraining model to select a mesh 18 network trained based on an image-mesh data set as an example of the convolution feature extraction layer, and embedding the attention module between all the convolution blocks of the mesh 18 network.
In this step, the tuning parameter configuration of each randomly generated convolutional neural network model is shown in table 2:
TABLE 2
S202, acquiring an eyeball AS-OCT image of the tested person.
S2031, taking the eyeball AS-OCT image AS a global image, and generating a key area image based on the global image.
In the embodiment based on the pre-training model, specifically, the size of the input eyeball AS-OCT image is 2000x1750, and an image block with the center of the image expanded by 320 pixels up, down, left and right is taken to extract the image block AS the key region image.
S2032, inputting the global image into the convolution feature extraction layer to generate a second global image feature.
And S2033, inputting the key region image into the convolution feature extraction layer to generate key region image features.
In this step, optionally, the global region branch includes an attention module, and in an embodiment based on the pre-training model described above, specifically, the second global image feature is extracted through the resnet18 network embedded with the CBAM module, and the key region image feature is extracted by using the resnet18 network.
S2034, inputting the second global image feature and the key area image feature into a full connection layer to generate a classification result.
The step of connecting the second global image feature and the key region image feature to obtain a final classification feature, and classifying the features by using a full connection layer. Specifically, 512 input hidden units and 2 output units are fully connected neural networks.
S204, judging whether the eyeball of the tested person is the left eye or the right eye based on the classification result.
The classification result is output as two values based on the usage scenario of the present invention, and the classification result is K, which is 1 or 0, for example. When k=1, the eyeball of the subject is determined to be the left eye, and k=0, the eyeball of the subject is determined to be the right eye.
According to the embodiment, the construction process of the classification model is increased, the most important areas for judging the left eye and the right eye in the visual areas are used as key areas through Grad-CAM visual technology, the weight of the key areas is improved, the classification result is more accurate, and meanwhile, the attention module is added on the overall area branch of the convolution feature extraction layer, so that the classification performance of the model is improved.
Example III
As shown in fig. 4, the present embodiment provides a neural network-based left-right eye classification device 5, including:
an acquisition module 501, configured to acquire an eyeball AS-OCT image of a subject;
the calculation module 502 is configured to input the eyeball AS-OCT image into a preset left-right eye classification model, and output a classification result;
a determining module 503, configured to determine whether the eyeball of the tested person is the left eye or the right eye based on the classification result.
In an alternative embodiment, as shown in fig. 5, the left and right eye classification model of the neural network-based left and right eye classification device 5 includes a convolution feature extraction layer and a fully connected layer connected in sequence, the convolution feature extraction layer including a critical region branch and a global region branch, the global region branch including an attention module, the neural network-based left and right eye classification device further including a training module 504, the training module 504 including:
an obtaining unit 5041, configured to obtain a training set, where the training set is a pre-stored AS-OCT image of an eyeball and a corresponding left-right eye determination result;
a first model generating unit 5042, configured to substitute the training set into a preset first classification model to perform parameter adjustment, so as to generate a second classification model;
a first computing unit 5043 configured to substitute the training set into the second classification model to obtain a first global image feature;
a second computing unit 5044 for substituting the first global image feature into the attention module to generate an attention profile;
a key region determining unit 5045 for determining the key region based on the attention profile;
a second model generating unit 5046 for constructing the left and right eye classification model based on the key region and the second classification model.
The left-right eye classification device based on the neural network provided by the embodiment of the invention can execute the left-right eye classification method based on the neural network provided by any embodiment of the invention, and has the corresponding execution method and beneficial effects of the functional module.
Example IV
As shown in fig. 6, the present embodiment provides a schematic structural diagram of a server, and as shown in the drawing, the server includes a processor 601, a memory 602, an input device 603, and an output device 604; the number of processors 601 in the server may be one or more, one processor 601 being illustrated in the figure; the processor 601, memory 602, input means 603 and output means 604 in the device/terminal/server may be linked by a bus or other means, in the figure by way of example.
The memory 602 is used as a computer readable storage medium for storing a software program, a computer executable program, and modules, such as program instructions/modules (e.g., acquisition module, 501, calculation module 502, etc.) corresponding to the gateway-based link generation method in the embodiment of the present invention. The processor 601 executes various functional applications of the device/terminal/server and data processing by running software programs, instructions and modules stored in the memory 602, i.e., implements the above-described neural network-based left-right eye classification method.
The memory 602 may include primarily a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for functionality; the storage data area may store data created according to the use of the terminal, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 602 may further include memory remotely located with respect to the processor 601, which may be linked to the device/terminal/server through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 603 may be used to receive input numeric or character information and to generate key signal inputs related to user settings of the device/terminal/server and function control. The output 604 may include a display device such as a display screen.
The embodiment of the invention provides a server capable of executing the left-right eye classification method based on the neural network, which is provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example five
The fifth embodiment of the present invention further provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the neural network-based left-right eye classification method according to any embodiment of the present invention:
acquiring an eyeball AS-OCT image of a tested person;
inputting the eyeball AS-OCT image into a preset left eye classification model and a preset right eye classification model, and outputting a classification result;
based on the classification result, whether the eyeball of the subject is the left eye or the right eye is determined.
The computer-readable storage media of embodiments of the present invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical link having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or terminal. In the case of remote computers, the remote computer may be linked to the user's computer through any sort of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it may be linked to an external computer (for example, through the Internet using an Internet service provider).
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (7)
1. A left-right eye classification method, comprising:
acquiring an eyeball AS-OCT image of a tested person;
inputting the eyeball AS-OCT image into a preset left eye classification model and a preset right eye classification model, and outputting a classification result;
judging whether the eyeball of the tested person is the left eye or the right eye based on the classification result;
the left eye classification model and the right eye classification model comprise a convolution feature extraction layer and a full connection layer which are sequentially connected, wherein the convolution feature extraction layer comprises a key region branch and a global region branch, and the global region branch comprises an attention module;
inputting the eyeball AS-OCT image into a preset left eye classification model and a preset right eye classification model, and outputting a classification result, wherein the method comprises the following steps:
taking the eyeball AS-OCT image AS a global image, and generating a key area image based on the global image;
inputting the global image into the convolution feature extraction layer to generate a second global image feature;
simultaneously, inputting the key region image into the convolution feature extraction layer to generate key region image features;
inputting the second global image features and the key region image features into the full-connection layer to generate a classification result;
inputting the second global image feature and the key region image feature into the full connection layer to generate a classification result, wherein the method comprises the following steps:
connecting the second global image feature and the key region image feature through the full connection layer to obtain a final classification feature;
using the full connection layer to classify left and right eyes of the input image according to the final classification characteristic;
wherein the full connection layer comprises 512 hidden units and 2 output units.
2. A left-right eye classification method according to claim 1, wherein the attention module comprises a channel attention extraction layer and a spatial attention extraction layer.
3. The left-right eye classification method according to claim 1, wherein the training method of the left-right eye classification model comprises:
acquiring a training set, wherein the training set is a pre-stored eyeball AS-OCT image and a corresponding left eye and right eye judgment result;
substituting the training set into a preset first classification model to perform parameter adjustment so as to generate a second classification model;
substituting the training set into the second classification model to obtain a first global image feature;
substituting the first global image feature into the attention module to generate an attention profile;
determining the critical area based on the attention profile;
and constructing the left and right eye classification models based on the key region and the second classification model.
4. A neural network-based left-right eye classification device, comprising:
the acquisition module is used for acquiring an eyeball AS-OCT image of the tested person;
the calculation module is used for inputting the eyeball AS-OCT image into a preset left-right eye classification model and outputting a classification result;
the judging module is used for judging whether eyeballs of the tested person are left eyes or right eyes based on the classification result;
the left eye classification model and the right eye classification model comprise a convolution feature extraction layer and a full connection layer which are sequentially connected, wherein the convolution feature extraction layer comprises a key region branch and a global region branch, and the global region branch comprises an attention module;
the computing module includes:
a key region image generation unit configured to generate a key region image based on the global image with the eyeball AS-OCT image AS the global image;
a third calculation unit, configured to input the global image into the convolution feature extraction layer, and generate a second global image feature;
a fourth calculation unit, configured to input the key region image into the convolution feature extraction layer at the same time, and generate a key region image feature;
the classification unit is used for inputting the second global image features and the key region image features into the full-connection layer to generate classification results;
the classification unit is specifically configured to connect the second global image feature and the key area image feature through the full connection layer, so as to obtain a final classification feature;
using the full connection layer to classify left and right eyes of the input image according to the final classification characteristic;
wherein the full connection layer comprises 512 hidden units and 2 output units.
5. The neural network-based left-right eye classification device of claim 4, further comprising a training module comprising:
the acquisition unit is used for acquiring a training set, wherein the training set is a pre-stored eyeball AS-OCT image and a corresponding left-right eye judgment result;
the first model generation unit is used for substituting the training set into a preset first classification model to perform parameter adjustment so as to generate a second classification model;
the first computing unit is used for substituting the training set into the second classification model to obtain a first global image feature;
a second computing unit configured to substitute the first global image feature into the attention module to generate an attention profile;
a key region determining unit configured to determine the key region based on the attention profile;
and the second model generating unit is used for constructing the left-right eye classification model based on the key region and the second classification model.
6. A server comprising a memory, a processor and a program stored on the memory and executable on the processor, wherein the processor implements the left-right eye classification method of any of claims 1-3 when the program is executed by the processor.
7. A terminal-readable storage medium, on which a program is stored, characterized in that the program, when executed by a processor, is capable of implementing the left-right eye classification method according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010935200.1A CN112101438B (en) | 2020-09-08 | 2020-09-08 | Left-right eye classification method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010935200.1A CN112101438B (en) | 2020-09-08 | 2020-09-08 | Left-right eye classification method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112101438A CN112101438A (en) | 2020-12-18 |
| CN112101438B true CN112101438B (en) | 2024-04-16 |
Family
ID=73751151
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010935200.1A Active CN112101438B (en) | 2020-09-08 | 2020-09-08 | Left-right eye classification method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112101438B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114155400B (en) * | 2022-02-10 | 2022-05-17 | 澄影科技(北京)有限公司 | Image processing method, device and equipment |
| CN115205956B (en) * | 2022-08-11 | 2023-04-07 | 北京万里红科技有限公司 | Left and right eye detection model training method, method and device for identifying left and right eyes |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011013315A1 (en) * | 2009-07-30 | 2011-02-03 | 株式会社トプコン | Fundus analysis device and fundus analysis method |
| CN105592829A (en) * | 2013-07-29 | 2016-05-18 | 拜尔普泰戈恩公司 | Procedural optical coherence tomography (OCT) for surgery and related systems and methods |
| EP3125135A1 (en) * | 2015-07-31 | 2017-02-01 | Xiaomi Inc. | Picture processing method and device |
| CN109635669A (en) * | 2018-11-19 | 2019-04-16 | 北京致远慧图科技有限公司 | Image classification method, the training method of device and disaggregated model, device |
| CN110197208A (en) * | 2019-05-14 | 2019-09-03 | 江苏理工学院 | A kind of textile flaw intelligent measurement classification method and device |
| JP2019150486A (en) * | 2018-03-06 | 2019-09-12 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
| CN110400288A (en) * | 2019-06-18 | 2019-11-01 | 中南民族大学 | A kind of sugar of fusion eyes feature nets sick recognition methods and device |
| CN110543802A (en) * | 2018-05-29 | 2019-12-06 | 北京大恒普信医疗技术有限公司 | Method and device for identifying left eye and right eye in fundus image |
| CN110555845A (en) * | 2019-09-27 | 2019-12-10 | 上海鹰瞳医疗科技有限公司 | Fundus OCT image identification method and equipment |
| CN110580482A (en) * | 2017-11-30 | 2019-12-17 | 腾讯科技（深圳）有限公司 | Image classification model training, image classification and personalized recommendation method and device |
| CN110717856A (en) * | 2019-09-03 | 2020-01-21 | 天津大学 | Super-resolution reconstruction algorithm for medical imaging |
| CN110826470A (en) * | 2019-11-01 | 2020-02-21 | 复旦大学 | Eye fundus image left and right eye identification method based on depth active learning |
| CN110993094A (en) * | 2019-11-19 | 2020-04-10 | 中国科学院深圳先进技术研究院 | Intelligent auxiliary diagnosis method and terminal based on medical images |
| CN111046939A (en) * | 2019-12-06 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | CNN (CNN) class activation graph generation method based on attention |
| CN111340087A (en) * | 2020-02-21 | 2020-06-26 | 腾讯医疗健康(深圳)有限公司 | Image recognition method, image recognition device, computer-readable storage medium and computer equipment |
| CN111402217A (en) * | 2020-03-10 | 2020-07-10 | 广州视源电子科技股份有限公司 | Image grading method, device, equipment and storage medium |
| CN111428807A (en) * | 2020-04-03 | 2020-07-17 | 桂林电子科技大学 | Image processing method and computer-readable storage medium |
| CN111428072A (en) * | 2020-03-31 | 2020-07-17 | 南方科技大学 | Ophthalmologic multimodal image retrieval method, apparatus, server and storage medium |
| CN111461218A (en) * | 2020-04-01 | 2020-07-28 | 复旦大学 | Sample data labeling system for fundus image of diabetes mellitus |
| CN111553205A (en) * | 2020-04-12 | 2020-08-18 | 西安电子科技大学 | Vehicle weight recognition method, system, medium and video monitoring system without license plate information |
-
2020
- 2020-09-08 CN CN202010935200.1A patent/CN112101438B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011013315A1 (en) * | 2009-07-30 | 2011-02-03 | 株式会社トプコン | Fundus analysis device and fundus analysis method |
| CN105592829A (en) * | 2013-07-29 | 2016-05-18 | 拜尔普泰戈恩公司 | Procedural optical coherence tomography (OCT) for surgery and related systems and methods |
| EP3125135A1 (en) * | 2015-07-31 | 2017-02-01 | Xiaomi Inc. | Picture processing method and device |
| CN110580482A (en) * | 2017-11-30 | 2019-12-17 | 腾讯科技（深圳）有限公司 | Image classification model training, image classification and personalized recommendation method and device |
| JP2019150486A (en) * | 2018-03-06 | 2019-09-12 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
| CN110543802A (en) * | 2018-05-29 | 2019-12-06 | 北京大恒普信医疗技术有限公司 | Method and device for identifying left eye and right eye in fundus image |
| CN109635669A (en) * | 2018-11-19 | 2019-04-16 | 北京致远慧图科技有限公司 | Image classification method, the training method of device and disaggregated model, device |
| CN110197208A (en) * | 2019-05-14 | 2019-09-03 | 江苏理工学院 | A kind of textile flaw intelligent measurement classification method and device |
| CN110400288A (en) * | 2019-06-18 | 2019-11-01 | 中南民族大学 | A kind of sugar of fusion eyes feature nets sick recognition methods and device |
| CN110717856A (en) * | 2019-09-03 | 2020-01-21 | 天津大学 | Super-resolution reconstruction algorithm for medical imaging |
| CN110555845A (en) * | 2019-09-27 | 2019-12-10 | 上海鹰瞳医疗科技有限公司 | Fundus OCT image identification method and equipment |
| CN110826470A (en) * | 2019-11-01 | 2020-02-21 | 复旦大学 | Eye fundus image left and right eye identification method based on depth active learning |
| CN110993094A (en) * | 2019-11-19 | 2020-04-10 | 中国科学院深圳先进技术研究院 | Intelligent auxiliary diagnosis method and terminal based on medical images |
| CN111046939A (en) * | 2019-12-06 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | CNN (CNN) class activation graph generation method based on attention |
| CN111340087A (en) * | 2020-02-21 | 2020-06-26 | 腾讯医疗健康(深圳)有限公司 | Image recognition method, image recognition device, computer-readable storage medium and computer equipment |
| CN111402217A (en) * | 2020-03-10 | 2020-07-10 | 广州视源电子科技股份有限公司 | Image grading method, device, equipment and storage medium |
| CN111428072A (en) * | 2020-03-31 | 2020-07-17 | 南方科技大学 | Ophthalmologic multimodal image retrieval method, apparatus, server and storage medium |
| CN111461218A (en) * | 2020-04-01 | 2020-07-28 | 复旦大学 | Sample data labeling system for fundus image of diabetes mellitus |
| CN111428807A (en) * | 2020-04-03 | 2020-07-17 | 桂林电子科技大学 | Image processing method and computer-readable storage medium |
| CN111553205A (en) * | 2020-04-12 | 2020-08-18 | 西安电子科技大学 | Vehicle weight recognition method, system, medium and video monitoring system without license plate information |
Non-Patent Citations (7)
| Title |
|---|
| Anterior Segment OCT;Jacqueline Sousa Asam 等;High Resolution Imaging in Microscopy and Ophthalmology;第258-299页 * |
| 前节光学相干断层扫描在眼科领域的应用;武芹;党光福;段练;;国际眼科杂志(09);第1689-1691页 * |
| 前节光学相干断层扫描在青光眼研究中的应用;景金霞;哈少平;范文燕;;国际眼科杂志(06);第1100-1102页 * |
| 基于卷积神经网络的左右眼识别;钟志权;袁进;唐晓颖;;计算机研究与发展(08);第1667-1673页 * |
| 屈光不正患者角膜上皮厚度的特征分析;蒋晶晶 等;汕头大学医学院学报;第32卷(第4期);第215-219页 * |
| 李腾 ; 张宝华 ; .基于局部二值特征与深度学习的人脸识别.内蒙古科技大学学报.2018,(02),163-169. * |
| 王立新 ; 江加和 ; .基于深度学习的显著性区域的图像检索研究.应用科技.2018,(06),63-67. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112101438A (en) | 2020-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11361192B2 (en) | Image classification method, computer device, and computer-readable storage medium | |
| US20210248751A1 (en) | Brain image segmentation method and apparatus, network device, and storage medium | |
| CN110399929B (en) | Fundus image classification method, fundus image classification apparatus, and computer-readable storage medium | |
| Zhu et al. | Anatomynet: Deep 3d squeeze-and-excitation u-nets for fast and fully automated whole-volume anatomical segmentation | |
| CN110889446A (en) | Face image recognition model training and face image recognition method and device | |
| WO2020022027A1 (en) | Learning device and learning method | |
| CN111402217B (en) | Image grading method, device, equipment and storage medium | |
| CN111881926A (en) | Image generation method, image generation model training method, image generation device, image generation equipment and image generation medium | |
| CN112529146B (en) | Neural network model training method and device | |
| CN112101438B (en) | Left-right eye classification method, device, server and storage medium | |
| US10803586B1 (en) | Image analysis in pathology | |
| DE112020002425T5 (en) | MOTION PREDICTION USING ONE OR MORE NEURAL NETWORKS | |
| CN115345938B (en) | Global-to-local-based head shadow mark point positioning method, equipment and medium | |
| US20230058876A1 (en) | Image processing method and apparatus based on image processing model, electronic device, storage medium, and computer program product | |
| JP6345332B1 (en) | Classification device, classification method, program, and information recording medium | |
| CN112446322A (en) | Eyeball feature detection method, device, equipment and computer-readable storage medium | |
| CN109949288A (en) | Tumor type determines system, method and storage medium | |
| CN106462974A (en) | Optimization of parameters for segmenting an image | |
| CN114998374A (en) | Image segmentation method, device and equipment based on position prior and storage medium | |
| CN110706780A (en) | Radiotherapy plan generation system and storage medium | |
| Wu et al. | Semiautomatic segmentation of glioma on mobile devices | |
| Ankireddy | Assistive diagnostic tool for brain tumor detection using computer vision | |
| TRIVEDI et al. | MOSAICFUSION: MERGING MODALITIES WITH PARTIAL DIFFERENTIAL EQUATION AND DISCRETE COSINE TRANSFORMATION | |
| CN115731240A (en) | Segmentation method, segmentation device, electronic equipment and storage medium | |
| CN112884706A (en) | Image evaluation system based on neural network model and related product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |