CN107977605A - Ocular Boundary characteristic extraction method and device based on deep learning - Google Patents
Ocular Boundary characteristic extraction method and device based on deep learning Download PDFInfo
- Publication number
- CN107977605A CN107977605A CN201711091696.3A CN201711091696A CN107977605A CN 107977605 A CN107977605 A CN 107977605A CN 201711091696 A CN201711091696 A CN 201711091696A CN 107977605 A CN107977605 A CN 107977605A
- Authority
- CN
- China
- Prior art keywords
- mrow
- network
- msub
- boundary
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Ophthalmology & Optometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of ocular Boundary characteristic extraction method and device based on deep learning, wherein, method includes:Training dataset is obtained, by manually demarcating four borders containing semantic information on eye image, and is stored in the boundary graph of four-way and obtains training dataset;A set of overall nested deep neural network for Boundary Extraction is improved, the loss function and network structure of deep neural network are improved, so that the semantic information comprising four edges circle can simultaneously export the boundary graph of corresponding four-way;By the method for deep learning, Training is carried out using the embedding deep neural network of the improved entirety of training data set pair, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.This method can extract four kinds of borders in eye image by the method for deep learning, so as to provide good basis for the real-time three-dimensional reconstruction of eyelid, and can provide the ocular boundary characteristic containing semantic information.
Description
Technical field
It is more particularly to a kind of based on deep learning the present invention relates to computer vision, computer graphics techniques field
Ocular Boundary characteristic extraction method and device.
Background technology
In computer graphics and computer vision field, it is always a weight to generate lively real mask animation
Will and it is challenging the problem of.Face is a most important part on human body, and eyes are more known as the window of soul, eye
The slight movement of portion's area skin expresses the emotion that the mankind enrich.Therefore, the real-time three-dimensional of eyelid is rebuild and can greatly be carried
The sense of reality of mask animation is risen, there is high scientific research and application value.
However, the area of ocular is very small, and contained in the movement of eyelid skin block and fold, for
Ocular shape and the Direct Modeling of movement are typically extremely difficult.Although for the people of different sex, races, age
The shape of group's ocular and movement are had nothing in common with each other, but there are feature same as below for the eyes of the mankind:Upper eyelid and lower eyelid,
Double-edged eyelid and sleeping silkworm (groups of people's presence), these features show as specific border in eye image, can be good at reflecting
The shape of eyelid and movement.
The content of the invention
It is contemplated that solve at least some of the technical problems in related technologies.
For this reason, an object of the present invention is to provide a kind of ocular Boundary characteristic extraction side based on deep learning
Method, this method can rebuild for the real-time three-dimensional of eyelid and provide good basis, and can provide the eye containing semantic information
Portion zone boundary feature.
It is another object of the present invention to propose a kind of ocular Boundary characteristic extraction device based on deep learning.
To reach above-mentioned purpose, one aspect of the present invention embodiment proposes a kind of ocular border based on deep learning
Feature extracting method, comprises the following steps:Training dataset is obtained, wherein, contained by four manually demarcated on eye image
The border of semantic information, and be stored in the boundary graph of four-way and obtain the training dataset;To a set of overall nested use
It is improved in the deep neural network of Boundary Extraction, wherein, improve the loss function and network knot of the deep neural network
Structure, so that the semantic information comprising four edges circle can simultaneously export the boundary graph of the corresponding four-way;Pass through deep learning
Method, carries out Training using the embedding deep neural network of the improved entirety of training data set pair, obtains network
Optimized parameter, to use four edges circle in neural network forecast eye image.
The ocular Boundary characteristic extraction method based on deep learning of the embodiment of the present invention, can pass through deep learning
Method extraction eye image in four kinds of borders so that for eyelid real-time three-dimensional rebuild provide good basis, and
Ocular boundary characteristic containing semantic information can be provided.
In addition, the ocular Boundary characteristic extraction method according to the above embodiment of the present invention based on deep learning may be used also
With with following additional technical characteristic:
Further, in one embodiment of the invention, it is described to contain language by manually demarcating four on eye image
The border of adopted information, further comprises:Control point is demarcated by hand in a line circle in office;Determined according to the control point corresponding
Spline curve;The spline curve of predetermined width is drawn in the boundary graph passage belonging to any bar border.
Further, in one embodiment of the invention, it is described to a set of overall nested depth for Boundary Extraction
Degree neutral net is improved, and is further comprised:Definition includes the loss function of the four edges circle semantic information, will be described whole
The convolution kernel depth for the convolutional layer being connected in the deep neural network of body nesting with side output and final output is changed to 4 by 1.
Further, in one embodiment of the invention, it is described embedding using the improved entirety of training data set pair
Deep neural network carry out Training, further comprise:Using the network of pre-training as initial value, the instruction is used
Practice the original image in data set as network inputs, with reference to deep learning platform using the boundary graph of the four-way of calibration to institute
The training that deep neural network carries out having supervision is stated, obtains the optimized parameter of the network.
Further, in one embodiment of the invention, the loss function is:
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network
The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network
Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
To reach above-mentioned purpose, another aspect of the present invention embodiment proposes a kind of ocular side based on deep learning
Boundary's feature deriving means, including:Acquisition module, for obtaining training dataset, wherein, by manually demarcating on eye image
Four borders containing semantic information, and be stored in the boundary graph of four-way and obtain the training dataset;Module is improved, is used for
A set of overall nested deep neural network for Boundary Extraction is improved, wherein, improve the deep neural network
Loss function and network structure so that the semantic information comprising four edges circle can simultaneously export the border of the corresponding four-way
Figure;Training module, for the method by deep learning, utilizes the embedding depth nerve of the improved entirety of training data set pair
Network carries out Training, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.
The ocular Boundary characteristic extraction device based on deep learning of the embodiment of the present invention, can pass through deep learning
Method extraction eye image in four kinds of borders so that for eyelid real-time three-dimensional rebuild provide good basis, and
Ocular boundary characteristic containing semantic information can be provided.
In addition, the ocular Boundary characteristic extraction device according to the above embodiment of the present invention based on deep learning may be used also
With with following additional technical characteristic:
Further, in one embodiment of the invention, the acquisition module, further comprises:Unit is demarcated, is used for
Control point is demarcated by hand in a line circle in office;Acquiring unit, for determining corresponding spline curve according to the control point;Paint
Unit processed, for drawing the spline curve of predetermined width in the boundary graph passage belonging to any bar border.
Further, in one embodiment of the invention, the improvement module, further comprises:Definition is comprising described
The loss function of four edges circle semantic information, will in the overall nested deep neural network with side output and final output
The convolution kernel depth of connected convolutional layer is changed to 4 by 1.
Further, in one embodiment of the invention, the training module, further comprises:Use pre-training
Network is as initial value, using the original image that the training data is concentrated as network inputs, with reference to deep learning platform profit
The training that has supervision is carried out to the deep neural network with the boundary graph of the four-way of calibration, obtains the optimal ginseng of the network
Number.
Further, in one embodiment of the invention, the loss function is:
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network
The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network
Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
Of the invention above-mentioned and/or additional aspect and advantage will become from the following description of the accompanying drawings of embodiments
Substantially and it is readily appreciated that, wherein:
Fig. 1 is the flow according to the ocular Boundary characteristic extraction method based on deep learning of the embodiment of the present invention
Figure;
Fig. 2 is the stream according to the ocular Boundary characteristic extraction method based on deep learning of one embodiment of the invention
Cheng Tu;
Fig. 3 is to predict four-way according to the improved overall nested networks with optimized parameter of one embodiment of the invention
The result schematic diagram of boundary graph;
Fig. 4 is to predict four-way according to the improved overall nested networks with optimized parameter of one embodiment of the invention
The result schematic diagram of the part calibration of boundary graph;
Fig. 5 is to be shown according to the structure of the ocular Boundary characteristic extraction device based on deep learning of the embodiment of the present invention
It is intended to.
Embodiment
The embodiment of the present invention is described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or has the function of same or like element.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and is not considered as limiting the invention.
The ocular boundary characteristic based on deep learning proposed according to embodiments of the present invention is described with reference to the accompanying drawings
Extracting method and device, describe the ocular based on deep learning proposed according to embodiments of the present invention with reference to the accompanying drawings first
Boundary characteristic extraction method.
Fig. 1 is the flow chart of the ocular Boundary characteristic extraction method based on deep learning of the embodiment of the present invention.
As shown in Figure 1, the ocular Boundary characteristic extraction method based on deep learning of being somebody's turn to do comprises the following steps:
In step S101, training dataset is obtained, wherein, by manually demarcating four on eye image containing semantic letter
The border of breath, and be stored in the boundary graph of four-way and obtain training dataset.
Further, in one embodiment of the invention, by manually demarcating four on eye image containing semantic letter
The border of breath, further comprises:Control point is demarcated by hand in a line circle in office;Determine that corresponding batten is bent according to control point
Line;The spline curve of predetermined width is drawn in boundary graph passage belonging to a line circle in office.
It is understood that with reference to shown in Fig. 1 and Fig. 2, the four-way side of training dataset, i.e. eye areas is manually demarcated
Boundary's figure.First, the embodiment of the present invention demarcates control point by hand on certain border, then determines its corresponding sample according to control point
Bar curve, finally draws the spline curve of one fixed width in the boundary graph passage belonging to the border, wherein, in boundary graph
The probability that the higher expression pixel of pixel value belongs to border is bigger, and therefore, above-mentioned line transect is plotted as in an all pixels
It is worth on the gray level image for 0 and draws the line transect that gray value is 255, represents the position on this border.Particularly, double-edged eyelid
In partial eye area image and it is not present with the lower boundary of sleeping silkworm, then the side corresponding to this two borders in this case
All image pixel values of boundary's figure passage are 0.
In step s 102, a set of overall nested deep neural network for Boundary Extraction is improved, wherein,
The loss function and network structure of deep neural network are improved, so that the semantic information comprising four edges circle can simultaneously export accordingly
The boundary graph of four-way.
Further, in one embodiment of the invention, to a set of overall nested depth god for Boundary Extraction
It is improved, further comprises through network:Definition includes the loss function of four edges circle semantic information, by overall nested depth
The convolution kernel depth for the convolutional layer being connected in neutral net with side output and final output is changed to 4 by 1.
Further, in one embodiment of the invention, loss function is:
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network
The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network
Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
It is understood that the embodiment of the present invention can be to a set of overall nested depth nerve net for Boundary Extraction
Network is improved.Wherein, which is generally used for all borders in extraction image, when for extracting ocular
Border when, on the one hand its extraction border in include noise information, the border of such as iris and sclera, eyebrow;On the other hand
Four edges circle that it is extracted are in same boundary graph, the semantic information not comprising four edges circle.Therefore, the embodiment of the present invention pair
The entirety nested networks are optimized as follows, and the semantic information that can include four edges circle can simultaneously export corresponding four-way
Boundary graph.For loss function, whether the embodiment of the present invention belongs to pixel in four edges circle border judgement by considering
Mistake and the punishment produced, the loss function being defined as follows:
Wherein
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output of network and side
Side exports, αfWithThe weight of respectively final loss and side loss.Loss function L calculate Pixel-level network output and
Cross entropy between legitimate reading,To belong to the set of the pixel on border in i-th border in legitimate reading, β is to belong to border
Pixel account for the ratios of all pixels,Belong to the probability of border i, Pr (g for pixel jj=0 | I;θ) it is picture
Plain j belongs to the probability of non-border pixel.
In step s 103, by the method for deep learning, the embedding depth god of the improved entirety of training data set pair is utilized
Training is carried out through network, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.
Further, in one embodiment of the invention, the embedding depth god of the improved entirety of training data set pair is utilized
Training is carried out through network, is further comprised:Using the network of pre-training as initial value, concentrated using training data
Original image as network inputs, with reference to deep learning platform using the four-way of calibration boundary graph to deep neural network into
Row has the training of supervision, obtains the optimized parameter of network.
It is understood that the embodiment of the present invention can be trained to obtain optimized parameter to above-mentioned network, for predicting
Four edges circle of ocular.The embodiment of the present invention is put down using the training dataset manually demarcated with reference to the deep learning increased income
Platform, carries out arameter optimization on the network model of pre-training, obtains optimal network parameter, has optimal ginseng so as to use
Several network models predicts the four-way boundary graph containing semantic information of any eye image.
In one particular embodiment of the present invention, as shown in figure 3, the embodiment of the present invention includes one group of eye image conduct
Training set, one group of eye image are implemented as test set and a set of overall depth of nesting neutral net being not optimised, the present invention
Example illustrate using improved overall nested networks prediction four-way boundary graph with optimized parameter as a result, specific steps such as
Under:
First, training dataset is manually demarcated.I.e. for each eye image in training set, four edges are demarcated respectively
The position on boundary.For each edge circle, 6 control points are demarcated by hand first, then determine that its corresponding batten is bent according to control point
Line, finally draws the spline curve of one fixed width in the boundary graph passage belonging to the border.Part calibration is illustrated in Fig. 4
As a result, wherein showing for convenience, each edge circle represents with different colours and be covered in (double-edged eyelid on original eye image:
Blueness 1, upper eyelid:Green 2, lower eyelid:Red 3, the lower boundary for the silkworm that crouches:Purple 4).In addition, the situation that double-edged eyelid are not present is such as
Shown in Fig. 4 (c), shown in the situation that sleeping silkworm is not present such as Fig. 4 (d), respectively not under double-edged eyelid and sleeping silkworm under both of these case
Border is demarcated.
Secondly, the embodiment of the present invention is improved a set of overall nested deep neural network for Boundary Extraction.
A convolutional layer is all connected with before the output of primitive network side and final output, its convolution kernel depth is 1, i.e., can only export single-pass
The boundary graph in road, and do not include semantic information.The convolution kernel depth of convolutional layer is improved to 4 in the present embodiment, with four-way
The boundary graph containing semantic information it is corresponding.
Finally, the embodiment of the present invention is trained to obtain optimized parameter to above-mentioned network, for predicting the four of ocular
Bar border.The present embodiment uses the model of a pre-training of above-mentioned overall nested networks as initial value, on its basis into
Row arameter optimization.The present embodiment uses input of the original eye image of training set as network, and uses the four-way of calibration
Boundary graph carries out network the training for having supervision, when iterations reaches 2000 times training restrained and by network at this time
Optimized parameter of the parameter as network.The present embodiment so using the overall nested networks prediction with optimized parameter containing semantic letter
The four-way ocular boundary graph of breath, prediction result are as shown in Figure 3.
The ocular Boundary characteristic extraction method based on deep learning proposed according to embodiments of the present invention, can pass through
Four kinds of borders in the method extraction eye image of deep learning, so as to provide good base for the real-time three-dimensional reconstruction of eyelid
Plinth, and the ocular boundary characteristic containing semantic information can be provided, the real-time three-dimensional reconstruction to eyelid has greater significance.
The ocular boundary characteristic based on deep learning proposed according to embodiments of the present invention referring next to attached drawing description
Extraction element.
Fig. 5 is that the structure of the ocular Boundary characteristic extraction device based on deep learning of one embodiment of the invention is shown
It is intended to.
As shown in figure 5, being somebody's turn to do the ocular Boundary characteristic extraction device 10 based on deep learning includes:Acquisition module 100,
Improve module 200 and training module 300.
Wherein, acquisition module 100 is used to obtain training dataset, wherein, by manually demarcating four on eye image
Border containing semantic information, and be stored in the boundary graph of four-way and obtain training dataset.Module 200 is improved to be used for a set of
The overall nested deep neural network for Boundary Extraction is improved, wherein, improve the loss function of deep neural network
And network structure, so that the semantic information comprising four edges circle can simultaneously export the boundary graph of corresponding four-way.Training module 300
For the method by deep learning, supervision instruction has been carried out using the embedding deep neural network of the improved entirety of training data set pair
Practice, obtain the optimized parameter of network, to use four edges circle in neural network forecast eye image.The device of the embodiment of the present invention
10 can extract four kinds of borders in eye image by the method for deep learning, so as to be provided for the real-time three-dimensional reconstruction of eyelid
Good basis, and the ocular boundary characteristic containing semantic information can be provided.
Further, in one embodiment of the invention, acquisition module 100, further comprise:Demarcate unit, obtain
Unit and drawing unit.Wherein, unit is demarcated to be used to demarcate control point by hand in a line circle in office.Acquiring unit is used for basis
Control point determines corresponding spline curve.Drawing unit is used to draw default width in the boundary graph passage belonging to a line circle in office
The spline curve of degree.
Further, in one embodiment of the invention, module 200 is improved, is further comprised:Definition includes four edges
The loss function of boundary's semantic information, the convolution being connected in overall nested deep neural network with side output and final output
The convolution kernel depth of layer is changed to 4 by 1.
Further, in one embodiment of the invention, training module 300, further comprise:Use the net of pre-training
Network, using the original image that training data is concentrated as network inputs, calibration is utilized with reference to deep learning platform as initial value
The boundary graph of four-way the training for having supervision is carried out to deep neural network, obtain the optimized parameter of network.
Further, in one embodiment of the invention, loss function is:
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network
The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network
Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
It should be noted that the foregoing explanation to the ocular Boundary characteristic extraction embodiment of the method based on deep learning
Illustrate the ocular Boundary characteristic extraction device based on deep learning for being also applied for the embodiment, details are not described herein again.
The ocular Boundary characteristic extraction device based on deep learning proposed according to embodiments of the present invention, can pass through
Four kinds of borders in the method extraction eye image of deep learning, so as to provide good base for the real-time three-dimensional reconstruction of eyelid
Plinth, and the ocular boundary characteristic containing semantic information can be provided, the real-time three-dimensional reconstruction to eyelid has greater significance.
In the description of the present invention, it is to be understood that term " " center ", " longitudinal direction ", " transverse direction ", " length ", " width ",
" thickness ", " on ", " under ", "front", "rear", "left", "right", " vertical ", " level ", " top ", " bottom " " interior ", " outer ", " up time
The orientation or position relationship of the instruction such as pin ", " counterclockwise ", " axial direction ", " radial direction ", " circumferential direction " be based on orientation shown in the drawings or
Position relationship, is for only for ease of and describes the present invention and simplify description, rather than indicates or imply that signified device or element must
There must be specific orientation, with specific azimuth configuration and operation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are only used for description purpose, and it is not intended that instruction or hint relative importance
Or the implicit quantity for indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, " multiple " are meant that at least two, such as two, three
It is a etc., unless otherwise specifically defined.
In the present invention, unless otherwise clearly defined and limited, term " installation ", " connected ", " connection ", " fixation " etc.
Term should be interpreted broadly, for example, it may be fixedly connected or be detachably connected, or integrally;Can be that machinery connects
Connect or be electrically connected;It can be directly connected, can also be indirectly connected by intermediary, can be in two elements
The connection in portion or the interaction relationship of two elements, unless otherwise restricted clearly.For those of ordinary skill in the art
For, the concrete meaning of above-mentioned term in the present invention can be understood as the case may be.
In the present invention, unless otherwise clearly defined and limited, fisrt feature can be with "above" or "below" second feature
It is that the first and second features directly contact, or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists
Second feature " on ", " top " and " above " but fisrt feature are directly over second feature or oblique upper, or be merely representative of
Fisrt feature level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be
One feature is immediately below second feature or obliquely downward, or is merely representative of fisrt feature level height and is less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment of the present invention or example.In the present specification, schematic expression of the above terms is not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
Combined in an appropriate manner in one or more embodiments or example.In addition, without conflicting with each other, the skill of this area
Art personnel can be tied the different embodiments or example described in this specification and different embodiments or exemplary feature
Close and combine.
Although the embodiment of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is impossible to limitation of the present invention is interpreted as, those of ordinary skill in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, changes, replacing and modification.
Claims (10)
- A kind of 1. ocular Boundary characteristic extraction method based on deep learning, it is characterised in that comprise the following steps:Training dataset is obtained, wherein, by manually demarcating four borders containing semantic information on eye image, and it is stored in The training dataset is obtained in the boundary graph of four-way;A set of overall nested deep neural network for Boundary Extraction is improved, wherein, improve the depth nerve The loss function and network structure of network, so that the semantic information comprising four edges circle can simultaneously export the corresponding four-way Boundary graph;AndBy the method for deep learning, prison has been carried out using the embedding deep neural network of the improved entirety of training data set pair Supervise and instruct white silk, obtain the optimized parameter of network, to use four edges circle in neural network forecast eye image.
- 2. the ocular Boundary characteristic extraction method according to claim 1 based on deep learning, it is characterised in that institute State by manually demarcating four borders containing semantic information on eye image, further comprise:Control point is demarcated by hand in a line circle in office;Corresponding spline curve is determined according to the control point;The spline curve of predetermined width is drawn in the boundary graph passage belonging to any bar border.
- 3. the ocular Boundary characteristic extraction method according to claim 1 based on deep learning, it is characterised in that institute State and a set of overall nested deep neural network for Boundary Extraction is improved, further comprise:Definition include the loss function of the four edges circle semantic information, by the deep neural network of the overall nesting with side The convolution kernel depth for the convolutional layer that side output is connected with final output is changed to 4 by 1.
- 4. the ocular Boundary characteristic extraction method according to claim 1 based on deep learning, it is characterised in that institute State and carry out Training using the embedding deep neural network of the improved entirety of training data set pair, further comprise:Using the network of pre-training as initial value, using the original image that the training data is concentrated as network inputs, knot The training that deep learning platform carries out the deep neural network to have supervision using the boundary graph of the four-way of calibration is closed, is obtained The optimized parameter of the network.
- 5. according to ocular Boundary characteristic extraction method of the claim 1-4 any one of them based on deep learning, it is special Sign is that the loss function is:<mrow> <mi>arg</mi> <mi> </mi> <msub> <mi>min</mi> <mi>&theta;</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>&alpha;</mi> <mi>f</mi> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&psi;</mi> <mi>f</mi> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>+</mo> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>5</mn> </msubsup> <msub> <mi>&alpha;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&psi;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>)</mo> </mrow> <mo>,</mo> </mrow>Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output of network and network Side exports, αfWithThe weight for weight and the side loss respectively finally lost, L (ψf(I, θ), G) it is the final of network Loss, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
- A kind of 6. ocular Boundary characteristic extraction device based on deep learning, it is characterised in that including:Acquisition module, for obtaining training dataset, wherein, by manually demarcating four on eye image containing semantic information Border, and be stored in the boundary graph of four-way and obtain the training dataset;Module is improved, for being improved to a set of overall nested deep neural network for Boundary Extraction, wherein, improve The loss function and network structure of the deep neural network, so that the semantic information comprising four edges circle can simultaneously export accordingly The boundary graph of the four-way;AndTraining module, for the method by deep learning, utilizes the embedding depth god of the improved entirety of training data set pair Training is carried out through network, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.
- 7. the ocular Boundary characteristic extraction device according to claim 6 based on deep learning, it is characterised in that institute Acquisition module is stated, is further comprised:Unit is demarcated, control point is demarcated by hand in a line circle in office;Acquiring unit, for determining corresponding spline curve according to the control point;Drawing unit, for drawing the spline curve of predetermined width in the boundary graph passage belonging to any bar border.
- 8. the ocular Boundary characteristic extraction device according to claim 6 based on deep learning, it is characterised in that institute Improvement module is stated, is further comprised:Definition include the loss function of the four edges circle semantic information, by the deep neural network of the overall nesting with side The convolution kernel depth for the convolutional layer that side output is connected with final output is changed to 4 by 1.
- 9. the ocular Boundary characteristic extraction device according to claim 6 based on deep learning, it is characterised in that institute Training module is stated, is further comprised:Using the network of pre-training as initial value, using the original image that the training data is concentrated as network inputs, knot The training that deep learning platform carries out the deep neural network to have supervision using the boundary graph of the four-way of calibration is closed, is obtained The optimized parameter of the network.
- 10. according to ocular Boundary characteristic extraction device of the claim 6-9 any one of them based on deep learning, it is special Sign is that the loss function is:<mrow> <mi>arg</mi> <mi> </mi> <msub> <mi>min</mi> <mi>&theta;</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>&alpha;</mi> <mi>f</mi> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&psi;</mi> <mi>f</mi> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>+</mo> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>5</mn> </msubsup> <msub> <mi>&alpha;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&psi;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>)</mo> </mrow> <mo>,</mo> </mrow>Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output of network and network Side exports, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) lost for the final of network, ψf The final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function, Lost for side,The side output of network in the case of for I and θ.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711091696.3A CN107977605B (en) | 2017-11-08 | 2017-11-08 | Eye region boundary feature extraction method and device based on deep learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711091696.3A CN107977605B (en) | 2017-11-08 | 2017-11-08 | Eye region boundary feature extraction method and device based on deep learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107977605A true CN107977605A (en) | 2018-05-01 |
CN107977605B CN107977605B (en) | 2020-04-24 |
Family
ID=62013116
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201711091696.3A Active CN107977605B (en) | 2017-11-08 | 2017-11-08 | Eye region boundary feature extraction method and device based on deep learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107977605B (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108830901A (en) * | 2018-06-22 | 2018-11-16 | 维沃移动通信有限公司 | A kind of image processing method and electronic equipment |
CN109389085A (en) * | 2018-10-09 | 2019-02-26 | 清华大学 | Lip reading identification model training method and device based on parametric curve |
CN110969191A (en) * | 2019-11-07 | 2020-04-07 | 吉林大学 | Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method |
CN112329546A (en) * | 2020-10-15 | 2021-02-05 | 杭州电子科技大学 | Eye height measuring method based on deep learning |
CN112799510A (en) * | 2021-01-27 | 2021-05-14 | 清华大学 | Automatic generation method and device for multi-style natural eyeball movement |
WO2021191692A1 (en) * | 2020-03-27 | 2021-09-30 | International Business Machines Corporation | Annotation of digital images for machine learning |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105718878A (en) * | 2016-01-19 | 2016-06-29 | 华南理工大学 | Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network |
CN105769120A (en) * | 2016-01-27 | 2016-07-20 | 深圳地平线机器人科技有限公司 | Fatigue driving detection method and device |
CN106650688A (en) * | 2016-12-30 | 2017-05-10 | 公安海警学院 | Eye feature detection method, device and recognition system based on convolutional neural network |
CN106960456A (en) * | 2017-03-28 | 2017-07-18 | 长沙全度影像科技有限公司 | A kind of method that fisheye camera calibration algorithm is evaluated |
CN107133616A (en) * | 2017-04-02 | 2017-09-05 | 南京汇川图像视觉技术有限公司 | A kind of non-division character locating and recognition methods based on deep learning |
CN107168527A (en) * | 2017-04-25 | 2017-09-15 | 华南理工大学 | The first visual angle gesture identification and exchange method based on region convolutional neural networks |
-
2017
- 2017-11-08 CN CN201711091696.3A patent/CN107977605B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105718878A (en) * | 2016-01-19 | 2016-06-29 | 华南理工大学 | Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network |
CN105769120A (en) * | 2016-01-27 | 2016-07-20 | 深圳地平线机器人科技有限公司 | Fatigue driving detection method and device |
CN106650688A (en) * | 2016-12-30 | 2017-05-10 | 公安海警学院 | Eye feature detection method, device and recognition system based on convolutional neural network |
CN106960456A (en) * | 2017-03-28 | 2017-07-18 | 长沙全度影像科技有限公司 | A kind of method that fisheye camera calibration algorithm is evaluated |
CN107133616A (en) * | 2017-04-02 | 2017-09-05 | 南京汇川图像视觉技术有限公司 | A kind of non-division character locating and recognition methods based on deep learning |
CN107168527A (en) * | 2017-04-25 | 2017-09-15 | 华南理工大学 | The first visual angle gesture identification and exchange method based on region convolutional neural networks |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108830901A (en) * | 2018-06-22 | 2018-11-16 | 维沃移动通信有限公司 | A kind of image processing method and electronic equipment |
CN108830901B (en) * | 2018-06-22 | 2020-09-25 | 维沃移动通信有限公司 | Image processing method and electronic equipment |
CN109389085A (en) * | 2018-10-09 | 2019-02-26 | 清华大学 | Lip reading identification model training method and device based on parametric curve |
CN109389085B (en) * | 2018-10-09 | 2020-09-25 | 清华大学 | Lip language recognition model training method and device based on parameterized curve |
CN110969191A (en) * | 2019-11-07 | 2020-04-07 | 吉林大学 | Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method |
CN110969191B (en) * | 2019-11-07 | 2022-10-25 | 吉林大学 | Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method |
WO2021191692A1 (en) * | 2020-03-27 | 2021-09-30 | International Business Machines Corporation | Annotation of digital images for machine learning |
US11205287B2 (en) | 2020-03-27 | 2021-12-21 | International Business Machines Corporation | Annotation of digital images for machine learning |
CN112329546A (en) * | 2020-10-15 | 2021-02-05 | 杭州电子科技大学 | Eye height measuring method based on deep learning |
CN112799510A (en) * | 2021-01-27 | 2021-05-14 | 清华大学 | Automatic generation method and device for multi-style natural eyeball movement |
CN112799510B (en) * | 2021-01-27 | 2022-09-09 | 清华大学 | Automatic generation method and device for multi-style natural eyeball movement |
Also Published As
Publication number | Publication date |
---|---|
CN107977605B (en) | 2020-04-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107977605A (en) | Ocular Boundary characteristic extraction method and device based on deep learning | |
CN110378985B (en) | Animation drawing auxiliary creation method based on GAN | |
CN112102283B (en) | Retina fundus blood vessel segmentation method based on depth multi-scale attention convolution neural network | |
CN106920227B (en) | The Segmentation Method of Retinal Blood Vessels combined based on deep learning with conventional method | |
CN112132817B (en) | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism | |
CN109919830B (en) | Method for restoring image with reference eye based on aesthetic evaluation | |
CN109829855A (en) | A kind of super resolution ratio reconstruction method based on fusion multi-level features figure | |
CN110197493A (en) | Eye fundus image blood vessel segmentation method | |
CN109671018A (en) | A kind of image conversion method and system based on production confrontation network and ResNets technology | |
CN109345538A (en) | A kind of Segmentation Method of Retinal Blood Vessels based on convolutional neural networks | |
CN107437092A (en) | The sorting algorithm of retina OCT image based on Three dimensional convolution neutral net | |
CN106408562A (en) | Fundus image retinal vessel segmentation method and system based on deep learning | |
CN109902558A (en) | A kind of human health deep learning prediction technique based on CNN-LSTM | |
CN111931908B (en) | Face image automatic generation method based on face contour | |
CN109544457A (en) | Image super-resolution method, storage medium and terminal based on fine and close link neural network | |
CN110473142A (en) | Single image super resolution ratio reconstruction method based on deep learning | |
CN108921169B (en) | A kind of eye fundus image blood vessel segmentation method | |
CN113724354B (en) | Gray image coloring method based on reference picture color style | |
CN112633288B (en) | Face sketch generation method based on painting brush touch guidance | |
CN110334719A (en) | The method and system of object image are built in a kind of extraction remote sensing image | |
CN113486944A (en) | Face fusion method, device, equipment and storage medium | |
CN110298898A (en) | Change the method and its algorithm structure of automobile image body color | |
CN114581434A (en) | Pathological image processing method based on deep learning segmentation model and electronic equipment | |
CN113763406A (en) | Infant brain MRI segmentation method based on semi-supervised learning | |
WO2021223042A1 (en) | Method for implementing machine intelligence similar to human intelligence |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |