CN110472521A - A kind of Pupil diameter calibration method and system - Google Patents
A kind of Pupil diameter calibration method and system Download PDFInfo
- Publication number
- CN110472521A CN110472521A CN201910674821.6A CN201910674821A CN110472521A CN 110472521 A CN110472521 A CN 110472521A CN 201910674821 A CN201910674821 A CN 201910674821A CN 110472521 A CN110472521 A CN 110472521A
- Authority
- CN
- China
- Prior art keywords
- pupil
- image
- circle
- point
- follows
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000001747 pupil Anatomy 0.000 title claims abstract description 216
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000001914 filtration Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 19
- 230000001629 suppression Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 3
- 229910052705 radium Inorganic materials 0.000 claims description 3
- HCWPIIXVSYCSAN-UHFFFAOYSA-N radium atom Chemical compound [Ra] HCWPIIXVSYCSAN-UHFFFAOYSA-N 0.000 claims description 3
- 230000004304 visual acuity Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims 2
- 241000208340 Araliaceae Species 0.000 claims 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims 1
- 235000003140 Panax quinquefolius Nutrition 0.000 claims 1
- 235000008434 ginseng Nutrition 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 7
- 238000002203 pretreatment Methods 0.000 abstract description 4
- 238000010606 normalization Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 6
- 230000007423 decrease Effects 0.000 description 2
- 230000001771 impaired effect Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/197—Matching; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of Pupil diameter calibration method and systems, the pre-treatment steps such as shelter removal, normalization, gray scale linear interpolation by the image for acquisition pupil, the image made has the effect of low-pass filtering, the influence that pupil region in image zooms in or out is eliminated to which the pupil region of centering carries out positioning calibration, effectively improves location efficiency and positioning accuracy；It is disturbed the part at left and right sides of pupil outer boundary less, borderline quality is higher, eliminates the influence that pupil region in image zooms in or out, and can effectively eliminate the high-frequency noise of image, quickly accurate positioning pupil edge, improves the efficiency and precision of Pupil diameter.
Description
Technical field
This disclosure relates to Pupil diameter, technical field of image processing, and in particular to a kind of Pupil diameter calibration method and be
System.
Background technique
In the case where external environment is more complicated, detecting human eye and positioning pupil position is a popular research class
Topic, realizing from classifier haarcascade for computer vision library OpenCV can be used in positioning pupil, in actual application
In, the shelter bring as caused by the objective factors such as image light, shooting angle, shelter block, glasses block is interfered
So that Pupil diameter matching technique becomes difficult.Secondly, video monitoring image background is complicated, and human face posture multiplicity, Pupil diameter
Key point be difficult to determine, further increase the difficulty of Pupil diameter technology, cannot make us full mostly so as to cause precision
Meaning.
In the prior art, Chinese Patent Application No.: CN201610810660.5 discloses a kind of pupil center's point location
Method obtains the maximum rectangular area comprising face from the gray image；By preset proportional region from the rectangle of face
Region inside casing selects ocular；Pupil center's point is calculated by the method for gradient intersection point calculation, is carried out using facial-feature analysis
Pupil diameter realizes and carries out Pupil diameter to the image of low resolution or video in the case where no ancillary equipment is intervened.
Chinese Patent Application No. CN201710146844.0 discloses a kind of method and device of Pupil diameter, obtains and eye template
With the highest area video image of degree；The profile that binaryzation extracts video image is carried out to area video image, profile is divided
For multiple segments, ellipse fitting is carried out to reference points multiple in the segment of selection and calculates the oval matching degree with profile, matching
It spends highest elliptical centre coordinate and is the position coordinates of pupil, and then have found the position of pupil, improve Pupil diameter
Efficiency and precision；Both methods improves the efficiency and precision of Pupil diameter, but is having shelter, image background multiple
Miscellaneous, when human face posture multiplicity, location efficiency, effect and the precision positioned is completely unsatisfactory.
Summary of the invention
To solve the above problems, the disclosure provides the technical solution of a kind of Pupil diameter calibration method and system, pass through needle
Pre-treatment steps, the images made such as shelter removal, normalization, gray scale linear interpolation to the image of acquisition pupil have
The effect of low-pass filtering eliminates the influence that pupil region in image zooms in or out to which the pupil region of centering is positioned
Calibration, effectively improves location efficiency and positioning accuracy.
To achieve the goals above, according to the one side of the disclosure, a kind of Pupil diameter calibration method, the method are provided
The following steps are included:
S100, the image for acquiring pupil obtain the first image；
S200 obtains the second image to the first image preprocessing；
S300 carries out positioning calibration to the pupil region in the second image.
Further, in S100, the method that acquires the image of pupil are as follows: can be adopted by images such as camera, cameras
Collection equipment to human eye portion carry out shooting, collecting obtain include pupil image.
Further, in s 200, the method that the second image is obtained to the first image preprocessing are as follows:
The problems such as shelter, image background are complicated, and human face posture is various affects the positioning of pupil, if without place
Reason will cause Pupil diameter recognition accuracy decline.
S210: shelter removal:
Shelter is lower compared to the other parts gray value of pupil region, and shelter is more and disperses, spy in irregular shape
Point, therefore from judging that the angle of gray value sets about removing.
If the threshold value for separating shelter is T, in the pupil region there may be shelter (or entire first
In image), if the gray value of the pixel of pixel is lower than T in the first image, judge that the pixel is shelter pixel,
Marking the pixel is noise spot, and the gray value of the pixel is updated to 255 (rgb value is white, i.e. rgb value
255):
Wherein, I (x, y) is the gray value for the pixel that coordinate is (x, y), and N (x, y) is the gray scale of updated pixel
Value；T=200；
S220: the inside and outside boundary of pupil region in the first image is obtained:
If P is the pupil center of circle, coordinate is (xp,yp)；I is the center of circle of pupil outer boundary, and coordinate is (xi,yi)；
Distance between the inside and outside center of circle are as follows:
The angle of circle center line connecting and horizontal direction are as follows:
The radius of pupil outer boundary are as follows: Ri, radius is R when being indicated with pupil center, then have:
Ri=(R cos θ+Δ r cos φ)2+(R sinθ+Δr sinφ)2； (4)
Then pupil radium R are as follows:
Wherein, for the first image, circle center line connecting distance, delta r and itself and two boundary of included angle and pupil of horizontal direction
Radius is constant, the expression-form of pupil outer boundary is indicated with pupil center, the inside and outside boundary of pupil can be in pupil
The heart is indicated for pole；
By any point R in pupil region in the first imagepLine of the position of (θ, α) as the inside and outside boundary point position of pupil
Property combination, it is as follows:
Rp(θ, α)=(1- α) r (θ)+α R (θ)； (6)
Wherein r (θ) is pupil inner boundary radius, and R (θ) is pupil outer boundary radius, and α ∈ [0,1] is weighting coefficient, with this
By entire pupil region with polar coordinate representation；Wherein, the identical point R of all weighting coefficient αp(θ) corresponds to same on circumference
A circumference, the identical point R of all angle, θsp(α) corresponds to the same radial upper point, and α=1 corresponds to pupil outer boundary, α
=0 corresponds to pupil boundary.
S230: pupil region is normalized to the identical rectangular area of dimension:
It is sampled in polar form pupil region with identical angular resolution and radial resolving power, and by pupil
Regional implementation is at the identical rectangular area of dimension；In the disclosure, the column of rectangular area indicate that angular samples, row indicate radial and adopt
Sample；
It is N in the hits of radial direction, then pupil region if pupil region is M in the hits of angle direction
It is converted to the rectangular area of N × M, specific steps are as follows:
S231: the coordinate parameters (x on the inside and outside boundary of pupil is obtained in the first imagep,yp,rp), (xi,yi,Ri)
S232: the center of circle of pupil outer circle and the line distance in the pupil center of circle and its angle with horizontal direction are calculated:
S233: all angle directions all phases when by the inner boundary radius of the pupil polar coordinate representation that the pupil center of circle is pole
Together, i.e. r (θ)=rp；When the pupil outer boundary polar coordinate representation that the pupil center of circle is pole, outer boundary point to pupil center away from
Change, specific formula from according to angle are as follows:
Wherein: j=1,2 ..., M；
S240: the first image progress gray-level interpolation is obtained into the second image:
When the polar coordinates point of the first image is mapped as rectangular co-ordinate point, if rectangular co-ordinate point (x, y) is not integer,
The gray value of image can not be then acquired, so to carry out gray-level interpolation processing；
The gray value of output pixel point is the average value of 2 × 2 neighborhood sampled points in image, utilizes four phases around sampled point
The gray value of adjacent pixel makees linear interpolation in both the horizontal and vertical directions；Enabling f (x, y) is the function of two variables, and its
Around sampled point four adjacent pixels formed square vertices value oneself know；It can be obtained by following bilinear interpolation equation
The functional value at any point in square:
F (x, y)=ax+by+cxy+d, in formula, a, b, tetra- parameters of c, d are determined by the functional value on known four vertex；
First image is carried out to the specific steps of gray-level interpolation are as follows:
S241: interpolation is carried out to two vertex of upper end first, is had:
F (x, 0)=f (0,0)+x (f (1,0)-f (0,0))； (9)
S242: interpolation is carried out to the bottom end vertex Liang Ge, is had:
F (x, 1)=f (0,1)+x (f (1,1)-f (0,1))； (10)
S243: in vertical direction:
F (x, y)=f (x, 0)+x (f (x, 1)-f (x, 0))； (11)
S244: it can be obtained according to the formula integration in S241 to S243:
By pre-treatment step, the second image made has the effect of low-pass filtering, eliminates pupil in the first image
The influence that bore region zooms in or out, but keep high fdrequency component impaired, it is thus possible to the profile that will lead to pupil has lesser degree
It is fuzzy.
Further, in S300, the method for carrying out positioning calibration to the pupil region in the second image is following steps:
S310 is neither destroyed the edge of pupil in the second image, also can by smooth second image of gauss low frequency filter
Eliminate the high-frequency noise of the second image；
S320 carries out smoothly, if two-dimensional Gaussian function the second image with the first derivative of two-dimensional Gaussian function are as follows:
The gradient vector of two-dimensional Gaussian function are as follows:
2 filtering convolution masks of two-dimensional Gaussian function G are decomposed into 2 one-dimensional ranks filters:
Wherein, k is constant, and σ is the parameter of Gaussian filter, indicates the smoothness of image；
S330 calculates amplitude and the direction of gradient:
Using 2 × 2 neighborhoods calculate the second image horizontal direction on gradient, to avoid detect in eyes except pupil it
Outer other parts；Wherein, 2 array P of x and y Directional partial derivativex[i, j] and Py[i, j] is respectively as follows:
X Directional partial derivative:
Px[i, j]=(I [i, j+1]-I [i, j]+I [i+1, j+1]-I [i+1, j])/2； (17)
Y Directional partial derivative:
Py[i, j]=0； (18)
With the amplitude of second order norm calculation gradient and direction, it is respectively as follows:
Amplitude is
Direction is θ [i, j]=arctan (Py[i,j]/Px[i,j])； (20)
S340 carries out non-maxima suppression to gradient magnitude:
It in order to accurately position pupil edge, then needs to calibrate gradient magnitude image M [i, j], retains amplitude part
Change maximum point, this process is exactly non-maxima suppression；
During non-maxima suppression, select to compare neighborhood within 1.5 pixel coverage of radius 3 × 3 sizes, include
The neighborhood in 8 directions carries out the interpolation of gradient magnitude to all pixels of gradient magnitude array M [i, j] along gradient direction, every
On one point, the center pixel m [i, j] of neighborhood is compared with the interpolation result of 2 gradient magnitudes along gradient direction, F [i,
J] it is fan-shaped region at neighborhood of pixels center along gradient direction, non-maxima suppression carries out in this region, if in neighborhood
The amplitude m [i, j] of heart point is big unlike 2 interpolation results on gradient direction, then by the corresponding pupil edge flag bit of m [i, j]
It is assigned a value of 0, obtains non-maxima suppression N [i, j]；
Relative to traditional non-maxima suppression, the pupil edge image that this programme obtains is relatively preferable, and pupil edge is big
In be equal to a pixel, be less than or equal to two pixels, than selecting error caused by contiguous range of the radius for 1 small, and with selection
The neighborhood pupil edge figure of radius 2 is compared, and pupil edge has obtained preferable refinement；
S350 detects by dual-threshold voltage and connects pupil edge；
S360 carries out pupil loop truss by Hough transform；
By the analytic curve f of Hough transform the second image of detection, (x, a)=0, wherein x is the point on analytic curve, a
For the point on parameter space；For pupil circle, if its radius is r, the center of circle of pupil is (a, b), then equation of a circle:
(xi-a)2+(yi-b)2=r2； (21)
The circle of pupil corresponds to a point in parameter space (a, b, r) in the image space of second image, each given
Point (xi,yi) constrain cluster Circle Parameters (a, b, r) by the point；
S370 calibrates the Pupil diameter in the second image:
The parametric equation of pupil circle is rewritten are as follows:
A=x-r cos θ, b=y-r sin θ； (22)
Pupil edge point in image space is substituted into above formula one by one, finds out parameter (a, b) value；If (a, b) is located at figure
As intermediate region 1.5 pixel coverage of radius in, then otherwise plus 0 the element in the battle array H (a, b) that adds up accordingly is added 1,；Find out H
It is r that the maximum value of element in (a, b), which is respective radius, and the center of circle is (a, b), and the most circle of edge point, as pupil on circumference
Border circle.
The present invention also provides a kind of Pupil diameter calibration system, the system comprises: memory, processor and storage
In the memory and the computer program that can run on the processor, the processor execute the computer program
It operates in the unit of following system:
Image acquisition units, the image for acquiring pupil obtain the first image；
Pretreatment unit, for obtaining the second image to the first image preprocessing；
Calibration unit is positioned, for carrying out positioning calibration to the pupil region in the second image.
The disclosure has the beneficial effect that the present invention provides a kind of Pupil diameter calibration method and system, makes pupil outer boundary
The part of the left and right sides is disturbed less, and borderline quality is higher, eliminates the influence that pupil region in image zooms in or out,
The high-frequency noise of image can be effectively eliminated, quickly accurate positioning pupil edge, improves the efficiency and precision of Pupil diameter.
Detailed description of the invention
By the way that the embodiment in conjunction with shown by attached drawing is described in detail, above-mentioned and other features of the disclosure will
More obvious, identical reference label indicates the same or similar element in disclosure attached drawing, it should be apparent that, it is described below
Attached drawing be only some embodiments of the present disclosure, for those of ordinary skill in the art, do not making the creative labor
Under the premise of, it is also possible to obtain other drawings based on these drawings, in the accompanying drawings:
Fig. 1 show a kind of flow chart of Pupil diameter calibration method；
Fig. 2 show the positioning result of pupil；
Fig. 3 show a kind of Pupil diameter calibration system structure chart.
Specific embodiment
It is carried out below with reference to technical effect of the embodiment and attached drawing to the design of the disclosure, specific structure and generation clear
Chu, complete description, to be completely understood by the purpose, scheme and effect of the disclosure.It should be noted that the case where not conflicting
Under, the features in the embodiments and the embodiments of the present application can be combined with each other.
It is as shown in Figure 1 to be illustrated according to a kind of flow chart of Pupil diameter calibration method of the disclosure below with reference to Fig. 1
According to a kind of Pupil diameter calibration method of embodiment of the present disclosure.
The disclosure proposes a kind of Pupil diameter calibration method, specifically includes the following steps:
S100, the image for acquiring pupil obtain the first image；
S200 obtains the second image to the first image preprocessing；
S300 carries out positioning calibration to the pupil region in the second image.
Further, in S100, the method that acquires the image of pupil are as follows: can be adopted by images such as camera, cameras
Collection equipment to human eye portion carry out shooting, collecting obtain include pupil image.
Further, in s 200, the method that the second image is obtained to the first image preprocessing are as follows:
The problems such as shelter, image background are complicated, and human face posture is various affects the positioning of pupil, if without place
Reason will cause Pupil diameter recognition accuracy decline.
S210: shelter removal:
Shelter is lower compared to the other parts gray value of pupil region, and shelter is more and disperses, spy in irregular shape
Point, therefore from judging that the angle of gray value sets about removing.
If the threshold value for separating shelter is T, in the pupil region there may be shelter (or entire first
In image), if the gray value of the pixel of pixel is lower than T in the first image, judge that the pixel is shelter pixel,
Marking the pixel is noise spot, and the gray value of the pixel is updated to 255 (rgb value is white, i.e. rgb value
255):
Wherein, I (x, y) is the gray value for the pixel that coordinate is (x, y), and N (x, y) is the gray scale of updated pixel
Value；T=200；
S220: the inside and outside boundary of pupil region in the first image is obtained:
If P is the pupil center of circle, coordinate is (xp,yp)；I is the center of circle of pupil outer boundary, and coordinate is (xi,yi)；
Distance between the inside and outside center of circle are as follows:
The angle of circle center line connecting and horizontal direction are as follows:
The radius of pupil outer boundary are as follows: Ri, radius is R when being indicated with pupil center, then have:
Ri=(R cos θ+Δ r cos φ)2+(R sinθ+Δr sinφ)2； (4)
Then pupil radium R are as follows:
Wherein, for the first image, circle center line connecting distance, delta r and itself and two boundary of included angle and pupil of horizontal direction
Radius is constant, the expression-form of pupil outer boundary is indicated with pupil center, the inside and outside boundary of pupil can be in pupil
The heart is indicated for pole；
By any point R in pupil region in the first imagepLine of the position of (θ, α) as the inside and outside boundary point position of pupil
Property combination, it is as follows:
Rp(θ, α)=(1- α) r (θ)+α R (θ)； (6)
Wherein r (θ) is pupil inner boundary radius, and R (θ) is pupil outer boundary radius, and α ∈ [0,1] is weighting coefficient, with this
By entire pupil region with polar coordinate representation；Wherein, the identical point R of all weighting coefficient αp(θ) corresponds to same on circumference
A circumference, the identical point R of all angle, θsp(α) corresponds to the same radial upper point, and α=1 corresponds to pupil outer boundary, α
=0 corresponds to pupil boundary.
S230: pupil region is normalized to the identical rectangular area of dimension:
It is sampled in polar form pupil region with identical angular resolution and radial resolving power, and by pupil
Regional implementation is at the identical rectangular area of dimension；In the disclosure, the column of rectangular area indicate that angular samples, row indicate radial and adopt
Sample；
It is N in the hits of radial direction, then pupil region if pupil region is M in the hits of angle direction
It is converted to the rectangular area of N × M, specific steps are as follows:
S231: the coordinate parameters (x on the inside and outside boundary of pupil is obtained in the first imagep,yp,rp), (xi,yi,Ri)
S232: the center of circle of pupil outer circle and the line distance in the pupil center of circle and its angle with horizontal direction are calculated:
S233: all angle directions all phases when by the inner boundary radius of the pupil polar coordinate representation that the pupil center of circle is pole
Together, i.e. r (θ)=rp；When the pupil outer boundary polar coordinate representation that the pupil center of circle is pole, outer boundary point to pupil center away from
Change, specific formula from according to angle are as follows:
Wherein: j=1,2 ..., M；
S240: the first image progress gray-level interpolation is obtained into the second image:
When the polar coordinates point of the first image is mapped as rectangular co-ordinate point, if rectangular co-ordinate point (x, y) is not integer,
The gray value of image can not be then acquired, so to carry out gray-level interpolation processing；
The gray value of output pixel point is the average value of 2 × 2 neighborhood sampled points in image, utilizes four phases around sampled point
The gray value of adjacent pixel makees linear interpolation in both the horizontal and vertical directions；Enabling f (x, y) is the function of two variables, and its
Around sampled point four adjacent pixels formed square vertices value oneself know；It can be obtained by following bilinear interpolation equation
The functional value at any point in square:
F (x, y)=ax+by+cxy+d, in formula, a, b, tetra- parameters of c, d are determined by the functional value on known four vertex；
First image is carried out to the specific steps of gray-level interpolation are as follows:
S241: interpolation is carried out to two vertex of upper end first, is had:
F (x, 0)=f (0,0)+x (f (1,0)-f (0,0))； (9)
S242: interpolation is carried out to the bottom end vertex Liang Ge, is had:
F (x, 1)=f (0,1)+x (f (1,1)-f (0,1))； (10)
S243: in vertical direction:
F (x, y)=f (x, 0)+x (f (x, 1)-f (x, 0))； (11)
S244: it can be obtained according to the formula integration in S241 to S243:
By pre-treatment step, the second image made has the effect of low-pass filtering, eliminates pupil in the first image
The influence that bore region zooms in or out, but keep high fdrequency component impaired, it is thus possible to the profile that will lead to pupil has lesser degree
It is fuzzy.
Further, in S300, the method for carrying out positioning calibration to the pupil region in the second image is following steps:
S310 is neither destroyed the edge of pupil in the second image, also can by smooth second image of gauss low frequency filter
Eliminate the high-frequency noise of the second image；
S320 carries out smoothly, if two-dimensional Gaussian function the second image with the first derivative of two-dimensional Gaussian function are as follows:
The gradient vector of two-dimensional Gaussian function are as follows:
2 filtering convolution masks of two-dimensional Gaussian function G are decomposed into 2 one-dimensional ranks filters:
Wherein, k is constant, and σ is the parameter of Gaussian filter, indicates the smoothness of image；
S330 calculates amplitude and the direction of gradient:
Using 2 × 2 neighborhoods calculate the second image horizontal direction on gradient, to avoid detect in eyes except pupil it
Outer other parts；Wherein, 2 array P of x and y Directional partial derivativex[i, j] and Py[i, j] is respectively as follows:
X Directional partial derivative:
Px[i, j]=(I [i, j+1]-I [i, j]+I [i+1, j+1]-I [i+1, j])/2； (17)
Y Directional partial derivative:
Py[i, j]=0； (18)
With the amplitude of second order norm calculation gradient and direction, it is respectively as follows:
Amplitude is
Direction is θ [i, j]=arctan (Py[i,j]/Px[i,j])； (20)
S340 carries out non-maxima suppression to gradient magnitude:
It in order to accurately position pupil edge, then needs to calibrate gradient magnitude image M [i, j], retains amplitude part
Change maximum point, this process is exactly non-maxima suppression；
During non-maxima suppression, select to compare neighborhood within 1.5 pixel coverage of radius 3 × 3 sizes, include
The neighborhood in 8 directions carries out the interpolation of gradient magnitude to all pixels of gradient magnitude array M [i, j] along gradient direction, every
On one point, the center pixel m [i, j] of neighborhood is compared with the interpolation result of 2 gradient magnitudes along gradient direction, F [i,
J] it is fan-shaped region at neighborhood of pixels center along gradient direction, non-maxima suppression carries out in this region, if in neighborhood
The amplitude m [i, j] of heart point is big unlike 2 interpolation results on gradient direction, then by the corresponding pupil edge flag bit of m [i, j]
It is assigned a value of 0, obtains non-maxima suppression N [i, j]；
Relative to traditional non-maxima suppression, the pupil edge image that this programme obtains is relatively preferable, and pupil edge is big
In be equal to a pixel, be less than or equal to two pixels, than selecting error caused by contiguous range of the radius for 1 small, and with selection
The neighborhood pupil edge figure of radius 2 is compared, and pupil edge has obtained preferable refinement；
S350 detects by dual-threshold voltage and connects pupil edge；
S360 carries out pupil loop truss by Hough transform；
By the analytic curve f of Hough transform the second image of detection, (x, a)=0, wherein x is the point on analytic curve, a
For the point on parameter space；For pupil circle, if its radius is r, the center of circle of pupil is (a, b), then equation of a circle:
(xi-a)2+(yi-b)2=r2； (21)
The circle of pupil corresponds to a point in parameter space (a, b, r) in the image space of second image, each given
Point (xi,yi) constrain cluster Circle Parameters (a, b, r) by the point；
S370 calibrates the Pupil diameter in the second image:
The parametric equation of pupil circle is rewritten are as follows:
A=x-r cos θ, b=y-r sin θ； (22)
Pupil edge point in image space is substituted into above formula one by one, finds out parameter (a, b) value；If (a, b) is located at figure
As intermediate region 1.5 pixel coverage of radius in, then otherwise plus 0 the element in the battle array H (a, b) that adds up accordingly is added 1,；Find out H
It is r that the maximum value of element in (a, b), which is respective radius, and the center of circle is (a, b), and the most circle of edge point, as pupil on circumference
Border circle.
A kind of embodiment of the disclosure:
(1) the pupil picture of the preparation of sample positive sample selection different people first, negative sample select non-pupil picture.Sample
Number is 1000, positive sample 700, and negative sample 300 is placed on positive negative sample under different files respectively, while training letter
Several and positive negative sample is put into together under a file；
(2) a kind of Pupil diameter calibration method of the disclosure such as is carried out to positive negative sample to handle；
(3) after generation pattern representation file pre-processes positive negative sample, positive sample vec file is created；
(4) training classifier is inputted in cmd order line: opencv_traincascade.exe-data xml-vec
pos.vec-bg neg\neg.txt-numpos 700-numneg 300-numstages 20-featureType LBP-
w24-h24；
In order to reduce the data volume of processing, when positioning pupil inner and outer boundary, pupil image is scaled first, and
And the circle of original image is converted into after scaling positioning again, original image size is 80 × 80 pixels.It is fixed since inner circle is relatively small
The outer bowlder scaling 0.3 in position, positioning interior bowlder scaling is 0.5；The result of positioning is as shown in Fig. 2, Fig. 2 show pupil
Positioning result.
A kind of Pupil diameter calibration system that embodiment of the disclosure provides, is illustrated in figure 3 a kind of pupil of the disclosure
A kind of Pupil diameter calibration system of locating calibration system structure chart, the embodiment includes: processor, memory and is stored in
In the memory and the computer program that can run on the processor, when the processor executes the computer program
Realize the step in a kind of above-mentioned Pupil diameter calibration system embodiment.
It can be transported in the memory and on the processor the system comprises: memory, processor and storage
Capable computer program, the processor execute the computer program and operate in the unit of following system:
Image acquisition units, the image for acquiring pupil obtain the first image；
Pretreatment unit, for obtaining the second image to the first image preprocessing；
Calibration unit is positioned, for carrying out positioning calibration to the pupil region in the second image.
A kind of Pupil diameter calibration system can run on desktop PC, notebook, palm PC and cloud
Server etc. calculates in equipment.A kind of Pupil diameter calibration system, the system that can be run may include, but be not limited only to, place
Manage device, memory.It will be understood by those skilled in the art that the example is only a kind of example of Pupil diameter calibration system,
The restriction to a kind of Pupil diameter calibration system is not constituted, may include component more more or fewer than example, or combination
Certain components or different components, such as a kind of Pupil diameter calibration system can also include input-output equipment, net
Network access device, bus etc..
Alleged processor can be central processing unit (Central Processing Unit, CPU), can also be it
His general processor, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng the processor is a kind of control centre of Pupil diameter calibration system operating system, utilizes various interfaces and route
A kind of entire Pupil diameter calibration system of connection can operating system various pieces.
The memory can be used for storing the computer program and/or module, and the processor is by operation or executes
Computer program in the memory and/or module are stored, and calls the data being stored in memory, described in realization
A kind of various functions of Pupil diameter calibration system.The memory can mainly include storing program area and storage data area,
In, storing program area can application program needed for storage program area, at least one function (such as sound-playing function, image
Playing function etc.) etc.；Storage data area, which can be stored, uses created data (such as audio data, phone directory according to mobile phone
Deng) etc..In addition, memory may include high-speed random access memory, it can also include nonvolatile memory, such as firmly
Disk, memory, plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital,
SD) block, flash card (Flash Card), at least one disk memory, flush memory device or other volatile solid-states
Part.
Although the description of the disclosure is quite detailed and especially several embodiments are described, it is not
Any of these details or embodiment or any specific embodiments are intended to be limited to, but should be considered as is by reference to appended
A possibility that claim provides broad sense in view of the prior art for these claims explanation, to effectively cover the disclosure
Preset range.In addition, the disclosure is described with inventor's foreseeable embodiment above, its purpose is to be provided with
Description, and those equivalent modifications that the disclosure can be still represented to the unsubstantiality change of the disclosure still unforeseen at present.
Claims (7)
1. a kind of Pupil diameter calibration method, which is characterized in that the described method comprises the following steps:
S100, the image for acquiring pupil obtain the first image；
S200 obtains the second image to the first image preprocessing；
S300 carries out positioning calibration to the pupil region in the second image.
2. a kind of Pupil diameter calibration method according to claim 1, which is characterized in that in S100, acquire pupil
The method of image are as follows: can by camera, camera to human eye portion carry out shooting, collecting obtain include pupil image.
3. a kind of Pupil diameter calibration method according to claim 2, which is characterized in that in s 200, described to first
The method that image preprocessing obtains the second image are as follows:
S210: shelter removal:
If the threshold value for separating shelter is T, in the pupil region there may be shelter, if pixel in the first image
The gray value of the pixel of point is lower than T, then judges that the pixel is shelter pixel, and marking the pixel is noise spot, and
The gray value of the pixel is updated to 255:
Wherein, I (x, y) is the gray value for the pixel that coordinate is (x, y), and N (x, y) is the gray value of updated pixel；T
=200；
S220: the inside and outside boundary of pupil region in the first image is obtained:
If P is the pupil center of circle, coordinate is (xp,yp)；I is the center of circle of pupil outer boundary, and coordinate is (xi,yi)；
Distance between the inside and outside center of circle are as follows:
The angle of circle center line connecting and horizontal direction are as follows:
The radius of pupil outer boundary are as follows: Ri, radius is R when being indicated with pupil center, then have:
Ri=(Rcos θ+Δ rcos φ)2+(Rsinθ+Δrsinφ)2； (4)
Then pupil radium R are as follows:
Wherein, for the first image, the radius of circle center line connecting distance, delta r and itself and two boundary of included angle and pupil of horizontal direction
It is constant；
By any point R in pupil region in the first imagepLinear group as the inside and outside boundary point position of pupil of the position of (θ, α)
It closes, as follows:
Rp(θ, α)=(1- α) r (θ)+α R (θ)； (6)
Wherein r (θ) is pupil inner boundary radius, and R (θ) is pupil outer boundary radius, and α ∈ [0,1] is weighting coefficient, will be whole with this
A pupil region is with polar coordinate representation；Wherein, the identical point R of all weighting coefficient αp(θ) corresponds to the same circle on circumference
Week, the identical point R of all angle, θsp(α) corresponds to the same radial upper point, and α=1 corresponds to pupil outer boundary, α=0 pair
It should be in pupil boundary；
S230: pupil region is normalized to the identical rectangular area of dimension；
S240: the first image progress gray-level interpolation is obtained into the second image.
4. a kind of Pupil diameter calibration method according to claim 3, which is characterized in that in S230, by pupil region
The method for being normalized to the identical rectangular area of dimension are as follows:
It is sampled in polar form pupil region with identical angular resolution and radial resolving power, and by pupil region
It is launched into the identical rectangular area of dimension；The column of rectangular area indicate that angular samples, row indicate radial sampling；
It is N in the hits of radial direction, then pupil region is converted if pupil region is M in the hits of angle direction
At the rectangular area of N × M, specific steps are as follows:
S231: the coordinate parameters (x on the inside and outside boundary of pupil is obtained in the first imagep,yp,rp), (xi,yi,Ri)
S232: the center of circle of pupil outer circle and the line distance in the pupil center of circle and its angle with horizontal direction are calculated:
S233: all angle directions when the inner boundary radius of the pupil polar coordinate representation that the pupil center of circle is pole are all identical,
That is r (θ)=rp；When the pupil outer boundary polar coordinate representation that the pupil center of circle is pole, the distance of outer boundary point to pupil center
Change, specific formula according to angle are as follows:
Wherein: j=1,2 ..., M.
5. a kind of Pupil diameter calibration method according to claim 3, which is characterized in that in S240, by the first image
Carry out the method that gray-level interpolation obtains the second image are as follows:
The gray value of output pixel point is the average value of 2 × 2 neighborhood sampled points in image, utilizes four adjacent pictures around sampled point
The gray value of element makees linear interpolation in both the horizontal and vertical directions；Enabling f (x, y) is the function of two variables, and it is being adopted
Around sampling point four adjacent pixels formed square vertices value oneself know；It is available just by following bilinear interpolation equation
The functional value at rectangular interior any point:
F (x, y)=ax+by+cxy+d, in formula, a, b, tetra- parameters of c, d are determined by the functional value on known four vertex；
First image is carried out to the specific steps of gray-level interpolation are as follows:
S241: interpolation, formula are carried out to two vertex of upper end first are as follows:
F (x, 0)=f (0,0)+x (f (1,0)-f (0,0))； (9)
S242: interpolation, formula are carried out to the bottom end vertex Liang Ge are as follows:
F (x, 1)=f (0,1)+x (f (1,1)-f (0,1))； (10)
S243: in vertical direction:
F (x, y)=f (x, 0)+x (f (x, 1)-f (x, 0))； (11)
S244: it can be obtained according to the formula integration in S241 to S243:
6. a kind of Pupil diameter calibration method according to claim 3, which is characterized in that in S300, to the second image
In pupil region carry out positioning calibration method be following steps:
S310 is neither destroyed the edge of pupil in the second image, can also be eliminated by smooth second image of gauss low frequency filter
The high-frequency noise of second image；
S320 carries out smoothly, if two-dimensional Gaussian function the second image with the first derivative of two-dimensional Gaussian function are as follows:
The gradient vector of two-dimensional Gaussian function are as follows:
2 filtering convolution masks of two-dimensional Gaussian function G are decomposed into 2 one-dimensional ranks filters:
Wherein, k is constant, and σ is the parameter of Gaussian filter, indicates the smoothness of image；
S330 calculates amplitude and the direction of gradient:
The gradient in the horizontal direction of the second image is calculated, using 2 × 2 neighborhoods to avoid detecting in eyes in addition to pupil
Other parts；Wherein, 2 array P of x and y Directional partial derivativex[i, j] and Py[i, j] is respectively as follows:
X Directional partial derivative:
Px[i, j]=(I [i, j+1]-I [i, j]+I [i+1, j+1]-I [i+1, j])/2； (17)
Y Directional partial derivative:
Py[i, j]=0； (18)
With the amplitude of second order norm calculation gradient and direction, it is respectively as follows:
Amplitude is
Direction is
S340 carries out non-maxima suppression to gradient magnitude:
During non-maxima suppression, select to compare neighborhood within 1.5 pixel coverage of radius 3 × 3 sizes, comprising 8
The neighborhood in direction carries out the interpolation of gradient magnitude to all pixels of gradient magnitude array M [i, j] along gradient direction, at each
On point, the center pixel m [i, j] of neighborhood is compared with the interpolation result of 2 gradient magnitudes along gradient direction, and F [i, j] is
Along the fan-shaped region of gradient direction at neighborhood of pixels center, non-maxima suppression carries out in this region, if centre of neighbourhood point
Amplitude m [i, j] it is big unlike 2 interpolation results on gradient direction, then by the corresponding pupil edge flag bit assignment of m [i, j]
It is 0, obtains non-maxima suppression N [i, j]；
S350 detects by dual-threshold voltage and connects pupil edge；
S360 carries out pupil loop truss by Hough transform；
By the analytic curve f of Hough transform the second image of detection, (x, a)=0, wherein x is the point on analytic curve, and a is ginseng
Point on number space；For pupil circle, if its radius is r, the center of circle of pupil is (a, b), then equation of a circle:
(xi-a)2+(yi-b)2=r2 (21)
The circle of pupil corresponds to a point in parameter space (a, b, r), each set point (x in the image space of second imagei,
yi) constrain cluster Circle Parameters (a, b, r) by the point；
S370 calibrates the Pupil diameter in the second image:
The parametric equation of pupil circle is rewritten are as follows:
A=x-rcos θ, b=y-rsin θ (22)
Pupil edge point in image space is substituted into above formula one by one, finds out parameter (a, b) value；If (a, b) is located in image
Between region 1.5 pixel coverage of radius in, then otherwise plus 0 the element in the battle array H (a, b) that adds up accordingly is added 1,；Find out H (a,
B) it is r that the maximum value of element in, which is respective radius, and the center of circle is (a, b), and the most circle of edge point on circumference, as pupil side
Boundary's circle.
7. a kind of Pupil diameter calibration system, which is characterized in that the system comprises: memory, processor and it is stored in institute
The computer program that can be run in memory and on the processor is stated, the processor executes the computer program operation
In the unit of following system:
Image acquisition units, the image for acquiring pupil obtain the first image；
Pretreatment unit, for obtaining the second image to the first image preprocessing；
Calibration unit is positioned, for carrying out positioning calibration to the pupil region in the second image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910674821.6A CN110472521B (en) | 2019-07-25 | 2019-07-25 | Pupil positioning calibration method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910674821.6A CN110472521B (en) | 2019-07-25 | 2019-07-25 | Pupil positioning calibration method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110472521A true CN110472521A (en) | 2019-11-19 |
| CN110472521B CN110472521B (en) | 2022-12-20 |
Family
ID=68509735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910674821.6A Active CN110472521B (en) | 2019-07-25 | 2019-07-25 | Pupil positioning calibration method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110472521B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111178289A (en) * | 2019-12-31 | 2020-05-19 | 中山市奥珀金属制品有限公司 | Method and system for shortening iris recognition time |
| CN112070028A (en) * | 2020-09-09 | 2020-12-11 | 苏州小艺物联科技有限公司 | Animal iris positioning method and system |
| CN112434675A (en) * | 2021-01-26 | 2021-03-02 | 西南石油大学 | Pupil positioning method for global self-adaptive optimization parameters |
| CN112989878A (en) * | 2019-12-13 | 2021-06-18 | Oppo广东移动通信有限公司 | Pupil detection method and related product |
| CN113190117A (en) * | 2021-04-29 | 2021-07-30 | 南昌虚拟现实研究院股份有限公司 | Pupil and light spot positioning method, data calculation method and related device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266645A (en) * | 2008-01-24 | 2008-09-17 | 电子科技大学中山学院 | Iris positioning method based on multi-resolutions analysis |
| CN102902967A (en) * | 2012-10-16 | 2013-01-30 | 第三眼（天津）生物识别科技有限公司 | Method for positioning iris and pupil based on eye structure classification |
| CN103440476A (en) * | 2013-08-26 | 2013-12-11 | 大连理工大学 | Locating method for pupil in face video |
| US20140022371A1 (en) * | 2012-07-20 | 2014-01-23 | Pixart Imaging Inc. | Pupil detection device |
| CN108053444A (en) * | 2018-01-02 | 2018-05-18 | 京东方科技集团股份有限公司 | Pupil positioning method and device, equipment and storage medium |
| CN109840449A (en) * | 2017-11-27 | 2019-06-04 | 上海聚虹光电科技有限公司 | It eliminates the pupil that environment shadow is rung and scales emotion judgment method |
-
2019
- 2019-07-25 CN CN201910674821.6A patent/CN110472521B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266645A (en) * | 2008-01-24 | 2008-09-17 | 电子科技大学中山学院 | Iris positioning method based on multi-resolutions analysis |
| US20140022371A1 (en) * | 2012-07-20 | 2014-01-23 | Pixart Imaging Inc. | Pupil detection device |
| CN102902967A (en) * | 2012-10-16 | 2013-01-30 | 第三眼（天津）生物识别科技有限公司 | Method for positioning iris and pupil based on eye structure classification |
| CN103440476A (en) * | 2013-08-26 | 2013-12-11 | 大连理工大学 | Locating method for pupil in face video |
| CN109840449A (en) * | 2017-11-27 | 2019-06-04 | 上海聚虹光电科技有限公司 | It eliminates the pupil that environment shadow is rung and scales emotion judgment method |
| CN108053444A (en) * | 2018-01-02 | 2018-05-18 | 京东方科技集团股份有限公司 | Pupil positioning method and device, equipment and storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112989878A (en) * | 2019-12-13 | 2021-06-18 | Oppo广东移动通信有限公司 | Pupil detection method and related product |
| CN111178289A (en) * | 2019-12-31 | 2020-05-19 | 中山市奥珀金属制品有限公司 | Method and system for shortening iris recognition time |
| CN111178289B (en) * | 2019-12-31 | 2023-08-25 | 张杰辉 | Method and system for shortening iris recognition time consumption |
| CN112070028A (en) * | 2020-09-09 | 2020-12-11 | 苏州小艺物联科技有限公司 | Animal iris positioning method and system |
| CN112070028B (en) * | 2020-09-09 | 2024-04-09 | 苏州小艺物联科技有限公司 | Animal iris positioning method and system |
| CN112434675A (en) * | 2021-01-26 | 2021-03-02 | 西南石油大学 | Pupil positioning method for global self-adaptive optimization parameters |
| CN113190117A (en) * | 2021-04-29 | 2021-07-30 | 南昌虚拟现实研究院股份有限公司 | Pupil and light spot positioning method, data calculation method and related device |
| CN113190117B (en) * | 2021-04-29 | 2023-02-03 | 南昌虚拟现实研究院股份有限公司 | Pupil and light spot positioning method, data calculation method and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110472521B (en) | 2022-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110472521A (en) | A kind of Pupil diameter calibration method and system | |
| CN108230252B (en) | Image processing method and device and electronic equipment | |
| CN111080529A (en) | Unmanned aerial vehicle aerial image splicing method for enhancing robustness | |
| CN107292242B (en) | Iris identification method and terminal | |
| CN103914676B (en) | A kind of method and apparatus used in recognition of face | |
| CN105260698B (en) | The method and apparatus that iris image is positioned | |
| CN105678806B (en) | A kind of live pig action trail automatic tracking method differentiated based on Fisher | |
| CN107945111B (en) | Image stitching method based on SURF (speeded up robust features) feature extraction and CS-LBP (local binary Pattern) descriptor | |
| CN107292922B (en) | A method of it is registrated for optics with diameter radar image | |
| EP2079054B1 (en) | Detection of blobs in images | |
| US20120082385A1 (en) | Edge based template matching | |
| CN108369473A (en) | Influence the method for the virtual objects of augmented reality | |
| CN107016646A (en) | One kind approaches projective transformation image split-joint method based on improved | |
| CN102713938A (en) | Scale space normalization technique for improved feature detection in uniform and non-uniform illumination changes | |
| Houshiar et al. | A study of projections for key point based registration of panoramic terrestrial 3D laser scan | |
| CN108010082B (en) | Geometric matching method | |
| CN105404868B (en) | The rapid detection method of text in a kind of complex background based on interaction platform | |
| CN108875504B (en) | Image detection method and image detection device based on neural network | |
| CN108550165A (en) | A kind of image matching method based on local invariant feature | |
| CN113780201B (en) | Hand image processing method and device, equipment and medium | |
| CN109242787A (en) | It paints in a kind of assessment of middle and primary schools' art input method | |
| CN114022554B (en) | Massage robot acupoint detection and positioning method based on YOLO | |
| CN107025449A (en) | A kind of inclination image linear feature matching process of unchanged view angle regional area constraint | |
| CN106296632B (en) | A kind of well-marked target detection method based on amplitude spectrum analysis | |
| CN112183517A (en) | Certificate card edge detection method, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: No.45 Zhaoyi Road, Dongsheng Town, Zhongshan City, Guangdong Province 528415 Applicant after: Guangdong AOPO smart home Co.,Ltd. Address before: No.45 Zhaoyi Road, Dongsheng Town, Zhongshan City, Guangdong Province 528415 Applicant before: ZHONGSHAN AOPO METAL MANUFACTORY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221123 Address after: No. 18, Tai'an West Lane, Yuelong Middle Road, Xiaolan Town, Zhongshan, Guangdong 528415 Applicant after: Zhang Jiehui Address before: No.45 Zhaoyi Road, Dongsheng Town, Zhongshan City, Guangdong Province 528415 Applicant before: Guangdong AOPO smart home Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |