CN103176607A - Eye-controlled mouse realization method and system - Google Patents
Eye-controlled mouse realization method and system Download PDFInfo
- Publication number
- CN103176607A CN103176607A CN2013101303929A CN201310130392A CN103176607A CN 103176607 A CN103176607 A CN 103176607A CN 2013101303929 A CN2013101303929 A CN 2013101303929A CN 201310130392 A CN201310130392 A CN 201310130392A CN 103176607 A CN103176607 A CN 103176607A
- Authority
- CN
- China
- Prior art keywords
- eyes image
- eye
- screen
- pupil
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention provides an eye-controlled mouse realization method and system. The method comprises the following steps: obtaining an eye image; detecting and positioning a pupil position in the eye image; establishing a mapping relation between an eye gazing direction and a screen; monitoring the eye gazing direction in real time, determining a fixation point of the eye sight on the screen according to the mapping relation between the eye gazing direction and the screen, determining the working mode of the mouse, displaying the mouse at a fixation point of the eye sight on the screen and working according to the working mode. The movement of the mouse can be accurately controlled by using the eye gazing direction, and the function of controlling the mouse by eye is realized.
Description
Technical field
The present invention relates to the Visual Tracking field, be specifically related to a kind of controlled mouse implementation method and system.
Background technology
The man-computer relation of exploring natural harmony has become a key areas of computer research, and natural, efficient, intelligentized human-computer interaction interface is the important trend of computer nowadays development.But, for physical disabilities, if the trick campaign is inconvenient, utilize conventional mouse to realize that man-machine interaction is just very difficult.
In field of human-computer interaction, eyes are as important information interaction passage, and sight line reaction people's attention direction, thereby line-of-sight applications had characteristics such as its naturality, substantivity and interactivity in field of human-computer interaction, enjoy people's concern, therefore how to realize utilizing the eye sight line direction to control and realize that the eye controlled mouse of mouse function is the technical matters of needing solution badly.
Summary of the invention
In order to overcome the defective that exists in above-mentioned prior art, the purpose of this invention is to provide a kind of controlled mouse implementation method and system, can utilize the eye sight line direction to control and realize mouse function.
In order to realize above-mentioned purpose of the present invention, according to an aspect of the present invention, the invention provides a kind of controlled mouse implementation method, comprise the steps:
S1: obtain eyes image;
S2: the pupil position in described eyes image is detected and locates;
S3: the mapping relations of setting up described eye direction of visual lines and screen;
S4: Real Time Monitoring eye direction of visual lines, according to the mapping relations of described eye direction of visual lines and screen, determine the blinkpunkt of described eye sight line on described screen;
S5: determine the mode of operation of mouse, described mode of operation comprises the mouse Move Mode, clicks pattern and double-click pattern;
S6: the blinkpunkt place of eye sight line shows mouse and carries out work according to described mode of operation on described screen.
Eye controlled mouse implementation method of the present invention is by setting up the mapping relations of eye direction of visual lines and screen, then determine pupil center, mapping relations according to eye direction of visual lines and screen, determine the blinkpunkt of eye sight line on described screen, the blinkpunkt place of eye sight line shows mouse and carries out work according to default mode of operation on screen, realizes rapidly and accurately the function of a controlled mouse.
In a preferred embodiment of the present invention, in described step S2, the localization method of pupil is:
S21: described eyes image is carried out pre-service, increase the overall brightness of described eyes image, and the noise of the described eyes image of filtering;
S22: the maximum between-cluster variance of all pixels of the described eyes image of statistics;
S23: according to described maximum between-cluster variance, calculate the optimal threshold of described eyes image binaryzation, realize the eyes image binary conversion treatment;
S24: extract pupil edge information;
S25: determine pupil center and radius.
Pupil positioning method of the present invention can position pupil rapidly and accurately, even in the situation that eye image is fuzzy, also the center of pupil can be located out fast.
In another kind of preferred embodiment of the present invention, in described eyes image was carried out pretreated step, the method that increases the overall brightness of described eyes image was:
S211: described eyes image is converted into corresponding histogram;
S212: the gray-scale value that goes out all pixels in described eyes image according to described statistics with histogram;
S213: determine the benchmark of grey scale pixel value, the gray-scale value of the pixel of described eyes image is carried out light compensation according to described benchmark.
The present invention is by increasing the overall brightness of eyes image, also can carry out rapidly and accurately the through hole location for some vague image vegetarian refreshments or blurred picture, improved accuracy.
In another preferred embodiment of the present invention, the method for the benchmark of described definite grey scale pixel value is: select the mean value of the gray-scale value of the pixel of 5%-10% in all pixels as benchmark.
The present invention selects the mean value of gray-scale value of the partial pixel point in all pixels as benchmark, and the ratio of choosing can be adjusted according to actual needs, has improved the dirigibility of method.
In a preferred embodiment of the present invention, in described eyes image is carried out pretreated step, adopt the noise of the described eyes image of median filter filtering of two dimension.
The present invention adopts the preprocess method of medium filtering and light compensation, has ability fuzzy, partially dark, promoted brightness and filtering noise by the eyes image of noise, has improved the accuracy of pupil location.
In another kind of preferred embodiment of the present invention, in described step S22, the method for the maximum between-cluster variance of all pixels of the described eyes image of statistics is:
S221: the histogram that obtains described eyes image;
S222: determine that in described histogram in described eyes image, number of pixels is zero grey scale pixel value;
S223: determine the non-vanishing grey scale pixel value of number of pixels in described eyes image, calculate the maximum between-cluster variance of the non-vanishing grey scale pixel value of described number of pixels.
Maximum between-cluster variance computing method of the present invention only need the maximum between-cluster variance of the non-vanishing grey scale pixel value of calculating pixel number, have improved computing velocity.
In a preferred embodiment of the present invention, choose the segmentation threshold of prospect and the background of described eyes image, when described maximum between-cluster variance was maximum, described segmentation threshold was the optimal threshold of cutting apart.
The present invention utilizes follow-on maximum variance between clusters to realize that adaptive threshold extracts, and has that threshold value is suitable, a self-adaptation, fireballing advantage.
In a preferred embodiment of the present invention, the method for extracting pupil edge information is:
If the pixel value of central pixel point is 255, no matter why the pixel value of all the other 8 adjacent pixels is worth, the pixel value that keeps without exception central pixel point is 255;
If the pixel value of central pixel point is 0, and the pixel value of 8 adjacent pixels is 0, the pixel value of central pixel point changed into 255;
During all the other situations, all the pixel value with central pixel point changes 0 into.
In another kind of preferred embodiment of the present invention, utilize least square method to determine pupil center and radius.
The extracting method of pupil edge information of the present invention and based on the accurate location that the ellipse fitting method of definite pupil center of least square method and radius realizes pupil has characteristics quickly and accurately.
In a preferred embodiment of the present invention, the step of setting up the mapping relations of described eye direction of visual lines and screen is:
S31: show successively M calibration point on screen, described M is positive integer;
S32: when eyes when watching each calibration point attentively, video camera detect light that infrared light supply sends through pupil center's reflection after N corresponding point on screen, described N is positive integer;
S33: N the corresponding point that each calibration point detects are analyzed, obtained an equivalent correspondence point;
S34: the corresponding relation of setting up pupil center and calibration point.
Thereby determine the blinkpunkt of eyes on screen by detecting the eye sight line direction, realize the function of eyelet mouse.
In order to realize above-mentioned purpose of the present invention, according to two aspects of the present invention, the invention provides a kind of controlled mouse and realize system, comprising: infrared light supply, video camera, screen and control module, described control module comprises eyes image acquisition module and core processing module; Described infrared light supply and video camera are set up the mapping relations of eye direction of visual lines and screen; Described eyes image acquisition module is used for gathering eyes image; Described core processing module is connected with described eyes image acquisition module, is used for receiving described eyes image and judges the center of pupil, and realize the work of eye controlled mouse and show by screen.
Eye controlled mouse of the present invention realizes that system can utilize the motion of mouse beacon exactly of eye sight line direction, realizes the function of eye controlled mouse.
in a preferred embodiment of the present invention, described core processing module comprises pupil center's judge module, demarcating module and eye tracking module, described pupil center judge module is connected with described eyes image acquisition module, the center that is used for receiving eyes image and judges pupil, described demarcating module is used for setting up the mapping relations of described eye direction of visual lines and screen, described eye tracking module is connected with described demarcating module with the pupil center judge module respectively, described eye tracking module is determined blinkpunkt and the analog mouse realization tracking operation of eyes on screen according to the pupil center of described pupil center judge module judgement and the mapping relations of described eye direction of visual lines and screen.Thereby utilize the motion of mouse beacon exactly of eye sight line direction, realize the function of eye controlled mouse.
In a preferred embodiment of the present invention, also comprise data conversion module, described data conversion module is connected with described core processing module with described eyes image acquisition module respectively, and the eyes image after also changing for the format conversion that realizes described eyes image data transfers to described core processing module.
Data conversion module of the present invention carries out format conversion, when the eyes image data of the data acquisition of eyes image acquisition module collection and data type that core processing module can be processed not simultaneously, utilize this data conversion module that the data layout of eyes image acquisition module collection is changed, improved the compatibility of pupil positioning system.
Additional aspect of the present invention and advantage part in the following description provide, and part will become obviously from the following description, or recognize by practice of the present invention.
Description of drawings
Above-mentioned and/or additional aspect of the present invention and advantage are from obviously and easily understanding becoming the description of embodiment in conjunction with following accompanying drawing, wherein:
Fig. 1 is the process flow diagram of eye controlled mouse implementation method of the present invention;
Fig. 2 is the process flow diagram of pupil positioning method of the present invention;
Fig. 3 is the corresponding histogram of eyes image in a kind of preferred implementation of the present invention;
Fig. 4 is the eyes image figure that gathers in a kind of preferred implementation of the present invention;
Fig. 5 is the pretreating effect figure to eyes image shown in Figure 4;
Fig. 6 is the binaryzation design sketch of eyes image shown in Figure 4;
Fig. 7 is the locating effect figure of pupil center of eyes image shown in Figure 4.
Embodiment
The below describes embodiments of the invention in detail, and the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment that is described with reference to the drawings, only be used for explaining the present invention, and can not be interpreted as limitation of the present invention.
In description of the invention, unless otherwise prescribed and limit, need to prove, term " installation ", " being connected ", " connection " should be done broad understanding, for example, can be mechanical connection or electrical connection, can be also the connection of two element internals, can be directly to be connected, and also can indirectly be connected by intermediary, for the ordinary skill in the art, can understand as the case may be the concrete meaning of above-mentioned term.
The invention provides a kind of controlled mouse implementation method, as shown in Figure 1, comprise the steps:
S1: obtain eyes image;
S2: the pupil position in eyes image is detected and locates;
S3: the mapping relations of setting up eye direction of visual lines and screen;
S4: Real Time Monitoring eye direction of visual lines, according to the mapping relations of eye direction of visual lines and screen, determine the blinkpunkt of eye sight line on screen;
S5: determine the mode of operation of mouse, wherein, mode of operation comprises the mouse Move Mode, clicks pattern and double-click pattern;
S6: the blinkpunkt place of eye sight line shows mouse and carries out work according to described mode of operation on screen.
In the present embodiment, the method to the pupil position in eyes image detects and locates as shown in Figure 2, comprises the steps:
S21: described eyes image is carried out pre-service, increase the overall brightness of described eyes image, and the noise of the described eyes image of filtering;
S22: the maximum between-cluster variance of all pixels of the described eyes image of statistics;
S23: according to described maximum between-cluster variance, calculate the optimal threshold of described eyes image binaryzation, realize the eyes image binary conversion treatment;
S24: extract pupil edge information;
S25: determine pupil center and radius.
In a kind of preferred implementation of the present invention, obtain eyes image, gather the eye video image of eye rotation by the eye movement instrument, and be stored in computing machine; Then utilize step S21 to gather each frame eye video image to step S23, realize that the pre-service of image and self-adaption binaryzation process; Utilize at last step S24 pupil to be positioned to step S25, specifically extract pupil edge information and determine pupil center and radius.Pupil positioning method of the present invention can position pupil rapidly and accurately, even in the situation that eye image is fuzzy, also the center of pupil can be located out fast.
In a kind of preferred implementation of the present invention, the concrete steps of this pupil positioning method are:
The first step: obtain eyes image, in the present embodiment, utilize the video camera of eye movement instrument obtain the eyes image video and be stored in the storer of computing machine, Fig. 4 shows the eyes image of the three width different angles that gathered by the eye movement instrument.
Eyes image is carried out pre-service, increase the overall brightness of eyes image, and the noise of filtering eyes image, image after processing as shown in Figure 5, in the present embodiment, gather each frame eyes image and do the image pre-service, increase integral image brightness and filtering noise by light compensation.In eyes image was carried out pretreated step, the method that increases the overall brightness of eyes image by light compensation was:
S211: eyes image is converted into corresponding histogram, in the present embodiment, as shown in Figure 3, in conjunction with software Visual studio2010, eyes image is converted into corresponding histogram by the image vision storehouse, its horizontal ordinate is gray-scale value, and its ordinate is the corresponding number of pixels of gray-scale value.
S212: the gray-scale value that goes out all pixels in eyes image according to statistics with histogram.
S213: determine the benchmark of grey scale pixel value, the gray-scale value benchmark of the pixel of eyes image is carried out light compensation.In the present embodiment, the method of determining the benchmark of grey scale pixel value is: select the mean value of the gray-scale value of the pixel of some in all pixels as benchmark, in an embodiment that is more preferably of the present invention, choose the mean value of gray-scale value of pixel of 5%-10% as benchmark.The mean value of the gray-scale value of the pixel of preferred employing 8% is as benchmark.The present invention selects the mean value of gray-scale value of the partial pixel point in all pixels as benchmark, and the ratio of choosing can be adjusted according to actual needs, has improved the dirigibility of method.
After determining benchmark, the gray-scale value benchmark of the pixel of eyes image is carried out light compensation.In the present embodiment, can carry out light compensation to the gray-scale value of all pixels, also can carry out light compensation to the gray-scale value of other all pixels except determining the selected pixel of benchmark.
In the present embodiment, the method of concrete light compensation can for but be not limited to the method that coefficient in proportion compensates, the certain proportion coefficient of getting the benchmark gray-scale value is added on the gray-scale value of all pixels, for example getting 50% of benchmark gray-scale value is added on the gray-scale value of all pixels, thereby realize the light compensation of the gray-scale value of all pixels has been strengthened the brightness of eyes image.The present invention is by increasing the overall brightness of eyes image, also can carry out rapidly and accurately the through hole location for some vague image vegetarian refreshments or blurred picture, improved accuracy.
After eyes image is carried out light compensation, the noise of filtering eyes image, in the present embodiment, wave filter adopts nonlinear filter, can adopt but be not limited to mean filter or and median filter, preferably adopt median filter, in a kind of embodiment that is more preferably of the present invention, adopt the noise of the median filter filtering eyes image of two dimension, concrete implementation procedure is as follows:
Slide on image with a moving window that contains odd point, the gray-scale value of window center being put the correspondence image pixel replaces with the intermediate value in window.
Because eyes image is two-dimentional, adopt two dimensional filter, this wave filter can be expressed from the next:
Wherein, S represents filter window, { f
i,jBe the sequence of image pixel, y
i,jOutput for wave filter.
The present invention adopts the preprocess method of medium filtering and light compensation, has ability fuzzy, partially dark, promoted brightness and filtering noise by the eyes image of noise, has improved the accuracy of pupil location.
Second step: the maximum between-cluster variance of all pixels of statistics eyes image, in another kind of preferred implementation of the present invention, also can adopt improved maximum between-cluster variance computing method, it is not namely the maximum between-cluster variance that calculates all pixels, but calculate the maximum between-cluster variance of certain one part of pixel point, concrete method can for but be not limited to according to the eyes image histogram, count the grey scale pixel value that does not occur in eyes image, calculate the maximum between-cluster variance of remaining pixel, in the present embodiment, the method for calculating maximum between-cluster variance is:
S231: the histogram that obtains eyes image;
S232: determine that in histogram in eyes image, number of pixels is zero grey scale pixel value;
S233: determine the non-vanishing grey scale pixel value of number of pixels in eyes image, the maximum between-cluster variance of the grey scale pixel value that the calculating pixel number is non-vanishing.
Maximum between-cluster variance computing method of the present invention only need the maximum between-cluster variance of the non-vanishing grey scale pixel value of calculating pixel number, have improved computing velocity.
In a preferred embodiment of the present invention, choose the segmentation threshold of prospect and the background of eyes image, when maximum between-cluster variance was maximum, segmentation threshold was the optimal threshold of cutting apart.The present invention utilizes follow-on maximum variance between clusters to realize that adaptive threshold extracts, and has that threshold value is suitable, a self-adaptation, fireballing advantage.
In the present embodiment, after in the first step, pretreated eyes image being done the histogram processing, then adopt improved maximum variance between clusters to ask for and count the grey scale pixel value that does not occur in image, only calculate the maximum between-cluster variance of remaining pixel.As shown in Figure 3, this histogram is that the corresponding number of pixels of this two part gray-scale value that reaches between 0-35 between 210-255 is all zero in the gradation of image value, therefore when asking for maximum between-cluster variance, first this two parts grey scale pixel value in image is come out, do not calculate its corresponding inter-class variance value, only the pixel of remainder is calculated.
The principle of calculating maximum between-cluster variance is as follows: meter t is the segmentation threshold of picture prospect and background, wherein to count and account for whole image scaled be w0 to prospect, its image averaging gray-scale value is u0, background is counted and accounted for image scaled is w1, its image averaging gray-scale value is u1, is u so can calculate image overall average gray scale.Traversal is t from the minimum gradation value to the maximum gradation value, and when t made inter-class variance value g maximum, this moment, t was the optimal threshold of cutting apart.
If the image size is the M*N pixel, L is the rank of gradation of image value, and in the present embodiment, the value of L is 1-255, and the gray-scale value of establishing pixel in image is denoted as N0 less than the number of pixels of t, and pixel grey scale is denoted as N1 greater than the number of pixels of t, can calculate:
N
0+N
1＝M×N （3）
w
0+w
1＝1 （4）
u＝w
0u
0+w
1u
1 （5）
g＝w
0(u
0-u)
2+w
1(u
1-u)
2 （6）
Formula (5) is brought in formula (6), obtains formula (7):
g＝w
0w
1(u
0-u
1)
2 （7）
T will be from 0 to L-1 value successively, and the t value when g gets maximum is the optimal threshold of image segmentation.
The 3rd step: according to maximum between-cluster variance, after the optimal threshold that the calculating eyes image is cut apart, carry out binary conversion treatment, gray-scale value is taken as p1 greater than the gray-scale value of the pixel of t value, and gray-scale value is taken as p2 less than the gray-scale value of the pixel of t value, and p1〉p2.Thereby realize the eyes image binary conversion treatment, result as shown in Figure 5.In the present embodiment, the optimal threshold of cutting apart according to the motion conditions Dynamic Acquisition eyes image of eye, and carry out dynamic binary conversion treatment according to optimal threshold.In the present embodiment, can also filter the eyes image after binary conversion treatment and make an uproar and repair, concrete method is:
extract through adaptive threshold the hole that can there be the pupil target that causes because cornea is reflective in the binaryzation eyes image that obtains, eyelash blocks the shade that brings, the noise that burr etc. causes and the situation of defective, therefore need to carry out image to the binaryzation eyes image and filter the repairing of making an uproar, in the present embodiment, adopt the corrosion operation, expansive working, opening operation and closed operation realize filtering the repairing of making an uproar, in a kind of embodiment that is more preferably of the present invention, the concrete employing corroded operation 5 times, 5 expansive workings, 1 opening operation, 1 closed operation realizes filtering preferably the repair efficiency of making an uproar, wherein, concrete function and the parameter that adopts is set to:
The corrosion handling function is: cvErode (threshold, threshold, NULL, 5);
The expansive working function is: cvDilate (threshold, threshold, NULL, 5);
The opening operation function is:
cvMorphologyEx(threshold,threshold,0,0,CV_MOP_OPEN,1);
The closed operation function is:
cvMorphologyEx(threshold,threshold,0,0,CV_MOP_CLOSE,1)。
The 4th step: extract pupil edge information on the eyes image after binary conversion treatment.In the present embodiment, the method for extraction pupil edge information is:
If the pixel value of central pixel point is 255, no matter why the pixel value of all the other 8 adjacent pixels is worth, the pixel value that keeps without exception central pixel point is 255;
If the pixel value of central pixel point is 0, and the pixel value of 8 adjacent pixels is 0, the pixel value of central pixel point changed into 255;
During all the other situations, all the pixel value with central pixel point changes 0 into.
The 5th step: determine pupil center and radius.In the present embodiment, as shown in Figure 7, utilize least square method to determine pupil center and radius.Employing realizes that based on least square method the basic thought of the match link of edge point is: ask each candidate point minimum to the quadratic sum of the upper distance of circle.If the elliptic equation of pupil is:
Ax
2+Bxy+Cy
2+Dx+Ey+F＝0 （8）
Its constraint condition is:
B
2-4AC＜0 （9）
Can be got by principle of least square method:
By extreme value theorem, the value of wanting to make f (A, B, C, D, E) is minimum, must have:
Can obtain thus a system of linear equations, then in conjunction with constraint condition, can solve equation coefficient A, B, C, D, E, the value of F.So oval centre coordinate (xp, yp), major axis and minor axis (a, b) all can be asked by following formula, thereby obtained radius and the centre coordinate value of pupil:
The extracting method of pupil edge information of the present invention and based on the accurate location that the ellipse fitting method of definite pupil center of least square method and radius realizes pupil has characteristics quickly and accurately.
Pupil positioning method based on the modified maximum variance between clusters of the present invention is by image pretreatment operation and the grey scale pixel value that exists in image is calculated the method for maximum between-cluster variance, fast and accurately asked for the optimal threshold of image segmentation, for the intact extraction of pupil edge information lays the foundation; Last adopt on this basis contour extraction method and realize the accurate location of pupil and improved arithmetic speed based on the ellipse fitting method of least square method.
S3: the mapping relations of setting up eye direction of visual lines and screen.In the present embodiment, step S3 can carry out before step S1, and the step of specifically setting up the mapping relations of eye direction of visual lines and screen is:
S31: show successively M calibration point on screen, described M is positive integer, in the present embodiment, preferably adopts 5 calibration points, and namely four angles and the central authorities at screen arrange respectively a calibration point; In the other preferred implementation of the present invention, adopt 9 calibration points, being about to screen divider is the sphere of movements for the elephants shape, the intersection point of every two lines is as a calibration point.
S32: when eyes when watching each calibration point attentively, N corresponding point after the light that video camera detection infrared light supply sends reflects through pupil center on screen, described N is positive integer, in the present embodiment, 30-50 corresponding point of preferred employing, in a kind of embodiment that is more preferably of the present invention, adopt 50 corresponding point, the raising corresponding point are got accuracy a little.
S33: N the corresponding point that each calibration point detects are analyzed, obtain an equivalent correspondence point, in the present embodiment, coordinate to N corresponding point is averaged, obtain the equivalent correspondence point, simultaneously N pupil center location corresponding to corresponding point also averaged, obtain equivalent pupil center, thereby the movement of pupil is reflected as the equivalent correspondence point movement that screen coordinate is fastened.
S34: the corresponding relation of setting up equivalent pupil center and calibration point.
After the mapping relations of setting up eye direction of visual lines and screen, be kept in calibration matrix, for the tracking that realizes follow-up sight line provides real time data.
S4: Real Time Monitoring eye direction of visual lines, according to the mapping relations of eye direction of visual lines and screen, determine the blinkpunkt of eye sight line on screen.
S5: determine the mode of operation of mouse, wherein, mode of operation comprises the mouse Move Mode, clicks pattern and double-click pattern.In the present embodiment, can define the movement of the corresponding mouse of pupil uniform motion, twice nictation of interval M1 time corresponding single-click operation, twice nictation of interval M2 time corresponding single-click operation, in an embodiment that is more preferably of the present invention, M1 is 1 second, and M2 is 2 seconds.
S6: the blinkpunkt place of eye sight line shows mouse and carries out work according to the mode of operation of determining on screen.
The present invention also provides a kind of controlled mouse to realize system, and it comprises infrared light supply, video camera, screen and control module, and control module comprises eyes image acquisition module and core processing module.The light that infrared light supply sends is used for shining pupil center, the light that video camera detects pupil center's reflection on screen reflection spot and reflective information is transferred to control module, control module shows reflection spot information on screen, and sets up the mapping relations of eye direction of visual lines and screen.The eyes image acquisition module is used for gathering eyes image; Core processing module is connected with described eyes image acquisition module, is used for receiving described eyes image and judges the center of pupil, realizes that the work of eye controlled mouse also shows by screen.
in the present embodiment, core processing module comprises pupil center's judge module, demarcating module and eye tracking module, described pupil center judge module is connected with described eyes image acquisition module, the center that is used for receiving eyes image and judges pupil, described demarcating module is used for setting up the mapping relations of described eye direction of visual lines and screen, described eye tracking module is connected with described demarcating module with the pupil center judge module respectively, described eye tracking module is determined blinkpunkt and the analog mouse realization tracking operation of eyes on screen according to the pupil center of described pupil center judge module judgement and the mapping relations of described eye direction of visual lines and screen.
In the present embodiment, core processing module is positioned at computing machine, also has storer in computing machine, be used for the eyes image that storage eyes image acquisition module gathers, the user can call the eyes image in storer and check at any time by image display by core processing module.
In a kind of preferred implementation of the present invention, also comprise data conversion module, data conversion module is connected with core processing module with the eyes image acquisition module respectively, and the eyes image after also changing for the format conversion that realizes the eyes image data transfers to described core processing module.Data conversion module of the present invention carries out format conversion, when the eyes image data of the data acquisition of eyes image acquisition module collection and data type that core processing module can be processed not simultaneously, utilize this data conversion module that the data layout of eyes image acquisition module collection is changed, improved the compatibility of pupil positioning system, concrete conversion method can be carried out according to prior art.
In the present embodiment, the eyes image acquisition module is eye movement instrument or optical sensor, preferably adopts the eye movement instrument, and the present invention adopts the eye movement instrument to gather eyes image, guarantees that eyes image is accurate, has improved accuracy and has been convenient to timely processing.
In the other preferred implementation of the present invention, this system also comprises power supply and peripheral circuit, is used to infrared light supply, video camera, screen and control module that electric energy is provided, and guarantees the normal operation of infrared light supply, video camera, screen and control module.
In the present embodiment, the eyes image acquisition module selects the MT9V011 chip of MICRON company as optical sensor SENSOR, image resolution ratio is 640*480, the frame per second of the eyes image of taking was 15 frame/seconds, was that fineness and the filming frequency of image all satisfies the usual oculomotor requirement of the mankind.Data conversion module is selected CP2102, has the full speed interface of USB2.0, satisfies the large transmission requirement of data volume, and compatible with the process chip of back.Core processing module is selected the ARM11 series of processes chip S3C6410 of Samsung, and speed and performance can satisfy the requirement of image and Algorithm Analysis.Power supply and peripheral circuit can be selected existing power supply chip, as long as the operating voltage that provides meets the demands.
In the description of this instructions, the description of reference term " embodiment ", " some embodiment ", " example ", " concrete example " or " some examples " etc. means to be contained at least one embodiment of the present invention or example in conjunction with specific features, structure, material or the characteristics of this embodiment or example description.In this manual, the schematic statement of above-mentioned term not necessarily referred to identical embodiment or example.And the specific features of description, structure, material or characteristics can be with suitable mode combinations in any one or more embodiment or example.
Although illustrated and described embodiments of the invention, those having ordinary skill in the art will appreciate that: in the situation that do not break away from principle of the present invention and aim can be carried out multiple variation, modification, replacement and modification to these embodiment, scope of the present invention is limited by claim and equivalent thereof.
Claims (10)
1. an eye controlled mouse implementation method, is characterized in that, comprises the steps:
S1: obtain eyes image;
S2: the pupil position in described eyes image is detected and locates;
S3: the mapping relations of setting up described eye direction of visual lines and screen;
S4: Real Time Monitoring eye direction of visual lines, according to the mapping relations of described eye direction of visual lines and screen, determine the blinkpunkt of described eye sight line on described screen;
S5: determine the mode of operation of mouse, described mode of operation comprises the mouse Move Mode, clicks pattern and double-click pattern;
S6: the blinkpunkt place of eye sight line shows mouse and carries out work according to described mode of operation on described screen.
2. eye controlled mouse implementation method as claimed in claim 1, is characterized in that, in described step S2, the localization method of pupil is:
S21: described eyes image is carried out pre-service, increase the overall brightness of described eyes image, and the noise of the described eyes image of filtering;
S22: the maximum between-cluster variance of all pixels of the described eyes image of statistics;
S23: according to described maximum between-cluster variance, calculate the optimal threshold of described eyes image binaryzation, realize the eyes image binary conversion treatment;
S24: extract pupil edge information;
S25: determine pupil center and radius.
3. eye controlled mouse implementation method as claimed in claim 2, is characterized in that, in described eyes image was carried out pretreated step, the method that increases the overall brightness of described eyes image was:
S211: described eyes image is converted into corresponding histogram;
S212: the gray-scale value that goes out all pixels in described eyes image according to described statistics with histogram;
S213: determine the benchmark of grey scale pixel value, the gray-scale value of the pixel of described eyes image is carried out light compensation according to described benchmark.
4. eye controlled mouse implementation method as claimed in claim 2, is characterized in that, the method for the benchmark of described definite grey scale pixel value is: select the mean value of the gray-scale value of the pixel of 5%-10% in all pixels as benchmark.
5. eye controlled mouse implementation method as claimed in claim 2, is characterized in that, in described step S22, the method for the maximum between-cluster variance of all pixels of the described eyes image of statistics is:
S221: the histogram that obtains described eyes image;
S222: determine that in described histogram in described eyes image, number of pixels is zero grey scale pixel value;
S223: determine the non-vanishing grey scale pixel value of number of pixels in described eyes image, calculate the maximum between-cluster variance of the non-vanishing grey scale pixel value of described number of pixels.
6. as 2 or 5 described controlled mouse implementation methods of claim, it is characterized in that, choose the segmentation threshold of prospect and the background of described eyes image, when described maximum between-cluster variance was maximum, described segmentation threshold was the optimal threshold of cutting apart.
7. eye controlled mouse implementation method as claimed in claim 2, is characterized in that, the method for extracting pupil edge information is:
If the pixel value of central pixel point is 255, no matter why the pixel value of all the other 8 adjacent pixels is worth, the pixel value that keeps without exception central pixel point is 255;
If the pixel value of central pixel point is 0, and the pixel value of 8 adjacent pixels is 0, the pixel value of central pixel point changed into 255;
During all the other situations, all the pixel value with central pixel point changes 0 into.
8. eye controlled mouse implementation method as claimed in claim 1, is characterized in that, the step of setting up the mapping relations of described eye direction of visual lines and screen is:
S31: show successively M calibration point on screen, described M is positive integer;
S32: when eyes when watching each calibration point attentively, video camera detect light that infrared light supply sends through pupil center's reflection after N corresponding point on screen, described N is positive integer;
S33: N the corresponding point that each calibration point detects are analyzed, obtained an equivalent correspondence point;
S34: the corresponding relation of setting up pupil center and calibration point.
9. an eye controlled mouse is realized system, it is characterized in that, comprising: infrared light supply, video camera, screen and control module, and described control module comprises eyes image acquisition module and core processing module;
Described infrared light supply and video camera are set up the mapping relations of eye direction of visual lines and screen according to the method shown in claim 8;
Described eyes image acquisition module is used for gathering eyes image;
Described core processing module is connected with described eyes image acquisition module, is used for receiving described eyes image and judges the center of pupil, and realize the work of eye controlled mouse and show by screen according to the described method of one of claim 1-7.
10. eye controlled mouse as claimed in claim 9 is realized system, it is characterized in that: described core processing module comprises pupil center's judge module, demarcating module and eye tracking module, described pupil center judge module is connected with described eyes image acquisition module, the center that is used for receiving eyes image and judges pupil, described demarcating module is used for setting up the mapping relations of described eye direction of visual lines and screen, described eye tracking module is connected with described demarcating module with the pupil center judge module respectively, described eye tracking module is determined blinkpunkt and the analog mouse realization tracking operation of eyes on screen according to the pupil center of described pupil center judge module judgement and the mapping relations of described eye direction of visual lines and screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310130392.9A CN103176607B (en) | 2013-04-16 | 2013-04-16 | A kind of eye-controlled mouse realization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310130392.9A CN103176607B (en) | 2013-04-16 | 2013-04-16 | A kind of eye-controlled mouse realization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103176607A true CN103176607A (en) | 2013-06-26 |
| CN103176607B CN103176607B (en) | 2016-12-28 |
Family
ID=48636542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310130392.9A Expired - Fee Related CN103176607B (en) | 2013-04-16 | 2013-04-16 | A kind of eye-controlled mouse realization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103176607B (en) |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103336581A (en) * | 2013-07-30 | 2013-10-02 | 黄通兵 | Human eye movement characteristic design-based human-computer interaction method and system |
| CN103425970A (en) * | 2013-08-29 | 2013-12-04 | 大连理工大学 | Human-computer interaction method based on head postures |
| CN104238120A (en) * | 2013-12-04 | 2014-12-24 | 全蕊 | Smart glasses and control method |
| CN104680122A (en) * | 2013-11-29 | 2015-06-03 | 展讯通信(天津)有限公司 | Tracking method and device based on skin color detection |
| CN104680552A (en) * | 2013-11-29 | 2015-06-03 | 展讯通信(天津)有限公司 | Tracking method and device based on skin color detection |
| CN104898823A (en) * | 2014-03-04 | 2015-09-09 | 中国电信股份有限公司 | Method and device for controlling sighting mark motion |
| CN105094604A (en) * | 2015-06-30 | 2015-11-25 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105138965A (en) * | 2015-07-31 | 2015-12-09 | 东南大学 | Near-to-eye sight tracking method and system thereof |
| CN105137601A (en) * | 2015-10-16 | 2015-12-09 | 上海斐讯数据通信技术有限公司 | Intelligent glasses |
| CN105425967A (en) * | 2015-12-16 | 2016-03-23 | 中国科学院西安光学精密机械研究所 | Sight tracking and human eye region-of-interest positioning system |
| CN105676458A (en) * | 2016-04-12 | 2016-06-15 | 王鹏 | Wearable calculation device and control method thereof, and wearable equipment with wearable calculation device |
| CN106020461A (en) * | 2016-05-13 | 2016-10-12 | 陈盛胜 | Video interaction method based on eyeball tracking technology |
| CN106502423A (en) * | 2016-11-21 | 2017-03-15 | 武汉理工大学 | Based on the automation microoperation method that human eye vision is positioned |
| CN106527705A (en) * | 2016-10-28 | 2017-03-22 | 努比亚技术有限公司 | Operation realization method and apparatus |
| CN106774862A (en) * | 2016-12-03 | 2017-05-31 | 西安科锐盛创新科技有限公司 | VR display methods and VR equipment based on sight line |
| CN106774863A (en) * | 2016-12-03 | 2017-05-31 | 西安中科创星科技孵化器有限公司 | A kind of method that Eye-controlling focus are realized based on pupil feature |
| CN103412643B (en) * | 2013-07-22 | 2017-07-25 | 深圳Tcl新技术有限公司 | Terminal and its method for remote control |
| CN107067441A (en) * | 2017-04-01 | 2017-08-18 | 海信集团有限公司 | Camera marking method and device |
| CN104007820B (en) * | 2014-05-26 | 2017-09-29 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN107436675A (en) * | 2016-05-25 | 2017-12-05 | 深圳纬目信息技术有限公司 | A kind of visual interactive method, system and equipment |
| CN107609516A (en) * | 2017-09-13 | 2018-01-19 | 重庆爱威视科技有限公司 | Adaptive eye moves method for tracing |
| CN107831900A (en) * | 2017-11-22 | 2018-03-23 | 中国地质大学（武汉） | The man-machine interaction method and system of a kind of eye-controlled mouse |
| CN108282699A (en) * | 2018-01-24 | 2018-07-13 | 北京搜狐新媒体信息技术有限公司 | A kind of display interface processing method and processing device |
| CN108427503A (en) * | 2018-03-26 | 2018-08-21 | 京东方科技集团股份有限公司 | Human eye method for tracing and human eye follow-up mechanism |
| CN109656373A (en) * | 2019-01-02 | 2019-04-19 | 京东方科技集团股份有限公司 | One kind watching independent positioning method and positioning device, display equipment and storage medium attentively |
| CN109960405A (en) * | 2019-02-22 | 2019-07-02 | 百度在线网络技术（北京）有限公司 | Mouse operation method, device and storage medium |
| CN110780739A (en) * | 2019-10-18 | 2020-02-11 | 天津理工大学 | Eye control auxiliary input method based on fixation point estimation |
| CN112183200A (en) * | 2020-08-25 | 2021-01-05 | 中电海康集团有限公司 | Eye movement tracking method and system based on video image |
| CN113891002A (en) * | 2021-11-12 | 2022-01-04 | 维沃移动通信有限公司 | Shooting method and device |
| CN114020155A (en) * | 2021-11-05 | 2022-02-08 | 沈阳飞机设计研究所扬州协同创新研究院有限公司 | High-precision sight line positioning method based on eye tracker |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101359365A (en) * | 2008-08-07 | 2009-02-04 | 电子科技大学中山学院 | Iris positioning method based on Maximum between-Cluster Variance and gray scale information |
| CN102012742A (en) * | 2010-11-24 | 2011-04-13 | 广东威创视讯科技股份有限公司 | Method and device for correcting eye mouse |
| CN102509095A (en) * | 2011-11-02 | 2012-06-20 | 青岛海信网络科技股份有限公司 | Number plate image preprocessing method |
| CN102830797A (en) * | 2012-07-26 | 2012-12-19 | 深圳先进技术研究院 | Man-machine interaction method and system based on sight judgment |
| CN102938058A (en) * | 2012-11-14 | 2013-02-20 | 南京航空航天大学 | Method and system for video driving intelligent perception and facing safe city |
-
2013
- 2013-04-16 CN CN201310130392.9A patent/CN103176607B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101359365A (en) * | 2008-08-07 | 2009-02-04 | 电子科技大学中山学院 | Iris positioning method based on Maximum between-Cluster Variance and gray scale information |
| CN102012742A (en) * | 2010-11-24 | 2011-04-13 | 广东威创视讯科技股份有限公司 | Method and device for correcting eye mouse |
| CN102509095A (en) * | 2011-11-02 | 2012-06-20 | 青岛海信网络科技股份有限公司 | Number plate image preprocessing method |
| CN102830797A (en) * | 2012-07-26 | 2012-12-19 | 深圳先进技术研究院 | Man-machine interaction method and system based on sight judgment |
| CN102938058A (en) * | 2012-11-14 | 2013-02-20 | 南京航空航天大学 | Method and system for video driving intelligent perception and facing safe city |
Non-Patent Citations (1)
| Title |
|---|
| 黄丽丽等: "基于改进型最大类间方差法的瞳孔定位方法", 《计算机工程与应用》 * |
Cited By (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103412643B (en) * | 2013-07-22 | 2017-07-25 | 深圳Tcl新技术有限公司 | Terminal and its method for remote control |
| CN103336581A (en) * | 2013-07-30 | 2013-10-02 | 黄通兵 | Human eye movement characteristic design-based human-computer interaction method and system |
| CN103425970A (en) * | 2013-08-29 | 2013-12-04 | 大连理工大学 | Human-computer interaction method based on head postures |
| CN104680122B (en) * | 2013-11-29 | 2019-03-19 | 展讯通信(天津)有限公司 | A kind of tracking and device based on Face Detection |
| CN104680122A (en) * | 2013-11-29 | 2015-06-03 | 展讯通信(天津)有限公司 | Tracking method and device based on skin color detection |
| CN104680552A (en) * | 2013-11-29 | 2015-06-03 | 展讯通信(天津)有限公司 | Tracking method and device based on skin color detection |
| CN104680552B (en) * | 2013-11-29 | 2017-11-21 | 展讯通信(天津)有限公司 | A kind of tracking and device based on Face Detection |
| CN104238120A (en) * | 2013-12-04 | 2014-12-24 | 全蕊 | Smart glasses and control method |
| CN104898823A (en) * | 2014-03-04 | 2015-09-09 | 中国电信股份有限公司 | Method and device for controlling sighting mark motion |
| CN104898823B (en) * | 2014-03-04 | 2018-01-23 | 中国电信股份有限公司 | The method and apparatus for controlling sighting target motion |
| CN104007820B (en) * | 2014-05-26 | 2017-09-29 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN105094604A (en) * | 2015-06-30 | 2015-11-25 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105138965B (en) * | 2015-07-31 | 2018-06-19 | 东南大学 | A kind of near-to-eye sight tracing and its system |
| CN105138965A (en) * | 2015-07-31 | 2015-12-09 | 东南大学 | Near-to-eye sight tracking method and system thereof |
| CN105137601A (en) * | 2015-10-16 | 2015-12-09 | 上海斐讯数据通信技术有限公司 | Intelligent glasses |
| CN105137601B (en) * | 2015-10-16 | 2017-11-14 | 上海斐讯数据通信技术有限公司 | A kind of intelligent glasses |
| CN105425967A (en) * | 2015-12-16 | 2016-03-23 | 中国科学院西安光学精密机械研究所 | Sight tracking and human eye region-of-interest positioning system |
| CN105425967B (en) * | 2015-12-16 | 2018-08-28 | 中国科学院西安光学精密机械研究所 | Sight tracking and human eye region-of-interest positioning system |
| CN105676458A (en) * | 2016-04-12 | 2016-06-15 | 王鹏 | Wearable calculation device and control method thereof, and wearable equipment with wearable calculation device |
| CN106020461A (en) * | 2016-05-13 | 2016-10-12 | 陈盛胜 | Video interaction method based on eyeball tracking technology |
| CN107436675A (en) * | 2016-05-25 | 2017-12-05 | 深圳纬目信息技术有限公司 | A kind of visual interactive method, system and equipment |
| CN106527705A (en) * | 2016-10-28 | 2017-03-22 | 努比亚技术有限公司 | Operation realization method and apparatus |
| CN106502423B (en) * | 2016-11-21 | 2019-04-30 | 武汉理工大学 | Automation microoperation method based on human eye vision positioning |
| CN106502423A (en) * | 2016-11-21 | 2017-03-15 | 武汉理工大学 | Based on the automation microoperation method that human eye vision is positioned |
| CN106774863A (en) * | 2016-12-03 | 2017-05-31 | 西安中科创星科技孵化器有限公司 | A kind of method that Eye-controlling focus are realized based on pupil feature |
| CN106774862A (en) * | 2016-12-03 | 2017-05-31 | 西安科锐盛创新科技有限公司 | VR display methods and VR equipment based on sight line |
| CN107067441A (en) * | 2017-04-01 | 2017-08-18 | 海信集团有限公司 | Camera marking method and device |
| CN107067441B (en) * | 2017-04-01 | 2020-02-11 | 海信集团有限公司 | Camera calibration method and device |
| CN107609516A (en) * | 2017-09-13 | 2018-01-19 | 重庆爱威视科技有限公司 | Adaptive eye moves method for tracing |
| CN107609516B (en) * | 2017-09-13 | 2019-10-08 | 重庆爱威视科技有限公司 | Adaptive eye movement method for tracing |
| CN107831900B (en) * | 2017-11-22 | 2019-12-10 | 中国地质大学（武汉） | human-computer interaction method and system of eye-controlled mouse |
| CN107831900A (en) * | 2017-11-22 | 2018-03-23 | 中国地质大学（武汉） | The man-machine interaction method and system of a kind of eye-controlled mouse |
| CN108282699A (en) * | 2018-01-24 | 2018-07-13 | 北京搜狐新媒体信息技术有限公司 | A kind of display interface processing method and processing device |
| CN108427503A (en) * | 2018-03-26 | 2018-08-21 | 京东方科技集团股份有限公司 | Human eye method for tracing and human eye follow-up mechanism |
| CN108427503B (en) * | 2018-03-26 | 2021-03-16 | 京东方科技集团股份有限公司 | Human eye tracking method and human eye tracking device |
| CN109656373A (en) * | 2019-01-02 | 2019-04-19 | 京东方科技集团股份有限公司 | One kind watching independent positioning method and positioning device, display equipment and storage medium attentively |
| US11205070B2 (en) | 2019-01-02 | 2021-12-21 | Beijing Boe Optoelectronics Technology Co., Ltd. | Method, an apparatus, a display device and a storage medium for positioning a gaze point |
| CN109960405A (en) * | 2019-02-22 | 2019-07-02 | 百度在线网络技术（北京）有限公司 | Mouse operation method, device and storage medium |
| CN110780739B (en) * | 2019-10-18 | 2023-11-03 | 天津理工大学 | Eye control auxiliary input method based on gaze point estimation |
| CN110780739A (en) * | 2019-10-18 | 2020-02-11 | 天津理工大学 | Eye control auxiliary input method based on fixation point estimation |
| CN112183200A (en) * | 2020-08-25 | 2021-01-05 | 中电海康集团有限公司 | Eye movement tracking method and system based on video image |
| CN112183200B (en) * | 2020-08-25 | 2023-10-17 | 中电海康集团有限公司 | Eye movement tracking method and system based on video image |
| CN114020155A (en) * | 2021-11-05 | 2022-02-08 | 沈阳飞机设计研究所扬州协同创新研究院有限公司 | High-precision sight line positioning method based on eye tracker |
| WO2023083279A1 (en) * | 2021-11-12 | 2023-05-19 | 维沃移动通信有限公司 | Photographing method and apparatus |
| CN113891002A (en) * | 2021-11-12 | 2022-01-04 | 维沃移动通信有限公司 | Shooting method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103176607B (en) | 2016-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103176607A (en) | Eye-controlled mouse realization method and system | |
| CN103136512A (en) | Pupil positioning method and system | |
| CN103340637B (en) | Move and driver's Alertness intelligent monitor system of brain electro' asion and method based on eye | |
| CN102508110B (en) | Texture-based insulator fault diagnostic method | |
| CN102692216B (en) | Real-time optical fiber winding defect detection method based on machine vision technology | |
| CN105184246A (en) | Living body detection method and living body detection system | |
| CN101241548A (en) | Eyelid detection apparatus and programs thereof | |
| CN104615978A (en) | Sight direction tracking method and device | |
| CN104766059A (en) | Rapid and accurate human eye positioning method and sight estimation method based on human eye positioning | |
| CN103105924B (en) | Man-machine interaction method and device | |
| CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
| CN104503472A (en) | Intelligent real-time human body position tracking energy saving heating system based on infrared radiation | |
| CN102980535A (en) | Angle measurement method and device | |
| CN103164022A (en) | Multi-finger touch method, device and portable type terminal device | |
| US20230057878A1 (en) | Industrial internet of things, control methods and storage medium based on machine visual detection | |
| CN103761011A (en) | Method, system and computing device of virtual touch screen | |
| CN104123728A (en) | Hough transform based round pipe detection method | |
| US11195490B1 (en) | Smart contact lens with adjustable light transmittance | |
| CN108634925A (en) | A kind of vision testing system based on gesture identification | |
| CN113160260A (en) | Head-eye double-channel intelligent man-machine interaction system and operation method | |
| CN105068643A (en) | Method and device for adjusting brightness of intelligent mirror | |
| CN107770359A (en) | A kind of obstacle based reminding method and mobile device | |
| CN106094969A (en) | The image aided diagnosis technique that a kind of maximum power point of photovoltaic power generation system is followed the trail of | |
| CN114661152B (en) | AR display control system and method for reducing visual fatigue | |
| CN103926922A (en) | Control and monitor system of smart vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20161228 Termination date: 20180416 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |