CN108829242A - Intelligent terminal and its non-touch operating method - Google Patents
Intelligent terminal and its non-touch operating method Download PDFInfo
- Publication number
- CN108829242A CN108829242A CN201810495419.7A CN201810495419A CN108829242A CN 108829242 A CN108829242 A CN 108829242A CN 201810495419 A CN201810495419 A CN 201810495419A CN 108829242 A CN108829242 A CN 108829242A
- Authority
- CN
- China
- Prior art keywords
- user
- eyeball
- display screen
- view region
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
- H04M1/72439—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for image or video messaging
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
- H04M1/72454—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/52—Details of telephonic subscriber devices including functional features of a camera
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of intelligent terminals and its non-touch operating method, this method to include：S1. the facial image of user is accomplished continuously or intermittently acquired, the facial image includes face depth image；S2. the facial image is handled, the eyeball fixes directional information and facial orientation information of the user are extracted, and tracks the user eyeball view region；S3. according to the user eyeball view region, predetermined registration operation is executed.By the facial image of acquisition user, the eyeball fixes directional information and facial orientation information of the user can be extracted, and track the user eyeball view region, it, can be with the operation of further progress non-touch according to user eyeball view region.Its eyeball view region combines the eyeball fixes directional information and facial orientation information of user, so that the eyeball view region accuracy being finally calculated is high, is more advantageous to the exact operations of non-touch operation, possesses better user experience.
Description
Technical field
The present invention relates to terminal applies field more particularly to a kind of intelligent terminal and its non-touch operating methods.
Technical background
With the development of the intelligent terminals such as smart phone and internet, people's lives mode has occurred that huge change
Change；Especially, the mode of information is obtained.Compared to traditional paper book or newspaper channel, people preferred through intelligent end
End obtains first-hand information on the net.
Currently, the intelligent terminals such as smart phone, tablet computer on the market, are mainly also switched on and hold screen or flexible screen
Carry out human-computer interaction.When the finger of user speckles with water, greasy dirt, the capacitive screens of interaction support are provided, often can not accurately be known
The intention (for example, being unable to complete successfully mobile phone web pages, page turning of e-book etc.) of other user, so as to cause bad user's body
It tests.In addition, excessively using capacitive screens or flexible screen, it will also result in the damage of unrepairable, accelerate the aging of smart phone
Deng.
Summary of the invention
To solve the above problems, the present invention provides a kind of intelligent terminal and its non-touch operating method, it can be to intelligence
Energy terminal executes non-touch operation, possesses better user experience.
The present invention provides a kind of non-touch operating method of intelligent terminal, including：S1. accomplished continuously or intermittently acquire user's
Facial image, the facial image include face depth image；S2. the facial image is handled, the eyeball of the user is extracted
Direction of gaze information and facial orientation information, and track the user eyeball view region；S3. it is regarded according to the user eyeball
Point region, executes predetermined registration operation.
Wherein, the facial image can also include one of RGB-D image, RGB image, infrared image or a variety of.
In some embodiments, step S2 determines the eyeball position letter of the user by the geometrical characteristic of human face five-sense-organ
Breath is to extract eyeball fixes directional information；Or the use is determined to the high reflectance of infrared beam by face cornea
The eyeball position information at family is to extract eyeball fixes directional information.
In some embodiments, step S2 reconstructs the three-dimensional face images of the user by the face depth image,
Extract the facial orientation information.
In some embodiments, the step S3 includes：Fallen in display screen according to the user eyeball view region or
Outside display screen, executes and keep operation or breath screen sleep operation.
If the user eyeball view region is fallen in outside display screen, calculates the user eyeball view region and leave display screen
Time T；When the time T is greater than or equal to preset threshold time T1, breath screen sleep operation is executed；When the time T is small
When preset threshold time T1, the current working condition of holding display screen.In some preferred embodiments, continuous by tracking
The n user eyeball view regions fall in the facial image outside display screen, in conjunction with the frequency acquisition of human face image collecting device
F calculates the time T=n/f that the user eyeball view region is fallen in outside display screen.
In some embodiments, the step S3 includes：If the user eyeball view region is fallen in display screen, calculate
The intersection location of the user eyeball view region and display screen；According to the relationship of the intersection location and default page turning region,
It executes page turn over operation or keeps operation.When the intersection location falls within the default page turning region, what the display screen was shown
Webpage or text execute page turn over operation upward or downward；When the intersection location is not at the default page turning region, holding is aobvious
The current display content of display screen.
The present invention also provides a kind of intelligent terminals, including：Display screen, for showing information and providing interaction；Acquisition dress
It sets, for acquiring the facial image of user, the facial image includes face depth image；Processor, for handling the people
Face image, extracts the eyeball fixes directional information and facial orientation information of the user, and tracks the user eyeball viewpoint
Region；According to the user eyeball view region, predetermined registration operation is executed.
Compared with prior art, beneficial effects of the present invention are：By acquiring the facial image of user, can extract described
The eyeball fixes directional information and facial orientation information of user, and the user eyeball view region is tracked, according to user's eye
Ball view region, can be with the operation of further progress non-touch.Its eyeball view region combines the eyeball fixes side of user
To information and facial orientation information, so that the eyeball view region accuracy being finally calculated is high, it is more advantageous to non-touch
The exact operations of operation possess better user experience.
Detailed description of the invention
Fig. 1 is that a kind of intelligence according to an embodiment of the present invention reads schematic diagram.
Fig. 2 is a kind of mobile terminal structure schematic diagram containing intelligent read function according to an embodiment of the present invention.
Fig. 3 is a kind of method flow diagram for realizing intelligence breath screen according to an embodiment of the present invention.
Fig. 4 is a kind of method flow diagram for realizing real-time printing according to an embodiment of the present invention.
Fig. 5 is a kind of smart phone structural schematic diagram containing intelligent read function according to an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached drawing to the present invention
Each embodiment be explained in detail.It should be understood that the embodiment is only a part of the embodiments of the present invention,
Rather than whole embodiments, it is not considered as limiting the invention.Based on the embodiments of the present invention, ordinary skill
Personnel's every other embodiment obtained without making creative work all should belong to the model that the present invention protects
It encloses.
It should be noted that in the embodiment of the present invention contained lid functional module, can according to need, pass through the prior art
One of or a variety of programming languages realize.For well known Programming Principle and software sharing, although not being elaborated hereafter,
But it should belong to a part of technical solution of the present invention.
Fig. 1 is that a kind of intelligence according to an embodiment of the present invention reads schematic diagram.In the embodiment, user 100 passes through intelligence
Mobile terminal 110 obtains information or reading electronic book or browsing webpage.Similar to traditional mobile terminal, user can be by aobvious
Display screen browses webpage, reading electronic book or viewing video etc.；Difference is, before intelligent mobile terminal 110 further includes at least one
It sets RGB-D camera 111 and user 100 is assisted to browse webpage, the intelligence of reading electronic book reads application.Specifically, it is assumed that intelligence
Energy reading application is in the open state, then intelligent mobile terminal accomplished continuously or intermittently obtains the people of user by preposition RGB-D camera
Face image (including face RGB image, depth image etc.), and from the facial image extract user eyeball characteristic information,
To determine the direction of gaze information and facial orientation information of user eyeball, so that it is determined that user eyeball view region, viewpoint area
Domain fixation time etc..By tracking user eyeball view region, view region fixation time, intelligent mobile terminal can identify use
The intention at family, and then independently execute the operation such as the suspend mode of breath screen, page turning.In order to make it easy to understand, in a kind of embodiment, when user's eye
When ball view region leaves the display screen time and is more than preset threshold time T1, intelligent mobile terminal executes breath screen sleep operation；When
When user eyeball view region falls within default page turning region, intelligent mobile terminal executes page turn over operation.It is understood that intelligence
The suspend mode of breath screen, page turn over operation performed by mobile terminal, essence is by the processor of intelligent mobile terminal by executing related call
Instruction, program, algorithm are completed.
It is to be appreciated that intelligence, which reads application developer, can be based on intelligent mobile terminal institute in a kind of embodiment
The operating system and its related api interface used is compiled, develops, to realize the contactless manipulation of intelligent mobile terminal.
In another equivalent embodiment, intelligence reads application developer can also be directly to the bottom operation of intelligent mobile terminal
System is compiled, develops, so that intelligent mobile terminal carries the function of intelligently reading, to reach, intelligent mobile terminal is non-to be connect
The purpose of touch manipulation.
Fig. 2 is a kind of mobile terminal structure schematic diagram containing intelligent read function according to an embodiment of the present invention.The intelligence
Mobile terminal generally comprises processor 200 and connected display screen 210, microphone 220, RGB-D camera 203, connects
Mouth 240, memory 250, power supply 260, audio devices 270, MEMS sensor 280, radio frequency and baseband processor 290, or also
Including equipment such as environment light/proximity sensors 300.It is to be appreciated that can be realized by circuit connection between distinct device
The exchange or processing of data.In addition, the structure constitutional diagram of intelligent mobile terminal shown in Fig. 2 is only schematic diagram, should not be understood as
Limitation to this hair content, in other equivalent embodiments, structure that intelligent mobile terminal is included, equipment can it is more or
Less.
Processor 200 is used for entirety control, data processing to intelligent mobile terminal, including unlatching/closing relevant device
Data, storing data are acquired, call, analyze data etc.；Processor 200 can be the processor of certain single equipment of control/handle,
It is also possible to integrated control/processing multiple equipment central processor CPU.
Display screen 210 is used for the displaying of information, so that user obtains desired information, including picture, video, text, symbol
Number, data etc.；In addition, the display screen of part intelligent mobile terminal can also be that the capacitance/resistance with touch-control input function is aobvious
Display screen/OLED (organic electroluminescent LED) flexible screen, so that user edits, handles the human-computer interactions such as associated display information behaviour
Make.
Microphone 220 is used to user/environment acoustic information being converted to power information, so as to user preferably with intelligent sliding
Dynamic terminal carries out information exchange.
RGB-D camera 230 is used to obtain all or part of image information of user's face, and is transmitted in processor, with
Just processor extracts the eyeball characteristic information of user.In the embodiment, RGB-D camera includes projection arrangement, depth image acquisition
The components such as device, RGB image acquisition device.A kind of space projection in embodiment, where projection arrangement to user or user
Wavelength is the infrared patterns light beam or visible pattern light beam of 850nm or 940nm, such as speckle light beam (speckle light figure
Case)；Depth image acquisition device captures the light pattern for falling within user's face surface, and is conveyed in processor；Processor passes through
The related algorithms such as triangulation obtain the face depth information of user, rebuild the face 3-D image of user；And RGB acquisition device
Directly acquire the color image of face.In some equivalent embodiments, flight time depth camera is also can be used in RGB-D camera
(TOF depth camera) and RGB camera replace.It should be understood that the projection arrangement of the TOF depth camera is with specific frequency/time
Spatial emission laser beam where period to user or user；What depth image acquisition device acquisition user or space reflection were returned
Laser beam feedback is to processor；Processor is by calculating laser beam from when being emitted to needed for being depth image by acquisition device capture
Between difference determine the depth information of user or space.In some other embodiments, intelligent mobile terminal can also include
It is any in the components such as binocular structure light depth camera, light-field camera, wide angle camera, focal length camera, zoom camera, fisheye camera
One or more combinations.
Interface 240 is for realizing connectivity port soft, hard inside or outside intelligent mobile terminal, to further realize data
The functions, such as I/O interface, network interface, USB interface, WIFI interface such as transmission, power transmission etc..
Memory 250 includes volatibility or the non-volatile readable storage mediums such as register, RAM, FLASH, hard disk, is used for
Temporarily or permanently save application program, data, access pointer, address etc..Specifically, the content of the storage of memory 250 includes
But it is not limited only to：Preset eyeball view region data information 251 is (for judging whether user eyeball view region falls within display
Important parameter in screen) and default eyeball fixes crash time data information 252 (for judging whether intelligent mobile terminal is held
The crash time parameter of row breath screen sleep operation)；In some other equivalent implementations, memory is also used to store user's eye
Ball to display screen optimum distance data information 253 (for judging user with the presence or absence of undesirable reading habit, to remind user
Pay attention to protecting eyes).In addition, memory 250 be also used to storage processor 200 when executing a certain task generated code and
Data.It is understood that a part of memory as intelligent mobile terminal, can also exist independently of mobile terminal, than
Such as cloud memory, the data saved can pass through certain specific communications protocol or access port storage/acquisition.
260 power supply equipment of power supply, for guaranteeing that it is necessary that intelligent mobile terminal inner/outer hardware, equipment provide
Current or voltage.Audio devices 270 are for realizing voice output, such as sound equipment, loudspeaker, buzzer.
MEMS sensor 280 is for obtaining the current status information of mobile terminal, such as position, direction, acceleration, gravity
Etc. information, therefore MEMS sensor may include but be not limited only to：The sensors such as accelerometer, gravitometer, gyroscope.At one
In embodiment, MEMS sensor 280 is used to activate certain applications of intelligent mobile terminal.For example, when MEMS sensor senses
When user picks up intelligent mobile terminal, the intelligent mobile terminal under dormant state is automatically waken up；Further, intelligent mobile is opened
The preposition RGB-D camera of terminal, the human face image information for acquiring user feed back to processor, so that processor extracts user eyeball
Characteristic information and judge whether user watches display screen attentively, and then determines whether to light display screen.
Radio frequency and frenulum processor 290 for example receive Base Transmitter for realizing the communication function of intelligent mobile terminal
Electromagnetic wave signal, and it is demodulated into associated video/voice signal, to realize the information exchange between mobile terminal.
Intelligent mobile terminal further includes environment light/proximity sensor, for sense the physics between user and display away from
From or environment light beam intensity.Generally, in a certain specifically interior apart from section [L1, L2], environment light/proximity sensor reception
Light beam, with the movement of user minor change can occur for light intensity.Environment light/proximity sensor is by by the change of beam intensity
Change the variation for being converted into electric signal, and be transmitted in processor 200, so that processor 200 wakes up the intelligent mobile of suspend mode indirectly
Terminal or the face identification system for opening intelligent mobile terminal.In some embodiments, it is obtained based on environment light/proximity sensor
The beam intensity information obtained, can also realize the adjust automatically of 210 brightness of display, comfortably show in order to provide more bright
Degree；In some other applications, also can realize the breath screen function of touch-control interactive display according to the strength signal of the light beam
Can, in order to avoid the mistake of display screen is caused to touch input.
Fig. 3 is a kind of method flow diagram for realizing intelligence breath screen according to an embodiment of the present invention.In the embodiment, intelligent sliding
The breath screen sleep operation of dynamic terminal, detailed process include：
Step 301, user independently selects activation/opening intelligently to read application.In some other embodiment, intelligently read
Application is read in default open state.
Step 302, the Front camera of processor driving intelligent mobile terminal, acquires the facial image of user, facial image
Including face depth image, in addition to this it is possible to including one of RGB-D image, RGB image, infrared image or a variety of,
And feed back to processor.In the present embodiment, it is preferred to use RGB-D camera, it is to be understood that preposition RGB-D camera collects
Facial image also typically include complicated background image.Therefore, processor is needed through associated picture Processing Algorithm to RGB-D
The image of camera acquisition is pre-processed, and the facial image of user is extracted from complicated background.Specifically, a kind of
In embodiment, processor can scan each pixel of acquired image, and according to its gray value by comparing algorithm
Face and background in described image are distinguished in the variation of (or depth value), extract the facial image of user.
Step 303, processor is extracted in the facial image by facial image described in image characteristics extraction algorithm process
Eyeball characteristic information, obtain user eyeball direction of gaze information；Facial orientation information is obtained simultaneously.Specifically, a kind of implementation
In mode, the geometrical characteristic based on human face five-sense-organ, such as the geometry in particular at face eyes, canthus, it determines in facial image
Eyeball position.In other implementations, processor it is infrared can also to find the face by related figure Processing Algorithm
" bright spot " position in light pattern is (since cornea is higher to infrared beam reflectivity, acquired in image collecting device
Infrared light pattern, corresponding eyeball position have more high brightness), and then determine user eyeball position and eyeball fixes
Bearing data information.In addition, the depth facial image based on the acquisition of RGB-D camera, in conjunction with triangulation algorithm, processor can be with
The three-dimensional face images of reconstructing user, and then determine the normal line vector of user's face place plane, obtain the facial orientation of user
Information.
Step 304, eyeball fixes bearing data information and facial orientation information of the processor based on the user, tracking
User eyeball view region, judges whether user's human eye watches display screen attentively.In a kind of embodiment, processor is by calculating user
The method in eyeball fixes direction (W1 coordinate system), user's facial orientation (W2 coordinate system) and plane (W3 coordinate system) where display screen
Whether the angular relationship between the three of line direction, confirmation user watch display screen attentively.Specifically, it is assumed that the facial orientation of user and aobvious
The normal vector angle of plane where display screen is N, where user's face between the eyeball fixes direction and facial orientation of coordinate system
Vector angle is L1, then when converting to coordinate system or plane where intelligent mobile terminal, reality of the eyeball relative to mobile terminal
The vector angle L of plane meets where border direction of gaze and display screen：L=N+L1.When the vector angle L meets 0 ° -180 °
In any angle (angle here refers to space multistory angle), can assert unauthorized people's eyeball fixes display.It can be with
Understand, when L=0 ° of the vector angle, watches the preferred plan of display screen attentively for user.The prior art is generally only with RGB phase
Machine obtains facial image；The facial image is only capable of providing a two-dimensional surface information, based on human eye iris to infrared beam
High reflectance feature, conventional solution only can determine that the eyeball fixes method of the relatively described two-dimension human face image, lack
Facial orientation information；Thus lacking suitable mechanism/coordinate system will establish accurately between user eyeball direction of gaze and display screen
Association.The human face image information that RGB-D camera obtains further includes depth information, carries out 3D reconstruct in the facial image to user
Afterwards, processor can be accurate to obtain relative to display panel by the conversion where display screen, face, eyeball between coordinate system
Actual user's eyeball view region, and then promoted processor judge whether user watches the precision of display screen attentively.
When user watches display screen attentively, step 305 is executed, that is, keeps the brightness of current display screen；Conversely, executing step
306。
Step 306, processor is instructed by relevant control, controls RGB-D camera, accomplished continuously or intermittently acquires face in space
Image, and by associated picture Processing Algorithm, feature extraction algorithm, analyze eyeball fixes direction position in multiple facial images
Variation and the variation of facial orientation information, realize the tracking of user eyeball view region；In addition, being directed to user eyeball viewpoint area
The case where display screen, is left in domain, calculates the time that its view region is left.In order to make it easy to understand, in a kind of embodiment, it is assumed that t
At the moment, user's facial image of RGB-D camera acquisition, eyeball view region initially moves off display screen, and RGB-D behind
N facial images of camera acquisition, eyeball view region is not still in the spatial dimension of screen；Then processor can pass through
This n facial images, accurately calculate the time T that user eyeball view region leaves display screen.It should be understood that due to RGB-
The frequency acquisition f of D camera constant (assuming that f=30 frame/second), then user eyeball view region leaves the time T=n/ of display screen
f.In other equivalent implementations, the facial image that RGB-D camera acquires user can also be controlled by the period, realize and use
The time that the tracking of family eyeball view region and user eyeball view region leave display screen calculates, detailed process and RGB phase
The scheme of machine continuous acquisition user's facial image is similar, does not do repetition herein and repeats.
Step 307, preset threshold time T1 allows user eyeball view region to leave display screen most as mobile terminal
Big time value；When user eyeball view region, which leaves display screen time T, is less than crash time T1, processor executes step
S305 continues the brightness and working condition that keep current display screen；Conversely, executing step S308.
Step S308, processor close display screen by associated control procedures, so as to be in suspend mode standby for mobile terminal
State, it is energy saving (electricity).
Fig. 4 is a kind of method flow diagram for realizing real-time printing according to an embodiment of the present invention.In the embodiment, intelligent sliding
The page turn over operation of dynamic terminal, detailed process include：
Step 401 obtains user's facial image for intelligent mobile terminal, obtains user eyeball direction of gaze to step 403
The process of data information and facial orientation information, it is similar to step 303 with the step 301 of Fig. 3 embodiment, weight is no longer done herein
It repeats again.
Step 404, based on user eyeball direction of gaze data (W1 coordinate system), facial orientation data (W2 coordinate system) and
The normal direction data of plane (W3 coordinate system) where display screen, processor may further determine that user eyeball view region is believed
Breath.It is to be appreciated that the view region information refers to：Coordinate system (W3 coordinate system), user where replacing display screen
The intersection location of eyeball view region and display screen.It should be understood that since intelligent mobile terminal is before batch production, it is preposition
The positional relationship of RGB-D camera and display screen is it has been determined that W3 coordinate where W4 coordinate system and display screen i.e. where RGB-D camera
The relationship of system has determined；Therefore, processor can pass through planar process where user eyeball direction of gaze data and display screen
Line vector calculates the intersection location information of user eyeball view region and display screen.Further, specific region is preset, is protected
It is stored in memory；Processor calls algorithm, alignment algorithm by related, and it is default to judge whether user eyeball view region falls within
Specific region.When user's view region falls within preset specific region, processor executes step 405, i.e. realization intelligent sliding
The automatic page turning operation of dynamic terminal；Conversely, processor executes step 406.
Step 406, processor instructs the face figure for controlling RGB-D camera and accomplished continuously or intermittently acquiring user by relevant operation
Picture, to track the eyeball view region of user, detailed process is similar to the step 304 of Fig. 3 embodiment, and it is superfluous not do repetition herein
It states.When the eyeball view region of user is still fallen within the scope of display screen, but view region is not being preset in specific page turning region
When, processor executes step 407, that is, continues the display of the working condition and the current page that keep display screen current；Conversely, executing
Step 408 is to step 410.
Step 408 calculates user eyeball view region for processor and leaves the display screen time, judge whether to step 410
The process of breath screen sleep operation is executed, it is similar to step 308 with the step 306 of Fig. 3 embodiment, it does not do repetition herein and repeats；Area
It is not, when the time T that user eyeball view region leaves display screen is less than preset threshold time T1, processor passes through correlation
The display of working condition and current page that control instruction, algorithm keep display screen current.
Fig. 5 is a kind of smart phone structural schematic diagram containing intelligent read function according to an embodiment of the present invention.Intelligent hand
Machine 500 includes front projection device 520, preposition image collecting device (RGB-D camera) 570.Wherein, projection arrangement 520 is used for
Infrared patterns light beam is projected to object space, image collecting device 570 is covered on object space/subject surface for acquiring
Infrared light pattern, smart phone further include processor (not shown), and processor is used to calculate after receiving infrared light pattern
The depth image of target out, such as the face depth information or profile information or colour information of user.
In some embodiments, smart phone further includes having infrared floodlight 530, can be launched and projection arrangement
The infrared light of 520 emitted infrared patterns light beam phase co-wavelengths can be by that will throw during carrying out man face image acquiring
Image device 520 obtains the depth image and infrared image of user with 530 time-shared switch of infrared floodlight respectively.It is acquired at this time
Infrared image be pure infrared image, for depth image, the facial feature information contained is more obvious.
In some embodiments, it can use the depth camera based on TOF (time flight method) technology, at this time projection arrangement
520 for emitting light pulse, and image collecting device 570 is by receiving light pulse, processor emit for recording impulse and
The utilized time is received, the depth image of user is calculated according to the time.Image collecting device 570 can obtain simultaneously at this time
The depth image and infrared image of user, and therebetween without any parallax.
Smart phone can also include the devices such as earpiece 540, ambient light sensor 550, proximity sensor 560 to realize more
More functions.Such as in some embodiments it is contemplated that infrared light to the harmfulness of human body, when face lean on it is excessively close when, can be with
It is detected by the degree of approach of the proximity sensor 560 to face, the projection of projective module group 520 is closed when too close or is reduced and is thrown
Shadow power.In some embodiments, automatic call can be realized in conjunction with recognition of face and earpiece, such as when device is communication dress
When setting, after device receives incoming call, depth camera required for starting face recognition application opens simultaneously and infrared camera acquisition are deep
Image and infrared image are spent, after being identified by, connects and converses and open the devices such as earpiece to realize call.
Smart phone can also include display screen 510, can be used to show the contents such as data/image/video or use
To carry out touch interaction.It should be understood that answered in intelligently reading, when the eyeball view region of user falls in display screen, i.e.,
When watching display screen attentively, smart phone display screen 510 keeps current display state；When the eyeball view region of user leaves display
When shielding, and being more than preset threshold time T1, smart phone executes breath screen sleep operation.In addition, when user eyeball view region is fallen
When presetting specific page turning region 511 or 512, smart phone executes page turn over operation, otherwise keeps current display page.It needs to manage
Solution, when the eyeball view region of user, which is fallen within, presets specific page turning region 511, processor executes "lower" page turn over operation；
It presets specific page turning region 512 when the eyeball view region of user is fallen within and is, processor executes "upper" page turn over operation.
Smart phone further includes memory (not shown), memory for store as preset threshold time T1 data,
The data informations such as specific page turning zone position information are preset, application program, instruction etc. can also be stored.Such as by intelligence noted earlier
The related applications such as energy reading, are remained in memory in a manner of software program, and when application program needs, processor calling is deposited
Instruction in reservoir simultaneously executes relevant operation.It is understood that application program can also directly be write in the form of instruction code
Enter to be formed into processor specific specific function processor functional module or corresponding independent processor, thus improve execute effect
Rate.In addition, with the continuous development of technology, the boundary between soft and hardware will fade away, therefore heretofore described method
It can be also possible to configure in the form of hardware in a device in a software form.
It is different from conventional terminal, the present invention provides a kind of intelligence and reads application and its intelligent terminal, the beneficial effect is that：It mentions
For the facial image of preposition RGB-D camera acquisition user, and feed back to processor；Processor is obtained by feature extraction algorithm and is used
The eyeball characteristic information at family, and the eyeball fixes directional information of user, facial orientation information are extracted, and then track the eyeball of user
View region (i.e. eyeball fixes display screen position information).Further, processor judges that user eyeball regards by alignment algorithm
The case where whether point region falls in display screen, leave display screen for user eyeball view region, calculates user eyeball viewpoint
Leave the time of display screen in region, it is determined whether execute breath screen sleep operation；Alternatively, processor judges to use by alignment algorithm
Whether family eyeball view region falls in preset specific page turning region, and then determines whether to execute automatic page turning operation.This hair
Bright intelligence reads application and its intelligent terminal is advantageously implemented the non-contact page turning of intelligent terminal, breath screen sleep operation or function,
Be conducive to the manipulation experience for further promoting user.
The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be said that
Specific implementation of the invention is only limited to these instructions.For those skilled in the art to which the present invention belongs, it is not taking off
Under the premise of from present inventive concept, several equivalent substitute or obvious modifications can also be made, and performance or use is identical, all answered
When being considered as belonging to protection scope of the present invention.
Claims (10)
1. a kind of non-touch operating method of intelligent terminal, which is characterized in that include the following steps：
S1. the facial image of user is accomplished continuously or intermittently acquired, the facial image includes face depth image；
S2. the facial image is handled, the eyeball fixes directional information and facial orientation information of the user are extracted, and is tracked
The user eyeball view region；
S3. according to the user eyeball view region, predetermined registration operation is executed.
2. non-touch operating method as described in claim 1, which is characterized in that the facial image further includes RGB-D figure
One of picture, RGB image, infrared image are a variety of.
3. non-touch operating method as described in claim 1, which is characterized in that the step S2 is several by human face five-sense-organ
What feature, determines the eyeball position information of the user to extract eyeball fixes directional information；
Or determine the eyeball position information of the user to extract the high reflectance of infrared beam by face cornea
Eyeball fixes directional information.
4. non-touch operating method as described in claim 1, which is characterized in that the step S2 passes through the face depth
Image reconstructs the three-dimensional face images of the user, extracts the facial orientation information.
5. non-touch operating method as described in claim 1, which is characterized in that the step S3 includes：According to the use
Family eyeball view region is fallen in display screen or outside display screen, is executed and is kept operation or breath screen sleep operation.
6. non-touch operating method as claimed in claim 5, which is characterized in that if the user eyeball view region is fallen in
Outside display screen, the time T that the user eyeball view region leaves display screen is calculated；
When the time T is greater than or equal to preset threshold time T1, breath screen sleep operation is executed；
When the time T is less than preset threshold time T1, the current working condition of holding display screen.
7. non-touch operating method as claimed in claim 6, which is characterized in that by tracking the continuous n user
Eyeball view region falls in the facial image outside display screen, in conjunction with the frequency acquisition f of human face image collecting device, calculates the use
Family eyeball view region falls in the time T=n/f outside display screen.
8. non-touch operating method as described in claim 1, which is characterized in that the step S3 includes：If the user
Eyeball view region is fallen in display screen, calculates the intersection location of the user eyeball view region and display screen；According to described
The relationship of intersection location and default page turning region executes page turn over operation or keeps operation.
9. non-touch operating method as claimed in claim 8, which is characterized in that when the intersection location falls within described preset
When page turning region, the webpage or text that the display screen is shown execute page turn over operation upward or downward；When the intersection location not
At the default page turning region, the current display content of holding display screen.
10. a kind of intelligent terminal, which is characterized in that including：
Display screen, for showing information and providing interaction；
Acquisition device, for acquiring the facial image of user, the facial image includes face depth image；
Processor extracts the eyeball fixes directional information and facial orientation letter of the user for handling the facial image
Breath, and track the user eyeball view region；According to the user eyeball view region, predetermined registration operation is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810495419.7A CN108829242A (en) | 2018-05-22 | 2018-05-22 | Intelligent terminal and its non-touch operating method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810495419.7A CN108829242A (en) | 2018-05-22 | 2018-05-22 | Intelligent terminal and its non-touch operating method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108829242A true CN108829242A (en) | 2018-11-16 |
Family
ID=64149084
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810495419.7A Pending CN108829242A (en) | 2018-05-22 | 2018-05-22 | Intelligent terminal and its non-touch operating method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108829242A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108471486A (en) * | 2018-03-09 | 2018-08-31 | 浙江工业大学 | A kind of intelligent reading operations method and device suitable for electronic viewing aid |
| CN109259434A (en) * | 2018-11-17 | 2019-01-25 | 北京华谛盟家具有限公司 | Intelligent U-shaped conference table |
| CN109727586A (en) * | 2018-12-29 | 2019-05-07 | 重庆秉为科技有限公司 | A kind of eyes protection display device |
| CN109739306A (en) * | 2019-01-04 | 2019-05-10 | 京东方科技集团股份有限公司 | A kind of intelligence desktop bracket and its control method |
| CN109919065A (en) * | 2019-02-26 | 2019-06-21 | 浪潮金融信息技术有限公司 | A method of focus is obtained on the screen using eyeball tracking technology |
| CN110046552A (en) * | 2019-03-21 | 2019-07-23 | 南京华捷艾米软件科技有限公司 | Protect the method for user's eyesight and the device of protection user's eyesight |
| CN110231871A (en) * | 2019-06-14 | 2019-09-13 | 腾讯科技（深圳）有限公司 | Page reading method, device, storage medium and electronic equipment |
| CN110297665A (en) * | 2019-06-10 | 2019-10-01 | 青岛小鸟看看科技有限公司 | A kind of switching method and camera of equipment operating mode |
| CN110333779A (en) * | 2019-06-04 | 2019-10-15 | Oppo广东移动通信有限公司 | Control method, terminal and storage medium |
| CN110488982A (en) * | 2019-08-26 | 2019-11-22 | 业成科技（成都）有限公司 | Through the device of eyeball tracking electronic whiteboard |
| CN112584280A (en) * | 2019-09-27 | 2021-03-30 | 百度在线网络技术（北京）有限公司 | Control method, device, equipment and medium for intelligent equipment |
| CN113473210A (en) * | 2021-07-15 | 2021-10-01 | 北京京东方光电科技有限公司 | Display method, apparatus and storage medium |
| CN113903317A (en) * | 2021-10-19 | 2022-01-07 | Oppo广东移动通信有限公司 | Screen brightness adjusting method and device of electronic equipment and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103135762A (en) * | 2011-11-29 | 2013-06-05 | 三星电子株式会社 | Method for operating user functions based on eye tracking and mobile device adapted thereto |
| CN103699308A (en) * | 2013-12-24 | 2014-04-02 | 惠州Tcl移动通信有限公司 | Automatic paging method and device |
| US20150091794A1 (en) * | 2013-10-02 | 2015-04-02 | Lg Electronics Inc. | Mobile terminal and control method therof |
| CN104598019A (en) * | 2013-10-28 | 2015-05-06 | 欧姆龙株式会社 | Screen operation apparatus and screen operation method |
| CN107909057A (en) * | 2017-11-30 | 2018-04-13 | 广东欧珀移动通信有限公司 | Image processing method, device, electronic equipment and computer-readable recording medium |
| CN107943280A (en) * | 2017-10-30 | 2018-04-20 | 深圳市华阅文化传媒有限公司 | The control method and device of e-book reading |
-
2018
- 2018-05-22 CN CN201810495419.7A patent/CN108829242A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103135762A (en) * | 2011-11-29 | 2013-06-05 | 三星电子株式会社 | Method for operating user functions based on eye tracking and mobile device adapted thereto |
| US20150091794A1 (en) * | 2013-10-02 | 2015-04-02 | Lg Electronics Inc. | Mobile terminal and control method therof |
| CN104598019A (en) * | 2013-10-28 | 2015-05-06 | 欧姆龙株式会社 | Screen operation apparatus and screen operation method |
| CN103699308A (en) * | 2013-12-24 | 2014-04-02 | 惠州Tcl移动通信有限公司 | Automatic paging method and device |
| CN107943280A (en) * | 2017-10-30 | 2018-04-20 | 深圳市华阅文化传媒有限公司 | The control method and device of e-book reading |
| CN107909057A (en) * | 2017-11-30 | 2018-04-13 | 广东欧珀移动通信有限公司 | Image processing method, device, electronic equipment and computer-readable recording medium |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108471486A (en) * | 2018-03-09 | 2018-08-31 | 浙江工业大学 | A kind of intelligent reading operations method and device suitable for electronic viewing aid |
| CN108471486B (en) * | 2018-03-09 | 2019-10-18 | 浙江工业大学 | A kind of intelligent reading operations method and device suitable for electronic viewing aid |
| CN109259434A (en) * | 2018-11-17 | 2019-01-25 | 北京华谛盟家具有限公司 | Intelligent U-shaped conference table |
| CN109259434B (en) * | 2018-11-17 | 2022-05-10 | 北京华谛盟家具有限公司 | Intelligent U-shaped conference table |
| CN109727586A (en) * | 2018-12-29 | 2019-05-07 | 重庆秉为科技有限公司 | A kind of eyes protection display device |
| CN109739306A (en) * | 2019-01-04 | 2019-05-10 | 京东方科技集团股份有限公司 | A kind of intelligence desktop bracket and its control method |
| CN109919065A (en) * | 2019-02-26 | 2019-06-21 | 浪潮金融信息技术有限公司 | A method of focus is obtained on the screen using eyeball tracking technology |
| CN110046552A (en) * | 2019-03-21 | 2019-07-23 | 南京华捷艾米软件科技有限公司 | Protect the method for user's eyesight and the device of protection user's eyesight |
| CN110333779A (en) * | 2019-06-04 | 2019-10-15 | Oppo广东移动通信有限公司 | Control method, terminal and storage medium |
| CN110297665A (en) * | 2019-06-10 | 2019-10-01 | 青岛小鸟看看科技有限公司 | A kind of switching method and camera of equipment operating mode |
| CN110297665B (en) * | 2019-06-10 | 2022-07-19 | 青岛小鸟看看科技有限公司 | Switching method of equipment working modes and camera |
| CN110231871A (en) * | 2019-06-14 | 2019-09-13 | 腾讯科技（深圳）有限公司 | Page reading method, device, storage medium and electronic equipment |
| CN110488982A (en) * | 2019-08-26 | 2019-11-22 | 业成科技（成都）有限公司 | Through the device of eyeball tracking electronic whiteboard |
| CN112584280A (en) * | 2019-09-27 | 2021-03-30 | 百度在线网络技术（北京）有限公司 | Control method, device, equipment and medium for intelligent equipment |
| CN113473210A (en) * | 2021-07-15 | 2021-10-01 | 北京京东方光电科技有限公司 | Display method, apparatus and storage medium |
| CN113903317A (en) * | 2021-10-19 | 2022-01-07 | Oppo广东移动通信有限公司 | Screen brightness adjusting method and device of electronic equipment and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108829242A (en) | Intelligent terminal and its non-touch operating method | |
| EP2984541B1 (en) | Near-plane segmentation using pulsed light source | |
| CN110495819B (en) | Robot control method, robot, terminal, server and control system | |
| EP3845282A1 (en) | Interaction method of application scenario, and mobile terminal and storage medium | |
| CN109660719A (en) | A kind of information cuing method and mobile terminal | |
| CN108712603B (en) | Image processing method and mobile terminal | |
| CN109325450A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN109361869A (en) | A kind of image pickup method and terminal | |
| CN109151180A (en) | A kind of object identifying method and mobile terminal | |
| CN109218648A (en) | A kind of display control method and terminal device | |
| CN109949412A (en) | A kind of three dimensional object method for reconstructing and device | |
| CN109348020A (en) | A kind of photographic method and mobile terminal | |
| CN107786811B (en) | A kind of photographic method and mobile terminal | |
| CN108989678A (en) | A kind of image processing method, mobile terminal | |
| CN109461124A (en) | A kind of image processing method and terminal device | |
| CN108881544A (en) | A kind of method taken pictures and mobile terminal | |
| CN109544445A (en) | A kind of image processing method, device and mobile terminal | |
| CN109688341A (en) | A kind of method for polishing and terminal device | |
| CN109688325A (en) | A kind of image display method and terminal device | |
| WO2022206494A1 (en) | Target tracking method and device | |
| CN107948503A (en) | A kind of photographic method, camera arrangement and mobile terminal | |
| CN110300286A (en) | A kind of image display method and terminal | |
| CN109947243A (en) | Based on the capture of intelligent electronic device gesture and identification technology for touching hand detection | |
| CN109542307A (en) | A kind of image processing method, equipment and computer readable storage medium | |
| CN108833791A (en) | A kind of image pickup method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 11-13 / F, joint headquarters building, high tech Zone, 63 Xuefu Road, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Applicant after: Obi Zhongguang Technology Group Co., Ltd Address before: 12 / F, joint headquarters building, high tech Zone, 63 Xuefu Road, Nanshan District, Shenzhen, Guangdong 518000 Applicant before: SHENZHEN ORBBEC Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181116 |