CN110069138A - The control method and equipment of internet of things equipment - Google Patents
The control method and equipment of internet of things equipment Download PDFInfo
- Publication number
- CN110069138A CN110069138A CN201910367613.1A CN201910367613A CN110069138A CN 110069138 A CN110069138 A CN 110069138A CN 201910367613 A CN201910367613 A CN 201910367613A CN 110069138 A CN110069138 A CN 110069138A
- Authority
- CN
- China
- Prior art keywords
- user
- information
- equipment
- local environment
- controlled
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
This application provides a kind of control method of internet of things equipment and equipment, are applied to the first equipment, by the environmental information for watching information and user's local environment attentively for obtaining user；Wherein, watch information attentively to be formed by equipment that user's sight is fallen in user's local environment；According to the environmental information for watching information and user's local environment attentively of user, apparatus to be controlled is identified；Obtain the control command that user issues；The control command that user is issued is sent by the communication channel between the first equipment and apparatus to be controlled to apparatus to be controlled.Since the application only needs user's sight to fall in the equipment in user's local environment, control of the user to apparatus to be controlled can be realized, compared with prior art, the application realizes the interaction of more convenient, efficient internet of things equipment.
Description
Technical field
The present invention relates to internet of things field more particularly to the control methods and equipment of a kind of internet of things equipment.
Background technique
Internet of Things is that all common objects that can exercise standalone feature is allowed to realize the network to interconnect.With Internet of Things skill
The development of art, Internet of Things are applied to transport and logistics field, industrial manufacture, health medical treatment territory, intelligent environment field, a
The multiple fields such as people and social field.
Currently, the interactive mode of the equipment in Internet of Things is main are as follows: selected manually by mobile devices such as mobile phone, tablet computers
Some equipment in connection list of devices is selected, and sends control command to the equipment, realizes the interaction of mobile device and the equipment.
For example, user can open or close TV, air-conditioning etc. by handset-selected.However, when the equipment that interconnection can be achieved is more and more
Later, user then need from contain it is multiple can manually select out some equipment in the connection list of devices of InterWorking Equipment, this
Interactive mode is obviously not convenient enough, efficient.
Summary of the invention
Based on above-mentioned the deficiencies in the prior art, present applicant proposes a kind of control method of internet of things equipment and equipment, with
Realize more convenient, efficient internet of things equipment interaction.
To achieve the goals above, it is proposed that scheme it is as follows:
First aspect present invention discloses a kind of control method of internet of things equipment, is applied to the first equipment, the Internet of Things
The control method of net equipment includes:
Obtain the environmental information for watching information and user's local environment attentively of user；Wherein, described to watch information attentively and regarded by user
Equipment that line is fallen in user's local environment and formed；
According to the environmental information for watching information and user's local environment attentively of the user, apparatus to be controlled is identified；
Obtain the control command that the user issues；
The control command that the user is issued passes through the communication channel between first equipment and the apparatus to be controlled
It is sent to the apparatus to be controlled.
Optionally, in the control method of above-mentioned internet of things equipment, the information of watching attentively of the user includes: described to be controlled set
The standby world coordinates information under user's local environment；The apparatus to be controlled is that user's sight falls sufficient equipment；Wherein, described
According to the environmental information for watching information and user's local environment attentively of the user, after identifying apparatus to be controlled, further includes:
World coordinates information of the apparatus to be controlled under user's local environment is converted, is that the part of the apparatus to be controlled is sat
Mark information；Wherein, the local coordinate information of the apparatus to be controlled is for illustrating user's blinkpunkt in the position of the apparatus to be controlled.
Optionally, in the control method of above-mentioned internet of things equipment, the control command for obtaining the user and issuing, packet
It includes:
Obtain the operation information of the user of the second equipment acquisition；
According to the operation information of the user and the corresponding relationship of control command, the operation information for obtaining the user is corresponding
Control command.
Optionally, in the control method of above-mentioned internet of things equipment, the operation information of the user, the human eye including user
The operation information of action message and user to second equipment.
Optionally, in the control method of above-mentioned internet of things equipment, the acquisition user's watches information attentively, comprising:
Obtain the user of the second equipment generation watches information attentively；Wherein, second equipment shoots to obtain user images, and
According to the user images, generation is corresponding with the user images to watch information attentively；Institute is included at least in the user images
State the eye figure information of user.
Optionally, in the control method of above-mentioned internet of things equipment, the environmental information for obtaining user's local environment, packet
It includes:
Obtain the image information of user's local environment, wherein the image information of user's local environment includes at least institute
State the image information of the equipment in user's local environment；
Wherein, the environmental information for watching information and user's local environment attentively according to the user, identify to
Control equipment, comprising:
It identifies that the user's watches information attentively, determines that user's sight falls sufficient image information；
User's sight is fallen into sufficient image information, is matched in the image information of user's local environment；
By in the image information of described image local environment, the figure that sufficient image information matches is fallen with user's sight
As corresponding equipment, as the apparatus to be controlled.
Optionally, in the control method of above-mentioned internet of things equipment, the environmental information for obtaining user's local environment, packet
It includes:
Obtain each of user's local environment world coordinates information of the equipment under user's local environment；
Wherein, the environmental information for watching information and user's local environment attentively according to the user, identify to
Control equipment, comprising:
It identifies that the user's watches information attentively, determines the blinkpunkt coordinate information of the user；
By the world coordinates of the blinkpunkt coordinate information of the user and each described equipment under user's local environment
Information is matched；
The equipment that the blinkpunkt coordinate information of world coordinates information and the user under user's local environment is matched,
As the apparatus to be controlled.
Second aspect of the present invention discloses a kind of control equipment of internet of things equipment, the control equipment of the internet of things equipment
For the first equipment, first equipment includes:
First acquisition unit, for obtaining the environmental information for watching information and user's local environment attentively of user；Wherein, described
Watch information attentively to be formed by equipment that user's sight is fallen in user's local environment；
Recognition unit, for the environmental information for watching information and user's local environment attentively according to the user, identification
Apparatus to be controlled out；
Second acquisition unit, the control command issued for obtaining the user；
Transmission unit, the control command for issuing the user pass through first equipment and the apparatus to be controlled
Between communication channel sent to the apparatus to be controlled.
Optionally, in the control equipment of above-mentioned internet of things equipment, the information of watching attentively of the user includes: described to be controlled set
The standby world coordinates information under user's local environment；The apparatus to be controlled is that user's sight falls sufficient equipment；Wherein, the object
The control equipment of networked devices, further includes:
Converting unit, for converting world coordinates information of the apparatus to be controlled under user's local environment, for it is described to
Control the local coordinate information of equipment；Wherein, the local coordinate information of the apparatus to be controlled is for illustrating user's blinkpunkt described
The position of apparatus to be controlled.
Optionally, in the control equipment of above-mentioned internet of things equipment, the second acquisition unit, comprising:
First obtains subelement, the operation information of the user for obtaining the acquisition of the second equipment；
Second acquisition subelement obtains institute for the corresponding relationship of operation information and control command according to the user
State the corresponding control command of operation information of user.
Optionally, in the control equipment of above-mentioned internet of things equipment, the operation information of the user, the human eye including user
The operation information of action message and user to second equipment.
Optionally, in the control equipment of above-mentioned internet of things equipment, the first acquisition unit executes the note for obtaining user
When visual information, it is used for:
Obtain the user of the second equipment generation watches information attentively；Wherein, second equipment shoots to obtain user images, and
According to the user images, generation is corresponding with the user images to watch information attentively；Institute is included at least in the user images
State the eye figure information of user；Second equipment is Eye-tracking devices.
Optionally, in the control equipment of above-mentioned internet of things equipment, the first acquisition unit, which executes, to be obtained locating for user
When the environmental information of environment, for obtaining the image information of user's local environment, wherein the image of user's local environment is believed
Breath includes at least the image information of the equipment in user's local environment；
Wherein, the recognition unit, comprising:
First identification subelement, the user watches information attentively for identification, determines that user's sight falls sufficient image
Information；
First matching unit, for user's sight to be fallen sufficient image information, in the figure of user's local environment
As information is matched；
First determination unit, for being fallen in the image information of described image local environment with user's sight sufficient
The corresponding equipment of the image that image information matches, as the apparatus to be controlled.
Optionally, in the control equipment of above-mentioned internet of things equipment, the first acquisition unit, which executes, to be obtained locating for user
When the environmental information of environment, for obtaining world coordinates of each of user's local environment equipment under user's local environment
Information；
Wherein, the recognition unit, comprising:
Second identification subelement, the user watches information attentively for identification, determines the blinkpunkt coordinate letter of the user
Breath；
Second matching unit, for by the blinkpunkt coordinate information of the user and each described equipment locating for the user
World coordinates information under environment is matched；
Second determination unit, for by under user's local environment world coordinates information and the user blinkpunkt coordinate
The equipment that information matches, as the apparatus to be controlled.
It can be seen from the above technical scheme that in the control method of internet of things equipment provided by the present application, for first
For equipment, the environmental information for watching information and user's local environment attentively of user need to be only obtained, so that it may which letter is watched attentively according to user
The environmental information of breath and user's local environment, identifies apparatus to be controlled.After obtaining the control command that user issues again, user is sent out
Control command out is sent by the communication channel between the first equipment and apparatus to be controlled to apparatus to be controlled.Wherein, watch information attentively
It is formed by equipment that user's sight is fallen in user's local environment.Since the application only needs user's sight to fall in ring locating for user
Control of the user to apparatus to be controlled can be realized in equipment in border, compared with prior art, that the application realizes is more convenient,
The interaction of efficient internet of things equipment.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow diagram of the control method of internet of things equipment disclosed by the embodiments of the present invention；
Fig. 2 is a kind of flow diagram of method for identifying apparatus to be controlled disclosed by the embodiments of the present invention；
Fig. 3 is the flow diagram of the method for another identification apparatus to be controlled disclosed by the embodiments of the present invention；
Fig. 4 is the flow diagram of the control method of another internet of things equipment disclosed by the embodiments of the present invention；
Fig. 5 is a kind of structural schematic diagram of the control equipment of internet of things equipment disclosed by the embodiments of the present invention；
Fig. 6 is the structural schematic diagram of the control equipment of another internet of things equipment disclosed by the embodiments of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The embodiment of the present application discloses a kind of control method of internet of things equipment, is applied to the first equipment, wherein first sets
Standby is to have the function of eyeball tracking, image sensing function and equipment with computing unit, and the first equipment is by obtaining user
The environmental information for watching information and user's local environment attentively, realize control to apparatus to be controlled.
The control method of internet of things equipment provided in this embodiment, refering to fig. 1, specifically includes the following steps:
S101, the environmental information for watching information and user's local environment attentively for obtaining user.
Wherein, watch information attentively to be formed by equipment that user's sight is fallen in user's local environment.Environment locating for user
Refer to the environment in user's range of visibility, the environmental information of user's local environment is the information obtained in user's range of visibility.
It include multiple equipment, such as computer, TV, air-conditioning etc. in user's local environment, the equipment that user's sight is fallen in is apparatus to be controlled.
Specifically, when executing step S101, information is watched attentively by eyeball tracking function generation user by the first equipment, by
First equipment generates the environmental information of user's local environment by image sensing function, then is obtained by the computing unit in the first equipment
Take the environmental information for watching information and user's local environment attentively at family.
Optionally, the information of watching attentively of user is world coordinates information of the apparatus to be controlled under user's local environment, to be controlled to set
It is standby to refer to that user's sight falls sufficient equipment.Wherein, world coordinates information is formed in equipment for illustrating that user's sight falls foot
Blinkpunkt in the position of user's local environment.It can indicate with by a coordinate value under X-axis, Y-axis and Z axis coordinate system,
It can also be indicated with a space vector.
Wherein, user watches information attentively by the eyeball tracking function of the first equipment to generate.Optionally, in the application one
In specific embodiment, the first equipment generates the embodiment for watching information attentively of user are as follows:
For first equipment by shooting user images, and according to user images, generation is corresponding with user images to watch letter attentively
Breath.Wherein, the eye figure information of user is included at least in user images.First equipment is believed by being fitted or extracting the eye figure of user
Breath can be obtained and corresponding with eye figure information watch information attentively.
Optionally, in the application another specific embodiment, the first equipment generates the embodiment for watching information attentively of user
Are as follows:
First equipment is calculated by the position of eyes of user and the three-dimensional direction of visual lines of eyes of user from eyes of user
The three-dimensional sight set out of position fall in apparatus to be controlled and watch information attentively.
Optionally, in one specific embodiment of the application, user's local environment of the computing unit acquisition in the first equipment
Environmental information can be the image information of user's local environment.It specifically, can since the first equipment has the function of image sensing
To shoot the image information for obtaining user's local environment by camera.Wherein, the image information of user's local environment is at least wrapped
Include the image information of the equipment in user's local environment.The image information of the equipment of user's local environment has corresponding close with equipment
System.
Optionally, in the application another specific embodiment, ring locating for the user that the computing unit in the first equipment obtains
The environmental information in border can also be world coordinates letter of each of user's local environment equipment under user's local environment
Breath.First equipment can obtain the world coordinates information of each of user's local environment equipment, i.e., with world coordinates information
Form record the position of each equipment.
It can thus be seen that optionally, in one specific embodiment of the application, when executing step S101:
First equipment can go to generate watching attentively for user there are two the eyeball tracking formula glasses of camera by a configuration
The environmental information of information and user's local environment.The first camera shooting user's eye figure information in first equipment, passes through first
Eyeball tracking function in equipment generate it is corresponding with eye figure information watch information attentively, and the second camera bat in the first equipment
Take the photograph the image information of user's local environment, wherein the image information of user's local environment includes at least in user's local environment
The image information of equipment.
It should be noted that obtaining the specific embodiment of the environmental information for watching information and user's local environment attentively of user
Very much, the scheme including but not limited in the present embodiment shown in step S101.
S102, the environmental information for watching information and user's local environment attentively according to user, identify apparatus to be controlled.
Wherein, apparatus to be controlled refers to that user's sight falls sufficient equipment.
Since, including computing unit, computing unit can watch information and user attentively according to the user of acquisition in the first equipment
The environmental information of local environment, may recognize that apparatus to be controlled.
Optionally, if in step S101, the environmental information of user's local environment of acquisition is the image of user's local environment
Information thens follow the steps a kind of concrete mode of S102, as shown in Figure 2, comprising:
S201, it identifies that user's watches information attentively, determines that user's sight falls sufficient image information.
Wherein, watch that information is the equipment that falls in user's local environment of user's sight and is formed attentively.User's sight falls sufficient
Point is blinkpunkt, it is also assumed that watching the relevant information that information is the blinkpunkt of user attentively.Computing unit root in first equipment
Information is watched attentively according to the user of acquisition, and the sight of available user falls sufficient position, and then can determine that user's sight falls foot
Position image information.It wherein, include user's sight in the image information of the user's local environment obtained in step S101
The image information of sufficient position is fallen, therefore user's sight can be fallen to the image information of sufficient position and the image of user's local environment
Information is further matched.
S202, user's sight is fallen to sufficient image information, is matched in the image information of user's local environment.
It include the image information of the equipment in user's local environment in the image information of user's local environment, and user's sight
Fall sufficient position be user's local environment in apparatus to be controlled position, therefore user's sight can be fallen sufficient image information with
It is matched in the image information of family local environment.
S203, by the image information of image local environment, fall the image that sufficient image information matches with user's sight
Corresponding equipment, as apparatus to be controlled.
Due to including the image information of equipment in user's local environment, and equipment in the image information of user's local environment
There is corresponding relationship with the image information of equipment, therefore according to the corresponding relationship of equipment and equipment image, identify and regarded with user
Line falls the corresponding equipment of image that sufficient image information matches, and using the equipment as apparatus to be controlled.
For example, the computing unit of the first equipment obtains the image of the equipment such as computer, TV, the air-conditioning in user's local environment
Information preserves image information corresponding to each equipment.When the sight of user falls television set of the foot in local environment
Some point on, then can determine that the image information of television set.Again by the figure of TV images information and user's local environment
As information is matched, the equipment matched is determined as television set, and using television set as apparatus to be controlled.
Optionally, if in step S101, the environmental information of user's local environment of acquisition is every in user's local environment
World coordinates information of one equipment under user's local environment, thens follow the steps a kind of concrete mode of S102, such as Fig. 3 institute
Show, comprising:
S301, it identifies that user's watches information attentively, determines the blinkpunkt coordinate information of user.
It due to watching information attentively is formed by equipment that user's sight is fallen in user's local environment, user's sight falls foot
Point be blinkpunkt, it is also assumed that watching the relevant information that information is the blinkpunkt of user attentively.Therefore according to watch information attentively can be with
Determine the blinkpunkt coordinate information of user.And blinkpunkt coordinate information is for illustrating blinkpunkt in the position of user's local environment.
S302, the world coordinates information by the blinkpunkt coordinate information and each equipment of user under user's local environment
It is matched.
Wherein, world coordinates information of each equipment under user's local environment is for illustrating each equipment in user
Position in local environment.By the blinkpunkt coordinate information of user and world coordinates of each equipment under user's local environment
Information is matched, it may be determined that the equipment that the blinkpunkt of user falls in user's local environment.
S303, it is set what the blinkpunkt coordinate information of world coordinates information and user under user's local environment matched
It is standby, as apparatus to be controlled.
The equipment that the blinkpunkt coordinate information of world coordinates information and user under user's local environment matches is user
Sight falls the equipment where sufficient blinkpunkt, as apparatus to be controlled.
It should also be noted that, there are many specific embodiment for executing step S102, including but not limited to the application is real
Apply the scheme of the step S102 exemplified.
Optionally, if the information of watching attentively of the user obtained in step S101 is that apparatus to be controlled is complete under user's local environment
Office coordinate information, then after executing the step S102, can also include:
World coordinates information of the apparatus to be controlled under user's local environment is converted, is the local coordinate information of apparatus to be controlled.
Wherein, the local coordinate information of apparatus to be controlled is for illustrating user's blinkpunkt in the position of the apparatus to be controlled.And
Apparatus to be controlled is then used to illustrate position of the apparatus to be controlled under user's local environment in the world coordinates information under user's local environment
It sets.Wherein, apparatus to be controlled is that user's sight falls sufficient equipment.Computing unit in first equipment is by apparatus to be controlled locating for the user
World coordinates information under environment is converted to the local coordinate information of apparatus to be controlled, may be implemented more diversified to apparatus to be controlled
Control.For example, user can be sat by watching cursor attentively according to part of the cursor in apparatus to be controlled when apparatus to be controlled is computer
Information is marked, realization carries out the cursor on computer display screen the control such as to move, click, and rather than just simple unlatching, closes
The control of computer.
S103, the control command that user issues is obtained.
Wherein, control command is the order that user allows apparatus to be controlled to execute, and can be and opening of device, closing etc. is allowed to order.
And for some equipment with a variety of specific functions, such as computer, TV etc., if after executing step S102, the first equipment
World coordinates information of the conversion apparatus to be controlled under user's local environment is also performed, is the local coordinate information of apparatus to be controlled
Step, the then control command that user issues can also be in addition to opening of device, closing etc. are ordered for some specific of equipment
The order of function.Movement, click of the cursor on computer display screen etc. are ordered for example, realizing.
Optionally, in one specific embodiment of the application, a kind of embodiment of step S103 is executed, comprising:
The operation information for obtaining user obtains user's according to the operation information of user and the corresponding relationship of control command
The corresponding control command of operation information.
Optionally, the operation information of user can be the human eye action information of user.The computing unit of first equipment can be pre-
The corresponding relationship of human eye action information and control command is first set, for example, blinking primary eye is the order that apparatus to be controlled is opened, blinks two
Secondary eye is the order that apparatus to be controlled is closed.The eyeball tracking function of first equipment can collect human eye action information, and first sets
Corresponding relationship of the computing unit according to preset human eye action information and control command in standby, obtains the control of user's sending
System order.
Optionally, the operation information of user can also be user to the operation information of the first equipment.For example, in the first equipment
Remote keying can be had, the first equipment can preset user to the operation of key and the corresponding relationship of control command, for example, pressing
Green key corresponds to the order of apparatus to be controlled unlatching, and the order of apparatus to be controlled closing is corresponded to by red key.Specifically, Ke Yishe
The signal for setting different key sending is different, and unlike signal corresponds to different control commands, the computing unit in the first equipment
According to the corresponding relationship of preset signal and control command, the different signals received are converted into control command.
It should be noted that obtain the concrete mode of the control command that user issues there are also very much, including but not limited to this
Apply for the mode in embodiment shown in step S103.
S104, the control command for issuing user, by the communication channel between the first equipment and apparatus to be controlled to be controlled
Equipment is sent.
Wherein, the communication channel between the first equipment and apparatus to be controlled refers to identification of first equipment by apparatus to be controlled
The communication informations such as number (identification, ID), apparatus to be controlled type and apparatus to be controlled network interface address are set up
The communication channel come.
Specifically, information needed for apparatus to be controlled establishes communication is stored in server, and the computing unit of the first equipment is logical
It crosses after step S103 identifies apparatus to be controlled, information needed for apparatus to be controlled establishes communication is obtained from server, according to be controlled
Information needed for equipment establishes communication establishes the communication channel between the first equipment and apparatus to be controlled, and the first equipment is logical by communication
Road sends control command to apparatus to be controlled.
In the control method of internet of things equipment provided by the present application, for the first equipment, the note of user need to be only obtained
The environmental information of visual information and user's local environment, so that it may be believed according to the environment for watching information and user's local environment attentively of user
Breath, identifies apparatus to be controlled.After obtaining the control command that user issues again, the control command that user is issued is set by first
The standby communication channel between apparatus to be controlled is sent to apparatus to be controlled.Wherein, watch information attentively to be fallen in locating for user as user's sight
Equipment in environment and formed.Since the application only needs user's sight to fall in the equipment in user's local environment, use can be realized
Control of the family to apparatus to be controlled, compared with prior art, the application realize the friendship of more convenient, efficient internet of things equipment
Mutually.
Another embodiment of the application discloses the control method of another internet of things equipment, by the first equipment and the second equipment
Cooperation executes, wherein the first equipment can be the terminal devices such as mobile phone, tablet computer comprising computing unit, the second equipment are
Eye-tracking devices with imaging sensor watch information and user institute attentively by what the first equipment and the second equipment obtained user
Locate the environmental information of environment, realizes the control to apparatus to be controlled.
The control method of another kind internet of things equipment provided by the embodiments of the present application specifically includes following step refering to Fig. 4
It is rapid:
S401, the second equipment generate the environmental information for watching information and user's local environment attentively of user.
Wherein, watch information attentively to be formed by equipment that user's sight is fallen in user's local environment.Optionally, the note of user
Visual information is world coordinates information of the apparatus to be controlled under user's local environment, and apparatus to be controlled refers to that user's sight falls sufficient set
It is standby.Wherein, world coordinates information is used to illustrate that user's sight to fall the sufficient blinkpunkt formed in equipment in user's local environment
Position.It can indicate with by a coordinate value under X-axis, Y-axis and Z axis coordinate system, can also be with a space vector come table
Show.
Since the second equipment is a kind of Eye-tracking devices, the second equipment can be by shooting user images, and root
According to user images, generation is corresponding with user images to watch information attentively.Wherein, believe in user images including at least the eye figure of user
Breath, the second equipment is by fitting or extracts the eye figure information of user, can be obtained and corresponding with eye figure information watches information attentively.
Second equipment can also be calculated by the position of eyes of user and the three-dimensional direction of visual lines of eyes of user from eyes of user
The three-dimensional sight that position is set out, which is fallen in apparatus to be controlled, watches information attentively.
Optionally, the environmental information of user's local environment can be the image information of user's local environment.It sets due to second
Standby is to have the Eye-tracking devices of imaging sensor, and imaging sensor can be camera, therefore can set by second
The image information of standby camera shooting user's local environment.Wherein, the image information of user's local environment includes at least user
The image information of equipment in local environment.The image information and equipment of the equipment of user's local environment have corresponding relationship.Example
If the image of TV corresponds to this equipment of TV, the image of computer corresponds to this equipment of computer etc..
Optionally, the environmental information of user's local environment, can also be each of user's local environment equipment with
World coordinates information under the local environment of family.The global of each of the available user's local environment of second equipment equipment is sat
Information is marked, i.e., records the position of each equipment in the form of world coordinates information.
S402, the first equipment obtain the environment letter for watching information and user's local environment attentively for the user that the second equipment generates
Breath.
It should be noted that step S402 can also regard that the second equipment watches information attentively to the first equipment transmission user as
With the environmental information of user's local environment.Wherein, the first equipment can be the terminal devices such as mobile phone, tablet computer comprising meter
Calculate unit.
S403, the first equipment identify to be controlled set according to the environmental information for watching information and user's local environment attentively of user
It is standby.
Wherein, the information of watching attentively of user includes: world coordinates information of the apparatus to be controlled under user's local environment, to be controlled to set
It is standby to fall sufficient equipment for user's sight.
It should be noted that step S403 is identical as the implementation procedure of step S102 shown in fig. 1 and principle, here not
It repeats again.
The world coordinates information of S404, the first equipment conversion apparatus to be controlled under user's local environment, is apparatus to be controlled
Local coordinate information.
Wherein, the local coordinate information of apparatus to be controlled is for illustrating user's blinkpunkt in the position of apparatus to be controlled.And it is to be controlled
Equipment is then used to illustrate position of the apparatus to be controlled under user's local environment in the world coordinates information under user's local environment.The
World coordinates information of the apparatus to be controlled under user's local environment is converted to the local coordinate information of apparatus to be controlled by one equipment, can
To realize to the more diversified control of apparatus to be controlled.For example, user can be by watching cursor, root attentively when apparatus to be controlled is computer
According to local coordinate information of the cursor in apparatus to be controlled, realization carries out the cursor on computer display screen the control such as to move, click,
Rather than just the simple control opened, close computer.
The operation information of S405, the second equipment acquisition user.
Wherein, the operation information of user can be the human eye action information of user.Since the second equipment sets for eye tracking
It is standby, therefore the human eye action information of user can be acquired by the second equipment.Wherein, human eye action information can be blink, eye closing etc.,
The control different to apparatus to be controlled is realized by the different human eye action of user.
The operation information of user can also be user to the operation information of the second equipment.Such as second in equipment with remote control
Key, the different capable of emitting different signals of remote keying give the first equipment.By user click different keys issue it is different
Signal, to realize control of the user to apparatus to be controlled.
S406, the first equipment obtain the operation information of the user of the second equipment acquisition.
It should be noted that can also regard that the second equipment sends the operation information of user to the first equipment as.
S407, the first equipment obtain the operation letter of user according to the operation information of user and the corresponding relationship of control command
Cease corresponding control command.
Optionally, in one specific embodiment of the application, if the human eye that the operation information that step S406 is obtained is user moves
Make information, then the specific implementation process of step S407 are as follows: the first equipment can preset human eye action information and control command
Corresponding relationship, for example, blinking primary eye is the order that apparatus to be controlled is opened, blinking eye twice is the order that apparatus to be controlled is closed.Second
The eyeball tracking function of equipment can collect human eye action information and be sent to the first equipment, and the first equipment foundation is preset
Human eye action information and control command corresponding relationship, obtain user sending control command.
Optionally, in one specific embodiment of the application, if the operation information of user is operation of the user to the second equipment
Information, the then specific implementation process of step S407 are as follows: the first equipment presets user to the operation information and control of the second equipment
The corresponding relationship of order is made, for example, corresponding to the order of apparatus to be controlled unlatching by green key, corresponds to apparatus to be controlled by red key
The order of closing.Specifically, the signal that different key sending can be set is different, and unlike signal corresponds to different control lives
It enables, operation information is converted into control command according to the corresponding relationship of preset signal and control command by the first equipment.
S408, the control command for issuing user, by the communication channel between the first equipment and apparatus to be controlled to be controlled
Equipment is sent.
It should be noted that step S408 is identical as the implementation procedure of step S104 shown in fig. 1 and principle, here not
It repeats again.
In the control method of internet of things equipment provided by the embodiments of the present application, for the first equipment, to be controlled set is converted
The standby world coordinates information under user's local environment is the local coordinate information of apparatus to be controlled, be may be implemented to apparatus to be controlled more
Diversified control.Wherein, the local coordinate information of apparatus to be controlled is for illustrating user's blinkpunkt in the position of apparatus to be controlled.
In control method based on internet of things equipment disclosed in above-mentioned the embodiment of the present application, the embodiment of the present application is also corresponding public
A kind of control equipment of internet of things equipment is opened, the control equipment of internet of things equipment is the first equipment, and refering to Fig. 5, Internet of Things is set
Standby control equipment 500 specifically includes that first acquisition unit 501, recognition unit 502, second acquisition unit 503 and sends single
Member 504.
First acquisition unit 501, for obtaining the environmental information for watching information and user's local environment attentively of user；Wherein,
Watch information attentively to be formed by equipment that user's sight is fallen in user's local environment.
Optionally, in one specific embodiment of the application, the information of watching attentively of user includes: apparatus to be controlled ring locating for user
World coordinates information under border.Apparatus to be controlled is that user's sight falls sufficient equipment.
Wherein, the control equipment 500 of internet of things equipment, further includes:
Converting unit is apparatus to be controlled for converting world coordinates information of the apparatus to be controlled under user's local environment
Local coordinate information.Wherein, the local coordinate information of apparatus to be controlled is for illustrating user's blinkpunkt in the position of apparatus to be controlled.
Optionally, in one specific embodiment of the application, the execution acquisition user's of first acquisition unit 501 watches information attentively
When, comprising:
Obtain the user of the second equipment generation watches information attentively.Wherein, the second equipment shoots to obtain user images, and according to
User images, generation is corresponding with user images to watch information attentively.In user images include at least user eye figure information, second
Equipment is Eye-tracking devices.
Recognition unit 502 identifies to be controlled for the environmental information for watching information and user's local environment attentively according to user
Equipment.
Optionally, in one specific embodiment of the application, first acquisition unit 501 executes the ring for obtaining user's local environment
When the information of border, it is used for:
Obtain the image information of user's local environment, wherein the image information of user's local environment includes at least user institute
Locate the image information of the equipment in environment.
Wherein, recognition unit 502, comprising:
First identification subelement, user watches information attentively for identification, determines that user's sight falls sufficient image information.
First matching unit, for user's sight to be fallen to sufficient image information, user's local environment image information into
Row matching.
First determination unit, for falling sufficient image information with user's sight in the image information of image local environment
The corresponding equipment of the image to match, as apparatus to be controlled.
Optionally, in one specific embodiment of the application, first acquisition unit 501 executes the ring for obtaining user's local environment
Border information, is used for:
Obtain each of user's local environment world coordinates information of the equipment under user's local environment.
Wherein, recognition unit 502, comprising:
Second identification subelement, user watches information attentively for identification, determines the blinkpunkt coordinate information of user.
Second matching unit, for by the blinkpunkt coordinate information of user and each equipment under user's local environment
World coordinates information is matched.
Second determination unit, for by the blinkpunkt coordinate information of world coordinates information and user under user's local environment
The equipment to match, as apparatus to be controlled.
Second acquisition unit 503, for obtaining the control command of user's sending.
Optionally, in one specific embodiment of the application, second acquisition unit 503, comprising:
First obtains subelement, the operation information of the user for obtaining the acquisition of the second equipment.
Second obtains subelement, for obtaining user's according to the operation information of user and the corresponding relationship of control command
The corresponding control command of operation information.
Optionally, in one specific embodiment of the application, the operation information of user include user human eye action information and
Operation information of the user to the second equipment.
Transmission unit 504, the control command for issuing user, passes through the communication between the first equipment and apparatus to be controlled
Channel is sent to apparatus to be controlled.
It the specific principle of each unit in the control equipment of internet of things equipment disclosed in the embodiments of the present invention and holds
Row process, it is identical as the control method of internet of things equipment disclosed in the embodiments of the present invention, reference can be made to aforementioned present invention is implemented
Corresponding part in the control method of internet of things equipment disclosed in example, is not discussed here.
In the control equipment of internet of things equipment provided by the present application, watching attentively for user is obtained by first acquisition unit 501
The environmental information of information and user's local environment, recognition unit 502 can watch information and user's local environment attentively according to user
Environmental information, identify apparatus to be controlled.The control command that user issues is obtained further through second acquisition unit 503, is sent single
The control command that member 504 issues user is sent by the communication channel between the first equipment and apparatus to be controlled to apparatus to be controlled.
Wherein, watch information attentively to be formed by equipment that user's sight is fallen in user's local environment.Since the application only needs user's sight
The equipment in user's local environment is fallen in, control of the user to apparatus to be controlled can be realized, compared with prior art, the application is real
The interaction of more convenient, efficient internet of things equipment is showed.
Refering to Fig. 6, the embodiment of the present application discloses the control equipment 600 of another internet of things equipment, internet of things equipment
Control equipment 600 is the first equipment, specifically includes that first acquisition unit 601, recognition unit 602, converting unit 603, second are obtained
Take unit 604 and transmission unit 605.
First acquisition unit 601, for obtaining the environmental information for watching information and user's local environment attentively of user.
Wherein, watch information attentively to be formed by equipment that user's sight is fallen in user's local environment.User's watches information attentively
It include: world coordinates information of the apparatus to be controlled under user's local environment.Apparatus to be controlled is that user's sight falls sufficient equipment.
It should be noted that the implementation procedure and principle of first acquisition unit 601 and the first acquisition unit shown in Fig. 5
501 is identical, and details are not described herein again.
Recognition unit 602 identifies to be controlled for the environmental information for watching information and user's local environment attentively according to user
Equipment.
It should be noted that the implementation procedure and principle of recognition unit 602 are identical as the recognition unit 502 shown in Fig. 5, this
Place repeats no more.
Converting unit 603 is apparatus to be controlled for converting world coordinates information of the apparatus to be controlled under user's local environment
Local coordinate information.Wherein, the local coordinate information of apparatus to be controlled is for illustrating user's blinkpunkt in the position of apparatus to be controlled.
Second acquisition unit 604, for obtaining the control command of user's sending.
It should be noted that the implementation procedure and principle of second acquisition unit 604 and the second acquisition unit shown in Fig. 5
503 is identical, and details are not described herein again.
Transmission unit 605, the control command for issuing user, passes through the communication between the first equipment and apparatus to be controlled
Channel is sent to the apparatus to be controlled.
It should be noted that the implementation procedure and principle of transmission unit 605 are identical as the transmission unit 504 shown in Fig. 5, this
Place repeats no more.
In the control equipment of internet of things equipment provided by the embodiments of the present application, converting unit 603 convert apparatus to be controlled with
World coordinates information under the local environment of family is the local coordinate information of apparatus to be controlled, be may be implemented more diversified to apparatus to be controlled
Control.Wherein, the local coordinate information of apparatus to be controlled is for illustrating user's blinkpunkt in the position of apparatus to be controlled.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (14)
1. a kind of control method of internet of things equipment, which is characterized in that be applied to the first equipment, the control of the internet of things equipment
Method includes:
Obtain the environmental information for watching information and user's local environment attentively of user；Wherein, described to watch information attentively and fallen by user's sight
Equipment in user's local environment and formed；
According to the environmental information for watching information and user's local environment attentively of the user, apparatus to be controlled is identified；
Obtain the control command that the user issues；
The control command that the user is issued, by the communication channel between first equipment and the apparatus to be controlled to institute
State apparatus to be controlled transmission.
2. the method according to claim 1, wherein the information of watching attentively of the user includes: the apparatus to be controlled
World coordinates information under user's local environment；The apparatus to be controlled is that user's sight falls sufficient equipment；Wherein, the basis
The environmental information for watching information and user's local environment attentively of the user, after identifying apparatus to be controlled, further includes:
World coordinates information of the apparatus to be controlled under user's local environment is converted, is that the local coordinate of the apparatus to be controlled is believed
Breath；Wherein, the local coordinate information of the apparatus to be controlled is for illustrating user's blinkpunkt in the position of the apparatus to be controlled.
3. the method according to claim 1, wherein the control command for obtaining the user and issuing, comprising:
Obtain the operation information of the user of the second equipment acquisition；
According to the operation information of the user and the corresponding relationship of control command, the corresponding control of operation information of the user is obtained
System order.
4. according to the method described in claim 3, it is characterized in that, the operation information of the user, the human eye including user are dynamic
Make information and user to the operation information of second equipment.
5. the method according to claim 1, wherein the acquisition user's watches information attentively, comprising:
Obtain the user of the second equipment generation watches information attentively；Wherein, second equipment shoots to obtain user images, and according to
The user images, generation is corresponding with the user images to watch information attentively；The use is included at least in the user images
The eye figure information at family.
6. the method according to claim 1, wherein the environmental information for obtaining user's local environment, comprising:
Obtain the image information of user's local environment, wherein the image information of user's local environment includes at least the use
The image information of equipment in the local environment of family；
Wherein, the environmental information for watching information and user's local environment attentively according to the user, identifies to be controlled set
It is standby, comprising:
It identifies that the user's watches information attentively, determines that user's sight falls sufficient image information；
User's sight is fallen into sufficient image information, is matched in the image information of user's local environment；
By in the image information of described image local environment, the image pair that sufficient image information matches is fallen with user's sight
The equipment answered, as the apparatus to be controlled.
7. the method according to claim 1, wherein the environmental information for obtaining user's local environment, comprising:
Obtain each of user's local environment world coordinates information of the equipment under user's local environment；
Wherein, the environmental information for watching information and user's local environment attentively according to the user, identifies to be controlled set
It is standby, comprising:
It identifies that the user's watches information attentively, determines the blinkpunkt coordinate information of the user；
By the world coordinates information of the blinkpunkt coordinate information of the user and each described equipment under user's local environment
It is matched；
The equipment that the blinkpunkt coordinate information of world coordinates information and the user under user's local environment is matched, as
The apparatus to be controlled.
8. a kind of control equipment of internet of things equipment, which is characterized in that the control equipment of the internet of things equipment is the first equipment,
First equipment includes:
First acquisition unit, for obtaining the environmental information for watching information and user's local environment attentively of user；Wherein, described to watch attentively
Information is formed by the equipment that user's sight is fallen in user's local environment；
Recognition unit, for the environmental information for watching information and user's local environment attentively according to the user, identify to
Control equipment；
Second acquisition unit, the control command issued for obtaining the user；
Transmission unit, the control command for issuing the user, by between first equipment and the apparatus to be controlled
Communication channel sent to the apparatus to be controlled.
9. equipment according to claim 8, which is characterized in that the information of watching attentively of the user includes: the apparatus to be controlled
World coordinates information under user's local environment；The apparatus to be controlled is that user's sight falls sufficient equipment；Wherein, the Internet of Things
The control equipment of net equipment, further includes:
Converting unit to be controlled is set to be described for converting world coordinates information of the apparatus to be controlled under user's local environment
Standby local coordinate information；Wherein, the local coordinate information of the apparatus to be controlled is for illustrating user's blinkpunkt described to be controlled
The position of equipment.
10. equipment according to claim 8, which is characterized in that the second acquisition unit, comprising:
First obtains subelement, the operation information of the user for obtaining the acquisition of the second equipment；
Second acquisition subelement obtains the use for the corresponding relationship of operation information and control command according to the user
The corresponding control command of the operation information at family.
11. equipment according to claim 10, which is characterized in that the operation information of the user, the human eye including user
The operation information of action message and user to second equipment.
12. equipment according to claim 8, which is characterized in that the first acquisition unit, which executes, obtains watching attentively for user
When information, it is used for:
Obtain the user of the second equipment generation watches information attentively；Wherein, second equipment shoots to obtain user images, and according to
The user images, generation is corresponding with the user images to watch information attentively；The use is included at least in the user images
The eye figure information at family；Second equipment is Eye-tracking devices.
13. equipment according to claim 8, which is characterized in that the first acquisition unit, which executes, obtains ring locating for user
When the environmental information in border, for obtaining the image information of user's local environment, wherein the image information of user's local environment
Including at least the image information of the equipment in user's local environment；
Wherein, the recognition unit, comprising:
First identification subelement, the user watches information attentively for identification, determines that user's sight falls sufficient image information；
First matching unit is believed for user's sight to be fallen sufficient image information in the image of user's local environment
Breath is matched；
First determination unit, for falling sufficient image with user's sight in the image information of described image local environment
The corresponding equipment of the image that information matches, as the apparatus to be controlled.
14. equipment according to claim 8, which is characterized in that the first acquisition unit, which executes, obtains ring locating for user
When the environmental information in border, for obtaining world coordinates letter of each of user's local environment equipment under user's local environment
Breath；
Wherein, the recognition unit, comprising:
Second identification subelement, the user watches information attentively for identification, determines the blinkpunkt coordinate information of the user；
Second matching unit, for by the blinkpunkt coordinate information of the user and each described equipment in user's local environment
Under world coordinates information matched；
Second determination unit, for by under user's local environment world coordinates information and the user blinkpunkt coordinate information
The equipment to match, as the apparatus to be controlled.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910367613.1A CN110069138B (en) | 2019-05-05 | 2019-05-05 | Control method and device for Internet of things equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910367613.1A CN110069138B (en) | 2019-05-05 | 2019-05-05 | Control method and device for Internet of things equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110069138A true CN110069138A (en) | 2019-07-30 |
| CN110069138B CN110069138B (en) | 2022-09-20 |
Family
ID=67369882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910367613.1A Active CN110069138B (en) | 2019-05-05 | 2019-05-05 | Control method and device for Internet of things equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110069138B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007206966A (en) * | 2006-02-01 | 2007-08-16 | Pioneer Electronic Corp | Equipment controller, equipment control method and equipment control program |
| CN105677040A (en) * | 2016-02-17 | 2016-06-15 | 深圳创维数字技术有限公司 | Terminal control method, device and wearable device |
| CN105867603A (en) * | 2015-12-08 | 2016-08-17 | 乐视致新电子科技（天津）有限公司 | Eye-controlled method and device |
| CN106462231A (en) * | 2014-03-17 | 2017-02-22 | Itu 商业发展公司 | Computer-implemented gaze interaction method and apparatus |
| CN108762492A (en) * | 2018-05-14 | 2018-11-06 | 歌尔科技有限公司 | Method, apparatus, equipment and the storage medium of information processing are realized based on virtual scene |
-
2019
- 2019-05-05 CN CN201910367613.1A patent/CN110069138B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007206966A (en) * | 2006-02-01 | 2007-08-16 | Pioneer Electronic Corp | Equipment controller, equipment control method and equipment control program |
| CN106462231A (en) * | 2014-03-17 | 2017-02-22 | Itu 商业发展公司 | Computer-implemented gaze interaction method and apparatus |
| CN105867603A (en) * | 2015-12-08 | 2016-08-17 | 乐视致新电子科技（天津）有限公司 | Eye-controlled method and device |
| CN105677040A (en) * | 2016-02-17 | 2016-06-15 | 深圳创维数字技术有限公司 | Terminal control method, device and wearable device |
| CN108762492A (en) * | 2018-05-14 | 2018-11-06 | 歌尔科技有限公司 | Method, apparatus, equipment and the storage medium of information processing are realized based on virtual scene |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110069138B (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104081317B (en) | Information processing equipment and information processing method | |
| CN107168537B (en) | Cooperative augmented reality wearable operation guidance method and system | |
| CN107390863B (en) | Device control method and device, electronic device and storage medium | |
| US20190129607A1 (en) | Method and device for performing remote control | |
| JP2017515253A (en) | Numerical transfer method, terminal and cloud server | |
| CN110275619A (en) | The method and its head-mounted display of real-world object are shown in head-mounted display | |
| CN107707825B (en) | A kind of panorama shooting method, mobile terminal and computer readable storage medium | |
| CN110248254A (en) | Display control method and Related product | |
| CN110209285B (en) | Sand table display system based on gesture control | |
| EP3547083B1 (en) | Information processing program, information processing method, and information processing system | |
| US20200209951A1 (en) | Information processing system, information processing method, and program | |
| CN104881114A (en) | Angle rotation real-time matching method based on try wearing of 3D (three dimensional) glasses | |
| CN110471804A (en) | Apparatus debugging method, device and electronic equipment | |
| KR20170012979A (en) | Electronic device and method for sharing image content | |
| CN109839827B (en) | Gesture recognition intelligent household control system based on full-space position information | |
| CN107426508A (en) | A kind of method that long-range security inspection is realized using AR | |
| CN102571624A (en) | Real-time communication system and relevant calculator readable medium | |
| CN105938665A (en) | Remote audio and video operation demonstration system | |
| CN107332977A (en) | The method and augmented reality equipment of augmented reality | |
| CN103257703B (en) | A kind of augmented reality device and method | |
| CN109753315A (en) | A kind of smart machine interactive content editor implementation method and storage medium | |
| CN112887601B (en) | Shooting method and device and electronic equipment | |
| CN104023207A (en) | Live communication terminal capable of automatically carrying out bidirectional communication after unidirectional call, method and tool | |
| CN105183142B (en) | A kind of digital information reproducing method of utilization space position bookbinding | |
| CN110069138A (en) | The control method and equipment of internet of things equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |