CN107506037A - A kind of method and apparatus of the control device based on augmented reality - Google Patents
A kind of method and apparatus of the control device based on augmented reality Download PDFInfo
- Publication number
- CN107506037A CN107506037A CN201710728237.5A CN201710728237A CN107506037A CN 107506037 A CN107506037 A CN 107506037A CN 201710728237 A CN201710728237 A CN 201710728237A CN 107506037 A CN107506037 A CN 107506037A
- Authority
- CN
- China
- Prior art keywords
- user
- information
- equipment
- target device
- visual range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Abstract
This application provides a kind of method of the control device based on augmented reality, applied in Intelligent worn device, this method includes：When detecting that user eyeball focuses on an equipment, it is equipment to be identified to determine the equipment；Equipment to be identified is identified as target device, and establishes and communicates to connect with the target device；The information of target device is obtained, and described information is added in the real scene of user's visual range and shown；When receiving user the control menu in target device information being operated by intelligent interaction mode, operation behavior is analyzed, generates the control instruction to target device；By the communication connection of foundation, the control instruction of generation is sent into target device performs target device.A kind of device of the control device based on enhancing display is also proposed based on same inventive concept, in the embodiment of the present application so that user is more convenient in free space to be interacted with smart machine, so as to improve Consumer's Experience.
Description
Technical field
The present invention relates to communication technical field, the method and dress of more particularly to a kind of control device based on augmented reality
Put.
Background technology
At present, with science and technology development and Consumer's Experience demand lifted, smart home become a hot discussion and
The topic of research, meanwhile, the product of smart home field emerges in an endless stream.
Though however, title of the current smart home product titled with " intelligence ", with " intelligence " truly but
Greatly differ from each other, most product is all user after simply building the communication network between mobile intelligent terminal and intelligent appliance
By mobile intelligent terminal, such as mobile phone, tablet personal computer etc., control command is sent to control intelligent appliance, it is substantially simply distant
Control the transplanting of device function；
In addition, smart home product user experience instantly is poor, the operation interface on mobile intelligent terminal is excessively cumbersome, greatly
Selected and operated by menu layer by layer more, this greatly reduces Consumer's Experience, and for the elderly, they are right
The use of smart mobile phone etc. is more not familiar, while learning ability is also relatively weaker, smart home product pair in the market
, it is necessary to which longer time goes the guidance of layman to go down to learn to use for them, in this case, old man may dislike
Bother and directly abandon using；In addition, for some illiterate crowds, such as child, it is also difficult to go to the current intelligent family of operation
Occupy the operation interface of product.
A kind of method that intelligent home device is controlled by VR equipment is provided in existing realization, VR equipment identification user is defeated
The gesture information entered, by the gesture information with prestoring for controlling the control gesture information of intelligent home device to be compared
Compared with being controlled according to the comparative result to intelligent home device.
The program identifies the gesture information of user's input by VR equipment, controls the switch of intelligent home device, realizes
Linkage between VR equipment and real world object, but use virtual reality technology, user without visual operation interface is provided
Bad with interacting for reality, user experience is bad.
A kind of visible i.e. controllable Intelligent housing scheme is additionally provided in existing realization, control system is utilized and caught on the spot
The instant image recognition of the intelligent appliance obtained goes out the intelligent appliance, and then generation is based on the instant image of the intelligent appliance or preset figure
The pseudo-entity operation interface of picture and/or auxiliary operation interface, then user is controllable intelligent appliance；
The program but this method need the mode of user manually to carry out equipment IMAQ, and use traditional side
Formula is interacted to control intelligent terminal with intelligent mobile terminal.
The content of the invention
In view of this, the application provides a kind of method and apparatus of the control device based on augmented reality, it is possible to increase uses
Experience at family.
In order to solve the above technical problems, what the technical scheme of the application was realized in：
A kind of method of the control device based on augmented reality, applied in Intelligent worn device, this method includes：
When detecting that user eyeball focuses on an equipment, it is equipment to be identified to determine the equipment；
Equipment to be identified is identified as target device, and establishes and communicates to connect with the target device；
The information of target device is obtained, and described information is added in the real scene of user's visual range and shown；
When receiving user the control menu in target device information being operated by intelligent interaction mode, to operation
Behavior is analyzed, and generates the control instruction to target device；
By the communication connection of foundation, the control instruction of generation is sent into target device performs target device.
A kind of device of the control device based on augmented reality, applied in Intelligent worn device, the device includes：It is determined that
Unit, recognition unit, establish unit, display unit, receiving unit, processing unit and transmitting element；
The determining unit, for when detecting that user eyeball focuses on an equipment, determining that the equipment sets to be to be identified
It is standby；
The recognition unit, for identifying the equipment to be identified of the determining unit determination as target device；
The connection unit, the target device for being identified with the recognition unit establish communication connection；
The display unit, it is added to user's visual range for obtaining the information of target device, and by described information
Shown in real scene；
The receiving unit, for receiving the operation of user；
The processing unit, for being believed when the receiving unit receives user by intelligent interaction mode target device
When control menu in breath is operated, operation behavior is analyzed, generates the control instruction to target device；
The transmitting element, for by the communication connection established unit and established, the processing unit to be generated
Control instruction, which is sent to target device, performs target device.
As can be seen from the above technical solution, the intelligence paid close attention in the application by user's ocular focusing point Intelligent Recognition user
Equipment, the information superposition of smart machine is shown in the real scene of user viewing area, there is provided intelligent interaction mode for
Family operates.The relevant information of multiple smart machines can be added in the real scene of user viewing area by the present invention so that use
Family is more convenient in free space to be interacted with smart machine, so as to improve Consumer's Experience.
Brief description of the drawings
Fig. 1 is the schematic flow sheet of the control device based on enhancing display in the embodiment of the present application；
Fig. 2 is that schematic diagram that equipment to be identified is refrigerator is identified in the embodiment of the present application；
Fig. 3 is a kind of schematic diagram for closing Overlapping display information in the embodiment of the present application in relative mode；
Fig. 4 is that the region of Overlapping display information under absolute mode in the embodiment of the present application is shown in outside user's visual range
Schematic diagram；
Fig. 5 is the schematic diagram of scalability information viewing area in the embodiment of the present application；
Fig. 6 is the schematic diagram of mobile message viewing area in the embodiment of the present application；
Fig. 7 is the apparatus structure schematic diagram for being applied to above-mentioned technology in the embodiment of the present application.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and examples,
Technical scheme is described in detail.
A kind of method of the control device based on augmented reality is provided in the embodiment of the present application, applied to intelligent helmet
On, by the smart machine of user's ocular focusing point Intelligent Recognition user concern, the information superposition of smart machine is shown to use
In the real scene of family viewing area, there is provided intelligent interaction mode operates for user.The present invention can be by the phase of multiple smart machines
Information superposition is closed into the real scene of user viewing area so that user is more convenient in free space to enter with smart machine
Row interaction, so as to improve Consumer's Experience.
It is used to control each intelligent home device using intelligent helmet in the embodiment of the present application, is being controlled it
Before, establish and communicate to connect with the intelligent home device, after finishing control, the communication connection established with it can be disconnected.
Wherein, intelligent helmet can be glasses, helmet etc..
Below in conjunction with the accompanying drawings, the process of the control device based on augmented reality in the embodiment of the present application is described in detail.
Referring to Fig. 1, Fig. 1 is the schematic flow sheet of the control device based on enhancing display in the embodiment of the present application.Specific step
Suddenly it is：
Step 101, when intelligent helmet detects that user eyeball focuses on an equipment, it is to be identified to determine the equipment
Equipment.
Intelligent helmet detects that user eyeball focuses on an equipment in the present embodiment, including：
Determine whether current user state or behavior meet preparatory condition, if it is, determining that the equipment is to be identified
Equipment.
Following either conditions that the preparatory condition configures for satisfaction：Default limb action, default voice, user eyeball are gathered
The time of a burnt equipment is up to the first preset time threshold.
For preset limb action (any position on the human body such as the action of hand, the action of eyes, action of head it is dynamic
Make), finger is such as used to some equipment, it is determined that the equipment of active user's eye gaze is the equipment that user eyeball focuses on；
It is refrigerator as keyword be present in sound that user is currently sent, it is determined that active user for presetting voice
The equipment of eye gaze is the equipment that user eyeball focuses on；
Above two preparatory condition, it is the situation that equipment in user while also using eye gaze.
For user eyeball focus on an equipment time up to the first preset time threshold, specifically, user eyeball focuses on
Time in the region of default size reaches the first preset time threshold, it is determined that the equipment of active user's ocular focusing one.
The above-mentioned several examples for being embodied as the application and user eyeball focusing one equipment offer being provided, however it is not limited to above-mentioned
Description.
Step 102, the intelligent helmet identifies equipment to be identified as target device, and is built with the target device
Vertical communication connection.
The intelligent helmet identifies that equipment to be identified is implemented as：Using intelligent helmet to be identified
Equipment is taken pictures, and the household of the intelligent home device with being locally stored carries out images match, identifies equipment tool to be identified
Which kind of intelligent home device body is.
The communication connection established between intelligent helmet and target device is wireless connection, specific connection procedure without
Limitation.
Referring to Fig. 2, Fig. 2 is that schematic diagram that equipment to be identified is refrigerator is identified in the embodiment of the present application.It is pre- when meeting
If during condition, the location shooting such as picture of refrigerator is realized in user, in case carrying out picture recognition.
Step 103, the intelligent helmet obtains the information of target device, and described information is added to the visual model of user
Shown in the real scene enclosed.
After intelligent helmet establishes communication connection with target device, the information of equipment can be obtained from target device,
When can also use target device every time, the relevant information of target device is being locally stored, it is necessary to when showing, directly in local
Obtain.
The information of target device includes：The essential information of equipment, the status information of equipment, control menu etc., for difference
Intelligent home device will also include some customized informations.
Such as air-conditioning, its information can include the essential information of air-conditioning：Indoor temperature, humidity；The status information of air-conditioning：Open,
Or close, it is presently in mode of operation；Control menu：Configure temperature, wind speed and wind direction etc..
If the bar number of the information of target device is more than default number of branches threshold value, i.e. the information content is relatively more, then provides the user
User slides or page turning display function, as user's Pagination Display, or all information is shown by way of drop-down.
Step 104, the intelligent helmet receives user by intelligent interaction mode to the control in target device information
When menu processed is operated, operation behavior is analyzed, generates the control instruction to target device.
Intelligent interaction mode in the embodiment of the present application is：Limb action, and/or, voice.
Limb action such as gesture motion, blink, shake the head.
As user by gesture motion (click) select control menu in opening refrigerator doors, then intelligent helmet should
Gesture motion generation control instruction be：Open refrigerator doors.
Step 105, the control instruction of generation is sent to target and set by the intelligent helmet by the communication connection of foundation
It is standby to perform target device.
In the embodiment of the present application, the information display methods of two kinds of superpositions can be provided the user：Relative mode and absolute mould
Formula.
When user selects relative mode, the region of Overlapping display information is shown in user's visual range always.
During specific implementation, this kind of stacked system can be arranged to default behavior, i.e., when user does not select, with this kind of side
The region of formula Overlapping display information.
This display mode is that user moves anyway, and the region of Overlapping display information is shown in the visual model of user all the time
In enclosing.
After user terminates to the control of target device, Overlapping display information is closed by intelligent interaction mode, and disconnect
With the connection of target device.
Referring to Fig. 3, Fig. 3 is a kind of schematic diagram for closing Overlapping display information in the embodiment of the present application in relative mode.
The information that gesture operation closes Overlapping display is given by Fig. 3, it is no longer aobvious in user's visual range after closing
Show the information of superposition.
When user selects absolute mode, and user's visual range changes, the region of Overlapping display information is shown in change
Outside user's visual range afterwards.
Here user's visual range change can set a threshold value, when changing much regions, such as laterally, or longitudinal direction
Change how long apart from when, make the information of Overlapping display no longer in user's visual range；User can also be regarded as to folded
The presentation of information added determines that this position no longer changes after certain position, and the visual range of user, which changes to, can no longer see
The information of this superposition.
Referring to Fig. 4, it is visual that Fig. 4 is that the region of Overlapping display information under absolute mode in the embodiment of the present application is shown in user
Schematic diagram outside scope.
In Fig. 4 due to user selection be absolute mode, when user visual range change after, the area of Overlapping display information
Domain can not be in the visual range of user.
When the information of Overlapping display is being more than the second preset time threshold at the visual range not shown time of user, close
The display of information is closed, and disconnects the communication connection with target device.
Here a kind of a kind of triggering mode of the display of closing information in absolute mode is provided, but not limited to this.
When can also be in visual range of the information in user of Overlapping display, the information of Overlapping display be actively closed, and is disconnected and mesh
The communication connection of marking device.
In specific implementation, it can also move, scale the information for being superimposed upon and being shown in the real scene of user's visual range
The region at place.
Referring to Fig. 5, Fig. 5 is the schematic diagram of scalability information viewing area in the embodiment of the present application.Pass through intelligent interaction in Fig. 5
Mode (gesture operation) scaling is superimposed upon the region (Overlapping display where the information shown in the real scene of user's visual range
Area), you can so that viewing area to be amplified, viewing area can also be reduced, to improve Consumer's Experience.
Referring to Fig. 6, Fig. 6 is the schematic diagram of mobile message viewing area in the embodiment of the present application.Pass through intelligent interaction in Fig. 6
Mode (gesture operation) moves superimposed on the region (Overlapping display where the information shown in the real scene of user's visual range
Area) position, i.e., viewing area is moved to user and conveniently browsed, or do not influence the position currently done work.
The control that above-described embodiment is carried out for an intelligent home device, in a particular application, can also be directed to simultaneously
Multiple intelligent home devices are controlled.
Three kinds of application scenarios are given below, to further describe the control device based on augmented reality of the application offer
Process.
Embodiment one
User cooks in kitchen, and operating desk is separated by a distance with refrigerator.User in refrigerator after dish is taken out, it is necessary to tie
Close the menu function that intelligent refrigerator provides to be cooked, select relevant information to be shown as relative mode under the application scenarios.
User wears intelligent helmet after operation edge of table finishes preparation, watches the intelligent refrigerator of distant place attentively；
Head-wearing type intelligent equipment identifies the ocular focusing refrigerator of user, by image recognition to smart machine progress
Match somebody with somebody, after identifying refrigerator, establish and communicate to connect with the refrigerator.
The relevant information of refrigerator is shown in the real scene of the viewing area of intelligent helmet, and user passes through intelligent friendship
Mutual mode selects the menu and Overlapping display of refrigerator recommendation.
Position and the size of menu overlap-add region are adjusted by gesture motion so that Overlapping display region will not block use
The sight at family.
In cooking process, if user thinks a few garnishes of casual addition, ice can be checked again by intelligent interaction mode
The information of case, check in refrigerator whether oneself desired vegetable.
After user cooks, pass through the connection between intelligent interaction mode (such as gesture) closing and intelligent refrigerator.
Embodiment two：
User is sitting in amusement and leisure in parlor, thinks that indoor temperature is too high, looks on the bright side of things and opens air-conditioning, adjusts indoor temperature.This should
Absolute mode is used with scene.
User wears head-wearing type intelligent equipment, watches the intelligent air condition of parlor corner installation attentively；
Intelligent helmet identifies that the time of the ocular focusing air-conditioning of user reaches preset time, passes through image recognition pair
Equipment is matched, and after identifying intelligent air condition, is established and is communicated to connect with the intelligent air condition；
User browses the intelligent air condition essential information got, includes the information such as temperature, humidity of current indoor.
User transfers the actions menu of intelligent air condition, configures suitable temperature, wind speed and wind direction etc..
User is after intelligent air condition normal work is confirmed, the state before direction of visual lines is returned to again；In absolute mould
Under formula, the information and menu of intelligent air condition are shown outside the visual line of sight of active user；After certain time, Overlapping display with
Connection is automatic to close.
Embodiment three
User's self-driving is gone on a journey, and the short time leaves vehicle without closing skylight, and the unexpected big change of weather, it is necessary to close day in time
Window.Absolute mode may be selected in the pattern.
User wears intelligent helmet, watches vehicle nearby attentively；
Head-wearing type intelligent equipment identifies the ocular focusing vehicle of user, by image recognition to smart machine progress
Match somebody with somebody, after identifying vehicle, establish and communicate to connect with the intelligent vehicle；
User transfers the actions menu of intelligent vehicle, and skylight is closed in selection.
User is after confirming that skylight is closed, the state before direction of visual lines is returned to again；In absolute mode, vehicle
Control menu is shown outside the visual line of sight of active user；After certain time, Overlapping display is with being connected automatic closing.
Based on same inventive concept, the application also proposes a kind of device of the control device based on increase display, application
In on intelligent helmet.Referring to Fig. 7, Fig. 7 is the apparatus structure schematic diagram for being applied to above-mentioned technology in the embodiment of the present application.Should
Device includes：Determining unit 701, recognition unit 702, establish unit 703, display unit 704, receiving unit 705, processing unit
706 and transmitting element 707；
Determining unit 701, for when detecting that user eyeball focuses on an equipment, determining that the equipment sets to be to be identified
It is standby；
Recognition unit 702, for identifying the equipment to be identified of the determination of determining unit 701 as target device；
Connection unit 703, the target device for being identified with recognition unit 702 establish communication connection；
Display unit 704, the information of the target device for obtaining the identification of recognition unit 702, and described information is superimposed
Shown into the real scene of user's visual range；
Receiving unit 705, for receiving the operation of user；
Processing unit 706, for receiving user by intelligent interaction mode to target device information when receiving unit 705
In control menu when being operated, operation behavior is analyzed, generates the control instruction to target device；
Transmitting element 707, for the communication connection established by establishing unit 703, the control that processing unit 706 is generated
Instruction, which is sent to target device, performs target device.
It is preferred that
Determining unit 701, specifically for detect user eyeball focus on an equipment when, however, it is determined that current user state or row
To meet preparatory condition, it is determined that the equipment is equipment to be identified.
It is preferred that
Following either conditions that the preparatory condition configures for satisfaction：Default limb action, default voice, user eyeball are gathered
The time of a burnt equipment is up to the first preset time threshold.
It is preferred that
Display unit 704, when being shown specifically for described information is added in the real scene of user's visual range, if
The bar number of described information is more than default number of branches threshold value, then provides the user user's slip or page turning display function.
It is preferred that
Display unit 704, be further used for when user selects relative mode, the region of Overlapping display information always with
In the visual range of family.
It is preferred that
Display unit 704, it is further used for when user selects absolute mode, and user's visual range changes, superposition is aobvious
Show that the region of information is shown in outside user's visual range after change.
It is preferred that
Display unit 704, it is further used for when the information of Overlapping display is big in the visual range not shown time of user
When the second preset time threshold, the display of closing information；
Connection unit 703, it is further used for when the display of the closing information of display unit 704, disconnects and target device
Communication connection.
It is preferred that
Processing unit 706, it is further used for scaling the true field for being superimposed upon user's visual range by intelligent interaction mode
Region where the information shown in scape.
It is preferred that
Processing unit 706, it is further used for moving superimposed in the true field of user's visual range by intelligent interaction mode
The position in the region where the information shown in scape.
It is preferred that
The intelligent interaction mode is：Limb action, and/or, voice.
The unit of above-described embodiment can be integrated in one, and can also be deployed separately；A unit is can be merged into, also may be used
To be further split into multiple subelements.
In summary, the smart machine that the application is paid close attention to by user's ocular focusing point Intelligent Recognition user, will intelligently set
Standby information superposition is shown in the real scene of user viewing area, there is provided intelligent interaction mode operates for user.The present invention
The relevant information of multiple smart machines can be added in the real scene of user viewing area so that user is in free space
It is more convenient to be interacted with smart machine, so as to improve Consumer's Experience.
Augmented reality, eyeball tracking, image recognition, gesture identification, speech recognition technology are combined in the embodiment of the present application, is led to
Cross intelligent helmet control intelligent home device.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
God any modification, equivalent substitution and improvements done etc., should be included within the scope of protection of the invention with principle.
Claims (20)
- A kind of 1. method of the control device based on augmented reality, applied in Intelligent worn device, it is characterised in that this method Including：When detecting that user eyeball focuses on an equipment, it is equipment to be identified to determine the equipment；Equipment to be identified is identified as target device, and establishes and communicates to connect with the target device；The information of target device is obtained, and described information is added in the real scene of user's visual range and shown；When receiving user the control menu in target device information being operated by intelligent interaction mode, to operation behavior Analyzed, generate the control instruction to target device；By the communication connection of foundation, the control instruction of generation is sent into target device performs target device.
- 2. according to the method for claim 1, it is characterised in that it is described to detect that user eyeball focuses on an equipment, including：If it is determined that current user state or behavior meet preparatory condition, it is determined that the equipment is equipment to be identified.
- 3. according to the method for claim 2, it is characterised in thatFollowing either conditions that the preparatory condition configures for satisfaction：Default limb action, default voice, user eyeball focus on one The time of equipment is up to the first preset time threshold.
- 4. according to the method for claim 1, it is characterised in that described that described information is added to the true of user's visual range When being shown in real field scape, if the bar number of described information is more than default number of branches threshold value, provides the user user's slip or page turning shows Show function.
- 5. according to the method for claim 1, it is characterised in thatWhen user selects relative mode, the information of Overlapping display is always in user's visual range.
- 6. according to the method for claim 1, it is characterised in thatWhen user selects absolute mode, and user's visual range changes, the user of the presentation of information of Overlapping display before change Outside visual range.
- 7. according to the method for claim 6, it is characterised in thatWhen the information of Overlapping display is being more than the second preset time threshold at the visual range not shown time of user, letter is closed The display of breath, and disconnect the communication connection with target device.
- 8. according to the method for claim 1, it is characterised in thatThe region being superimposed upon where the information shown in the real scene of user's visual range is scaled by intelligent interaction mode.
- 9. according to the method for claim 1, it is characterised in thatThe region where the information shown in the real scene of user's visual range is moved superimposed on by intelligent interaction mode Position.
- 10. according to the method described in claim any one of 1-9, it is characterised in thatThe intelligent interaction mode is：Limb action, and/or, voice.
- A kind of 11. device of the control device based on augmented reality, applied in Intelligent worn device, it is characterised in that the dress Put including：Determining unit, recognition unit, establish unit, display unit, receiving unit, processing unit and transmitting element；The determining unit, for when detecting that user eyeball focuses on an equipment, it to be equipment to be identified to determine the equipment；The recognition unit, for identifying the equipment to be identified of the determining unit determination as target device；The connection unit, the target device for being identified with the recognition unit establish communication connection；The display unit, it is added to the true of user's visual range for obtaining the information of target device, and by described information Shown in scene；The receiving unit, for receiving the operation of user；The processing unit, for receiving user by intelligent interaction mode in target device information when the receiving unit Control menu when being operated, operation behavior is analyzed, generates the control instruction to target device；The transmitting element, for by it is described establish unit establish communication connection, the control that the processing unit is generated Instruction, which is sent to target device, performs target device.
- 12. device according to claim 11, it is characterised in thatThe determining unit, specifically for detect user eyeball focus on an equipment when, however, it is determined that current user state or behavior Meet preparatory condition, it is determined that the equipment is equipment to be identified.
- 13. device according to claim 12, it is characterised in thatFollowing either conditions that the preparatory condition configures for satisfaction：Default limb action, default voice, user eyeball focus on one The time of equipment is up to the first preset time threshold.
- 14. device according to claim 11, it is characterised in thatThe display unit, when being shown specifically for described information is added in the real scene of user's visual range, if institute The bar number for stating information is more than default number of branches threshold value, then provides the user user's slip or page turning display function.
- 15. device according to claim 11, it is characterised in thatThe display unit, it is further used for when user selects relative mode, the region of Overlapping display information is always in user In visual range.
- 16. device according to claim 11, it is characterised in thatThe display unit, it is further used for when user selects absolute mode, and user's visual range changes, Overlapping display letter The region of breath is shown in outside user's visual range after change.
- 17. device according to claim 16, it is characterised in thatThe display unit, it is further used for the information when Overlapping display and is more than the in the visual range not shown time of user During two preset time thresholds, the display of closing information；The connection unit, it is further used for when the display of the display unit closing information, disconnects logical with target device Letter connection.
- 18. device according to claim 11, it is characterised in thatThe processing unit, is further used for scaling by intelligent interaction mode and is superimposed upon in the real scene of user's visual range Region where the information of display.
- 19. device according to claim 1, it is characterised in thatThe processing unit, it is further used for moving superimposed in the real scene of user's visual range by intelligent interaction mode The position in the region where the information of display.
- 20. according to the device described in claim any one of 1-9, it is characterised in thatThe intelligent interaction mode is：Limb action, and/or, voice.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710728237.5A CN107506037B (en) | 2017-08-23 | 2017-08-23 | Method and device for controlling equipment based on augmented reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710728237.5A CN107506037B (en) | 2017-08-23 | 2017-08-23 | Method and device for controlling equipment based on augmented reality |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107506037A true CN107506037A (en) | 2017-12-22 |
| CN107506037B CN107506037B (en) | 2020-08-28 |
Family
ID=60692475
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710728237.5A Active CN107506037B (en) | 2017-08-23 | 2017-08-23 | Method and device for controlling equipment based on augmented reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107506037B (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108628449A (en) * | 2018-04-24 | 2018-10-09 | 北京小米移动软件有限公司 | Apparatus control method, device, electronic equipment and computer readable storage medium |
| CN109324748A (en) * | 2018-09-05 | 2019-02-12 | 联想(北京)有限公司 | A kind of method, electronic equipment and the storage medium of equipment control |
| CN109361727A (en) * | 2018-08-30 | 2019-02-19 | Oppo广东移动通信有限公司 | Information sharing method, device, storage medium and wearable device |
| CN109656364A (en) * | 2018-08-15 | 2019-04-19 | 亮风台（上海）信息科技有限公司 | It is a kind of for the method and apparatus of augmented reality content to be presented on a user device |
| CN109979012A (en) * | 2017-12-27 | 2019-07-05 | 北京亮亮视野科技有限公司 | Show the method and device of message informing |
| CN110286754A (en) * | 2019-06-11 | 2019-09-27 | Oppo广东移动通信有限公司 | Projective techniques and relevant device based on eyeball tracking |
| CN110418051A (en) * | 2018-04-28 | 2019-11-05 | 北京京东尚科信息技术有限公司 | Image processing apparatus, intelligent refrigerator and intelligent helmet for intelligent refrigerator |
| CN110910508A (en) * | 2019-11-20 | 2020-03-24 | 三星电子（中国）研发中心 | Image display method, device and system |
| CN111480134A (en) * | 2018-06-01 | 2020-07-31 | 苹果公司 | Attention-aware virtual assistant cleanup |
| US10901689B1 (en) | 2019-07-11 | 2021-01-26 | International Business Machines Corporation | Dynamic augmented reality interface creation |
| CN112684893A (en) * | 2020-12-31 | 2021-04-20 | 上海电气集团股份有限公司 | Information display method and device, electronic equipment and storage medium |
| WO2021107506A1 (en) * | 2019-11-25 | 2021-06-03 | Samsung Electronics Co., Ltd. | Electronic device for providing augmented reality service and operating method thereof |
| CN113687721A (en) * | 2021-08-23 | 2021-11-23 | Oppo广东移动通信有限公司 | Device control method and device, head-mounted display device and storage medium |
| US11467802B2 (en) | 2017-05-11 | 2022-10-11 | Apple Inc. | Maintaining privacy of personal information |
| US11487364B2 (en) | 2018-05-07 | 2022-11-01 | Apple Inc. | Raise to speak |
| CN115291734A (en) * | 2022-10-08 | 2022-11-04 | 深圳市天趣星空科技有限公司 | Intelligent equipment control method and system based on intelligent glasses |
| US11538469B2 (en) | 2017-05-12 | 2022-12-27 | Apple Inc. | Low-latency intelligent automated assistant |
| US11557310B2 (en) | 2013-02-07 | 2023-01-17 | Apple Inc. | Voice trigger for a digital assistant |
| WO2023051305A1 (en) * | 2021-09-29 | 2023-04-06 | 歌尔股份有限公司 | Smart device control method and system, electronic device, and storage medium |
| WO2023082655A1 (en) * | 2021-11-15 | 2023-05-19 | 青岛海尔空调电子有限公司 | Target control method and apparatus, and device and medium |
| US11675491B2 (en) | 2019-05-06 | 2023-06-13 | Apple Inc. | User configurable task triggers |
| US11696060B2 (en) | 2020-07-21 | 2023-07-04 | Apple Inc. | User identification using headphones |
| US11699448B2 (en) | 2014-05-30 | 2023-07-11 | Apple Inc. | Intelligent assistant for home automation |
| US11705130B2 (en) | 2019-05-06 | 2023-07-18 | Apple Inc. | Spoken notifications |
| US11749275B2 (en) | 2016-06-11 | 2023-09-05 | Apple Inc. | Application integration with a digital assistant |
| US11783815B2 (en) | 2019-03-18 | 2023-10-10 | Apple Inc. | Multimodality in digital assistant systems |
| US11790914B2 (en) | 2019-06-01 | 2023-10-17 | Apple Inc. | Methods and user interfaces for voice-based control of electronic devices |
| US11809886B2 (en) | 2015-11-06 | 2023-11-07 | Apple Inc. | Intelligent automated assistant in a messaging environment |
| US11838734B2 (en) | 2020-07-20 | 2023-12-05 | Apple Inc. | Multi-device audio adjustment coordination |
| US11837237B2 (en) | 2017-05-12 | 2023-12-05 | Apple Inc. | User-specific acoustic models |
| US11838579B2 (en) | 2014-06-30 | 2023-12-05 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US11888791B2 (en) | 2019-05-21 | 2024-01-30 | Apple Inc. | Providing message response suggestions |
| US11893992B2 (en) | 2018-09-28 | 2024-02-06 | Apple Inc. | Multi-modal inputs for voice commands |
| US11900936B2 (en) | 2008-10-02 | 2024-02-13 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
| US11914848B2 (en) | 2020-05-11 | 2024-02-27 | Apple Inc. | Providing relevant data items based on context |
| US11954405B2 (en) | 2015-09-08 | 2024-04-09 | Apple Inc. | Zero latency digital assistant |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104102678A (en) * | 2013-04-15 | 2014-10-15 | 腾讯科技（深圳）有限公司 | Method and device for realizing augmented reality |
| CN105955456A (en) * | 2016-04-15 | 2016-09-21 | 深圳超多维光电子有限公司 | Virtual reality and augmented reality fusion method, device and intelligent wearable equipment |
| US20170061696A1 (en) * | 2015-08-31 | 2017-03-02 | Samsung Electronics Co., Ltd. | Virtual reality display apparatus and display method thereof |
| CN106924970A (en) * | 2017-03-08 | 2017-07-07 | 网易（杭州）网络有限公司 | Virtual reality system, method for information display and device based on virtual reality |
-
2017
- 2017-08-23 CN CN201710728237.5A patent/CN107506037B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104102678A (en) * | 2013-04-15 | 2014-10-15 | 腾讯科技（深圳）有限公司 | Method and device for realizing augmented reality |
| US20170061696A1 (en) * | 2015-08-31 | 2017-03-02 | Samsung Electronics Co., Ltd. | Virtual reality display apparatus and display method thereof |
| CN105955456A (en) * | 2016-04-15 | 2016-09-21 | 深圳超多维光电子有限公司 | Virtual reality and augmented reality fusion method, device and intelligent wearable equipment |
| CN106924970A (en) * | 2017-03-08 | 2017-07-07 | 网易（杭州）网络有限公司 | Virtual reality system, method for information display and device based on virtual reality |
Cited By (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11900936B2 (en) | 2008-10-02 | 2024-02-13 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
| US11862186B2 (en) | 2013-02-07 | 2024-01-02 | Apple Inc. | Voice trigger for a digital assistant |
| US11557310B2 (en) | 2013-02-07 | 2023-01-17 | Apple Inc. | Voice trigger for a digital assistant |
| US11699448B2 (en) | 2014-05-30 | 2023-07-11 | Apple Inc. | Intelligent assistant for home automation |
| US11838579B2 (en) | 2014-06-30 | 2023-12-05 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US11954405B2 (en) | 2015-09-08 | 2024-04-09 | Apple Inc. | Zero latency digital assistant |
| US11809886B2 (en) | 2015-11-06 | 2023-11-07 | Apple Inc. | Intelligent automated assistant in a messaging environment |
| US11749275B2 (en) | 2016-06-11 | 2023-09-05 | Apple Inc. | Application integration with a digital assistant |
| US11467802B2 (en) | 2017-05-11 | 2022-10-11 | Apple Inc. | Maintaining privacy of personal information |
| US11538469B2 (en) | 2017-05-12 | 2022-12-27 | Apple Inc. | Low-latency intelligent automated assistant |
| US11862151B2 (en) | 2017-05-12 | 2024-01-02 | Apple Inc. | Low-latency intelligent automated assistant |
| US11837237B2 (en) | 2017-05-12 | 2023-12-05 | Apple Inc. | User-specific acoustic models |
| CN109979012A (en) * | 2017-12-27 | 2019-07-05 | 北京亮亮视野科技有限公司 | Show the method and device of message informing |
| CN108628449A (en) * | 2018-04-24 | 2018-10-09 | 北京小米移动软件有限公司 | Apparatus control method, device, electronic equipment and computer readable storage medium |
| CN110418051B (en) * | 2018-04-28 | 2024-03-08 | 北京京东尚科信息技术有限公司 | Image processing device for intelligent refrigerator, intelligent refrigerator and intelligent helmet |
| CN110418051A (en) * | 2018-04-28 | 2019-11-05 | 北京京东尚科信息技术有限公司 | Image processing apparatus, intelligent refrigerator and intelligent helmet for intelligent refrigerator |
| US11487364B2 (en) | 2018-05-07 | 2022-11-01 | Apple Inc. | Raise to speak |
| US11907436B2 (en) | 2018-05-07 | 2024-02-20 | Apple Inc. | Raise to speak |
| CN111480134A (en) * | 2018-06-01 | 2020-07-31 | 苹果公司 | Attention-aware virtual assistant cleanup |
| US11360577B2 (en) | 2018-06-01 | 2022-06-14 | Apple Inc. | Attention aware virtual assistant dismissal |
| US11630525B2 (en) | 2018-06-01 | 2023-04-18 | Apple Inc. | Attention aware virtual assistant dismissal |
| CN109656364B (en) * | 2018-08-15 | 2022-03-29 | 亮风台（上海）信息科技有限公司 | Method and device for presenting augmented reality content on user equipment |
| CN109656364A (en) * | 2018-08-15 | 2019-04-19 | 亮风台（上海）信息科技有限公司 | It is a kind of for the method and apparatus of augmented reality content to be presented on a user device |
| CN109361727A (en) * | 2018-08-30 | 2019-02-19 | Oppo广东移动通信有限公司 | Information sharing method, device, storage medium and wearable device |
| CN109324748B (en) * | 2018-09-05 | 2021-12-24 | 联想(北京)有限公司 | Equipment control method, electronic equipment and storage medium |
| CN109324748A (en) * | 2018-09-05 | 2019-02-12 | 联想(北京)有限公司 | A kind of method, electronic equipment and the storage medium of equipment control |
| US11893992B2 (en) | 2018-09-28 | 2024-02-06 | Apple Inc. | Multi-modal inputs for voice commands |
| US11783815B2 (en) | 2019-03-18 | 2023-10-10 | Apple Inc. | Multimodality in digital assistant systems |
| US11675491B2 (en) | 2019-05-06 | 2023-06-13 | Apple Inc. | User configurable task triggers |
| US11705130B2 (en) | 2019-05-06 | 2023-07-18 | Apple Inc. | Spoken notifications |
| US11888791B2 (en) | 2019-05-21 | 2024-01-30 | Apple Inc. | Providing message response suggestions |
| US11790914B2 (en) | 2019-06-01 | 2023-10-17 | Apple Inc. | Methods and user interfaces for voice-based control of electronic devices |
| CN110286754A (en) * | 2019-06-11 | 2019-09-27 | Oppo广东移动通信有限公司 | Projective techniques and relevant device based on eyeball tracking |
| US10901689B1 (en) | 2019-07-11 | 2021-01-26 | International Business Machines Corporation | Dynamic augmented reality interface creation |
| CN110910508A (en) * | 2019-11-20 | 2020-03-24 | 三星电子（中国）研发中心 | Image display method, device and system |
| CN110910508B (en) * | 2019-11-20 | 2023-04-25 | 三星电子（中国）研发中心 | Image display method, device and system |
| WO2021107506A1 (en) * | 2019-11-25 | 2021-06-03 | Samsung Electronics Co., Ltd. | Electronic device for providing augmented reality service and operating method thereof |
| US11222477B2 (en) | 2019-11-25 | 2022-01-11 | Samsung Electronics Co., Ltd. | Electronic device for providing augmented reality service and operating method thereof |
| US11914848B2 (en) | 2020-05-11 | 2024-02-27 | Apple Inc. | Providing relevant data items based on context |
| US11838734B2 (en) | 2020-07-20 | 2023-12-05 | Apple Inc. | Multi-device audio adjustment coordination |
| US11750962B2 (en) | 2020-07-21 | 2023-09-05 | Apple Inc. | User identification using headphones |
| US11696060B2 (en) | 2020-07-21 | 2023-07-04 | Apple Inc. | User identification using headphones |
| CN112684893A (en) * | 2020-12-31 | 2021-04-20 | 上海电气集团股份有限公司 | Information display method and device, electronic equipment and storage medium |
| CN113687721A (en) * | 2021-08-23 | 2021-11-23 | Oppo广东移动通信有限公司 | Device control method and device, head-mounted display device and storage medium |
| WO2023051305A1 (en) * | 2021-09-29 | 2023-04-06 | 歌尔股份有限公司 | Smart device control method and system, electronic device, and storage medium |
| WO2023082655A1 (en) * | 2021-11-15 | 2023-05-19 | 青岛海尔空调电子有限公司 | Target control method and apparatus, and device and medium |
| CN115291734A (en) * | 2022-10-08 | 2022-11-04 | 深圳市天趣星空科技有限公司 | Intelligent equipment control method and system based on intelligent glasses |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107506037B (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506037A (en) | A kind of method and apparatus of the control device based on augmented reality | |
| US11418699B1 (en) | User interfaces for altering visual media | |
| KR102184272B1 (en) | Glass type terminal and control method thereof | |
| CN104410883B (en) | The mobile wearable contactless interactive system of one kind and method | |
| US11417296B2 (en) | Information processing device, information processing method, and recording medium | |
| US20170083116A1 (en) | Electronic device and method of adjusting user interface thereof | |
| US20230388665A1 (en) | User interfaces for altering visual media | |
| CN114391163A (en) | Gesture detection system and method | |
| JP2014509758A (en) | Real-time virtual reflection | |
| WO2021244145A1 (en) | Head-mounted display device interaction method, terminal device, and storage medium | |
| CN111898407B (en) | Human-computer interaction operating system based on human face action recognition | |
| CN105042789B (en) | The control method and system of a kind of intelligent air condition | |
| US11016558B2 (en) | Information processing apparatus, and information processing method to determine a user intention | |
| CN205507687U (en) | Wear -type virtual reality equipment and virtual reality system | |
| CN110138401A (en) | A kind of double screen wearable device based on coaxial overturning | |
| CN105892965A (en) | Screen display method and device | |
| CN109814722A (en) | A kind of multi-modal man-machine interactive system and exchange method | |
| CN107514751A (en) | Control method, device and the air conditioner of air conditioner | |
| JP6109288B2 (en) | Information processing apparatus, information processing method, and program | |
| US20140240226A1 (en) | User Interface Apparatus | |
| CN111199583A (en) | Virtual content display method and device, terminal equipment and storage medium | |
| CN104793732A (en) | Intelligent glasses operation method and intelligent glasses | |
| CN107620996A (en) | A kind of intelligent range hood and its application method | |
| US20220314114A1 (en) | Game processing method and record medium | |
| CN110096132A (en) | A kind of method and intelligent glasses for eliminating intelligent glasses message informing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |