WO2016202034A1

WO2016202034A1 - Reality enhancement-based smart home interaction method, and device and system utilizing same

Info

Publication number: WO2016202034A1
Application number: PCT/CN2016/077195
Authority: WO
Inventors: 郑少华; 黎剑辉; 张圳; 朱一伟; 罗海彬; 湛浩
Original assignee: 丰唐物联技术（深圳）有限公司
Priority date: 2015-06-18
Filing date: 2016-03-24
Publication date: 2016-12-22
Also published as: CN105204347A; CN105204347B

Abstract

A reality enhancement-based smart home interaction method, and device and system utilizing the same. The method comprises: obtaining a screen of a real scene generated by monitoring, in real time and via a smart gateway, a smart home, and processing the same (S101); obtaining a geographic location of the smart gateway and location information of a terminal, wherein the location information comprise a geographic location coordinate of the terminal and a facing direction of the terminal, and are used to instruct the terminal to determine, according to the geographic location of the smart gateway, a corresponding relationship of the geographic location of the smart gateway and the location information of the terminal (S102); simultaneously displaying, on the terminal and according to the corresponding relationship, the processed screen of the real scene generated by monitoring via the smart gateway in real time the smart home and information of the terminal (S103); and receiving interaction associated with the displayed screen on the terminal (S104). The method provides an application and implementation of reality simulation for those who work outside by remotely interacting with the smart home using a reality enhancement technique, increasing user experience.

Description

Description: Inventive name: Intelligent home interaction method, device and system based on augmented reality technology

[0001] The present invention relates to the field of smart homes, and in particular, to a smart home interaction method, apparatus and system based on augmented reality technology.

Background technique

[0002] In the current Internet of Things environment, people who work outside often use smart phones, tablets, and other terminals to interact remotely with smart homes. They can see the working scenes of smart home devices at a glance, but they don’t feel at home. People outside are inconvenient, so the current remote interaction method is not smart enough, and the user experience is not good.

technical problem

[0003] In view of the above, it is necessary to provide a smart home interaction method, device and system based on augmented reality technology, aiming at realizing the purpose of remotely interacting with smart home through augmented reality technology, and providing a real presence for people who work outside. The application in the home reflects the user experience.

Problem solution

Technical solution

[0004] In a first aspect, an embodiment of the present invention provides a smart home interaction method based on augmented reality technology, where the method includes the following steps:

[0005] Obtaining a smart gateway to monitor and process the real scene of the smart home;

[0006] acquiring a geographic location of the smart gateway and location information of the terminal, where the location information includes a geographic location coordinate of the terminal and an orientation orientation of the terminal, where the location information is used to indicate that the terminal is configured according to the smart gateway The geographic location determines a correspondence between a geographic location of the smart gateway and location information of the terminal;

[0007] according to the corresponding relationship, the processed smart gateway real-time monitoring scene of the smart home real scene and the terminal information are displayed on the terminal simultaneously;

[0008] interacting with a screen displayed on the terminal.

[0009] In combination with the first aspect, in the first implementation manner, the intelligence of the smart gateway is monitored. Before the process of the home real scene and the processing step, the method further includes: establishing a communication connection between the smart gateway and the terminal, and transmitting, by using the communication connection, protocol information that the smart gateway communicates with the terminal, the terminal Protocol information 2 for communicating with the intelligent gateway.

[0010] In combination with the first aspect and the foregoing implementation manner, in a second implementation manner of the first aspect, the processing, by the smart gateway, the monitored smart home real scene according to the corresponding relationship The steps displayed on the terminal include:

[0011] calculating a position correspondence relationship between the terminal and the smart gateway according to a geographic location of the smart gateway and location information of the terminal;

[0012] displaying, according to the corresponding relationship, the processed smart gateway real-time monitoring scene of the smart home real scene is displayed on the terminal, and simultaneously displaying the information of the terminal on the screen.

[0013] In combination with the first aspect and the foregoing implementation manner, in a third implementation manner of the first aspect, the processed smart gateway is configured to monitor a real scene of the smart home according to the corresponding relationship. Displayed on the terminal, the step of displaying the information of the terminal on the screen also includes:

[0014] The information of the terminal is superimposed on the screen of the real scene of the smart home monitored by the processed smart gateway.

[0015] In a second aspect, an embodiment of the present invention provides a smart home interaction device based on an augmented reality technology, where the device includes:

[0016] a picture obtaining unit, configured to acquire a picture of the real scene of the smart home monitored by the smart gateway and process the same;

[0017] a location obtaining unit, configured to acquire a geographic location of the smart gateway and location information of the terminal, where the location information includes a geographic location coordinate of the terminal and an orientation orientation of the terminal, where the location information is used to indicate the Determining, by the terminal, a correspondence between a geographic location of the smart gateway and location information of the terminal according to a geographic location of the smart gateway;

[0018] a screen display unit, configured to display, on the terminal, the screen of the real scene of the smart home and the information of the terminal that are processed by the processed smart gateway according to the corresponding relationship;

[0019] an interaction unit, configured to interact with a screen displayed on the terminal.

[0020] With reference to the second aspect, in a first implementation manner of the second aspect, the device further includes:

[0021] a communication unit, configured to establish a communication connection between the smart gateway and the terminal, and connect through the communication The protocol information for transmitting the communication between the intelligent gateway and the terminal is the protocol information of the communication between the terminal and the intelligent gateway.

[0022] In combination with the second aspect and the foregoing implementation manner, in a second implementation manner of the second aspect, the screen display unit further includes:

[0023] a calculating unit, configured to calculate a position correspondence relationship between the terminal and the smart gateway according to a geographic location of the smart gateway and location information of the terminal;

[0024] a sub-screen display unit, configured to display, on the terminal, a screen of the smart home real scene that is processed by the smart gateway according to the correspondence, and display the information of the terminal on the screen. .

[0025] In combination with the second aspect and the foregoing implementation manner, in a third implementation manner of the second aspect, the sub-picture display unit further includes:

[0026] The superimposing unit, the information for the terminal is superimposed on the screen of the real scene of the smart home monitored by the processed smart gateway.

[0027] In a third aspect, an embodiment of the present invention provides a smart home interaction system based on augmented reality technology, where the system includes:

[0028] The smart gateway, the terminal, the augmented reality technology-based smart home interaction device, wherein the augmented reality technology-based smart home interaction device comprises the second aspect or any one of the possible implementation manners of the second aspect The smart home interaction device based on the augmented reality technology is integrated with the terminal, or is disposed separately from the terminal. Advantageous effects of the invention

Beneficial effect

[0029] The embodiment of the invention provides a smart home interaction method, device and system based on augmented reality technology, and establishes a communication connection between the intelligent gateway and the terminal, and obtains a picture of the real scene of the smart home monitored by the intelligent gateway. And processing, obtaining the geographic location of the intelligent gateway, and the location information of the terminal, and superimposing the information of the terminal on the processed smart gateway according to the correspondence between the geographic location of the intelligent gateway and the location information of the terminal. Monitoring the scene of the real scene of the smart home, and displaying it on the terminal, interacting with the screen displayed on the terminal, thereby achieving the purpose of remotely interacting with the smart home through augmented reality technology, and providing a person for working outside Really at home The application in the program enhances the user experience.

Brief description of the drawing

DRAWINGS

1 is a flow chart of a smart home interaction method based on augmented reality technology according to a first embodiment of the present invention;

2 is a structural diagram of a smart home interaction device based on augmented reality technology according to a second embodiment of the present invention;

3 is a structural diagram of a smart home interaction system based on augmented reality technology according to a third embodiment of the present invention.

Embodiments of the invention

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0034] The terms "a", "an", and the like, if any, are used in the <RTI ID=0.0> It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments described herein can be implemented in a sequence other than what is illustrated or described herein. In addition, the terms "comprises" and "comprising" and "comprises" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units, but may include other steps or orders not explicitly listed or inherent to such processes, methods, products or equipment

[0035] In the following embodiments of the smart home interaction method, device and system based on the augmented reality technology, the terminal described may be any smart terminal such as smart glasses, smart watches, smart phones, etc.; As the core of the smart home system, the convergence of various smart home devices to the home network is a hub for controlling various smart home devices. A person skilled in the art can understand that the specific expression form of the terminal and the smart gateway is not limited in the embodiment of the present invention as long as the idea of the smart home interaction method based on the augmented reality technology can be implemented. Related descriptions of terminals and intelligent gateways As described above, in the following specific embodiments for describing the corresponding technical solutions, the above content will not be described again.

[0036] The specific implementation of the present invention is described in detail below with reference to specific embodiments:

[0037] Embodiment 1:

1 shows a smart home interaction method based on augmented reality technology according to a first embodiment of the present invention, which is described in detail as follows:

[0039] In step S101, a picture of the real scene of the smart home monitored by the smart gateway is obtained and processed.

[0040] In a specific implementation process, a communication connection between the smart gateway and the terminal is first established, and protocol information for communicating between the smart gateway and the terminal is transmitted through the communication connection, and the terminal and the smart gateway perform The protocol information of the communication is two, wherein the protocol information that the intelligent gateway communicates with the terminal may be information such as the real scene image of the smart home that the acquired intelligent gateway actually monitors, and is transformed into a terminal identifiable during transmission to the terminal. The algorithm for transforming the information corresponding to the information such as the smart scene real-time scene of the smart home is also an algorithm for processing the acquired real-life scene of the smart home, such as: image processing The algorithm, after which the processed intelligent gateway actually transmits the real scene picture of the smart home to the terminal, and the protocol information of the communication between the terminal and the intelligent gateway may be interactive command information or other information input through the terminal. During the process of sending the smart gateway, it is transformed into A transform algorithm that the smart gateway can identify information corresponding to the interactive command information or other information.

[0041] In some preferred embodiments, the communication connection between the smart gateway and the terminal may be connected through a WiFi, a Bluetooth, another wireless mode, or a wired connection. The communication connection mode is not specifically limited by the present invention.

[0042] After establishing a communication connection between the intelligent gateway and the terminal, the intelligent gateway automatically starts the camera that is provided with the camera, and monitors the real scene in front of the camera of the intelligent gateway, and determines according to the range monitored by the camera provided by the intelligent gateway. The range of the screen of the real scene being monitored, when the orientation, inclination, etc. of the camera provided by the intelligent gateway are adjusted, the screen of the monitored real scene changes correspondingly, and the screen of the monitored real scene can be Processing, such as: storage, image compression, transmission, decomposition, recombination, etc., correspondingly, when adjusting the orientation, tilt, etc. of the camera that comes with the smart gateway The processed intelligent gateway realizes that the display of the real scene of the smart home is also changed correspondingly in the display of the terminal, and the screen of the monitored real scene can also be subjected to computer processing technology to obtain the simulated and monitored images. The virtual content corresponding to the picture of the real scene can be understood by those skilled in the art. The embodiment of the present invention does not specifically limit the processing of the picture of the smart home real scene of the smart gateway that is acquired, as long as the implementation of the present invention can be implemented. The idea of a smart home interaction method based on augmented reality technology can be used.

[0043] In step S102, acquiring a geographic location of the smart gateway and location information of the terminal, where the location information includes a geographic location coordinate of the terminal and an orientation orientation of the terminal, where the location information is used to indicate the terminal Determining, according to a geographic location of the smart gateway, a correspondence between a geographic location of the smart gateway and location information of the terminal.

[0044] In a specific implementation process, after establishing a communication connection between the smart gateway and the terminal, acquiring the geographic location of the intelligent gateway and the location information of the terminal, where the geographic location of the intelligent gateway includes at least the geographic coordinates of the location of the smart gateway, The geographic coordinates are obtained by the home network, and the geographic coordinates can be obtained by the base station. The geographic coordinates can be obtained by other methods. The embodiment of the present invention does not specifically limit the geographic location of the smart gateway. The location information of the terminal may include at least a geographic location coordinate of the terminal and an orientation orientation of the terminal, and may acquire a geographic location coordinate of the terminal by using a network manner of the terminal, or acquire a geographic location coordinate of the terminal by using GPS positioning. The location coordinates of the terminal may be located in other manners. The method for acquiring the geographic location coordinates of the terminal is not specifically limited by the embodiment of the present invention, and the orientation orientation of the terminal refers to the phase. For the terminal used by the user, the orientation of the display interface of the terminal may be used to identify the orientation of the terminal through the electronic compass of the terminal, and the terminal may determine the geographic location of the intelligent gateway and the location information of the terminal according to the geographic location of the intelligent gateway. The correspondence between the smart gateway and the smart gateway can be adjusted by the terminal to send a control command to the intelligent gateway, so that the obtained smart gateway can monitor the real scene of the smart home and the orientation of the camera of the smart gateway. The change of the inclination and the like are correspondingly changed, and the processed smart gateway actually monitors the real scene of the smart home, and the screen of the smart gateway changes accordingly according to the change of the orientation, the inclination, and the like of the camera of the smart gateway, wherein the corresponding The relationship may be a relative position coordinate, and the location information of the terminal is used to indicate that the terminal determines the location according to the geographic location of the smart gateway. The corresponding relationship between the geographic location of the intelligent gateway and the location information of the terminal may be displayed on the display interface of the terminal and the orientation of the terminal, and the processed smart gateway monitors the real scene of the smart home and the information of the terminal together. On the terminal interface, that is, the information of the terminal is superimposed on the screen of the real scene of the smart home that is processed by the intelligent gateway, and the image of the real scene of the smart home is enhanced.

[0045] In addition, through the display interface of the terminal and the orientation of the terminal, the processed smart gateway monitors the real scene of the smart home and the information of the terminal together on the observation surface in front of the terminal, that is, the terminal The information superimposed on the screen of the real scene of the smart home monitored by the intelligent gateway is displayed on the observation surface in front of the terminal, thereby realizing enhancement of the real scene picture of the smart home, wherein the front of the terminal The observation surface may be a projection surface in front of the terminal, such as a plane such as a ground or a wall, that is, superimposing the information of the terminal on the screen of the real scene of the smart home that is processed by the intelligent gateway, and the screen is projected to the terminal. The projection surface.

[0046] In step S103, according to the corresponding relationship, the processed smart gateway real-time monitoring scene of the smart home real scene and the terminal information are displayed on the terminal.

[0047] In a specific implementation process, determining, according to the correspondence between the geographic location of the smart gateway and the location information of the terminal, that the processed smart gateway actually monitors the location of the real scene of the smart home corresponding to the location on the display interface of the terminal,显示 Display the information of the terminal on the interface of the terminal.

[0048] In some preferred embodiments, the step S103 specifically includes:

[0049] calculating, according to the geographic location of the smart gateway and the location information of the terminal, a location correspondence between the terminal and the smart gateway; and processing, according to the correspondence, the processed smart gateway to monitor the real scene of the smart home The screen is displayed on the terminal, and the information of the terminal is displayed on the screen.

[0050] It should be noted that, according to the geographic location of the intelligent gateway and the location information of the terminal, the location correspondence between the terminal and the smart gateway is calculated, and the processed smart gateway is determined according to the location correspondence relationship. The location of the scene of the scene is displayed on the display interface of the terminal, and the information of the terminal is displayed on the processed real scene of the smart home. The information of the terminal may be the intelligence of the smart gateway to be acquired. The virtual reality content of the real scene of the home is simulated by computer processing technology. The information of the terminal can be distributed according to the position correspondence between the terminal and the intelligent gateway, and the smart gateway can be adjusted by sending a control command to the intelligent gateway through the orientation orientation of the terminal. Adjusting the position and inclination of the camera to adjust the position of the terminal and the intelligent gateway, so as to adjust the distribution of the information of the terminal on the processed real scene of the smart home, and enhance the real scene of the smart home, that is, the enhanced The reality is applied to the scene of the processed real scene of the smart home.

[0051] wherein the augmented reality is a method of applying virtual information to a real world by means of computer graphics technology and visualization technology, so that the real environment and the virtual object are actually superimposed on the same picture or space. , integrated 3D display technology, interactive technology, sensor technology, computer vision technology and multimedia technology.

[0052] Further, according to the correspondence, the processed smart gateway real-time monitoring real-life scene of the smart home is displayed on the terminal, and the step of displaying the information of the terminal on the screen includes :

[0053] The information of the terminal is superimposed on the screen of the real scene of the smart home monitored by the processed smart gateway.

[0054] In the embodiment of the present invention, the augmented reality, the processed virtual home real scene picture and the acquired intelligent gateway real-time monitoring of the real scene of the smart home real-time simulation of the virtual content after computer simulation technology simulation Displaying, that is, displaying the processed real-life scene picture of the smart home and the information of the terminal simultaneously, the processed smart home real scene picture and the terminal information complement and superimpose each other, and can be layered The information of the terminal is superimposed on the screen of the real scene of the smart home monitored by the processed intelligent gateway, and the information of the terminal can be placed on the upper layer, and the processed intelligent gateway is actually placed on the lower layer of the real scene of the smart home. It is also possible to put the picture of the real scene of the smart home monitored by the processed intelligent gateway on the upper layer, and the information of the terminal is placed on the lower layer. It should be understood that the manner of the layer is not specifically limited in the embodiment of the present invention, as long as the augmented reality can be highlighted. The effect is fine.

[0055] In step S104, interaction with the screen displayed on the terminal is performed.

[0056] In the specific implementation process, the smart home working scene control App supported by the terminal can display the working scene of the smart home device at a glance, and display the superimposed screen of the processed smart home real scene screen and the terminal information of the terminal. The image of the acquired real scene of the smart home is enhanced, and the user can superimpose the information of the real scene scene and the terminal of the processed smart home displayed on the terminal. The smart home scene presented by the screen performs remote interaction, and the smart home work scene can be changed by changing the working state of the smart home device, thereby realizing interaction with the smart home work scene, such as: control of the smart home work scene supported by the terminal The app interacts remotely with the smart home.

[0057] The embodiment of the present invention provides a smart home interaction method based on augmented reality technology, and establishes a communication connection between the intelligent gateway and the terminal, and obtains a picture of the real scene of the smart home monitored by the intelligent gateway, and performs processing, Acquiring the geographic location of the intelligent gateway and the location information of the terminal, and superimposing the information of the terminal on the processed real scene of the smart home according to the corresponding relationship between the geographic location of the intelligent gateway and the location information of the terminal, and displaying On the terminal, interacting with the screen displayed on the terminal, thereby achieving the purpose of remotely interacting with the smart home through the augmented reality technology, providing a real-life application expression for the person working outside, improving the user experience. effect.

[0058] Embodiment 2:

2 shows a structure of a smart home interaction device based on augmented reality technology according to a third embodiment of the present invention, which is described in detail as follows:

[0060] The picture obtaining unit 21 is configured to obtain a picture of the real scene of the smart home monitored by the smart gateway and process the same.

[0061] In a specific implementation process, the communication connection between the smart gateway and the terminal is first established, and the smart home interaction device based on the augmented reality technology further includes:

[0062] The communication unit 20 is configured to establish a communication connection between the smart gateway and the terminal, and transmit, by using the communication connection, protocol information that the smart gateway communicates with the terminal, and the terminal communicates with the intelligent gateway. Agreement information II.

[0063] the communication unit establishes a communication connection between the intelligent gateway and the terminal, and transmits protocol information for communicating between the intelligent gateway and the terminal through the communication connection, and a protocol information for communicating between the terminal and the intelligent gateway. The protocol information that the smart gateway communicates with the terminal may be information such as the real scene image of the smart home that is monitored by the smart gateway by the screen acquiring unit, and is converted into the terminal identifiable and The algorithm for transforming the information corresponding to the information such as the real scene picture of the intelligent home in the smart gateway may also be an algorithm for processing the real scene picture of the smart home that is obtained by the intelligent gateway, such as: an image processing algorithm, and then Transmitting the real scene image of the smart home monitored by the processed intelligent gateway to the terminal, and the terminal and the intelligent gateway The protocol information 2 for performing communication may be an interaction command information or other information input through the terminal, and converted into information corresponding to the interactive command information or other information that can be identified by the intelligent gateway during transmission to the intelligent gateway. algorithm.

[0064] In some preferred embodiments, the communication connection between the smart gateway and the terminal may be connected through WiFi, Bluetooth, other wireless methods, or a wired manner. The communication connection manner is not specifically limited in the present invention.

[0065] After the communication connection is established between the smart gateway and the terminal, the smart gateway automatically starts the camera that is provided with the camera, and the screen acquisition unit monitors the real scene in front of the camera of the smart gateway, and monitors according to the camera provided by the intelligent gateway. The range determines the range of the screen of the real scene to be monitored. When the orientation, inclination, etc. of the camera that comes with the smart gateway are adjusted, the screen of the monitored real scene changes accordingly, and the monitored reality can be The scene of the scene is processed, such as: storage, image compression, transmission, decomposition, recombination, etc., correspondingly, when the orientation, inclination, etc. of the camera that comes with the intelligent gateway are adjusted, the processed intelligent gateway is monitored. The display of the real scene of the smart home changes correspondingly in the display of the terminal, and the screen of the monitored real scene can also be subjected to computer processing technology to obtain a virtual corresponding to the screen of the monitored real scene after the simulation. Content, those skilled in the art can understand that the present invention Intelligent Gateway example of processing real-inch monitor to a real scene smart home screen acquired embodiment is not particularly limited as long as the present embodiment can be implemented based on the inventive concept can smart home augmented reality interaction method.

[0066] The location obtaining unit 22 is configured to acquire a geographic location of the smart gateway and location information of the terminal, where the location information includes a geographic location coordinate of the terminal and an orientation orientation of the terminal, where the location information is used to indicate The terminal determines a correspondence between the geographic location of the smart gateway and the location information of the terminal according to the geographic location of the smart gateway.

[0067] In a specific implementation process, after the communication connection between the smart gateway and the terminal is established, the location obtaining unit acquires the geographic location of the smart gateway and the location information of the terminal, where the geographic location of the smart gateway includes at least the geographic location of the smart gateway. Coordinates, the geographic coordinates can be obtained through the home network, and the geographic coordinates can be obtained by the base station. The geographic coordinates can be obtained by other means. The person skilled in the art can understand that the method for acquiring the geographic location of the intelligent gateway is not used in the embodiment of the present invention. Specifically, the location information of the terminal may include at least a geographic location coordinate of the terminal and a direction of the terminal The orientation can be obtained by acquiring the geographic location coordinates of the terminal through the network of the terminal, and the geographic location coordinates of the terminal can be obtained by using the GPS positioning, and the geographic location coordinates of the terminal can be obtained by other methods, which can be understood by those skilled in the art. The embodiment of the present invention does not specifically limit the manner in which the geographic location coordinates of the terminal are obtained. The orientation orientation of the terminal refers to the orientation of the display interface of the terminal relative to the terminal used by the user, and can be identified by the electronic compass of the terminal. The orientation of the terminal, the terminal may determine the correspondence between the geographic location of the smart gateway and the location information of the terminal according to the geographic location of the smart gateway, and may send an interactive command to the smart gateway to adjust the orientation and tilt of the camera of the smart gateway. Degree, etc., so that the acquired intelligent gateway actually monitors the scene of the real scene of the smart home and changes correspondingly with the change of the orientation, inclination, etc. of the camera of the smart gateway.

The processed intelligent gateway is configured to monitor the real scene of the smart home, and the screen of the smart gateway is changed according to the orientation, the inclination, and the like of the camera of the smart gateway, where the corresponding relationship may be a relative position coordinate, The location information of the terminal is used to indicate that the terminal determines the correspondence between the geographic location of the smart gateway and the location information of the terminal according to the geographic location of the smart gateway, and may pass through the display interface of the terminal and the orientation of the terminal. The processed intelligent gateway implements the real-life scene of the smart home and the information of the terminal together on the terminal interface, that is, the information of the terminal is superimposed on the processed smart gateway to monitor the real scene of the smart home. The upper picture is presented on the terminal interface, thereby realizing the enhancement of the real scene picture of the smart home.

[0068] In addition, through the display interface of the terminal and the orientation of the terminal, the processed smart gateway monitors the real scene of the smart home and the information of the terminal together on the observation surface in front of the terminal, that is, the terminal The information superimposed on the screen of the real scene of the smart home monitored by the intelligent gateway is displayed on the observation surface in front of the terminal, thereby realizing enhancement of the real scene picture of the smart home, wherein the front of the terminal The observation surface may be a projection surface in front of the terminal, such as a plane such as a ground or a wall, that is, superimposing the information of the terminal on the screen of the real scene of the smart home that is processed by the intelligent gateway, and the screen is projected to the terminal. The projection surface.

[0069] The screen display unit 23 is configured to display, on the terminal, the screen of the real scene of the smart home and the information of the terminal that are processed by the processed smart gateway according to the corresponding relationship.

[0070] In a specific implementation process, the screen display unit 23 may determine, according to the correspondence between the geographic location of the smart gateway and the location information of the terminal, the processed smart gateway to monitor the real scene of the smart home. The face corresponds to the position on the display interface of the terminal, and the information of the terminal is displayed on the interface of the terminal.

[0071] In some preferred embodiments, the screen display unit 23 further includes:

[0072] The calculating unit 230 is configured to calculate a position correspondence relationship between the terminal and the smart gateway according to a geographic location of the smart gateway and location information of the terminal;

[0073] The sub-screen display unit 231 is configured to display, on the terminal, a screen of the smart home real scene that is processed by the smart gateway according to the correspondence, and display the information of the terminal on the screen. on.

[0074] It is to be noted that the calculation unit calculates the position correspondence relationship between the terminal and the smart gateway according to the geographic location of the smart gateway and the location information of the terminal, and the sub-screen display unit determines the processed intelligent gateway according to the location correspondence relationship.吋 Detecting the location of the real scene of the smart home in the display interface of the terminal, and simultaneously displaying the information of the terminal on the processed real scene of the smart home, the information of the terminal may be the acquired intelligence The virtual reality content of the real scene of the smart home monitored by the gateway is simulated by the computer processing technology. The information of the terminal can be distributed according to the position correspondence between the terminal and the intelligent gateway, and can be sent to the intelligent gateway through the orientation of the terminal. The interactive command adjusts the orientation and inclination of the camera of the smart gateway to adjust the position correspondence between the terminal and the intelligent gateway, thereby adjusting the distribution of the information of the terminal on the processed real scene of the smart home, and realizing the real scene picture of the smart home. Enhance That is the reality applied to enhance the processed real scene smart home screen.

[0075] wherein the augmented reality is a method of applying virtual information to a real world by means of computer graphics technology and visualization technology, so that the real environment and the virtual object are actually superimposed on the same picture or space. , integrated 3D display technology, interactive technology, sensor technology, computer vision technology and multimedia technology.

[0076] Further, the sub-screen display unit 231 further includes:

[0077] The superimposing unit 2310, the information for the terminal is superimposed on the screen of the smart home real scene monitored by the processed smart gateway.

[0078] In the embodiment of the present invention, the augmented reality, the processed virtual home real scene picture and the acquired intelligent gateway are monitored by the virtual home real scene, and the virtual content is simulated by computer processing technology. Display, that is, the processed smart home real scene picture and terminal The information is displayed at the same time, and the superimposing unit complements and superimposes the processed information of the real scene of the smart home and the information of the terminal, and superimposes the information of the terminal on the processed intelligent gateway in a layer manner. On the screen of the real scene of the smart home, the information of the terminal can be placed on the upper layer, and the processed intelligent gateway can be used to monitor the real scene of the smart home. The processed intelligent gateway can also monitor the intelligence. The image of the real scene of the home is placed on the upper layer, and the information of the terminal is placed on the lower layer. It should be understood that the manner of the layer is not specifically limited in the embodiment of the present invention, as long as the effect of the augmented reality can be highlighted.

[0079] The interaction unit 24 is configured to interact with a screen displayed on the terminal.

[0080] In the specific implementation process, through the smart home work scene interactive App supported by the terminal, the user can clearly view the working scene of the smart home device, and display the superimposed screen of the processed smart home real scene screen and the terminal information of the terminal. The enhanced real-life scene of the smart home is enhanced, and the user can remotely interact with the smart home scene presented by the overlay of the processed smart home real scene image and the terminal information displayed on the terminal through the interaction unit, which can be changed. The working status of the smart home device changes the working scene of the smart home, thereby realizing interaction with the smart home work scene, such as: remote interaction between the smart home work scene interactive app supported by the terminal and the smart home.

[0081] The embodiment of the present invention provides a smart home interaction device based on augmented reality technology, and establishes a communication connection between the intelligent gateway and the terminal through the communication unit, and the screen acquisition unit acquires the real scene of the intelligent home in the intelligent gateway. The location acquisition unit acquires the geographic location of the intelligent gateway and the location information of the terminal, and the screen display unit superimposes the information of the terminal on the processing according to the correspondence between the geographic location of the intelligent gateway and the location information of the terminal. The intelligent gateway is implemented on the screen of the real scene of the smart home, and displayed on the terminal, and the interaction unit interacts according to the screen displayed on the terminal, thereby achieving the purpose of realizing remote interactive smart home through augmented reality technology, People who work outside provide a real-life application that enhances the user experience.

Embodiment 3:

3 shows a smart home interactive system structure based on augmented reality technology according to a third embodiment of the present invention, which is described in detail as follows:

[0084] The augmented reality technology-based smart home interaction system includes: an intelligent gateway 30, based on augmented reality a smart home interaction device 31, a terminal 32, wherein

[0085] The smart home interaction device 31 based on the augmented reality technology is a smart home interaction device based on the augmented reality technology as shown in FIG. 2, and may be integrated with the terminal 32, or may be connected to the terminal. 32 is set as a separate part.

[0086] The terminal may be any smart terminal such as smart glasses, smart watches, smart phones, etc.; the smart gateway, as the core of the smart home system, converges various smart home devices to the home network, and is various intelligent A hub for interactive home devices. A person skilled in the art can understand that the specific expression of the terminal and the smart gateway is not limited as long as the idea of the smart home interaction method based on the augmented reality technology can be implemented.

[0087] The embodiment of the present invention provides a smart home interaction system based on augmented reality technology, which establishes a communication connection between the intelligent gateway and the terminal, and acquires and processes a real scene of the smart home real intelligent scene. Obtaining the geographic location of the intelligent gateway and the location information of the terminal, and superimposing the information of the terminal on the processed smart gateway to monitor the real scene of the smart home according to the corresponding relationship between the geographic location of the intelligent gateway and the location information of the terminal On the screen, and displayed on the terminal, interacting according to the screen displayed on the terminal, thereby achieving the purpose of realizing remote interactive smart home through augmented reality technology, and providing a real-life application for the person who is working outside. , improve user experience.

[0088] Those skilled in the art can understand that all or part of the steps of implementing the above embodiments may be completed by a program instructing related hardware, and the program may be stored in a computer readable storage medium. The storage medium described, such as a ROM/RAM, a magnetic disk, an optical disk, or the like.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the present invention. Within the scope of protection of the invention.

Claims

Claim

[Attachment 1] A smart home interaction method based on augmented reality technology, characterized in that the method comprises the steps of:

Acquiring and processing the smart gateway to monitor the real scene of the smart home; obtaining the geographic location of the intelligent gateway and the location information of the terminal, where the location information includes the geographic location coordinates of the terminal and the orientation of the terminal, The location information is used to indicate that the terminal determines a correspondence between a geographic location of the smart gateway and location information of the terminal according to a geographic location of the smart gateway;

According to the corresponding relationship, the processed smart gateway real-time monitoring scene of the smart home real scene and the terminal information are displayed on the terminal simultaneously; and interact with the screen displayed on the terminal.

[Claim 2] The augmented reality technology-based smart home interaction method according to claim 1, wherein before the acquiring the smart gateway to monitor the real scene of the smart home real scene and performing the processing steps, the method further includes Establishing a communication connection between the smart gateway and the terminal, and transmitting protocol information for communicating between the smart gateway and the terminal through the communication connection, and a protocol information 2 for the terminal to communicate with the intelligent gateway.

The Augmented Reality Technology-based smart home interaction method according to claim 1, wherein the processed intelligent gateway really monitors the smart home real scene according to the corresponding relationship The screen shows the steps on the terminal, including:

Calculating a position correspondence relationship between the terminal and the smart gateway according to a geographic location of the smart gateway and location information of the terminal;

And displaying, according to the correspondence, the processed smart home real-time scene of the smart home real-time scene is displayed on the terminal, and simultaneously displaying the information of the terminal on the screen.

[Claim 4] The augmented reality technology-based smart home interaction method according to claim 3, wherein the processed smart gateway is configured to monitor the real scene of the smart home according to the corresponding relationship The screen is displayed on the terminal, and the step of displaying the information of the terminal on the screen includes:

The information of the terminal is superimposed on the processed smart gateway to monitor the smart home On the picture of the real scene.

[Claim 5] A smart home interaction device based on augmented reality technology, wherein the device comprises:

The picture obtaining unit is configured to obtain a picture of the real scene of the smart home monitored by the intelligent gateway and process the same;

a location obtaining unit, configured to acquire a geographic location of the smart gateway and location information of the terminal, where the location information includes a geographic location coordinate of the terminal and an orientation orientation of the terminal, where the location information is used to indicate that the terminal is configured according to the location The geographic location of the smart gateway determines a correspondence between a geographic location of the smart gateway and location information of the terminal;

a screen display unit, configured to display, on the terminal, the interactive unit of the real-time scene of the smart home real-time monitored by the processed smart gateway according to the corresponding relationship, for displaying The screen of the terminal interacts.

[Claim 6] The augmented reality technology-based smart home interaction device according to claim 5, wherein the device further comprises:

a communication unit, configured to establish a communication connection between the intelligent gateway and the terminal, and pass the communication

The protocol information for transmitting the communication between the intelligent gateway and the terminal is as follows: 1. The protocol information of the communication between the terminal and the intelligent gateway.

The Augmented Reality-based smart home interaction device according to claim 5, wherein the screen display unit further comprises:

a calculating unit, configured to calculate a location correspondence between the terminal and the smart gateway according to a geographic location of the smart gateway and location information of the terminal;

The sub-screen display unit is configured to display, on the terminal, a screen of the smart home real scene that is processed by the smart gateway according to the correspondence, and display the information of the terminal on the screen.

The augmented reality technology-based smart home interaction device according to claim 7, wherein the sub-screen display unit further comprises: Superimposing unit, information for the terminal is superimposed on the processed intelligent gateway to monitor

The scene of the real scene of the smart home.

[Claim 9] A smart home interaction system based on augmented reality technology, wherein the system comprises:

The smart home device, the terminal, and the augmented reality technology-based smart home interaction device, wherein the augmented reality technology-based smart home interaction device comprises the augmented reality technology-based smart home interaction device according to any one of claims 5-8. It is provided integrally with the terminal, or is disposed separately from the terminal.