CN111045344A

CN111045344A - Control method of household equipment and electronic equipment

Info

Publication number: CN111045344A
Application number: CN201911418226.2A
Authority: CN
Inventors: 高桦
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2019-12-31
Filing date: 2019-12-31
Publication date: 2020-04-21

Abstract

The embodiment of the invention provides a control method of home equipment and electronic equipment, relates to the technical field of communication, and aims to solve the problem that the process of controlling the home equipment through the electronic equipment is complicated. The scheme comprises the following steps: receiving a first input of a user on a shooting preview interface; responding to the first input, and displaying a target AR control, wherein the target AR control is an AR control generated based on an input track of the first input; receiving a second input of the target AR control by the user; responding to the second input, and controlling the target household equipment to execute target control operation corresponding to the target AR control; the shooting preview interface comprises target household equipment. The method is applied to a scene that the household equipment is controlled through the electronic equipment.

Description

Control method of household equipment and electronic equipment

Technical Field

The embodiment of the invention relates to the technical field of communication, in particular to a control method of household equipment and electronic equipment.

Background

With the development of communication technology, the functions of electronic equipment are more and more abundant, for example, the electronic equipment can be connected with home equipment, so that a user can control the home equipment through the electronic equipment.

Generally, when a user needs to control a certain home device through an electronic device, the user may first trigger the electronic device to download a dedicated application program provided by a manufacturer of the home device, and then the user may trigger the electronic device to install the dedicated application program. After the electronic device successfully installs the dedicated application program, the user may control the home device through the dedicated application program, for example, the user may control the home device to turn on or off by inputting the dedicated application program.

However, according to the above method, since the user needs to trigger the electronic device to download and install the dedicated application program to control the corresponding home devices, when there are many home devices that the user needs to control, the electronic device may need to install a plurality of dedicated application programs, which results in a complicated and complicated process for controlling the home devices through the electronic device.

Disclosure of Invention

The embodiment of the invention provides a control method of household equipment and electronic equipment, and aims to solve the problem that the process of controlling the household equipment through the electronic equipment is complicated and complex.

In order to solve the technical problem, the invention is realized as follows:

in a first aspect, an embodiment of the present invention provides a control method for home equipment, where the method may be applied to electronic equipment, and the method includes receiving a first input of a user on a shooting preview interface; and in response to the first input, displaying a target Augmented Reality (AR) control and receiving a second input of the target AR control by the user; responding to the second input, and controlling the target household equipment to execute target control operation corresponding to the target AR control; the target AR control is generated based on the input track of the first input; the shooting preview interface comprises target household equipment.

In a second aspect, an embodiment of the present invention provides an electronic device, including: the device comprises a receiving module, a display module and a processing module. The receiving module is used for receiving a first input of a user on the shooting preview interface; the display module is used for responding to the first input received by the receiving module and displaying the target AR control, and the target AR control is generated based on the input track of the first input; the receiving module is also used for receiving second input of the target AR control displayed by the display module by the user; the processing module is used for responding to the second input received by the receiving module and controlling the target household equipment to execute target control operation corresponding to the target AR control; the shooting preview interface is an acquired image of a target area, and the target area comprises target household equipment.

In a third aspect, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a computer program that is stored in the memory and is executable on the processor, and when the computer program is executed by the processor, the steps of the control method for the home equipment in the first aspect are implemented.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the control method for home equipment in the first aspect are implemented.

In the embodiment of the invention, the electronic equipment can receive a first input of a user on a shooting preview interface (including a target household device); and in response to the first input, displaying a target AR control (an AR control generated for an input trajectory based on the first input); receiving a second input of the target AR control by the user; and responding to the second input, and controlling the target household equipment to execute target control operation corresponding to the target AR control. According to the scheme, the electronic equipment can generate and display the AR control used for controlling the target household equipment to execute the target control operation corresponding to the target AR control according to the input track input by the user on the shooting preview interface, so that the user can directly trigger the electronic equipment to control the target household equipment to execute the target control operation through the input of the target AR control, and the user does not need to trigger the electronic equipment to control the household equipment through a specific application program installed on the electronic equipment. Therefore, the electronic equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

Furthermore, in the embodiment of the invention, as the home equipment seen by the user on the display screen of the electronic equipment is the same as the home equipment seen by the user in a real scene, and the user directly operates the home equipment in the environment through the input on the shooting preview interface, the user can see and obtain the information, and the user can see and control the information, so that the flexibility and convenience of controlling the home equipment through the electronic equipment can be further improved on the basis of simplifying the process of controlling the home equipment through the electronic equipment, and the man-machine interaction performance can be improved.

Drawings

FIG. 1 is a block diagram of a possible operating system according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a control method for home equipment according to an embodiment of the present invention;

fig. 3 is one of schematic interface diagrams of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 4 is a second schematic interface diagram of an application of a control method for home equipment according to an embodiment of the present invention;

fig. 5 is a third schematic interface diagram of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 6 is a second flowchart of a control method for home equipment according to an embodiment of the present invention;

fig. 7 is a fourth schematic interface diagram of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 8 is a fifth schematic interface diagram of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 9 is a sixth schematic interface diagram of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 10 is a seventh schematic interface diagram of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 11 is an eighth schematic interface diagram of an application of a control method of home equipment according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

fig. 13 is a hardware schematic diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The term "and/or" herein is an association relationship describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The symbol "/" herein denotes a relationship in which the associated object is or, for example, a/B denotes a or B.

The terms "first" and "second," and the like, in the description and in the claims of the present invention are used for distinguishing between different objects and not for describing a particular order of the objects. For example, the first input and the second input, etc. are for distinguishing different inputs, rather than for describing a particular order of inputs.

In the embodiments of the present invention, words such as "exemplary" or "for example" are used to mean serving as examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "e.g.," an embodiment of the present invention is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the description of the embodiments of the present invention, unless otherwise specified, "a plurality" means two or more, for example, a plurality of elements means two or more elements, and the like.

The following first explains some of the nouns or terms referred to in the claims and the specification of the present invention.

The acquisition range of the camera is as follows: refers to the range of images that the camera can capture of an object (e.g., a person, or an object). For example, assuming that the acquisition range of the camera is range a, the camera may acquire images of objects within range a, but cannot acquire images of objects outside range a.

The embodiment of the invention provides a control method of household equipment and electronic equipment, wherein the electronic equipment can receive first input of a user on a shooting preview interface (comprising target household equipment); and in response to the first input, displaying a target AR control (an AR control generated for an input trajectory based on the first input); receiving a second input of the target AR control by the user; and responding to the second input, and controlling the target household equipment to execute target control operation corresponding to the target AR control. According to the scheme, the electronic equipment can generate and display the target AR control for controlling the target household equipment to execute the target control operation corresponding to the target AR control according to the input track of the user input on the shooting preview interface, so that the user can directly trigger the electronic equipment to control the target household equipment to execute the target control operation through the input of the target AR control, and therefore the user does not need to trigger the electronic equipment to control the household equipment through a specific application program installed on the electronic equipment. Therefore, the electronic equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved, and the man-machine interaction performance can be improved.

The electronic device in the embodiment of the present invention may be an electronic device having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, and embodiments of the present invention are not limited in particular.

A software environment applied to the control method for the home device provided by the embodiment of the invention is described below by taking a possible operating system as an example.

Fig. 1 is a schematic diagram of a possible operating system according to an embodiment of the present invention. In fig. 1, the architecture of the operating system includes 4 layers, respectively: an application layer, an application framework layer, a system runtime layer, and a kernel layer (specifically, a Linux kernel layer).

The application layer comprises various application programs (including system application programs and third-party application programs) in an operating system. Illustratively, the application layer may include an application program, such as a camera application program, for displaying a shooting preview interface in an embodiment of the present invention.

The application framework layer is a framework of the application, and a developer can develop some applications based on the application framework layer under the condition of complying with the development principle of the framework of the application. For example, an application displaying a shooting preview interface in the embodiment of the present invention may be developed based on an application framework layer.

Generally, an application program in the embodiment of the present invention may include two parts, where, taking a camera application program as an example, one part refers to a camera service (service) running in a background of an electronic device, and is used to detect an input triggered by a user, determine an image display area (for example, a display area of a target AR control in the embodiment of the present invention), and the like; another portion refers to content displayed on a display screen of the electronic device, such as a target AR control displayed on the display screen of the electronic device.

The system runtime layer includes a library (also referred to as a system library) and an operating system runtime environment. The library mainly provides various resources required by the operating system. The operating system runtime environment is used to provide a software environment for the operating system.

The kernel layer is the operating system layer of the operating system and belongs to the lowest layer of the operating system software layer. The kernel layer provides kernel system services and hardware-related drivers for the operating system based on the Linux kernel.

In the embodiment of the present invention, a developer may develop a software program for implementing the control method of the home equipment provided in the embodiment of the present invention based on the system architecture of the operating system shown in fig. 1, so that the control method of the home equipment may operate based on the operating system shown in fig. 1. That is, the processor or the electronic device may implement the control method of the home device provided by the embodiment of the present invention by running the software program in the operating system.

The electronic device in the embodiment of the invention can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and the non-mobile electronic device may be a Personal Computer (PC), a Television (TV), a teller machine, a self-service machine, and the like, and the embodiment of the present invention is not particularly limited.

The execution main body of the control method for the home equipment provided by the embodiment of the present invention may be the electronic equipment, or may also be a functional module and/or a functional entity capable of implementing the control method for the home equipment in the electronic equipment, which may be specifically determined according to actual use requirements, and the embodiment of the present invention is not limited. The following takes an electronic device as an example to exemplarily describe a control method of a home device provided by the embodiment of the present invention.

In the embodiment of the invention, when a user needs to control the home equipment to execute a certain operation through the electronic equipment, the user can trigger the electronic equipment to acquire the preview image of the area where the home equipment is located and display the preview image on a shooting preview interface of the electronic equipment, so that the user can input the shooting preview image, the electronic equipment is triggered to display the AR control generated based on the input track input by the user, and then the user can trigger the electronic equipment to control the target home equipment to execute the target control operation corresponding to the target AR control through the input of the AR control. Therefore, the electronic equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

The following describes an exemplary control method for home equipment according to an embodiment of the present invention with specific reference to the drawings.

As shown in fig. 2, an embodiment of the present invention provides a method for controlling home devices, where the method may include steps 201 to 204 described below.

Step 201, the electronic device receives a first input of a user on a shooting preview interface.

In an embodiment of the present invention, the shooting preview interface may be an interface in a camera application program in the electronic device. The shooting preview interface can include a preview image of the target area acquired by a camera of the electronic device. The target area may include at least one household device therein. For convenience of description, in the following embodiments, an image of at least one home device is included in a shooting preview interface, which is simply referred to as a target shooting preview interface including at least one home device, that is, including a device in an interface or an image means that an image of the device is included in the interface or the image.

Optionally, in this embodiment of the present invention, the first input may be a gesture input (one possible implementation manner) performed by a user within an acquisition range of a camera of the electronic device, or may be a touch gesture input (another possible implementation manner) performed by the user on a display screen of the electronic device, which may be determined specifically according to actual use requirements, and this embodiment of the present invention is not limited.

For the description of the acquisition range of the camera, reference may be made to the related description of the noun explanation part, which is not described herein again.

In the embodiments of the present invention, unless otherwise specified, the environments in the following embodiments all refer to spatial environments within the acquisition range of the camera, and the two environments have the same meaning and may be interchanged sometimes. For example, an input in the environment refers to an input within the capture range of a camera, and the two are synonymous and may be interchanged.

Optionally, in this embodiment of the present invention, after the user executes the first input, the electronic device may obtain an input trajectory of the first input, and generate an AR control (for example, a target AR control described below) according to the input trajectory.

Optionally, in the embodiment of the present invention, the AR control refers to a control displayed by using an AR technology.

Optionally, in this embodiment of the present invention, the input trajectory of the first input may include at least one of a trajectory shape of the first input and a trajectory direction of the first input. That is, the input trajectory of the first input may be the trajectory shape of the first input, the trajectory direction of the first input, or both the trajectory shape of the first input and the trajectory direction of the first input.

Alternatively, in the embodiment of the present invention, the shape of the trajectory of the first input may be any possible shape such as a shape of a number "2", a shape of a letter "on", a shape of a triangle "△", a shape of a pentagon ", a shape of a pair" √ "a, a shape of an arrow" → ", a shape of a volume horn, and a shape of an error" × ".

The above-mentioned upward direction, downward direction, leftward direction, rightward direction, counterclockwise direction, and clockwise direction are all illustrated with respect to the display screen of the electronic device when the display screen of the electronic device faces the user.

Optionally, in the embodiment of the present invention, when the input trajectory includes a trajectory shape and a trajectory direction, if the trajectory shapes of two inputs sequentially executed by the user are the same and the trajectory directions are different, the electronic device may determine that the two inputs are two different inputs; alternatively, if the trajectory shapes of two inputs sequentially executed by the user are different and the trajectory directions are the same, the electronic device may determine that the two inputs are two different inputs.

For example, assuming that the trajectory shapes are both arrow-shaped two gesture inputs, if the trajectory direction of one gesture input is a rightward direction and the trajectory direction of the other gesture input is a leftward direction, the two gesture inputs are two different gesture inputs.

For another example, assuming that the trajectory directions are both two gesture inputs in the left direction, if one gesture input is a straight line (trajectory shape) drawn from right to left (trajectory direction) and the other gesture input is an arrow pointing to the left side, the two gesture inputs are also different gesture inputs.

Optionally, in an embodiment of the present invention, in a possible implementation manner, the first input is a gesture input performed by a user within a collection range of a camera of the electronic device.

Optionally, in an embodiment of the present invention, in a possible implementation manner, the electronic device may sequentially collect, through a camera of the electronic device, multiple frames of images including the user gesture, and obtain the user gesture in each frame of image, so as to obtain a complete gesture performed by the user, that is, obtain the input trajectory of the first input.

Step 202, the electronic device responds to the first input and displays the target AR control.

The target AR control may be an AR control generated by the electronic device based on the input trajectory of the first input.

Optionally, in the embodiment of the present invention, the target AR control may be an AR control customized by a user. For example, an identifier for triggering a custom AR control may be set in a camera application or a settings application of the electronic device. Therefore, the user can trigger the electronic device to display the AR control user-defined interface by inputting the identifier, and when the electronic device displays the AR control user-defined interface, if the user executes an input (for example, drawing a circle), the electronic device can generate an AR control with a shape corresponding to the input track and store the AR control in the database, so that the user-defined target AR control can be realized.

In the embodiment of the invention, the target AR control can be customized by the user, namely, the user can set the target AR control into any possible shape according to the actual use requirement of the user, so that the display effect of the target AR control can be ensured to better meet the requirement of the user.

Optionally, in the embodiment of the present invention, the shape of the target AR control may be the same as or different from the shape of the trajectory of the first input. The method can be determined according to actual use requirements, and the embodiment of the invention is not limited.

For convenience of description, the following embodiments are exemplified by taking the case where the shape of the target AR control is the same as the shape of the trajectory of the first input.

It should be noted that, in the embodiment of the present invention, a corresponding relationship between an input trajectory of an input (for example, a first input) and a target control operation may be pre-stored in the electronic device, so that when the electronic device generates a target AR control based on an input trajectory of a user (that is, the input trajectory of the first input) and displays the target AR control, the electronic device may further obtain the target control operation corresponding to the input trajectory of the first input; then, when the user performs the second input on the target AR control, the electronic device may control the target home equipment to perform the target control operation (i.e., control the target home equipment to perform the target control operation corresponding to the target AR control).

In the embodiment of the present invention, the electronic device may correspond the input trajectory of the first input to the target control operation in the following two modes, i.e., mode a and mode B.

Optionally, in the embodiment of the present invention, in the mode a, only the corresponding relationship between the input trajectory of the first input and the target control operation is pre-stored in the electronic device; in this case, for the same first input, the target control operation corresponding to the input trajectory of the first input is the same regardless of whether the target home devices are the same. Or, in the mode B, the electronic device may pre-store a corresponding relationship between target information and target control operation, where the target information includes the first input track and the target home device; in this case, for the same first input, if the target home devices are different, the target control operations corresponding to the input trajectory of the first input may be the same or different.

The following describes the above-described mode a and the above-described mode B, respectively.

For convenience of description, in example 1 and example 2 described below, the input trajectory of the first input is illustrated as an example of the trajectory shape of the first input.

Example 1, in the above-described mode a, it is assumed that the track shape of the first input is "√" and that the track shape "√" corresponds to an opening operation. Then, when the user performs gesture input with a track of "√" shape, the electronic device may display a target AR control in the shape of "√" shape on the shooting preview interface, and the target AR control corresponds to the opening operation; at this time, when the user performs the second input on the target AR control, if the target home device is a television, the electronic device may control the television to perform a start operation (target control operation); if the target household equipment is the intelligent air conditioner, the electronic equipment can control the intelligent air conditioner to execute the starting operation.

Example 2, in the above-described mode B, it is assumed that the track shape of the first input is "→", and when the target household device is a television, that is, the target information is "→" and the television, "→" can be changed to the operation of the next television station; when the target household device is a smart light, that is, the target information is "→" and the smart light, "→" can correspond to the dimming operation. Then, when the user performs the gesture input with the track of "→", the electronic device may display the "→" shaped target AR control on the shooting preview interface, and at this time, if the target home device is a television, after the user performs the second input, the electronic device may control the television to perform the operation of tuning to the next television station; if the target household device is an intelligent lamp, after the user performs the second input, the electronic device may control the intelligent lamp to perform the operation of reducing the brightness.

Optionally, in the embodiment of the present invention, the electronic device may display the target AR control on a shooting preview interface.

Optionally, in this embodiment of the present invention, in a case that the input trajectory of the first input corresponds to the target control operation, the target AR control may include a first target sub AR control and a second target sub AR control, where the first target sub AR control may be an AR control generated based on the input trajectory of the first input, and the second target sub AR control may be an AR control generated based on a control type of the first target sub AR control.

Optionally, in the embodiment of the present invention, the step 202 may be specifically implemented by the following steps 202a to 202 c.

Step 202a, the electronic device displays a first target sub-AR control.

Step 202b, the electronic device determines a control type of the first target sub-AR control.

Step 202c, the electronic device generates a second target sub-AR control based on the control type of the first target sub-AR control, and displays the second target sub-AR control on the first target sub-AR control. The second input described below may be a user input to the second target sub-AR control.

In the embodiment of the present invention, the electronic device may determine the type of the first target sub-AR control according to the control operation corresponding to the first input trajectory. For example, assuming that an input trajectory of a first input is associated with a brightness adjustment function of a television, and as shown in (a) of fig. 3, the shooting preview interface 30 includes an image of the television, if a user draws a vertical line on the shooting interface, that is, the electronic device receives the first input, the electronic device may generate a vertical line-shaped AR control 31 (i.e., a first target sub-AR control), display the vertical line-shaped AR control 31 on the shooting preview interface 30, and determine that the type of the vertical line-shaped AR control 31 is a volume adjustment size, so that as shown in (b) of fig. 3, the electronic device may generate an AR volume adjustment block 32 (i.e., a second target sub-AR control) according to the type of the vertical line-shaped AR control 31, and display the AR volume adjustment block 32 on the vertical line-shaped AR control 31. In this way, the user may trigger the electronic device to control the television to adjust the volume of the television to the volume indicated by the AR volume adjustment block 32 through an input (e.g., a second input described below) to the AR volume adjustment block 32.

In the embodiment of the invention, the electronic equipment can automatically generate the second target sub-AR control corresponding to the control type of the first target sub-AR control according to the type of the first target sub-AR control, so that a user can only draw part of the target AR control, the process of drawing the target AR control by the user can be simplified, and the man-machine interaction performance is improved.

Further, since the electronic device may automatically match a part of the AR controls (e.g., the first target sub-AR control) in the target AR control according to a part of the AR controls (e.g., the second target sub-AR control) in the target AR control customized by the user, a process of triggering the electronic device to display the target AR control may be simplified.

And step 203, the electronic device receives a second input of the target AR control from the user.

Optionally, in this embodiment of the present invention, the second input may be used to determine that the user has performed the first input, and the second input may be used to indicate that the user has finished drawing the AR control (for example, the target AR control).

Optionally, in this embodiment of the present invention, the second input may be a gesture input executed by a user within an acquisition range of a camera of the electronic device, or may be a touch gesture input of the user on a display screen of the electronic device, which may be specifically determined according to actual use requirements, and this embodiment of the present invention is not limited.

In the embodiment of the present invention, each region on the display screen of the electronic device may correspond to a spatial region within the capture range of the camera of the electronic device, so that the electronic device determines, according to the gesture image, a mapping region (hereinafter referred to as a first mapping region) of the input region of the second input on the display screen of the electronic device.

Optionally, in this embodiment of the present invention, when the second input is a gesture input performed by the user in the acquisition range of the camera of the electronic device, if the mapping area of the input area of the second input on the display screen of the electronic device is located in the display area of the target AR control, the electronic device may perform the following step 204.

And step 204, the electronic equipment responds to the second input, and controls the target household equipment to execute target control operation corresponding to the target AR control.

The shooting preview interface may include target home equipment. The target household equipment is household equipment in the at least one household equipment.

Optionally, in the embodiment of the present invention, the number of the target home devices may be one or multiple, and may be specifically determined according to actual use requirements.

It should be noted that, in the embodiment of the present invention, the number of target home devices is not limited except for the specific description.

Optionally, in the embodiment of the present invention, the electronic device may send a control message to the target home equipment to control the target home equipment to execute a target control operation corresponding to the target AR control, so that the target home equipment may execute an operation corresponding to the control message after receiving the control message.

In practical implementation, the control method for the home device provided by the embodiment of the present invention may also implement that the electronic device controls the target home device to execute the target control operation corresponding to the target AR control in any other possible manner, which may be specifically determined according to actual use requirements, and the embodiment of the present invention is not limited.

Optionally, in the embodiment of the present invention, the control message may be control instruction information or may be an indication message, which may be determined specifically according to an actual use requirement, and the embodiment of the present invention is not limited.

For example, when the control message is control instruction information, after the user performs a second input on the target AR control, the electronic device may send the control instruction information corresponding to the target AR control to the target home device, so that after receiving the control instruction information, the target home device may perform an operation corresponding to the control instruction information. When the control message is an indication message, after the user performs a second input on the target AR control, the electronic device may send an indication message corresponding to the target AR control to the target home device, so that after receiving the indication message, the target home device may generate a control instruction message according to the indication message, and perform an operation corresponding to the control instruction message.

Optionally, in this embodiment of the present invention, the target home device may be an intelligent home device (i.e., a home device supporting network signal control, such as an intelligent refrigerator and a television). Alternatively, the target home device may be a non-smart home device (e.g., a television) supporting infrared signal control. The method can be determined according to actual use requirements, and the embodiment of the invention is not limited.

Specifically, when the target home device is an intelligent home device, the control message may be a network control signal; when the target home device is a non-intelligent home device, the control message may be an infrared control signal.

Optionally, in the embodiment of the present invention, different AR controls, for example, different target AR controls (generated based on an input trajectory input by a user) may correspond to different control operations. For example, a "check" shaped AR control may correspond to a control operation to turn on a home device (e.g., the target home device described above), and an "x" shaped AR control may correspond to a control operation to turn off the home device. Therefore, different AR controls correspond to different control operations, so that a user can trigger the electronic device to generate different AR controls through different gesture inputs so as to control the household device to execute different control operations.

Optionally, in the embodiment of the present invention, when the target home device is a television, after the electronic device receives a first input of a user on the shooting preview interface, it may be detected whether a control operation corresponding to an input track of the first input is associated with the shooting preview interface. If the input track of the first input is associated with the shooting preview interface, the electronic equipment responds to the second input and sends a target control message to the target household equipment; if the input track of the first input is not associated with the shooting preview interface, the electronic device may display the second information.

The target control message is a control message corresponding to a target control operation, where the target control message may be used to instruct the target home equipment to display first information, and the first information is information displayed after the target home equipment executes the target control operation, and may be, for example, an interface of a program played by a television. The second information may include at least one alternative operation option, where the at least one alternative operation option is information of a control operation associated with the home device corresponding to the device image (for example, may be an operation menu of the home device corresponding to the device image).

In the embodiment of the invention, the first input track is associated with the shooting preview interface, which can be understood as that the control operation corresponding to the first input track is associated with the household equipment corresponding to the equipment image in the shooting preview interface; that is, the household device corresponding to the device image in the shooting preview interface can successfully execute the control operation corresponding to the first input track. Correspondingly, the shooting preview interface with the input track not related to the first input can be understood as home equipment corresponding to the equipment image in the shooting preview interface with the control operation corresponding to the input track not related to the first input; that is, the household device corresponding to the device image in the shooting preview interface cannot successfully execute the control operation corresponding to the first input track.

Illustratively, the home equipment in the shooting preview interface is taken as a television. As shown in (a) of fig. 4, the shooting preview interface 30 of the electronic device includes a television 35 (specifically, an image of the television 35), the user writes "none" in the environment, that is, the input track of the first input is "none", and the target AR control is a "none" font AR control, if the input track is "none" and is associated with five of the televisions 35, the electronic device may send a target control message to the television 35, and after receiving the target control message, the television 35 may tune to five of the televisions (that is, the target control message is used to instruct the target home devices to display the first information), as shown in (b) of fig. 4. If the input trace is "no" and is not associated with any function in the television 35, that is, the television 35 cannot execute the control operation associated with the input trace being "no", then, as shown in (c) in fig. 4, the electronic device may display an operation menu of the television 35 on the shooting preview interface 30 (that is, the electronic device displays the second information), so that the user may trigger to control the television to execute the corresponding operation by inputting the operation menu. Therefore, the flexibility of controlling the household equipment through the electronic equipment can be improved.

In the embodiment of the invention, on one hand, because the electronic device can display at least one alternative operation option of the home equipment under the condition that the home equipment in the target area cannot successfully execute the control operation corresponding to the first input track, even if the first input executed by the user is incorrect, the user can trigger the electronic device to control the home equipment in the target area through the input of the at least one alternative operation option; on the other hand, the electronic device can directly control the household equipment to execute the control operation under the condition that the household equipment in the target area can execute the control operation corresponding to the input track of the first input; therefore, the flexibility of controlling the household equipment through the electronic equipment can be improved.

A control method of home equipment according to an embodiment of the present invention is exemplarily described below with reference to fig. 5.

Illustratively, the control method of the household equipment provided by the embodiment of the invention can be applied to a scene that a user controls the intelligent air conditioner through the electronic equipment. Specifically, it is assumed that the target household equipment is an intelligent air conditioner corresponding to an image of the intelligent air conditioner in the shooting preview interface, and the intelligent air conditioner is in a closed state; then, as shown in (a) of fig. 5, the photographing preview interface 40 of the electronic device includes an image 41 of the smart air conditioner, the user can draw "√" within the capture range of the camera of the electronic device, that is, the electronic device receives a first input on the capture preview interface 40 received by the user, and then the electronic device responds to the first input, a "√" shaped AR control 42 (i.e., a target AR control) can be displayed on the capture preview interface 40, then, as shown in fig. 5 (b), the user performs a click gesture input within the capture range of the camera of the electronic device, which corresponds to the user clicking on the AR control 42, that is, the electronic device receives a second input to the target AR control from the user, so that the electronic device can respond to the second input, control the intelligent air conditioner to be turned on, namely, the electronic equipment controls the target household equipment to execute the target control operation corresponding to the target AR control. Therefore, the electronic equipment can be triggered to control the intelligent air conditioner without inputting a specific application program by a user, so that the flexibility and convenience of controlling the intelligent air conditioner through the electronic equipment can be improved.

In the control method of the home equipment provided by the embodiment of the invention, the electronic equipment can generate and display the target AR control for controlling the target home equipment to execute the target control operation corresponding to the target AR control according to the input track input by the user on the shooting preview interface, so that the user can directly trigger the electronic equipment to control the target home equipment to execute the target control operation through the input of the target AR control, and therefore, the user does not need to trigger the electronic equipment to control the home equipment through a specific application program installed on the electronic equipment. Therefore, the electronic equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

Optionally, in the embodiment of the present invention, when a user needs to simultaneously control one or more home devices in a certain area (for example, a target area) to perform a certain operation through an electronic device, the user may first trigger the electronic device to acquire a preview image of the target area, and display the image in a shooting preview interface (generally, the image of the target area acquired by the electronic device through a camera in real time), and then the electronic device may recognize the preview image in the shooting preview interface to recognize the device in the target area.

Optionally, in this embodiment of the present invention, after the electronic device identifies the device in the target area, the electronic device may display an identifier (for example, at least one identifier described below) indicating the device in the target area on the shooting preview interface.

For example, in the embodiment of the present invention, with reference to fig. 2, as shown in fig. 6, before step 201, the method for controlling home equipment according to the embodiment of the present invention may further include step 205 and step 206 described below.

And step 205, the electronic equipment performs equipment identification on the shooting preview interface, and identifies at least one piece of home equipment.

The at least one piece of home equipment may include the target home equipment.

It should be noted that, in the embodiment of the present invention, the at least one piece of home equipment is a piece of home equipment in the target area, and thus the shooting preview interface includes an image of the at least one piece of home equipment.

The following describes an exemplary manner of performing device identification on the shooting preview interface by the electronic device in two different scenes.

In a first scenario: at least one household device in the target area is different household devices. In this case, the database may store in advance an image of each of the at least one home device, which is hereinafter referred to as each first preset image. The electronic equipment can compare a preview image acquired through a camera of the electronic equipment with each first preset image in a shooting preview interface so as to identify an image of at least one piece of home equipment in the preview image.

Specifically, taking an example that the electronic device compares a preview image with a first preview image, the electronic device may obtain first target feature information of each home device from the preview image, and compare the first target feature information with feature information of the home device in a first preset image (hereinafter referred to as first preset feature information), and if the first target feature information matches (that is, the first target feature information and the first preset feature information are the same, or a matching degree of the first target feature information and the first preset feature information is greater than a first preset threshold), the electronic device may determine to shoot an image of the home device indicated by the first preview image in the preview interface, so that the electronic device may identify the home device. It can be understood that, in the embodiment of the present invention, after the electronic device identifies the home device, the electronic device may obtain device information corresponding to the first preset image, establish a connection with the home device indicated by the device information according to the device information, and display an identifier (which is an identifier in at least one identifier described below) used for indicating the home device.

In a second scenario, at least one household device in the target area includes the same household device (e.g., the same lamp type). In this case, the three-dimensional image of the target area may be stored in the database in advance. The electronic device may then map the three-dimensional image onto a display screen of the electronic device using projection imaging principles in computer vision. Specifically, the electronic device may map three-dimensional coordinate information of at least one piece of home equipment in the three-dimensional image on a display screen of the electronic device, and if the coordinate information of the three-dimensional coordinate information mapped on the display screen of the electronic device is the mapped coordinate information, the electronic device may compare coordinate information (hereinafter, referred to as target coordinate information) of the image of the at least one piece of home equipment in the shooting preview interface on the display screen of the electronic device with the mapped coordinate information, and if the target coordinate information matches (the target coordinate information is the same as the target coordinate information, or the matching degree of the target coordinate information is greater than a second preset threshold) the electronic device may determine that the shooting preview interface includes the image of the home equipment corresponding to the mapped coordinate information, so that the electronic device may recognize the home equipment. It can be understood that, in the embodiment of the present invention, after the electronic device identifies the home device, the electronic device may obtain device information corresponding to the first preset image, establish a connection with the home device indicated by the device information according to the device information, and display an identifier (which is an identifier in at least one identifier described below) used for indicating the home device.

Optionally, in the embodiment of the present invention, the database may be a database in the electronic device, or may also be a database in the server, which may be determined specifically according to actual use requirements, and the embodiment of the present invention is not limited.

Optionally, in the embodiment of the present invention, the image of at least one piece of home equipment and the three-dimensional image of the target area, which are pre-stored in the database, may be manually triggered and imported into the database for the user (for example, the importing may be triggered in a setting interface), or may be automatically entered by the user through a camera of the electronic device, which may be specifically determined according to actual use requirements, and the embodiment of the present invention is not limited.

It is to be appreciated that after the electronic device identifies the at least one home device, the electronic device can display at least one indicator indicating the at least one home device on the photo preview interface. Therefore, the user can input the at least one identification to trigger the electronic equipment to determine the target household equipment.

And step 206, the electronic equipment displays at least one mark on the shooting preview interface.

Each mark in the at least one mark can be used for indicating one household device in the shooting preview interface. It should be noted that each identifier may be specifically used to indicate a home device corresponding to a home device image in a shooting preview interface.

It should be noted that the home devices indicated by each identifier in the at least one identifier are home devices in the at least one home device identified by the electronic device.

Optionally, in this embodiment of the present invention, each piece of home equipment indicated by the at least one identifier may be home equipment that establishes a connection with the electronic device. Namely, the user can control the household equipment indicated by the at least one identifier through the electronic equipment.

Optionally, in this embodiment of the present invention, the electronic device may display the at least one identifier in any area of the shooting preview interface. For example, as shown in fig. 7 (a), the electronic device may display 2 an indicator 73 (i.e., at least one indicator) in an area where an image of at least one home device in the photographing preview interface 70 is located. Alternatively, as shown in fig. 7 (b), the electronic device may display 2 the logo 73 (i.e., one less logo) in an area other than the area where the image of at least one home device in the photographing preview interface 70 is located. The method can be determined according to actual use requirements, and the embodiment of the invention is not limited.

It should be noted that the dashed box or the dashed box in fig. 7 and the text in the dashed box (for example, "smart air conditioner" and "television" shown in (b) in fig. 7) are only used for exemplary illustration of at least one identifier, and in actual implementation, at least one identifier may be any other possible identifier.

In the embodiment of the invention, the electronic equipment can identify the at least one piece of home equipment by identifying the shooting preview interface, so that after the electronic equipment is successfully identified, at least one mark for indicating the at least one piece of home equipment can be displayed to prompt a user which home equipment in an image acquisition range of the electronic equipment can be operated by the electronic equipment, therefore, the operation of the home equipment by the user through the electronic equipment can be improved, and the man-machine interaction performance is improved.

Optionally, in the embodiment of the present invention, after the electronic device displays the at least one identifier in the shooting preview interface, a user may trigger the electronic device to determine the target home device by inputting an identifier (for example, a target identifier) in the at least one identifier.

For example, in the embodiment of the present invention, after the step 206, the method for controlling home equipment according to the embodiment of the present invention may further include the following steps 207 and 208.

Step 207, the electronic device receives a third input from the user.

Optionally, in this embodiment of the present invention, the third input may be an input of a second identifier of the at least one identifier by a user, and the third input may be used to cancel display of identifiers (for example, the following first identifier) other than the second identifier of the at least one identifier. The home devices indicated by the second identifier may be the target home devices.

Optionally, in this embodiment of the present invention, the third input may be a gesture input executed by a user within an acquisition range of a camera of the electronic device, or may be a touch gesture input of the user on a display screen of the electronic device, which may be specifically determined according to actual use requirements, and this embodiment of the present invention is not limited.

For the description of the input form of the third input, reference may be specifically made to the related description of the input form of the first input in the foregoing embodiment, and details are not repeated here to avoid repetition.

And step 208, the electronic equipment responds to the third input and cancels the display of the first identifier.

The first identifier is an identifier of the at least one identifier, except for the second identifier, and the second identifier may be used to indicate the target home device.

The above steps 205-208 are exemplified below with reference to the specific example of fig. 7.

For example, the control method for the home devices provided by the embodiment of the invention can be applied to a scene in which the user triggers the electronic device to determine the target home device through gesture input in the environment. Specifically, as shown in fig. 7 (a) and fig. 7 (b), the image 71 of the smart air conditioner and the image 72 of the television are included in the shooting preview interface 70 of the electronic device, and 2 identifiers 73 (i.e., the at least one identifier) are displayed on the shooting preview interface 70, and the 2 identifiers 73 are respectively used for indicating the smart air conditioner and the television; assuming that the third input is a gesture input of the user in the environment, as shown in (a) of fig. 7, the hand of the user may be pointed to the television in the target area in the environment (at this time, the camera of the electronic device may capture the gesture of the user), that is, the electronic device receives the third input of the user; or the third input is a touch gesture input of the user on the display screen, as shown in (b) of fig. 7, the user may click on the identifier 73 indicating the television, that is, the electronic device receives the third input of the user; then, the electronic device may cancel the display of the identification indicating the smart air conditioner 71 and maintain the display of the identification 73 indicating the television 72, that is, the electronic device cancels the display of the first identification, in response to the third input, as shown in (c) of fig. 7. Therefore, the user can flexibly determine the household equipment required to be operated by the user through the electronic equipment, and the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

Optionally, in the embodiment of the present invention, the home device indicated by the second identifier may be a home device in a target area, and a user needs to be controlled by an electronic device. The home equipment indicated by the first identifier may be in a target area, and a user does not need to use home equipment controlled by electronic equipment.

In the embodiment of the invention, the user can trigger the electronic equipment to cancel the display of the first identifier by inputting the second identifier, namely, the user can flexibly determine the household equipment required to be operated by the user through the electronic equipment, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

Optionally, in the embodiment of the present invention, it is assumed that the electronic device performs device identification on the shooting preview interface and identifies at least one piece of home equipment, then the user may directly input an image of the home equipment in the shooting preview interface, so as to trigger the electronic device to determine home equipment (for example, the following M pieces of home equipment) corresponding to the user input as the target home equipment. Wherein N and M are positive integers, and M is less than N.

For example, in the embodiment of the present invention, before the step 201, the method for controlling home equipment according to the embodiment of the present invention may further include the following step 209 and step 210.

And step 209, the electronic device receives a fourth input of the user to the M household devices in the shooting preview interface.

And step 210, the electronic equipment responds to the fourth input, and the M household equipment are determined as target household equipment.

Optionally, in the embodiment of the present invention, the M pieces of home equipment are home equipment identified by the electronic device.

Optionally, in an embodiment of the present invention, the fourth input may be a gesture input performed in an environment or on a display screen along a second preset trajectory. For the description of the input form of the fourth input, reference may be specifically made to the related description of the input form of the first input in the foregoing embodiment, and details are not repeated here to avoid repetition.

Optionally, in the embodiment of the present invention, the electronic device may determine, as the target home devices, M home devices located in the fourth input track in the shooting preview interface. The image of each household device in the M household devices may be entirely located in the input track of the fourth input, or partially located in the input track of the fourth input, and may be determined specifically according to an actual use requirement, which is not limited in the embodiment of the present invention.

For example, the control method for the home devices provided by the embodiment of the invention can be applied to a scene in which a user triggers the electronic device to determine a plurality of home devices as target home devices by inputting the plurality of home devices in a shooting preview interface. Specifically, assuming that the second preset trajectory is a circle, as shown in (a) of fig. 8, if the shooting preview interface 80 of the electronic device includes an image 81 of a smart lamp, an image 82 of a television, and an image 83 of a smart air conditioner, the user may draw a circle (i.e., the second preset trajectory) on the shooting preview interface 80, that is, the electronic device receives a fourth input of the user to M home devices in the shooting preview interface, and then the electronic device may determine the smart lamp and the television as target home devices in response to the fourth input, that is, the electronic device determines the M home devices as target home devices. Therefore, the electronic equipment can trigger the electronic equipment through one input to determine the intelligent lamp and the television as the target household equipment, so that the efficiency of determining the intelligent lamp and the television by the electronic equipment can be improved, and the human-computer interaction performance is improved.

It should be noted that, after the electronic device determines the smart lamp and the television as the target home device, as shown in (b) in fig. 8, the user may draw a √ (i.e., the first input mentioned above) on the shooting preview interface 80, trigger the electronic device to display an AR control in the shape of the √ (i.e., the second input mentioned above), and then the user may click on the AR control in the shape of the √ (i.e., the second input mentioned above) to trigger the electronic device to control the smart lamp and the television to turn on.

In the embodiment of the invention, the user can trigger the electronic equipment to determine the plurality of household equipment in the shooting preview interface as the target household equipment through the input on the shooting preview interface, namely, the user can trigger the electronic equipment to determine the plurality of household equipment as the target household equipment through one input, so that the efficiency of determining the target household equipment by the electronic equipment can be improved, and the man-machine interaction performance is improved.

Optionally, in the embodiment of the present invention, after the electronic device displays the target AR control, the electronic device may further obtain environment information of an environment where the target device is currently located, and update the target control operation corresponding to the target AR control according to the environment information, so that after the user performs the second input, the electronic device may control the target device to perform the updated target control operation.

For example, in the embodiment of the present invention, after the step 202, the method for controlling home equipment according to the embodiment of the present invention may further include the following steps 211 to 213.

Step 211, the electronic device obtains an input track of the first input.

Step 212, the electronic device obtains the target environment information when the first input trajectory is the first preset trajectory.

And step 213, the electronic device updates the target control operation corresponding to the target AR control according to the target environment information.

The target environment information may be used to indicate an environment state of an environment where the target home device is currently located.

Optionally, in the embodiment of the present invention, the electronic device updates the target control operation corresponding to the target AR control according to the target environment information, which may be understood as that the electronic device performs scene drawing, so that the working state of the home device changes based on scene change.

For example, the control method for the home equipment provided by the embodiment of the invention can be applied to a scene that the user needs the electronic equipment to control the working state of the target home equipment based on the environment state. For example, assuming that the first preset trajectory is a pattern of a bed, if the user draws the bed (that is, the input trajectory of the first input is the first preset trajectory), it indicates that the user needs to sleep, and then the electronic device may determine the environmental state of the current environment of the target home device according to the current time and the current brightness, and adjust the most suitable sleep scenario for the user. If the current time is day, the electronic device can control the curtain to be completely pulled up, and if the current time is night, the electronic device can close all sounding household devices (such as a television, an intelligent sound and the like). For another example, assuming that the first preset trajectory is a pattern of a moon, if the user draws a moon, the electronic device may turn off other home devices except the lamps and the home devices (e.g., refrigerator) that must be turned on among the target home devices. Therefore, the flexibility of controlling the target household equipment through the electronic equipment can be improved.

In the embodiment of the present invention, since the electronic device may update the target control operation corresponding to the target AR control according to the environmental state of the current environment in which the target home device is located, after the user executes the second input, the electronic device may control the target device to execute the updated target control operation. Therefore, the flexibility of controlling the target household equipment through the electronic equipment can be improved.

Optionally, in this embodiment of the present invention, after the electronic device displays the target AR control, that is, after the step 202 is executed, when the electronic device exits from the shooting preview interface (for example, the electronic device closes the camera application, or runs the camera application in the background of the electronic device), the electronic device may keep displaying the target AR control on the display screen of the electronic device. Therefore, the user can input the target AR control on other interfaces except the shooting preview interface, and the electronic equipment is triggered to control the target household equipment to execute the target control operation corresponding to the target AR control.

In the embodiment of the invention, the electronic equipment can keep the display of the target AR control after the electronic equipment exits from the shooting preview interface, so that a user can directly input the generated target AR control to trigger the electronic equipment to control the target household equipment to execute the target control operation corresponding to the target AR control, and the electronic equipment can be triggered to control the target household equipment to execute the control operation after the target AR control is generated without triggering again, so that the operation convenience and flexibility of controlling the household equipment can be further improved.

Optionally, in the embodiment of the present invention, after the electronic device displays the target AR control, that is, after the step 202 is executed, the electronic device may generate the first control according to the target control operation that is executed by the user when the electronic device is triggered, and display the first control on the display screen of the electronic device, so that the user may trigger the electronic device to execute the target control operation through input to the first control, and thus, convenience in controlling the home equipment may be further improved.

It can be understood that, because one target control operation may correspond to one first control, when a plurality of target control operations are triggered and executed by a user, the number of the first controls is also multiple, that is, a plurality of first controls may be displayed on the display screen of the electronic device.

Optionally, in the embodiment of the present invention, each first control may include a name of the corresponding home device and a target control operation corresponding to the input trajectory of the first input. For example, if the first control is used to trigger the electronic device to control the smart air conditioner to adjust the temperature to 25 ℃, then the first control may display: "intelligent air conditioner: 25 ℃ in text.

For example, when the number of first controls displayed on the display screen of the electronic device may be P, where P is a positive integer, the M first controls may be generated by the electronic device according to P target control operations that are recently triggered by the user. For example, it is assumed that the number of times that the user historically triggers the target control operation is 5, and the 5 target control operations are sequentially: turning on a television, turning off the television, starting an intelligent washing machine, turning off an intelligent lamp and adjusting the temperature of an intelligent air conditioner to 25 ℃, if M target control operations displayed on a display screen of the electronic equipment are generated according to target control operations triggered by a user for the last 3 times (namely M is 3); then, the 3 first controls displayed on the display screen of the electronic device are: the intelligent air conditioner comprises a control for triggering and starting the intelligent washing machine, a control for triggering and closing the intelligent lamp and a control for triggering and adjusting the temperature of the intelligent air conditioner to be 25 ℃.

Optionally, in the embodiment of the present invention, the electronic device may display the first control on the display screen of the electronic device in the following manner a and manner B.

Mode A: the electronic device directly displays the one or more first controls on a display screen of the electronic device.

Mode B: the electronic device may display one or more first controls on a display screen of the electronic device in a hidden manner. Specifically, the electronic device may display one or more first controls in another control (hereinafter referred to as a second control), and display the second control on a display screen of the electronic device. In this manner, a user may trigger the electronic device to display one or more first controls via input to the second control.

For example, as shown in (a) of fig. 9, the electronic device may hover to display a second control 91 on a desktop 90 of the electronic device with a preset transparency so that a user may click on the second control 91 to trigger the electronic device to display a first control, and then as shown in (b) of fig. 9, the electronic device may display a first control 92 for instructing the television to turn on and a first control 93 for instructing the smart air conditioner to adjust the temperature to 25 ℃.

Optionally, in the embodiment of the present invention, after the electronic device displays the first control, the second control may keep displaying or cancel displaying (fig. 9 (b) illustrates an example of keeping displaying the second control 91), which may be specifically determined according to actual use requirements, and the embodiment of the present invention is not limited.

Further, after the electronic device displays the first control, when the second control remains displayed, the user may cancel the display of the first control by clicking on the second control or clicking in a blank of the currently displayed interface; when the second control cancels the display, the user can click on a blank of the currently displayed interface to cancel the display of the first control.

In the embodiment of the invention, the electronic equipment can generate and display the first control according to the target control operation, so that a user can trigger the electronic equipment to control the household equipment indicated by the first control to execute the target control operation corresponding to the first input through the input of the first control under the condition that the electronic equipment displays any interface. Therefore, the operation convenience of controlling the household equipment can be further improved.

Optionally, in this embodiment of the present invention, after the electronic device determines that the target AR device, that is, the electronic device performs step 208, or after the user performs step 210, the electronic device may further display a function menu of the target home device. Therefore, the user can directly input the function menu of the target household equipment, and the electronic equipment can be triggered to control the target household equipment to execute the operation corresponding to the input of the user, so that the user does not need to trigger and generate the target AR control.

Optionally, in the embodiment of the present invention, the function menu may include information of the target home device corresponding to the function menu, for example, the name of the target home device, so that a user may quickly and accurately distinguish the function menu of each target home device.

For example, assuming that the target home devices are smart pressure cookers and smart lamps, after the electronic device determines the target home devices, as shown in fig. 10, the electronic device may display a function menu 95 of the smart pressure cookers and a function menu 96 of the smart lamps on images of the respective target home devices in the shooting preview interface 94.

Optionally, in the embodiment of the present invention, the function menu of the target home device may include at least one menu item, where the at least one menu item corresponds to at least one function of the target home device. For example, as shown in fig. 10, assuming that the target household device is a smart lamp 95, a menu item "power" in a function menu of the smart lamp may correspond to a power key in a remote controller of the television, that is, a user may trigger the electronic device to control the smart lamp to turn on or off by inputting the menu item "power".

Optionally, in the embodiment of the present invention, the function menu is equivalent to a virtual controller of the target home device, that is, a user may operate the virtual controller to operate the target home device.

Optionally, in the embodiment of the present invention, the at least one menu item may be a menu item of all functions of the target home equipment; or, the at least one menu item may be a menu item of a partial function of the target home device. The method can be determined according to actual use requirements, and the embodiment of the invention is not limited.

Optionally, in the embodiment of the present invention, the function menu may be pre-stored in a database (for example, a function menu of a television), or may be recognized from a preset image for the electronic device, and may be determined specifically according to an actual use requirement, which is not limited in the embodiment of the present invention.

Optionally, in the embodiment of the present invention, the electronic device may display the function menu in an AR form.

In the embodiment of the invention, as the electronic equipment can display the function menu of the target household equipment, the user does not need to remember the corresponding gesture of the corresponding target control operation to trigger the electronic equipment to accurately control the target household equipment like a gesture input mode, so that the operation control process can be further simplified on the basis of accurately controlling the household equipment.

In order to more clearly describe the control method of the home device provided by the embodiment of the present invention, an exemplary description is given below of a manner in which a user triggers the electronic device to enter device information of the home device.

Optionally, in the embodiment of the present invention, the user may implement the entry of the device information in a manual input manner (the first manner), or implement the entry of the device information in an image recognition manner (the second manner).

First mode

In the embodiment of the invention, the user can complete the input of the equipment information of the household equipment on the equipment management interface of the electronic equipment.

Specifically, the shooting preview interface of the electronic device may include an entry identifier of a device management interface, and a user may trigger entry into the device management interface through input of the entry identifier, as shown in fig. 11 (a), the device management interface 120 may include at least one of an AR-controlled start identifier 121, a device identifier 122 of a logged-in device, a device add identifier, and a global gesture identifier.

The start identifier 121 may be configured to trigger turning on or turning off a function of controlling the home device by the electronic device; the device identification 122 of the entered device may be used to trigger a device details interface (123 shown in fig. 11 (b)) into the device; the equipment adding identifier can be used for recording the household equipment; the global gesture identification may be used to trigger setting a global gesture (the global gesture may take effect on all home devices that have been entered).

Optionally, in this embodiment of the present invention, as shown in fig. 11 (b), the device detail interface 123 may include a virtual controller 124 (i.e., a function menu) identifier and a custom command identifier of a certain entered home device. The user can check the functional description of the home equipment corresponding to the virtual controller by clicking the virtual controller identifier; the user can complete the input of the input track of the first input and the target control operation through the self-defined command identifier (which can be determined according to the actual use requirement).

Second mode

In the embodiment of the invention, the user can trigger the electronic equipment to acquire the image of the household equipment input by the user, so that the electronic equipment can compare the equipment image acquired by the electronic equipment with the preset image, and under the condition that the acquired equipment image conforms to the preset image, the electronic equipment can acquire the equipment information corresponding to the preset image, thereby realizing the automatic input of the equipment information.

It should be noted that, in the first manner and the second manner, after the electronic device obtains the device information of one device, the electronic device may match the home device indicated by the device information, and after the electronic device is successfully matched with the home device, the electronic device may display the device identifier of the home device on the device management interface. Therefore, the user can control the household equipment through the electronic equipment.

Optionally, in the embodiment of the present invention, after the electronic device successfully enters one piece of home equipment, the electronic device may obtain a control menu of the home equipment from a database (in a server or in the electronic device); or the user may also customize the corresponding relationship between the input trajectory input by the user and the target control operation, which may be specifically determined according to the actual use requirement, and the embodiment of the present invention is not limited.

In the embodiment of the present invention, the control method of the home equipment shown in the above method drawings is exemplarily described by referring to one drawing in the embodiment of the present invention. In specific implementation, the control method for the home equipment shown in the above method drawings may also be implemented by combining any other drawings that may be combined and are illustrated in the above embodiments, and details are not described here.

As shown in fig. 12, an embodiment of the present invention provides an electronic device, which may include: a receiving module 131, a display module 132 and a processing module 133. A receiving module 131, configured to receive a first input of a user on a shooting preview interface; a display module 132, configured to display, in response to the first input received by the receiving module 131, a target AR control, where the target AR control may be an AR control generated based on an input trajectory of the first input; the receiving module 131 may be further configured to receive a second input of the target AR control displayed by the display module 132 from the user; the processing module 133 may be configured to, in response to the second input received by the receiving module 131, control the target home equipment to execute a target control operation corresponding to the target AR control; the shooting preview interface can be a collected image of a target area, and the target area can include target household equipment.

In the embodiment of the invention, the electronic equipment can generate and display the target AR control for controlling the target household equipment to execute the target control operation corresponding to the target AR control according to the input track input by the user on the shooting preview interface, so that the user can directly trigger the electronic equipment to control the target household equipment to execute the target control operation through the input of the target AR control, and therefore, the user does not need to trigger the electronic equipment to control the household equipment through a specific application program installed on the electronic equipment. Therefore, the electronic equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

Optionally, in this embodiment of the present invention, the electronic device may further include an identification module. The receiving module 131 may be further configured to perform device identification on the shooting preview interface before the receiving module 131 receives the first input, and identify at least one piece of home equipment, where the at least one piece of home equipment includes a target home equipment; the display module can be further used for displaying at least one mark on the shooting preview interface, and each mark can be used for indicating one household device in the shooting preview interface.

Optionally, in this embodiment of the present invention, the receiving module 131 may be further configured to receive a third input of the user after the displaying module displays at least one identifier on the shooting preview interface; the display module may be further configured to cancel, in response to the third input received by the receiving module 131, display of the first identifier, where the first identifier may be an identifier of the at least one identifier, except for the second identifier, and the second identifier may be used to indicate the target home device.

Optionally, in this embodiment of the present invention, the electronic device may further include a determination module. The shooting preview interface can comprise N household devices. The receiving module 131 may be further configured to receive a fourth input of the user to the M home devices in the shooting preview interface before the first input of the user on the shooting preview interface is received; a determining module, configured to determine, in response to the fourth input received by the receiving module 131, the M home devices as target home devices.

Optionally, in the embodiment of the present invention, the target home device may be a television, and the electronic device may further include an obtaining module and a sending module. The sending module may be configured to, after the receiving module 131 receives a first input of a user on the shooting preview interface, obtain an input track of the first input; the sending module may be configured to send a target control message to the target home device under the condition that the first input track is associated with the shooting preview interface, where the target control message may be used to instruct the target home device to display the first information; the display module 132 may be further configured to display second information after the receiving module 131 receives the first input, where an input track of the first input is not associated with the shooting preview interface, where the second information may include at least one alternative operation option.

Optionally, in this embodiment of the present invention, the electronic device may further include an obtaining module, an updating module, and an obtaining module. The acquisition module may be configured to acquire target environment information after the display module displays the target AR control and under the condition that the first input trajectory is a first preset trajectory, where the target environment information may be used to indicate an environment state of an environment where the target home equipment is currently located; and the updating module can be used for updating the target control operation corresponding to the target AR control according to the target environment information acquired by the acquiring module.

Optionally, in this embodiment of the present invention, the target AR control may include a first target sub AR control and a second target sub AR control; the first target sub-AR control may be an AR control generated based on an input trajectory of the first input. The display module can comprise a display submodule, a determination submodule and a generation submodule; the display sub-module can be used for displaying and displaying the first target sub-AR control; the determining sub-module may be configured to determine a control type of the first target sub-AR control after the display sub-module displays the first target sub-AR control; the generation module can generate a second target sub-AR control based on the control type of the first target sub-AR control determined by the determination sub-module; the display sub-module can be further used for displaying a second target sub-AR control generated by the generation sub-module on the first target sub-AR control; wherein the second input may be an input to a second target sub-AR control.

In the embodiment of the invention, the electronic equipment can automatically generate the second target sub-AR control corresponding to the control type of the first target sub-AR control according to the type of the first target sub-AR control, so that the process of generating the target AR control by the electronic equipment can be simplified, and the man-machine interaction performance is improved.

Optionally, in this embodiment of the present invention, the display module may be further configured to, after the target AR control is displayed, keep displaying the target AR control on a display screen of the electronic device when the shooting preview interface is exited.

The electronic device 130 provided in the embodiment of the present invention can implement each process implemented by the electronic device shown in the foregoing method embodiment, and is not described here again to avoid repetition.

The embodiment of the invention provides electronic equipment, which can receive first input of a user on a shooting preview interface (including target household equipment); and in response to the first input, displaying a target AR control (an AR control generated for an input trajectory based on the first input); receiving a second input of the target AR control by the user; and responding to the second input, and controlling the target household equipment to execute target control operation corresponding to the target AR control. According to the scheme, the electronic equipment can generate and display the target AR control for controlling the target household equipment to execute the target control operation corresponding to the target AR control according to the input track input by the user on the shooting preview interface, so that the user can directly trigger the electronic equipment to control the target household equipment to execute the target control operation through the input of the target AR control, and therefore the user does not need to trigger the electronic equipment to control the household equipment through a specific application program installed on the electronic equipment. Therefore, the electronic equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the electronic equipment can be improved.

Fig. 13 is a schematic diagram of a hardware structure of an electronic device implementing various embodiments of the present invention. As shown in fig. 13, the electronic device 100 includes, but is not limited to: a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, a power supply 111, and an image capturing apparatus 112. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 13 does not constitute a limitation of the electronic device, and that the electronic device may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, a pedometer, and the like.

The user input unit 107 is used for receiving a first input of a user on the shooting preview interface; a display unit 106, configured to display, in response to the first input received by the user input unit 107, a target AR control, where the target AR control is an AR control generated based on an input trajectory of the first input; the user input unit 107 is further configured to receive a second input of the target AR control displayed by the display unit 106 from the user; the processor 110 is configured to, in response to the second input received by the user input unit 107, control the target home equipment to perform a target control operation corresponding to the target AR control; the shooting preview interface comprises target household equipment.

It can be understood that, in the embodiment of the present invention, the receiving module 131 in the structural schematic diagram of the electronic device (for example, fig. 12) may be implemented by the user input unit 107. The display module 132 in the structural schematic diagram of the electronic device (for example, fig. 12) can be implemented by the display unit 106. The processing module 133 in the structural schematic diagram of the electronic device (for example, fig. 12) may be implemented by the processor 110.

It should be understood that, in the embodiment of the present invention, the radio frequency unit 101 may be used for receiving and sending signals during a message transmission or call process, and specifically, after receiving downlink data from a base station, the downlink data is processed by the processor 110; in addition, the uplink data is transmitted to the base station. Typically, radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 can also communicate with a network and other devices through a wireless communication system.

The electronic device provides wireless broadband internet access to the user via the network module 102, such as assisting the user in sending and receiving e-mails, browsing web pages, and accessing streaming media.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the network module 102 or stored in the memory 109 into an audio signal and output as sound. Also, the audio output unit 103 may also provide audio output related to a specific function performed by the electronic apparatus 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 includes a speaker, a buzzer, a receiver, and the like.

The input unit 104 is used to receive an audio or video signal. The input unit 104 may include a Graphics Processing Unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes image data of a still picture or a video obtained by an image capturing apparatus in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphic processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the network module 102. The microphone 1042 may receive sound and may be capable of processing such sound into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 101 in case of a phone call mode.

The electronic device 100 also includes at least one sensor 105, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and a proximity sensor that can turn off the display panel 1061 and/or the backlight when the electronic device 100 is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 105 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.

The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), or the like.

The user input unit 107 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 107 includes a touch panel 1071 and other input devices 1072. Touch panel 1071, also referred to as a touch screen, may collect touch operations by a user on or near the touch panel 1071 (e.g., operations by a user on or near touch panel 1071 using a finger, stylus, or any suitable object or attachment). The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 110, and receives and executes commands sent by the processor 110. In addition, the touch panel 1071 may be implemented in various types, such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.

Further, the touch panel 1071 may be overlaid on the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or nearby, the touch panel 1071 transmits the touch operation to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Although in fig. 13, the touch panel 1071 and the display panel 1061 are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the electronic device, and is not limited herein.

The interface unit 108 is an interface for connecting an external device to the electronic apparatus 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic apparatus 100 or may be used to transmit data between the electronic apparatus 100 and the external device.

The memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 109 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 110 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the electronic device. Processor 110 may include one or more processing units; alternatively, the processor 110 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

The electronic device 100 may further include a power supply 111 (e.g., a battery) for supplying power to each component, and optionally, the power supply 111 may be logically connected to the processor 110 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system.

In addition, the electronic device 100 includes some functional modules that are not shown, and are not described in detail herein.

Optionally, an embodiment of the present invention further provides an electronic device, which includes the processor 110 shown in fig. 13, the memory 109, and a computer program stored in the memory 109 and capable of running on the processor 110, where the computer program, when executed by the processor 110, implements the processes of the foregoing method embodiment, and can achieve the same technical effect, and details are not described here to avoid repetition.

Optionally, in this embodiment of the present invention, the electronic device in the above embodiment may be an AR device. Specifically, when the electronic device in the above embodiment (for example, the electronic device shown in fig. 13) is an AR device, the AR device may include all or part of the functional modules in the electronic device. Of course, the AR device may further include a functional module not included in the electronic device.

It is to be understood that, in the embodiment of the present invention, when the electronic device in the above embodiment is an AR device, the electronic device may be an electronic device integrated with AR technology. The AR technology is a technology for realizing the combination of a real scene and a virtual scene. By adopting the AR technology, the visual function of human can be restored, so that human can experience the feeling of combining a real scene and a virtual scene through the AR technology, and further the human can experience the experience of being personally on the scene better.

The AR equipment can comprise the camera, so that the AR equipment can be combined with the virtual picture to display and interact on the basis of the picture shot by the camera. For example, in the embodiment of the present invention, the AR device may receive a first input of a user on a shooting preview interface (including a target home device); and in response to the first input, displaying a target AR control (an AR control generated for an input trajectory based on the first input); receiving a second input of the target AR control by the user; and responding to the second input, and controlling the target household equipment to execute target control operation corresponding to the target AR control. According to the scheme, the AR equipment can generate and display the AR control used for controlling the target household equipment to execute the target control operation corresponding to the target AR control according to the input track input by the user on the shooting preview interface, so that the user can directly trigger the electronic equipment to control the target household equipment to execute the target control operation through the input of the target AR control, and the user is not required to trigger the AR equipment to control the household equipment through a specific application program installed on the AR equipment. Therefore, the AR equipment can be triggered to control the household equipment without inputting a specific application program by a user, so that the flexibility and convenience of controlling the household equipment through the AR equipment can be improved.

Furthermore, in the embodiment of the invention, as the home equipment seen by the user on the display screen of the AR equipment is the same as the home equipment seen by the user in a real scene, and the user directly operates the home equipment in the environment through the input on the shooting preview interface, the user can see and obtain the information, and the user can see and control the information, so that the flexibility and convenience of controlling the home equipment through the AR equipment can be further improved on the basis of simplifying the process of controlling the home equipment through the AR equipment, and the man-machine interaction performance can be improved.

An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the processes of the foregoing method embodiments, and can achieve the same technical effects, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may include a read-only memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, and the like.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling an electronic device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A control method of household equipment is applied to electronic equipment, and is characterized by comprising the following steps:

receiving a first input of a user on a shooting preview interface;

responding to the first input, and displaying a target AR control, wherein the target AR control is an AR control generated based on an input track of the first input;

receiving a second input of the target AR control by the user;

responding to the second input, and controlling target household equipment to execute target control operation corresponding to the target AR control;

and the shooting preview interface comprises the target household equipment.

2. The method of claim 1, wherein prior to receiving the first input from the user on the capture preview interface, the method further comprises:

performing equipment identification on the shooting preview interface, and identifying at least one piece of home equipment, wherein the at least one piece of home equipment comprises the target home equipment;

and displaying at least one mark on the shooting preview interface, wherein each mark is used for indicating one household device in the shooting preview interface.

3. The method of claim 2, wherein after displaying the at least one indicator on the capture preview interface, the method further comprises:

receiving a third input of the user;

and canceling the display of a first identifier in response to the third input, wherein the first identifier is an identifier of the at least one identifier except for a second identifier, and the second identifier is used for indicating the target household equipment.

4. The method according to claim 1, wherein the shooting preview interface comprises N home devices;

before the receiving the first input of the user on the shooting preview interface, the method further comprises:

receiving fourth input of the user to the M household devices in the shooting preview interface;

and responding to the fourth input, and determining the M household devices as the target household devices.

5. The method of claim 1, wherein the target home device is a television, and wherein after receiving the first input from the user on the preview screen, the method further comprises:

acquiring an input track of the first input;

under the condition that the first input track is associated with the shooting preview interface, sending a target control message to the target household equipment, wherein the target control message is used for indicating the target household equipment to display first information;

and displaying second information under the condition that the input track of the first input is not associated with the shooting preview interface, wherein the second information comprises at least one alternative operation option.

6. The method of claim 1, wherein after the displaying the target AR control, the method further comprises:

acquiring target environment information under the condition that the first input track is a first preset track, wherein the target environment information is used for indicating the environment state of the current environment where the target household equipment is located;

and updating the target control operation corresponding to the target AR control according to the target environment information.

7. The method of claim 1, wherein the target AR control comprises a first target sub AR control and a second target sub AR control; the first target sub-AR control is an AR control generated based on the input track of the first input;

the displaying a target AR control comprises:

displaying the first target sub-AR control;

determining a control type of the first target sub-AR control;

generating a second target sub-AR control based on the control type of the first target sub-AR control, and displaying the second target sub-AR control on the first target sub-AR control;

wherein the second input is an input to the second target sub-AR control.

8. The method of claim 1, wherein after displaying a target AR control in response to the first input, the method further comprises:

and under the condition that the shooting preview interface is quitted, the target AR control is kept displayed on the display screen of the electronic equipment.

9. An electronic device, characterized in that the electronic device comprises: the device comprises a receiving module, a display module and a processing module;

the receiving module is used for receiving a first input of a user on a shooting preview interface;

the display module is used for responding to the first input received by the receiving module and displaying a target AR control, and the target AR control is generated based on an input track of the first input;

the receiving module is further configured to receive a second input of the target AR control from the user;

the processing module is used for responding to the second input received by the receiving module and controlling the target household equipment to execute the target control operation corresponding to the target AR control;

and the shooting preview interface comprises the target household equipment.

10. An electronic device, comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the control method of the home device according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when being executed by a processor, implements the steps of the control method of a home appliance according to any one of claims 1 to 8.