CN114721310A

CN114721310A - Intelligent device control method, expansion device control device and storage medium

Info

Publication number: CN114721310A
Application number: CN202210369431.XA
Authority: CN
Inventors: 刘宇超; 赵非
Original assignee: Beijing Youzhuju Network Technology Co Ltd
Current assignee: Beijing Youzhuju Network Technology Co Ltd
Priority date: 2022-04-08
Filing date: 2022-04-08
Publication date: 2022-07-08

Abstract

The disclosure relates to an intelligent device control method, an extended device control device and a storage medium, wherein the intelligent device control method comprises the following steps: acquiring behavior information of a user under the condition that the expansion equipment and the user meet preset conditions; if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention; and sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information. The method and the device can improve the operation efficiency of the user, and further improve the user experience.

Description

Intelligent device control method, expansion device control device and storage medium

Technical Field

The present disclosure relates to the field of intelligent control technologies, and in particular, to an intelligent device control method, an extended device control apparatus, and a storage medium.

Background

With the popularization of smart devices, people's life and working ways have changed greatly, and compared with the past that users usually use one smart device alone, target users can use a plurality of smart devices at the same time.

However, when a user uses multiple kinds of smart devices at the same time, the use is inconvenient.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, the present disclosure provides an intelligent device control method, including:

acquiring behavior information of a user under the condition that the expansion equipment and the user meet preset conditions;

if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention;

and sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information.

In a second aspect, the present disclosure provides an expansion device control method, including:

responding to control information sent by the intelligent equipment, and executing an action corresponding to the control information;

the control information is obtained by acquiring behavior information of a user when detecting that both the expansion equipment and the user meet preset conditions; and generating according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

In a third aspect, the present disclosure provides an intelligent device control apparatus, including:

the behavior information acquisition module is used for acquiring the behavior information of the user under the condition that the expansion equipment and the user are detected to meet the preset conditions;

the control information generation module is used for generating control information according to the appointed behavior intention if the user is determined to have the appointed behavior intention according to the behavior information of the user;

and the sending module is used for sending the control information to the expansion equipment, and the control information is used for the expansion equipment to execute the action corresponding to the control information.

In a fourth aspect, the present disclosure provides an extension device control apparatus, including:

the execution module is used for responding to the control information sent by the intelligent equipment and executing the action corresponding to the control information;

the control information is behavior information of the user obtained by the intelligent equipment when the intelligent equipment detects that both the expansion equipment and the user meet preset conditions; and generating according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

In a fifth aspect, the present disclosure provides a computer readable medium having stored thereon a computer program which, when executed by a processing apparatus, performs the steps of the method of the first or second aspect.

In a sixth aspect, the present disclosure provides a smart device comprising:

a storage device having one or more computer programs stored thereon;

one or more processing devices for executing the one or more computer programs in the storage device to implement the steps of the method of the first aspect.

In a seventh aspect, the present disclosure provides an expansion device, including:

a storage device having one or more computer programs stored thereon;

one or more processing devices for executing the one or more computer programs in the storage device to implement the steps of the method of the second aspect.

By the technical scheme, the behavior information of the user is acquired under the condition that the expansion equipment and the user meet the preset conditions; if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention; and sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information. Therefore, the intelligent device can determine whether the user and the expansion device are near the intelligent device according to whether the user and the expansion device meet the preset condition, so as to determine whether the user has the environment condition for using the expansion device, and under the condition that the environment condition is met, the behavior intention of the user is identified through the intelligent device, the expansion device is automatically controlled to make corresponding actions according to the behavior intention of the user, so that the step of manually starting the expansion device by the user is omitted, the user can directly use the expansion device, the use efficiency of the expansion device is improved, and the user experience is further improved.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale. In the drawings:

fig. 1 is a schematic diagram of an application environment of an intelligent device control method and an extended device control method according to an exemplary embodiment.

Fig. 2 is a flow chart illustrating a smart device control method according to an exemplary embodiment.

Fig. 3 is a flowchart illustrating a smart device control method according to another exemplary embodiment.

Fig. 4 is a flow chart illustrating an expansion device control method according to an example embodiment.

Fig. 5 is a flowchart illustrating an expansion device control method according to another exemplary embodiment.

Fig. 6 is an interface schematic diagram of a display interface of the expansion device shown in the embodiment of fig. 5.

Fig. 7 is a flowchart illustrating a control method of a smart device system according to an exemplary embodiment.

Fig. 8 is a block diagram illustrating an intelligent device control apparatus according to an example embodiment.

Fig. 9 is a block diagram illustrating an expansion device control apparatus according to an example embodiment.

Fig. 10 is a schematic structural diagram of an electronic device according to an exemplary embodiment.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a" or "an" in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will appreciate that references to "one or more" are intended to be exemplary and not limiting unless the context clearly indicates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

In the internet of everything, communication connection can be established between intelligent devices with different functions to realize linkage use, for example, an intelligent lamp can realize functions of word searching, voice interaction, sitting posture detection and the like, communication connection can be established through Wireless Fidelity (WiFi) or bluetooth, and a tablet personal computer or a mobile phone is used to enable the tablet personal computer or the mobile phone to become an extension screen thereof, and the tablet personal computer can display a User Interface (UI) of the intelligent lamp to provide better operation experience.

However, in the related art, when the intelligent lamp is connected to an extended screen (a tablet computer or a mobile phone, etc.), it is troublesome to actively unlock a screen of the extended screen and start corresponding application software by a user. If the number of the expansion devices needing linkage is more, the user needs to operate the expansion devices one by one, and the user experience is reduced.

In order to solve the above problems, the present disclosure provides an intelligent device control method, an expansion device control device, and a storage medium, which can improve the use efficiency of the expansion device and improve the use experience of a user.

An application environment of the intelligent device control method and the expansion device control method in this embodiment is described below, as shown in fig. 1, the application environment may be an intelligent device system 10, and the intelligent device system 10 may include an intelligent device 11 and expansion devices 12, where the intelligent device 11 may correspond to one or more expansion devices 12 in advance, each of the intelligent device 11 and the expansion devices 12 has a communication device, and a communication link may be established between the intelligent device 11 and the expansion device 12 that correspond to each other through the communication device, so as to perform data transmission. The smart device 11 may further include one or more detection devices, such as a human body detection device (e.g., an infrared sensor, an ultrasonic sensor, etc.), an image detection device (e.g., a high definition camera, a depth camera, etc.), an audio detection device (e.g., a microphone), and the like.

Optionally, the expansion devices 12 corresponding to different smart devices 11 in advance may be the same or different, and are not limited herein.

Optionally, the smart device 11 and the expansion device 12 include, but are not limited to: intelligent lamps and lanterns, panel computer, smart mobile phone, intelligent table chair, intelligent audio amplifier, intelligent TV etc..

Fig. 2 is a flowchart illustrating a smart device control method according to an exemplary embodiment, which may be applied to the smart device shown in fig. 1, as shown in fig. 2, and which may include the steps of:

110. and acquiring the behavior information of the user under the condition that the expansion equipment and the user are detected to meet the preset conditions.

In some embodiments, if the smart device detects that the distance between the expansion device and the smart device is less than or equal to a preset distance, and the distance between the user and the smart device is less than or equal to the preset distance, it is determined that the expansion device and the user are detected to satisfy the preset condition. It will be appreciated that the above-described,

it is understood that, in the present embodiment, when the distance between the expansion device and the smart device is smaller than the preset distance, the expansion device may be regarded as being in the vicinity of the smart device. When the distance between the user and the smart device is less than the preset distance, the user can be considered to be in the vicinity of the smart device.

As a manner, the smart device may detect whether a distance between the user and the smart device is smaller than a preset distance through a detection device of the smart device, for example, taking a smart lamp (e.g., a smart desk lamp) as an example, the detection device of the smart lamp includes an infrared human body sensor, where the infrared human body sensor is capable of sensing whether a person is present in a range with the smart lamp as a center and the preset distance as a radius, and if the person is present, it may be determined that the user meets a preset condition, so as to determine that the user is near the smart device.

As a manner, the smart device may detect whether the distance between the expansion device and the smart device is smaller than a preset distance through its communication device, for example, the expansion device is a tablet computer with a bluetooth communication function, the communication device of the smart device is a bluetooth communication module, and the smart device may search for other bluetooth devices through the bluetooth communication module. In some embodiments, when the smart device detects that the extension device and the user satisfy the preset condition, the behavior information of the user may be detected by the detection apparatus. For example, voice information of the user is detected by a microphone, and the voice information is taken as behavior information. For another example, the image information including the user is detected by a camera, and the posture and the motion information of the user are analyzed from the image information as the behavior information.

In other embodiments, the preset condition may be that the smart device detects that the user is in its vicinity.

As an example, the smart device may determine that the extension device and the user satisfy the preset condition when detecting that the user is in the vicinity thereof, acquire behavior information of the user, and then detect whether the extension device is in the vicinity of the smart device, and if the extension device is in the vicinity of the smart device, may continue to perform the subsequent step 120. 120. And if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention.

In some embodiments, the smart device may query a pre-configured behavioral intention database storing associations between a plurality of behavioral information and a plurality of representations of specified behavioral intentions according to the behavioral information, and determine whether the user has the specified behavioral intention. Illustratively, for example, if the behavior information is voice information of "dictionary look-up", it is detected that there is a behavior intention representation of "want to use dictionary application software" corresponding to the voice information in the behavior intention database, it may be determined that the user has a specified behavior intention. Conversely, if no behavioral intent representation corresponding to the speech information is detected in the behavioral intent database, it may be determined that the user does not have the specified behavioral intent. Alternatively, a given behavioral intent representation may correspond to one or more behavioral information. Alternatively, the designated behavioral intent representation may be a string of characters, with different designated behavioral intents corresponding to different characters.

After determining that the user has the designated behavior intention, the intelligent device may search, according to the representation of the designated behavior intention, control information corresponding to the designated behavior intention from a preset control information database, where the control information database stores in advance an association relationship between a plurality of control information and a plurality of representations of the designated behavior intention. Following the above example, for example, if the behavioral intent of "want to use dictionary application" indicates the corresponding control information a, the smart device may generate the control information a.

Optionally, the behavioral and intention database and the control information database may be disposed locally in the smart device, or may be disposed in a cloud server in communication connection with the smart device, which is not limited herein.

Optionally, after acquiring the behavior information, the smart device may determine, through its own processor, whether the user has a specified behavior intention. The behavior information may also be uploaded to the cloud server, and the cloud server determines whether the user has the designated behavior intention, and receives the determination result of the cloud server, which is not limited herein.

130. And sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information.

In some embodiments, the smart device may establish a communication link with the expansion device through the communication device, for example, establish a communication link with the expansion device through a WiFi connection, and then send the control information to the expansion device through the communication link, and the expansion device may perform an action corresponding to the control information after receiving the control information, for example, run dictionary application software on the tablet computer, so that when the user takes the tablet computer, the user may directly start using the dictionary application software.

As can be seen, in this embodiment, the behavior information of the user is obtained by detecting that the extension device and the user satisfy the preset condition; if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention; and sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information. Therefore, the intelligent device can determine whether the user and the expansion device are near the intelligent device according to whether the user and the expansion device meet the preset condition, so as to determine whether the user has the environment condition for using the expansion device, and under the condition that the environment condition is met, the behavior intention of the user is identified through the intelligent device, the expansion device is automatically controlled to make corresponding actions according to the behavior intention of the user, so that the step of manually starting the expansion device by the user is omitted, the user can directly use the expansion device, the use efficiency of the expansion device is improved, and the user experience is further improved.

Fig. 3 is a flowchart illustrating a smart device control method according to another exemplary embodiment, which may be applied to the smart device shown in fig. 1, as shown in fig. 3, and which may include the steps of:

210. and detecting whether the expansion equipment and the user meet preset conditions.

In some embodiments, the smart device includes an object detection apparatus, and the specific user detection in step 210 may include:

and if the preset target is detected by the target detection device within the latest specified time period, determining that the distance between the user and the intelligent equipment is less than the preset distance.

For example, the target detection device may include a human body detection device, the preset target may be a human body of the user, and the smart device may check whether the human body detection device detects the user within the last 1 minute, for example, the current time is 9:00, the smart device may check whether the human body detecting apparatus detects the user for a designated time period of 8:59 to 9:00, and if it detects, may determine that the distance between the user and the smart device is less than a preset distance, that is, the user is in the vicinity of the smart device, and determine that the user satisfies a preset condition. If not, the intelligent device can continue to detect through the human body detection device. Alternatively, the human body detecting means may include a proximity sensor, an infrared sensor, an ultrasonic sensor, a camera, and the like.

Optionally, the object detection device may further include a sound detection device, the preset object may be audio information of the user, and if the sound detection device detects the audio information of the user within a last specified time period, it may be determined that the user is in the vicinity of the smart device, and it is determined that the user satisfies the preset condition.

As one way, the method may further include: and if the current time is within the preset use time period of the intelligent equipment, starting the target detection device.

Illustratively, for example, the preset use time period is 18:00-21:00, and if the current time is 20:00, the target detection device may be started to detect the preset target. For another example, if the current time is 16:00, the target detection device may be turned off. Optionally, the preset usage time period may be set by a user through an interactive interface of the smart device, or may be determined by the smart device according to the historical usage time and the historical usage habit of the user on the smart device, for example, the time of using the smart device by the user is usually 18:00-21:00 per day, and the smart device may automatically set the time of using 18:00-21:00 as the preset usage time period.

Considering that the time period of using the intelligent device by the user is usually relatively fixed, for example, a student generally uses the intelligent lamp after school, if the intelligent device continuously detects through the human body detection device, unnecessary power consumption can be generated, in this embodiment, the human body detection device is turned on if the current time is in the preset time period of using the intelligent device, so that the power consumption of the intelligent device can be effectively saved.

In other embodiments, the smart device includes an interaction device, and the embodiment of specifically detecting the user in step 210 may include:

if the intelligent device detects that the interaction device is used in the latest specified time period, the distance between the user and the intelligent device is determined to be smaller than the preset distance, and therefore the user is determined to be in the vicinity of the intelligent device.

Illustratively, the interaction means may comprise a physical key, a touch screen, a microphone, etc., e.g. if the smart device detects that the user has used the touch screen within the last 10 seconds, this may indicate that the user is not far from the smart device, so that it may be accurately determined that the user is in the vicinity of the smart device.

In some embodiments, the smart device may detect the user through the plurality of detection apparatuses at the same time, and if at least one of the plurality of detection apparatuses detects the user, it may be determined that a distance between the user and the smart device is less than a preset distance, and it may be determined that the user is in the vicinity of the smart device.

In some embodiments, the smart device includes a short-range communication module, and the specific detection of the extension device of step 210 may include:

211. and acquiring the signal strength between the expansion equipment and the intelligent equipment through the short-distance communication module.

Illustratively, the short-range communication module may be a bluetooth module, and the smart device may measure the surrounding bluetooth devices through the bluetooth module, for example, the measured bluetooth device list includes: the device comprises a device a, a device B, and a device C, wherein if the device B is an extended device, the device B can further detect a bluetooth Signal Strength (RSSI) between the smart device and the device B.

Optionally, the short-range communication module may further include a WiFi module, a ZigBee module, or the like.

212. And if the signal intensity between the expansion equipment and the intelligent equipment is greater than or equal to the signal intensity threshold value, determining that the distance between the expansion equipment and the intelligent equipment is less than the preset distance.

Following the above example, if the detected bluetooth signal strength between the device B and the smart device is a2 and the bluetooth signal strength threshold is a, if a2 is greater than or equal to a, it may be determined that the distance between the expansion device and the smart device is less than the preset distance, and thus it is determined that the expansion device is in the vicinity of the smart device.

Considering that most smart devices have a short-range communication module, and the bluetooth signal strength between the devices is negatively correlated with the distance, the greater the signal strength, the smaller the distance between the devices is, in this embodiment, the bluetooth signal strength between the expansion device and the smart device is obtained by measuring the expansion device through the short-range communication module. If the Bluetooth signal intensity between the expansion device and the intelligent device is larger than or equal to the Bluetooth signal intensity threshold value, the distance between the expansion device and the intelligent device is determined to be smaller than the preset distance, and the intelligent device is determined to be close to the intelligent device, so that whether the expansion device is close to the intelligent device can be accurately and effectively detected by using a module of the intelligent device.

In other embodiments, if a placement area for the expansion device is set on the smart device, whether the expansion device is in the vicinity of the smart device may be determined by detecting whether the expansion device is placed in the placement area.

Illustratively, for example, a rack for placing an expansion device (such as a mobile phone) is arranged on a smart device (such as a smart lamp), a pressure sensor is arranged on the rack, and if a pressure value detected by the pressure sensor exceeds a pressure threshold value, it can be determined that the expansion device is placed on the rack, and thus it can be determined that the expansion device is in the vicinity of the smart device.

For another example, a wireless charging area for charging the expansion device is provided on the smart device (e.g., a smart desktop or a smart light), and if the smart device detects that the wireless charging area is charging the device, it may be determined that the expansion device is in the vicinity of the smart device.

220. And acquiring the behavior information of the user under the condition that the expansion equipment and the user are detected to meet the preset conditions.

In some embodiments, the smart device may use voice information of a user collected by an audio collection device thereof as behavior information, may use image information collected by a camera thereof as behavior information, and may use operation information of the user on the smart device as behavior information.

230. And if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention.

In some embodiments, the behavior information includes voice information, and the specific implementation of step 230 may include:

231a, performing semantic recognition processing on the voice information to obtain a recognition result.

Illustratively, the intelligent device may perform recognition processing on the detected voice information by using a semantic recognition technology, and specifically, may convert the voice information into corresponding text information, and use the text information as a recognition result.

232a, if the specified keywords exist in the identification result, determining that the user has the specified behavior intention, and generating the control information according to the specified behavior intention.

For example, the preset behavior intention database further stores a plurality of association relations among the specified key, the specified behavior intention representations and the control information, and the specific association relations may be as shown in table 1:

TABLE 1

Specifying keywords	Specifying behavioral intent representations	Control information
			Keyword a1, keyword a2	Behavioral intent representation A	Control information A
Keyword b	Behavioral intent representation B	Control information B
			……	……	……

As can be seen, the smart device may determine whether the user has the specified behavior intention by querying table 1, and if the specified behavior intention representation corresponding to the specified keyword can be found from table 1, may determine that the user has the specified behavior intention. For example, if the specified keyword a1 exists in the recognition result, it can be determined that the user has a behavior intention corresponding to the behavior intention representation a. Further, the smart device may also query the control information a corresponding to the representation a for the purpose from table 1, thereby generating the control information a.

For example, the behavior intention of the user is "to open a dictionary", and the keywords corresponding to the behavior intention representation may include "dictionary", "word query", and the like.

In the embodiment, the voice information of the user is used as the behavior information to determine the behavior intention of the user, so that the behavior intention of the user can be accurately and effectively determined.

In some embodiments, where the behavior information includes user image information, specific embodiments of step 230 may include:

231b, the image information is subjected to precession action analysis processing to obtain action information.

Illustratively, the smart device may input one or more frames of image information into a pre-trained model, and then obtain motion information output by the model, where the motion information may be positions of skeletal key points of a user, and the model is pre-trained by a plurality of frames of image samples containing a human body and a plurality of motion information samples.

232b, if the action information is matched with the specified action information, determining that the user has the specified action intention, and generating the control information according to the specified action intention.

Illustratively, the action information is a bone key point of the user, and the smart device may compare the currently measured bone key point with a specified bone key point, and if the bone key point is matched with the specified bone key point, determine that the user has a specified behavior intention corresponding to the specified bone key point. Optionally, when the distances between the plurality of currently measured bone key points and the corresponding key points in the designated bone key points are all less than or equal to the distance threshold, it may be determined that the bone key points match the designated bone key points. As an example, for example, a user makes a seeking action, and the skeletal key point of the seeking action may correspond to a behavior intention representation of "to use an expansion device" in advance, the behavior intention representation corresponds to control information C in advance, and the control information C is used for automatic unlocking of the expansion device, connection of a smart device and the like.

Optionally, the action information may further include gesture information of the user, and different gesture information may correspond to different behavior intention representations and different control information in advance.

In some embodiments, the behavior information includes an operation sequence performed by the user for a plurality of operable components of the smart device, and the specific implementation of step 230 may include:

231c, if the operation sequence is the designated operation sequence, determining that the user has the designated action intention corresponding to the designated operation sequence.

Exemplarily, taking the smart device as a smart lamp and the expansion device as a tablet computer as an example, the operable components may include a touch screen, a brightness adjustment button, a wireless charging switch, and the like, if the user performs the following operations in sequence: the brightness of the intelligent lamp is reduced through the brightness adjusting key, the touch screen is awakened, the wireless charging switch is turned off, and then the fact that the user probably needs to use the connection tablet computer is indicated, so that the user can be determined to have the designated behavior intention of using the tablet computer.

232c, generating control information according to the specified action intention.

Following the above example, the control information generated according to the specified behavior intent may be used for the tablet computer to perform actions such as unlocking, actively establishing a connection with the smart device, and the like.

In view of the fact that the behavior intention of the user cannot be effectively recognized under the condition that the intelligent device cannot acquire audio and images, in the embodiment, when the operation sequence is the designated operation sequence, it is determined that the user has the designated behavior intention corresponding to the designated operation sequence, and the control information is generated according to the designated behavior intention, so that the behavior intention of the user can be more flexibly determined.

240. And sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information.

In some embodiments, specific embodiments of steps 230 to 240 may include:

if the user is determined to have the appointed behavior intention according to the behavior information of the user, determining target scene information corresponding to the appointed behavior intention from a plurality of preset application scene information, wherein the application scene information comprises control information aiming at different extension devices; and sending the target scene information to corresponding expansion equipment.

Illustratively, for example, the association relationship between the behavior intention representation specifying the behavior intention, the plurality of preset application scenario information, the different extension devices, and the control information may be as shown in table 2:

TABLE 2

As can be seen from table 2, after the smart device determines the behavior intention specified by the user, corresponding target application scenario information (such as an identifier of a target application scenario), extension device information (such as an identifier of an extension device), and control information may be found from table 2 according to the behavior intention representation of the specified behavior intention. For example, after determining that the user has the designated behavior intention corresponding to the behavior intention representation c, the smart device may send the control information c1 to the expansion device c1, and send the control information c2 to the expansion device c2, so as to implement the scene linkage control corresponding to the application scene information c.

In the embodiment, if it is determined that the user has the designated behavior intention according to the behavior information of the user, target scene information corresponding to the designated behavior intention is determined from a plurality of preset application scene information, wherein the application scene information includes control information for different extension devices; the target scene information is sent to the corresponding expansion equipment, so that linkage control of one or more expansion equipment can be realized through intention identification, the operation efficiency of a user is further improved, and the user experience is further improved.

Fig. 4 is a flowchart illustrating a method for controlling an expansion device according to an exemplary embodiment, where as shown in fig. 4, the method may be applied to the expansion device shown in fig. 1, and the method may include the following steps:

310. responding to control information sent by the intelligent equipment, and executing an action corresponding to the control information, wherein the control information is behavior information of the user obtained by the intelligent equipment when the intelligent equipment detects that the expansion equipment and the user meet preset conditions; and is generated according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

Illustratively, the expansion device is a tablet computer or a mobile phone, for example, and the expansion device may push a message according to the control information, switch the working state, run an application software program on the expansion device, and the like. For another example, if the expansion device is a sound box, the expansion device may play audio according to the control information.

In some embodiments, specific embodiments of step 310 may include:

and starting the voice assistant application software on the expansion equipment. And if the voice instruction sent by the intelligent equipment is received through the voice assistant application software, responding to the voice instruction, and starting the application software corresponding to the voice instruction.

Illustratively, for example, a voice assistant application software is installed on the expansion device, and the voice assistant application software can make the expansion device execute corresponding actions by recognizing the received voice instructions. The control information sent by the smart device may be used to instruct the expansion device to turn on its voice assistant application software.

The expansion device may, upon receiving the control information, launch the voice assistant application in response to the control information. And then, the expansion device can receive the voice instruction sent by the intelligent device in real time through the started voice assistant application software, and if the voice instruction is received, the expansion device can respond to the voice instruction to start the application software corresponding to the voice instruction. For example, the smart device may automatically issue a "turn on XX learning software" voice command upon recognizing that the user has an intention to use the learning software, and may turn on XX learning software installed on the expansion device after the expansion device receives and recognizes the voice command through the turned-on voice assistant application software.

Fig. 5 is a flowchart illustrating an expansion device control method according to another exemplary embodiment, which may be applied to the expansion device shown in fig. 1, as shown in fig. 5, and which may include the steps of:

410. and responding to the target control information sent by the intelligent device, and outputting inquiry request information, wherein the inquiry request information is used for inquiring whether the user executes the action corresponding to the control information.

As shown in fig. 6, for example, the action corresponding to the control information may be to display query request information of "please confirm whether XX application software is started" on the display interface of the expansion apparatus. Alternatively, the expansion device may play the inquiry request information through an audio playing device

420. And executing the action corresponding to the control information in response to the confirmation operation made by the user for the inquiry request information.

Following the above example, if the extension device receives an operation of "yes" in the click display interface made by the user or an operation of "confirm start" spoken by the user, an action corresponding to the control information may be performed, and the XX application software will be started on the extension device.

Optionally, the expansion device performing the action corresponding to the control information may include: starting application software corresponding to the control information, pushing a message corresponding to the control information, and switching the working state of the expansion equipment to the working state corresponding to the control information.

Optionally, the operating state of the expansion device may include a power-on state, a power-off state, a standby state, and the like.

430. And if the operation information made by the user on the expansion equipment is not received within the specified time length, stopping executing the action corresponding to the control information.

Illustratively, for example, the specified time duration is 5 minutes, if the operation information made by the user on the expansion device is not received within 5 minutes of the execution of the action by the expansion device, the expansion device may stop executing the action corresponding to the control information (for example, disconnecting the connection with the smart device), thereby achieving the effect of saving power consumption of the expansion device.

Fig. 7 is a flowchart illustrating a control method of a smart device system according to an exemplary embodiment, which may be applied to the smart device system illustrated in fig. 1, as illustrated in fig. 7, and which may include the steps of:

510. and the intelligent equipment acquires the behavior information of the user under the condition that the expansion equipment and the user meet the preset conditions.

520. And the intelligent equipment generates control information according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

530. And the intelligent equipment sends the control information to the expansion equipment.

540. And the expansion equipment responds to the control information sent by the intelligent equipment and executes the action corresponding to the control information.

The specific implementation of steps 510 to 540 may refer to the intelligent device control method and the extended device control method in the above embodiments, and therefore, the detailed description thereof is omitted here.

Fig. 8 is a block diagram illustrating an apparatus for controlling a smart device according to an exemplary embodiment, where as shown in fig. 8, the apparatus 600 may be used in the smart device shown in fig. 1, and the apparatus 600 may include: a behavior information acquisition module 610, a control information generation module 620, and a transmission module 630.

The behavior information acquiring module 610 is configured to acquire behavior information of the user when detecting that the extension device and the user meet a preset condition.

And the control information generating module 620 is configured to generate control information according to the specified behavior intention if it is determined that the user has the specified behavior intention according to the behavior information of the user.

A sending module 630, configured to send control information to the extension device, where the control information is used for the extension device to execute an action corresponding to the control information.

In some embodiments, the behavior information obtaining module 610 is further configured to determine that the extension device and the user are detected to meet the preset condition when it is detected that the distance between the extension device and the smart device is smaller than the preset distance and the distance between the user and the smart device is smaller than the preset distance.

In some embodiments, the smart device comprises an object detection apparatus, the apparatus 600 comprising:

the first detection module is used for determining that the distance between the user and the intelligent device is smaller than the preset distance if the preset target is detected by the target detection device within the latest specified time period.

In some embodiments, the first detection module is further configured to start the target detection device if the current time is within a preset usage time period of the smart device.

In some embodiments, the smart device comprises an interaction apparatus, the apparatus 600 comprising:

and the second detection module is used for determining that the distance between the user and the intelligent equipment is less than the preset distance if the intelligent equipment detects that the interaction device is used in the latest specified time period.

In some embodiments, the smart device includes a short-range communication module, the apparatus 600 including:

the third detection module is used for acquiring the signal strength between the expansion equipment and the intelligent equipment through the short-distance communication module; and if the signal intensity between the expansion equipment and the intelligent equipment is greater than or equal to the signal intensity threshold value, determining that the distance between the expansion equipment and the intelligent equipment is smaller than a preset distance.

In some embodiments, the behavior information includes voice information, and the control information generation module 620 includes:

and the semantic recognition submodule is used for carrying out semantic recognition processing on the voice information to obtain a recognition result.

And the first control information generation sub-module is used for determining that the user has the appointed behavior intention if the appointed keyword exists in the recognition result, and generating the control information according to the appointed behavior intention.

In some embodiments, the behavior information includes user image information, and the control information generation module 620 includes:

the action information determining submodule is used for analyzing and processing the precession action of the image information to obtain action information;

and the second control information generation submodule is used for determining that the user has the appointed action intention if the action information is matched with the appointed action information, and generating the control information according to the appointed action intention.

In some embodiments, the behavior information includes a sequence of operations that a user makes with respect to a plurality of operable components of the smart device, and the control information generation module 620 includes:

the appointed behavior intention detection submodule is used for determining that the user has an appointed behavior intention corresponding to an appointed operation sequence if the operation sequence is the appointed operation sequence;

and the third control information generation submodule is used for generating control information according to the specified behavior intention.

In some embodiments, the control information generating module 620 includes:

the target scene information determining submodule is used for determining target scene information corresponding to the appointed behavior intention from a plurality of preset application scene information if the user is determined to have the appointed behavior intention according to the behavior information of the user, and the application scene information comprises control information aiming at different extension equipment;

accordingly, the sending module 630 is specifically configured to send the target scene information to the corresponding extension device.

Fig. 9 is a block diagram illustrating an expansion device control apparatus according to an exemplary embodiment, and as shown in fig. 9, the apparatus 700 may be used in the expansion device shown in fig. 1, and the apparatus 700 may include:

and the execution module 710 is configured to execute an action corresponding to the control information in response to the control information sent by the smart device.

The control information is behavior information of the user obtained by the intelligent equipment when the intelligent equipment detects that the expansion equipment and the user meet preset conditions; and is generated according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

In some embodiments, the execution module 710 includes:

and the query submodule is used for responding to the target control information sent by the intelligent equipment and outputting query request information, and the query request information is used for querying whether the user executes the action corresponding to the control information.

And the execution sub-module is used for responding to the confirmation operation made by the user for the inquiry request information and executing the action corresponding to the control information.

In some embodiments, the apparatus 700 further comprises:

and the closing module is used for stopping executing the action corresponding to the control information if the operation information made by the user on the expansion equipment is not received within the specified time length.

In some embodiments, the executing module 710 is specifically configured to at least one of start application software corresponding to the control information, push a message corresponding to the control information, and switch the operating state of the expansion device to the operating state corresponding to the control information.

In some embodiments, the execution module 710 is specifically configured to start a voice assistant application software on the expansion device; and if the voice instruction sent by the intelligent equipment is received through the voice assistant application software, responding to the voice instruction, and starting the application software corresponding to the voice instruction.

Referring now to FIG. 10, a schematic diagram of an electronic device (e.g., the smart device or the expansion device of FIG. 1) 800 suitable for implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, smart desk lamps, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 10, the electronic device 800 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 801 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage means 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the electronic apparatus 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

Generally, the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 807 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, or the like; storage 808 including, for example, magnetic tape, hard disk, etc.; and a communication device 809. The communication means 809 may allow the electronic device 800 to communicate wirelessly or by wire with other devices to exchange data. While fig. 10 illustrates an electronic device 800 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 809, or installed from the storage means 808, or installed from the ROM 802. The computer program, when executed by the processing apparatus 801, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some implementations, the electronic devices may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring behavior information of a user under the condition that the expansion equipment and the user are detected to meet preset conditions; if the user is determined to have the appointed behavior intention according to the behavior information of the user, generating control information according to the appointed behavior intention; and sending the control information to the expansion equipment, wherein the control information is used for the expansion equipment to execute the action corresponding to the control information.

Alternatively, the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: responding to the control information sent by the intelligent equipment, and executing an action corresponding to the control information; the control information is behavior information of the user obtained by the intelligent equipment when the intelligent equipment detects that the expansion equipment and the user meet preset conditions; and is generated according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the name of a module in some cases does not constitute a limitation on the module itself.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. An intelligent device control method, comprising:

acquiring behavior information of a user under the condition that the expansion equipment and the user are detected to meet preset conditions;

2. The method of claim 1, further comprising:

if the distance between the expansion equipment and the intelligent equipment is smaller than a preset distance and the distance between the user and the intelligent equipment is smaller than the preset distance, it is determined that the expansion equipment and the user meet preset conditions.

3. The method of claim 2, wherein the smart device includes an object detection apparatus, the method further comprising:

and if a preset target is detected by the target detection device within the latest specified time period, determining that the distance between the user and the intelligent equipment is less than a preset distance.

4. The method according to claim 3, wherein before determining that the distance between the user and the smart device is less than the preset distance if the preset target is detected by the target detection apparatus within the latest specified time period, further comprising:

and if the current time is within the preset use time period of the intelligent equipment, starting the target detection device.

5. The method of claim 2, wherein the smart device comprises an interaction means, the method further comprising:

if the intelligent device detects that the interaction device is used in the latest specified time period, determining that the distance between the user and the intelligent device is smaller than a preset distance.

6. The method of claim 2, wherein the smart device comprises a short-range communication module, the method further comprising:

acquiring signal strength between the expansion device and the intelligent device through the short-distance communication module;

and if the signal intensity between the expansion equipment and the intelligent equipment is greater than or equal to a signal intensity threshold value, determining that the distance between the expansion equipment and the intelligent equipment is less than a preset distance.

7. The method according to any one of claims 1 to 6, wherein the behavior information includes voice information, and the generating control information according to the specified behavior intention if it is determined that the user has the specified behavior intention according to the behavior information of the user includes:

performing semantic recognition processing on the voice information to obtain a recognition result;

and if the identification result contains the specified keyword, determining that the user has the specified behavior intention, and generating control information according to the specified behavior intention.

8. The method according to any one of claims 1 to 6, wherein the behavior information includes the user image information, and the generating of the control information according to the specified behavior intention if it is determined that the user has the specified behavior intention according to the behavior information of the user includes:

analyzing and processing the precession action of the image information to obtain action information;

and if the action information is matched with the specified action information, determining that the user has the specified action intention, and generating control information according to the specified action intention.

9. The method according to any one of claims 1 to 6, wherein the behavior information includes an operation sequence made by the user for a plurality of operable components of the smart device, and if it is determined from the behavior information of the user that the user has a specified behavior intention, generating control information according to the specified behavior intention includes:

if the operation sequence is a designated operation sequence, determining that the user has a designated behavior intention corresponding to the designated operation sequence;

and generating control information according to the specified behavior intention.

10. The method according to any one of claims 1 to 6, wherein if it is determined that the user has a specified behavior intention according to the behavior information of the user, generating control information according to the specified behavior intention comprises:

if the user is determined to have the appointed behavior intention according to the behavior information of the user, determining target scene information corresponding to the appointed behavior intention from a plurality of preset application scene information, wherein the application scene information comprises control information aiming at different extension devices;

the sending the control information to the expansion device includes:

and sending the target scene information to corresponding expansion equipment.

11. An expansion device control method, comprising:

the control information is behavior information of the user obtained by the intelligent equipment when the intelligent equipment detects that the expansion equipment and the user meet preset conditions; and generating according to the specified behavior intention under the condition that the user is determined to have the specified behavior intention according to the behavior information of the user.

12. The method of claim 11, wherein the performing, in response to the control information sent by the smart device, an action corresponding to the control information comprises:

responding to target control information sent by the intelligent equipment, and outputting inquiry request information, wherein the inquiry request information is used for inquiring whether a user executes an action corresponding to the control information;

and responding to the confirmation operation made by the user for the inquiry request information, and executing the action corresponding to the control information.

13. The method of claim 11, wherein after the performing the action corresponding to the control information in response to the control information sent by the smart device, further comprising:

and if the operation information made by the user on the expansion equipment is not received within the specified time length, stopping executing the action corresponding to the control information.

14. The method of any of claims 11 to 13, wherein the performing the action corresponding to the control information comprises:

starting application software corresponding to the control information, pushing a message corresponding to the control information, and switching the working state of the expansion equipment to at least one of the working states corresponding to the control information.

15. The method according to any of claims 11 to 13, wherein the performing the action corresponding to the control information comprises:

starting voice assistant application software on the expansion equipment;

and if the voice instruction sent by the intelligent equipment is received through the voice assistant application software, responding to the voice instruction, and starting the application software corresponding to the voice instruction.

16. An intelligent device control apparatus, comprising:

17. An expansion device control apparatus, characterized by comprising:

18. A computer-readable medium, on which a computer program is stored, characterized in that the program, when being executed by processing means, carries out the steps of the method of any one of claims 1 to 15.

19. A smart device, comprising:

a storage device having one or more computer programs stored thereon;

one or more processing devices for executing the one or more computer programs in the storage device to perform the steps of the method of any one of claims 1 to 10.

20. An expansion device, comprising:

a storage device having one or more computer programs stored thereon;

one or more processing devices for executing the one or more computer programs in the storage device to implement the steps of the method of any one of claims 11 to 15.