CN114385000A - Intelligent equipment control method, device, server and storage medium - Google Patents
Intelligent equipment control method, device, server and storage medium Download PDFInfo
- Publication number
- CN114385000A CN114385000A CN202111450709.8A CN202111450709A CN114385000A CN 114385000 A CN114385000 A CN 114385000A CN 202111450709 A CN202111450709 A CN 202111450709A CN 114385000 A CN114385000 A CN 114385000A
- Authority
- CN
- China
- Prior art keywords
- target
- virtual object
- environment
- digital
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Abstract
The embodiment of the invention provides an intelligent device control method, an intelligent device control device, a server and a storage medium, wherein the method comprises the following steps: acquiring a digital virtual environment, wherein the digital virtual environment is obtained by mapping a physical environment where the intelligent equipment is located; mapping a target real object in a physical environment to be a target virtual object; if the digital virtual environment does not contain the target virtual object, determining reaction information required to be made to the target virtual object based on the type of the target virtual object; and outputting the reaction information to the intelligent device so that the intelligent device makes a corresponding reaction action based on the reaction information. The scheme provided by the invention reduces manual intervention, improves the naturalness of the behavior of the intelligent equipment and improves the use flexibility of the intelligent equipment.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an intelligent device control method, an intelligent device control device, a server and a storage medium.
Background
Some smart devices in the related art may perform corresponding actions according to instructions given by a user. For example, if the user says "play song a" to the smart speaker, the smart speaker will tune the audio corresponding to song a and play the audio. The smart devices in the related art are passive, and the smart devices are always in a silent state if the user does not input any instruction to the smart devices. Therefore, the smart device in the related art has poor flexibility of use.
Disclosure of Invention
The embodiment of the invention provides an intelligent device control method, an intelligent device control device, a server and a storage medium, which are used for improving the use flexibility of intelligent devices.
In a first aspect, an embodiment of the present invention provides an intelligent device control method, which is applied to a server, and the method includes:
acquiring a digital virtual environment, wherein the digital virtual environment is obtained by mapping a physical environment where intelligent equipment is located;
mapping a target real object in the physical environment to a target virtual object;
if the target virtual object is not contained in the digital virtual environment, determining reaction information required to be made to the target virtual object based on the type of the target virtual object;
and outputting the reaction information to the intelligent device so that the intelligent device makes a corresponding reaction action based on the reaction information.
Optionally, if the digital virtual environment does not include the target virtual object, the method further includes:
adding the target virtual object to the digital virtual environment.
Optionally, before acquiring the digital virtual environment, the method further comprises:
mapping a physical environment in which the intelligent device is located into the digital virtual environment through a digital twinning technology;
the mapping of the target real object in the physical environment to the target virtual object comprises:
mapping a target real object in the physical environment to a target virtual object by the digital twinning technique.
Optionally, if the digital virtual environment does not include the virtual object, determining, based on the type of the virtual object, information that needs to be reacted to the virtual object, including:
acquiring each virtual object contained in the digital virtual environment, wherein each virtual object contained in the digital virtual environment is obtained by mapping each real object in the physical environment before mapping the target virtual object;
and if the target virtual object is not matched with each virtual object contained in the digital virtual environment, determining reaction information required to be made to the virtual object based on the type of the virtual object.
Optionally, if the target virtual object does not match any virtual object included in the digital virtual environment, determining, based on the type of the virtual object, information that needs to be responded to the virtual object, where the information includes:
acquiring physical characteristics corresponding to each virtual object contained in the digital virtual environment;
identifying physical characteristics corresponding to the target virtual object;
and if the physical characteristics corresponding to the target virtual object are not matched with the physical characteristics corresponding to the virtual objects in the digital virtual environment, determining reaction information required to be made to the virtual objects based on the types of the virtual objects.
Optionally, if the target real object corresponding to the target virtual object is a person, the physical feature corresponding to the target virtual object includes at least one of a face feature and a pose of the person.
Optionally, the reaction information includes a target interface of a state machine;
the determining, based on the type of the target virtual object, information that needs to be reacted to the target virtual object includes:
determining a target event triggered by the type of the target virtual object;
determining a target task corresponding to the target event according to a preset corresponding relation between the event and the task;
and executing the target task to obtain the target interface of the state machine, wherein the target interface is used for driving the intelligent equipment to make a reaction action.
Optionally, the reaction information includes voice information and action information;
the outputting the reaction information to the smart device to enable the smart device to make a corresponding reaction action based on the reaction information includes:
and outputting the response information to the intelligent equipment so that the intelligent equipment plays the voice information and moves based on the action information.
Optionally, the action information is a target animation;
the outputting the reaction information to the smart device to cause the smart device to perform a motion based on the motion information includes:
calling the target animation, wherein the target animation comprises an action executed by a virtual object corresponding to the intelligent equipment in the digital virtual environment;
and controlling the intelligent equipment to synchronously move with the virtual object corresponding to the intelligent equipment according to the action set in the target animation.
In a second aspect, an embodiment of the present invention provides an intelligent device control apparatus, including:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a digital virtual environment, and the digital virtual environment is obtained by mapping a physical environment where the intelligent equipment is located;
a mapping module for mapping a target real object in the physical environment to a target virtual object;
a determining module, configured to determine, if the target virtual object is not included in the digital virtual environment, response information that needs to be made to the target virtual object based on a type of the target virtual object;
and the output module is used for outputting the reaction information to the intelligent equipment so that the intelligent equipment makes corresponding reaction actions based on the reaction information.
Optionally, if the target virtual object is not included in the digital virtual environment, the apparatus further includes an adding module;
the adding module is used for:
adding the target virtual object to the digital virtual environment.
Optionally, the mapping module is configured to:
mapping a physical environment in which the intelligent device is located into the digital virtual environment through a digital twinning technology;
mapping a target real object in the physical environment to a target virtual object by the digital twinning technique.
Optionally, the determining module is configured to:
acquiring each virtual object contained in the digital virtual environment, wherein each virtual object contained in the digital virtual environment is obtained by mapping each real object in the physical environment before mapping the target virtual object;
and if the target virtual object is not matched with each virtual object contained in the digital virtual environment, determining reaction information required to be made to the virtual object based on the type of the virtual object.
Optionally, the determining module is configured to:
acquiring physical characteristics corresponding to each virtual object contained in the digital virtual environment;
identifying physical characteristics corresponding to the target virtual object;
and if the physical characteristics corresponding to the target virtual object are not matched with the physical characteristics corresponding to the virtual objects in the digital virtual environment, determining reaction information required to be made to the virtual objects based on the types of the virtual objects.
Optionally, if the target real object corresponding to the target virtual object is a person, the physical feature corresponding to the target virtual object includes at least one of a face feature and a pose of the person.
Optionally, the reaction information includes a target interface of a state machine;
the determining module is configured to:
determining a target event triggered by the type of the target virtual object;
determining a target task corresponding to the target event according to a preset corresponding relation between the event and the task;
and executing the target task to obtain the target interface of the state machine, wherein the target interface is used for driving the intelligent equipment to make a reaction action.
Optionally, the reaction information includes voice information and action information;
the output module is configured to:
and outputting the response information to the intelligent equipment so that the intelligent equipment plays the voice information and moves based on the action information.
Optionally, the action information is a target animation;
the output module is configured to:
calling the target animation, wherein the target animation comprises an action executed by a virtual object corresponding to the intelligent equipment in the digital virtual environment;
and controlling the intelligent equipment to synchronously move with the virtual object corresponding to the intelligent equipment according to the action set in the target animation.
In a third aspect, an embodiment of the present invention provides a server, which includes a processor and a memory, where the memory stores executable codes, and when the executable codes are executed by the processor, the processor may implement at least the intelligent device control method in the first aspect.
In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of a server, causes the processor to implement at least the smart device control method of the first aspect.
By adopting the method and the device, the target real object appearing in the physical environment can be mapped into the target virtual object, and then whether the target virtual object is a virtual object newly appearing in the digital virtual environment is determined by detecting whether the digital virtual environment contains the target virtual object. If the target virtual object appears in the digital virtual environment, the target virtual object represents that a new real object correspondingly appears in the physical environment. After detecting a new real object, the smart device may be controlled to autonomously react accordingly to different real objects. Compared with the scheme that the user needs to input the instruction equipment to respond correspondingly in the related art, the scheme provided by the invention reduces manual intervention, improves the naturalness of the behavior of the intelligent equipment and improves the use flexibility of the intelligent equipment.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a control method for an intelligent device according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a control method for an intelligent device according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an intelligent device control apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "a plurality" typically includes at least two.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
In addition, the sequence of steps in each method embodiment described below is only an example and is not strictly limited.
Fig. 1 is a flowchart of a method for controlling an intelligent device according to an embodiment of the present invention, where the method is applied to a server and implemented by an intelligent device. In some optional embodiments, the server may be a cloud server, and the smart device may be a robot, etc. As shown in fig. 1, the method comprises the steps of:
101. and acquiring a digital virtual environment, wherein the digital virtual environment is obtained by mapping a physical environment where the intelligent equipment is located.
102. And mapping the target real object in the physical environment into a target virtual object.
103. And if the digital virtual environment does not contain the target virtual object, determining the response information required to be made to the target virtual object based on the type of the target virtual object.
104. And outputting the reaction information to the intelligent device so that the intelligent device makes a corresponding reaction action based on the reaction information.
It should be noted that the digital virtual environment is obtained by simulating and mapping a real physical environment. The real physical environment is a real entity, and the digital virtual environment is established for the server, so that the server observes the real physical environment by looking at the digital virtual environment.
It can be understood that a variety of sensors and image acquisition devices may be disposed in the smart device, images, depths, and the like of a physical environment in which the smart device is located may be acquired by the sensors and the image acquisition devices, the smart device may transmit the acquired images and depths to the server, and the server may perform simulation, mapping, reconstruction, and the like on the physical environment in which the smart device is located based on the images and depths.
Alternatively, the process of establishing a digital virtual environment may be implemented as: and mapping the physical environment in which the intelligent device is positioned into a digital virtual environment through a digital twinning technology.
The digital twin technology fully utilizes data such as a physical model, sensor updating, operation history and the like, integrates a multidisciplinary, multi-physical quantity, multi-scale and multi-probability simulation process, and finishes mapping in a virtual space so as to reflect the full life cycle process of a corresponding real object. By adopting the digital twinning technology, the modeling of the appearance of a real object can be realized, and the simulation of the geometric structure, the spatial position, the physical structure constraint, the physical characteristics (such as friction coefficient, gravity and the like) of the real object can also be realized. In general, 1: 1, the cost of virtual and real migration can be reduced.
In the embodiment of the invention, firstly, a physical environment where the intelligent device is located needs to be mapped into a digital virtual environment through a digital twin technology. In some specific application scenarios, assuming that the smart device is placed in a certain room, the ceiling, floor, aisle, etc. in the room can be used as the object to be mapped in the physical environment. Through the mapping of the physical environment, the intelligent device can know what the spatial structure is located, which places can be passed through for the movement of the ontology, and the like.
It should be noted that the digital virtual environment further includes an intelligent device body, which can be mapped, so that a virtual object corresponding to the intelligent device can also appear in the digital virtual environment.
Then, if the physical environment has no other real objects (e.g., an empty room) in the initial stage, the physical environment may be mapped to the digital virtual environment only. If other real objects exist in the physical environment in the initial stage, the other real objects arranged in the physical environment can be mapped into corresponding virtual objects, and the virtual objects are added into the digital virtual environment. The real object is stationary or otherwise does not change over a period of time. In some specific application scenarios, it is still assumed that the smart device is placed in a certain room, and then objects such as a table, a chair, a cabinet, and the like, which are placed in the room, can be used as the above real objects.
It should be noted that, in practical applications, it is not necessary to map all the real objects present in the physical environment into the digital virtual environment. For example, if there is a cup on the table, the cup may not be mapped if it does not affect the smart device performing the task. Therefore, the calculation amount in the mapping process can be reduced, and the number of the mapped first virtual objects can also be reduced, so that the interference of the intelligent device in the task execution process is reduced.
It will be appreciated that after the digital virtual environment has been created and the virtual objects included within the digital virtual environment have been determined, the digital virtual environment and virtual objects may be stored and recalled for subsequent use of the data.
After the digital virtual environment is established, a target real object in the physical environment may be mapped to a target virtual object. It should be noted that the target real object may be any real object currently present in the physical environment.
Alternatively, a target real object in the physical environment may be mapped to a target virtual object by digital twinning techniques.
After reconstructing the target virtual object, it may be determined whether the target virtual object is contained in the digital virtual environment. If the digital virtual environment does not contain the target virtual object, which means that the target virtual object is newly appeared in the digital virtual environment, the information to be responded to the target virtual object can be determined based on the type of the target virtual object.
For example, assume that a person has walked into a room and the person comes within proximity of the smart device. For smart devices, the person has not been detected before and is not reconstructed. The virtual object corresponding to the character is unlikely to appear in the digital virtual environment, so the digital virtual environment does not include the virtual object corresponding to the character.
It is further noted that it is only necessary for the smart device to react to a specific type of target virtual object. For example, if the target virtual object is a character, it is necessary to react to the character. If the target virtual object is a table that is newly moved into the room, no reaction to the table is required.
Optionally, if the target virtual object is not contained in the digital virtual environment, the target virtual object may also be added to the digital virtual environment.
In such a way, the digital virtual environment is continuously updated iteratively, and as a new real object enters the physical environment, the correspondingly reconstructed virtual object is correspondingly added to the digital virtual environment.
Optionally, if the digital virtual environment does not include the virtual object, the process of determining the response information to the virtual object based on the type of the virtual object may be implemented as follows: acquiring each virtual object contained in the digital virtual environment, wherein each virtual object contained in the digital virtual environment is obtained by mapping each real object in the physical environment before mapping the target virtual object; if the target virtual object does not match each virtual object contained in the digital virtual environment, determining reaction information required to be made to the virtual object based on the type of the virtual object.
As previously mentioned, the digital virtual environment is continuously updated iteratively, and may not contain any virtual objects therein, or may add some virtual objects through updating. If no virtual object is contained in the digital virtual environment, the target virtual object is not contained in the digital virtual environment. If the digital virtual environment includes virtual objects, the target virtual object and each virtual object included in the digital virtual environment may be compared and matched to determine whether there is a target virtual object in the virtual objects included in the digital virtual environment.
Alternatively, if the target virtual object does not match each virtual object included in the digital virtual environment, the process of determining the response information to the virtual object based on the type of the virtual object may be implemented as follows: acquiring physical characteristics corresponding to each virtual object contained in the digital virtual environment; identifying physical characteristics corresponding to the target virtual object; and if the physical characteristics corresponding to the target virtual object are not matched with the physical characteristics corresponding to the virtual objects in the digital virtual environment, determining the response information to be made to the virtual objects based on the types of the virtual objects.
In some optional embodiments, if the real object corresponding to the virtual object is a human figure, the physical features include, but are not limited to, facial features and a pose of the human figure.
In practical applications, when the server detects a virtual object twice, the server may compare and match the detected virtual object to avoid repeatedly mapping the same virtual object detected twice to the digital virtual environment.
If the real object corresponding to the virtual object is a person, whether the persons detected in the two times are the same person can be determined by comparing and matching the face features of the persons detected in the two times and the poses of the persons detected in the two times. If no person is detected in the front but a person is detected in the rear, or if the person detected in the rear does not match any of the persons detected in the front, it indicates that the person detected in the rear is a newly appearing person.
It should be noted that, in the process of comparing the poses of the matched persons, if the pose difference between the persons detected in the two previous and subsequent times is within the preset range and the facial features of the persons detected in the two previous and subsequent times are matched, it indicates that the persons detected in the two previous and subsequent times are the same person. The reason why the difference in the poses of the two persons detected before and after is limited to a certain range is that the same person may be moved and adjusted by a small amount between the two detection intervals.
In addition, the continuity of the man-machine conversation process can be ensured by comparing and matching the characters detected before and after. For example, suppose that when a smart device detects a newly appearing character, the character is actively called. Then if two newly appearing characters are detected, the two characters may be individually called in turn. There will then be the following scenarios:
the smart device says to person a: you good!
The smart device turns to character B saying: you good!
If the smart device does not compare and match the detected people before and after, then the smart device will mistakenly think that the person A is a new person when detecting the person A again, and then turn to the person A to say "hello! ", so the dialog flow cannot proceed. And when the intelligent device can determine that the character A detected before and after is the same character in a comparison matching mode, the intelligent device can continue the conversation process. For example, the smart device may turn to character a saying "ask what can help you? "
After the new target object in the digital virtual environment is determined to appear in the above manner, the information to be responded to the target object may be determined based on the object type of the target object. For example, the target object is a person, and it is necessary to make a visual following and a call reaction to a newly appearing person.
Optionally, the reaction information may include a target interface of the state machine. Accordingly, as shown in fig. 2, based on the type of the target virtual object, the process of determining the reaction information required to be made to the target virtual object may be implemented as follows: determining a target event triggered by the type of the target virtual object; determining a target task corresponding to a target event according to a preset corresponding relation between the event and the task; and executing the target task to acquire a target interface of the state machine, wherein the target interface is used for driving the intelligent equipment to make a reaction action.
The preset corresponding relationship between the event and the task can be implemented as a behavior tree, that is, the target event is input into the behavior tree, the behavior tree judges an incoming branch according to the target event, different branches represent different tasks, and the branch corresponding to the target event is the target task, so that the behavior tree can output the target task corresponding to the target event.
The state machine is a control center that is configured by a state register and a combinational logic circuit, can perform state transition in accordance with a predetermined state in response to a control signal, and performs a specific operation by coordinating operations of the relevant signals.
Finally, the reaction information can be output to the intelligent device, so that the intelligent device can make corresponding reaction actions based on the reaction information.
Alternatively, the reaction information may include voice information and motion information. Outputting the reaction information to the smart device, so that the process of the smart device making the corresponding reaction action based on the reaction information can be implemented as follows: and outputting the response information to the intelligent device so that the intelligent device plays the voice information and moves based on the action information.
Specifically, the server may be caused to send a control instruction to the smart device by calling a corresponding target interface in the state machine, so that the smart device plays the voice information and performs a motion based on the motion information.
For example, assume that the server detects a new presence of a person, triggering an event to see the person. The event of seeing the person is input into a behavior tree, and the behavior tree judges to enter the branch of seeing the person according to the event of seeing the person. And if the task corresponding to the branch is the call task, calling a corresponding call interface in the state machine by executing the call task. The state machine then enters the state of calling. Through calling the interface of calling out, the server sends the control command of calling out to the intelligent equipment for the intelligent equipment obtains "hello" audio frequency and plays this audio frequency, simultaneously, makes the intelligent equipment see the face to the personage that needs to call out, and control robotic arm sways to make the action of calling out.
In some optional embodiments, in the digital virtual environment, a virtual character model corresponding to a character needing to be called can be mapped. The virtual character model is subjected to simulation mapping by referring to a real character, and the position of the virtual character model in the digital virtual environment corresponds to the position of the character standing in the physical environment. In addition, the height of the virtual character model corresponds to the height of the character.
Based on the position and the height of the virtual object corresponding to the intelligent device in the digital virtual environment and the position and the height of the virtual character model, the angle to which the virtual object corresponding to the intelligent device needs to face and the pitch angle of the head of the virtual object can be determined, so that the virtual object corresponding to the intelligent device can face the virtual character model in the digital virtual environment. By controlling the intelligent device and the virtual object corresponding to the intelligent device to perform synchronous action, the intelligent device can finally face the person needing to call.
Alternatively, for some smart devices that are constructed with a skeleton, the motion information may be a target animation. Accordingly, the reaction information is output to the smart device, so that the process of the smart device performing motion based on the motion information may be implemented as: calling a target animation, wherein the target animation comprises an action executed by a virtual object corresponding to the intelligent equipment in a digital virtual environment; and controlling the intelligent equipment to synchronously move with the virtual object corresponding to the intelligent equipment according to the action set in the target animation.
In some alternative embodiments, the greeting action may be pre-designed and fabricated as a model skeletal animation. It should be noted that the virtual character model may be composed of a plurality of bones, and the virtual character model may complete a preset action by controlling the bones of the virtual character model to adjust the corresponding poses. The process of adjusting the skeleton pose of the virtual character model can be recorded in the skeleton animation. In addition to the examples herein, different motions can be animated into different bones. When a certain target motion is required to be used, the skeleton animation corresponding to the target motion can be called.
It should be noted that the smart device may be composed of a plurality of mechanical structures, the mechanical structures may serve as the skeleton of the smart device, and the skeleton in the virtual character model corresponds to the skeleton of the smart device. Therefore, through the synchronization of the intelligent device and the virtual object corresponding to the intelligent device, when a certain skeleton in the virtual character model adjusts the corresponding pose, the same pose is also adjusted by the corresponding skeleton of the intelligent device, and the intelligent device can simulate the action set in the skeleton animation to move.
By adopting the method and the device, the target real object appearing in the physical environment can be mapped into the target virtual object, and then whether the target virtual object is a virtual object newly appearing in the digital virtual environment is determined by detecting whether the digital virtual environment contains the target virtual object. If the target virtual object appears in the digital virtual environment, the target virtual object represents that a new real object correspondingly appears in the physical environment. After detecting a new real object, the smart device may be controlled to autonomously react accordingly to different real objects. Compared with the scheme that the user needs to input the instruction equipment to respond correspondingly in the related art, the scheme provided by the invention reduces manual intervention, improves the naturalness of the behavior of the intelligent equipment and improves the use flexibility of the intelligent equipment.
The smart device control apparatus according to one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these intelligent device control means may be constructed by configuring the steps taught in the present scheme using commercially available hardware components.
Fig. 3 is a schematic structural diagram of an intelligent device control apparatus according to an embodiment of the present invention, and as shown in fig. 3, the apparatus includes:
an obtaining module 31, configured to obtain a digital virtual environment, where the digital virtual environment is obtained by mapping a physical environment where an intelligent device is located;
a mapping module 32 for mapping a target real object in the physical environment to a target virtual object;
a determining module 33, configured to determine, if the target virtual object is not included in the digital virtual environment, response information that needs to be made to the target virtual object based on the type of the target virtual object;
an output module 34, configured to output the response information to the smart device, so that the smart device makes a corresponding response action based on the response information.
Optionally, if the target virtual object is not included in the digital virtual environment, the apparatus further includes an adding module;
the adding module is used for:
adding the target virtual object to the digital virtual environment.
Optionally, the mapping module 32 is configured to:
mapping a physical environment in which the intelligent device is located into the digital virtual environment through a digital twinning technology;
mapping a target real object in the physical environment to a target virtual object by the digital twinning technique.
Optionally, the determining module 33 is configured to:
acquiring each virtual object contained in the digital virtual environment, wherein each virtual object contained in the digital virtual environment is obtained by mapping each real object in the physical environment before mapping the target virtual object;
and if the target virtual object is not matched with each virtual object contained in the digital virtual environment, determining reaction information required to be made to the virtual object based on the type of the virtual object.
Optionally, the determining module 33 is configured to:
acquiring physical characteristics corresponding to each virtual object contained in the digital virtual environment;
identifying physical characteristics corresponding to the target virtual object;
and if the physical characteristics corresponding to the target virtual object are not matched with the physical characteristics corresponding to the virtual objects in the digital virtual environment, determining reaction information required to be made to the virtual objects based on the types of the virtual objects.
Optionally, if the target real object corresponding to the target virtual object is a person, the physical feature corresponding to the target virtual object includes at least one of a face feature and a pose of the person.
Optionally, the reaction information includes a target interface of a state machine;
the determining module 33 is configured to:
determining a target event triggered by the type of the target virtual object;
determining a target task corresponding to the target event according to a preset corresponding relation between the event and the task;
and executing the target task to obtain the target interface of the state machine, wherein the target interface is used for driving the intelligent equipment to make a reaction action.
Optionally, the reaction information includes voice information and action information;
the output module 34 is configured to:
and outputting the response information to the intelligent equipment so that the intelligent equipment plays the voice information and moves based on the action information.
Optionally, the action information is a target animation;
the output module 34 is configured to:
calling the target animation, wherein the target animation comprises an action executed by a virtual object corresponding to the intelligent equipment in the digital virtual environment;
and controlling the intelligent equipment to synchronously move with the virtual object corresponding to the intelligent equipment according to the action set in the target animation.
The apparatus shown in fig. 3 may execute the intelligent device control method provided in the foregoing embodiments shown in fig. 1 to fig. 2, and the detailed execution process and technical effect refer to the description in the foregoing embodiments, which are not described herein again.
In a possible design, the structure of the intelligent device control apparatus shown in fig. 3 may be implemented as a server, as shown in fig. 4, where the server may include: a processor 91, and a memory 92. Wherein the memory 92 has stored thereon executable code, which when executed by the processor 91, makes the processor 91 at least implement the intelligent device control method as provided in the foregoing embodiments shown in fig. 1 to 2.
Optionally, the server may further include a communication interface 93 for communicating with other devices.
In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored, and when the executable code is executed by a processor of a server, the processor is enabled to implement at least the intelligent device control method provided in the foregoing embodiments shown in fig. 1 to 2.
The above-described apparatus embodiments are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course, can also be implemented by a combination of hardware and software. With this understanding in mind, the above-described aspects and portions of the present technology which contribute substantially or in part to the prior art may be embodied in the form of a computer program product, which may be embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including without limitation disk storage, CD-ROM, optical storage, and the like.
The intelligent device control method provided in the embodiment of the present invention may be executed by a certain program/software, the program/software may be provided by a network side, the server mentioned in the foregoing embodiment may download the program/software into a local nonvolatile storage medium, and when it needs to execute the intelligent device control method, the program/software is read into a memory by a CPU, and then the CPU executes the program/software to implement the intelligent device control method provided in the foregoing embodiment, and an execution process may refer to the illustration in fig. 1 to fig. 2.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (20)
1. An intelligent device control method is applied to a server, and comprises the following steps:
acquiring a digital virtual environment, wherein the digital virtual environment is obtained by mapping a physical environment where intelligent equipment is located;
mapping a target real object in the physical environment to a target virtual object;
if the target virtual object is not contained in the digital virtual environment, determining reaction information required to be made to the target virtual object based on the type of the target virtual object;
and outputting the reaction information to the intelligent device so that the intelligent device makes a corresponding reaction action based on the reaction information.
2. The method of claim 1, wherein if the target virtual object is not contained in the digital virtual environment, the method further comprises:
adding the target virtual object to the digital virtual environment.
3. The method of claim 1, wherein prior to obtaining the digital virtual environment, the method further comprises:
mapping a physical environment in which the intelligent device is located into the digital virtual environment through a digital twinning technology;
the mapping of the target real object in the physical environment to the target virtual object comprises:
mapping a target real object in the physical environment to a target virtual object by the digital twinning technique.
4. The method of claim 1, wherein determining information needed to react to the virtual object based on the type of the virtual object if the virtual object is not included in the digital virtual environment comprises:
acquiring each virtual object contained in the digital virtual environment, wherein each virtual object contained in the digital virtual environment is obtained by mapping each real object in the physical environment before mapping the target virtual object;
and if the target virtual object is not matched with each virtual object contained in the digital virtual environment, determining reaction information required to be made to the virtual object based on the type of the virtual object.
5. The method of claim 4, wherein determining information needed to react to the virtual object based on the type of the virtual object if the target virtual object does not match any of the virtual objects included in the digital virtual environment comprises:
acquiring physical characteristics corresponding to each virtual object contained in the digital virtual environment;
identifying physical characteristics corresponding to the target virtual object;
and if the physical characteristics corresponding to the target virtual object are not matched with the physical characteristics corresponding to the virtual objects in the digital virtual environment, determining reaction information required to be made to the virtual objects based on the types of the virtual objects.
6. The method according to claim 5, wherein if the target real object corresponding to the target virtual object is a human figure, the physical feature corresponding to the target virtual object comprises at least one of a human face feature and a pose of the human figure.
7. The method of claim 1, wherein the reaction information comprises a target interface of a state machine;
the determining, based on the type of the target virtual object, information that needs to be reacted to the target virtual object includes:
determining a target event triggered by the type of the target virtual object;
determining a target task corresponding to the target event according to a preset corresponding relation between the event and the task;
and executing the target task to obtain the target interface of the state machine, wherein the target interface is used for driving the intelligent equipment to make a reaction action.
8. The method of claim 1, wherein the response information comprises voice information and action information;
the outputting the reaction information to the smart device to enable the smart device to make a corresponding reaction action based on the reaction information includes:
and outputting the response information to the intelligent equipment so that the intelligent equipment plays the voice information and moves based on the action information.
9. The method of claim 8, wherein the action information is a target animation;
the outputting the reaction information to the smart device to cause the smart device to perform a motion based on the motion information includes:
calling the target animation, wherein the target animation comprises an action executed by a virtual object corresponding to the intelligent equipment in the digital virtual environment;
and controlling the intelligent equipment to synchronously move with the virtual object corresponding to the intelligent equipment according to the action set in the target animation.
10. An intelligent device control apparatus, comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a digital virtual environment, and the digital virtual environment is obtained by mapping a physical environment where the intelligent equipment is located;
a mapping module for mapping a target real object in the physical environment to a target virtual object;
a determining module, configured to determine, if the target virtual object is not included in the digital virtual environment, response information that needs to be made to the target virtual object based on a type of the target virtual object;
and the output module is used for outputting the reaction information to the intelligent equipment so that the intelligent equipment makes corresponding reaction actions based on the reaction information.
11. The apparatus of claim 10, further comprising an adding module if the target virtual object is not included in the digital virtual environment;
the adding module is used for:
adding the target virtual object to the digital virtual environment.
12. The apparatus of claim 10, wherein the mapping module is configured to:
mapping a physical environment in which the intelligent device is located into the digital virtual environment through a digital twinning technology;
mapping a target real object in the physical environment to a target virtual object by the digital twinning technique.
13. The apparatus of claim 10, wherein the determining module is configured to:
acquiring each virtual object contained in the digital virtual environment, wherein each virtual object contained in the digital virtual environment is obtained by mapping each real object in the physical environment before mapping the target virtual object;
and if the target virtual object is not matched with each virtual object contained in the digital virtual environment, determining reaction information required to be made to the virtual object based on the type of the virtual object.
14. The apparatus of claim 13, wherein the determining module is configured to:
acquiring physical characteristics corresponding to each virtual object contained in the digital virtual environment;
identifying physical characteristics corresponding to the target virtual object;
and if the physical characteristics corresponding to the target virtual object are not matched with the physical characteristics corresponding to the virtual objects in the digital virtual environment, determining reaction information required to be made to the virtual objects based on the types of the virtual objects.
15. The apparatus according to claim 14, wherein if the target real object corresponding to the target virtual object is a human figure, the physical feature corresponding to the target virtual object comprises at least one of a human face feature and a pose of the human figure.
16. The apparatus of claim 10, wherein the reaction information comprises a target interface of a state machine;
the determining module is configured to:
determining a target event triggered by the type of the target virtual object;
determining a target task corresponding to the target event according to a preset corresponding relation between the event and the task;
and executing the target task to obtain the target interface of the state machine, wherein the target interface is used for driving the intelligent equipment to make a reaction action.
17. The apparatus of claim 10, wherein the response information comprises voice information and action information;
the output module is configured to:
and outputting the response information to the intelligent equipment so that the intelligent equipment plays the voice information and moves based on the action information.
18. The apparatus of claim 17, wherein the action information is a target animation;
the output module is configured to:
calling the target animation, wherein the target animation comprises an action executed by a virtual object corresponding to the intelligent equipment in the digital virtual environment;
and controlling the intelligent equipment to synchronously move with the virtual object corresponding to the intelligent equipment according to the action set in the target animation.
19. A server, comprising: a memory, a processor; wherein the memory has stored thereon executable code which, when executed by the processor, causes the processor to perform the smart device control method of any of claims 1-9.
20. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of a server, causes the processor to perform the smart device control method of any one of claims 1-9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111450709.8A CN114385000A (en) | 2021-11-30 | 2021-11-30 | Intelligent equipment control method, device, server and storage medium |
| PCT/CN2022/105815 WO2023098090A1 (en) | 2021-11-30 | 2022-07-14 | Smart device control method and apparatus, server, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111450709.8A CN114385000A (en) | 2021-11-30 | 2021-11-30 | Intelligent equipment control method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114385000A true CN114385000A (en) | 2022-04-22 |
Family
ID=81195980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111450709.8A Pending CN114385000A (en) | 2021-11-30 | 2021-11-30 | Intelligent equipment control method, device, server and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114385000A (en) |
| WO (1) | WO2023098090A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023098090A1 (en) * | 2021-11-30 | 2023-06-08 | 达闼机器人股份有限公司 | Smart device control method and apparatus, server, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019033845A1 (en) * | 2017-08-18 | 2019-02-21 | 南京阿凡达机器人科技有限公司 | Active interaction method and system of robot |
| WO2019169854A1 (en) * | 2018-03-09 | 2019-09-12 | 南京阿凡达机器人科技有限公司 | Human-computer interaction method, and interactive robot |
| CN110968194A (en) * | 2019-11-28 | 2020-04-07 | 北京市商汤科技开发有限公司 | Interactive object driving method, device, equipment and storage medium |
| WO2020221445A1 (en) * | 2019-04-30 | 2020-11-05 | Siemens Aktiengesellschaft | Method and system for finding objects |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107223271B (en) * | 2016-12-28 | 2021-10-15 | 达闼机器人有限公司 | Display data processing method and device |
| CN111216127A (en) * | 2019-12-31 | 2020-06-02 | 深圳优地科技有限公司 | Robot control method, device, server and medium |
| CN111243101B (en) * | 2019-12-31 | 2023-04-18 | 浙江省邮电工程建设有限公司 | Method, system and device for increasing AR environment immersion degree of user based on artificial intelligence |
| CN111857335A (en) * | 2020-07-09 | 2020-10-30 | 北京市商汤科技开发有限公司 | Virtual object driving method and device, display equipment and storage medium |
| CN114385000A (en) * | 2021-11-30 | 2022-04-22 | 达闼机器人有限公司 | Intelligent equipment control method, device, server and storage medium |
-
2021
- 2021-11-30 CN CN202111450709.8A patent/CN114385000A/en active Pending
-
2022
- 2022-07-14 WO PCT/CN2022/105815 patent/WO2023098090A1/en unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019033845A1 (en) * | 2017-08-18 | 2019-02-21 | 南京阿凡达机器人科技有限公司 | Active interaction method and system of robot |
| WO2019169854A1 (en) * | 2018-03-09 | 2019-09-12 | 南京阿凡达机器人科技有限公司 | Human-computer interaction method, and interactive robot |
| WO2020221445A1 (en) * | 2019-04-30 | 2020-11-05 | Siemens Aktiengesellschaft | Method and system for finding objects |
| CN110968194A (en) * | 2019-11-28 | 2020-04-07 | 北京市商汤科技开发有限公司 | Interactive object driving method, device, equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023098090A1 (en) * | 2021-11-30 | 2023-06-08 | 达闼机器人股份有限公司 | Smart device control method and apparatus, server, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023098090A1 (en) | 2023-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10888785B2 (en) | Method and system for real-time animation generation using machine learning | |
| US8976184B2 (en) | System and method for controlling animation by tagging objects within a game environment | |
| US9805493B2 (en) | Social identity models for automated entity interactions | |
| JP2022531057A (en) | Interactive target drive methods, devices, devices, and recording media | |
| US20200122039A1 (en) | Method and system for a behavior generator using deep learning and an auto planner | |
| CN105141587A (en) | Virtual doll interaction method and device | |
| CN111095170B (en) | Virtual reality scene, interaction method thereof and terminal equipment | |
| JP2023525173A (en) | Conversational AI platform with rendered graphical output | |
| CN114241100B (en) | Skinning method, skinning device, skinning apparatus, skinning medium, and program product for virtual object | |
| JP2022530935A (en) | Interactive target drive methods, devices, devices, and recording media | |
| JP2022530726A (en) | Interactive target drive methods, devices, devices, and recording media | |
| WO2014051584A1 (en) | Character model animation using stored recordings of player movement interface data | |
| CN112673400A (en) | Avatar animation | |
| CN114419205B (en) | Driving method of virtual digital person and training method of pose acquisition model | |
| CN114385000A (en) | Intelligent equipment control method, device, server and storage medium | |
| US20230177755A1 (en) | Predicting facial expressions using character motion states | |
| CN113633983A (en) | Method, device, electronic equipment and medium for controlling expression of virtual character | |
| Fu et al. | Real-time multimodal human–avatar interaction | |
| CN114385002B (en) | Intelligent device control method, intelligent device control device, server and storage medium | |
| US20230267668A1 (en) | Joint twist generation for animation | |
| US20230259540A1 (en) | Conversational ai platform with extractive question answering | |
| CN115617429A (en) | Data processing method and related equipment | |
| CN115564803B (en) | Animation processing method, device, equipment, storage medium and product | |
| CN115222871B (en) | Model evaluation method, model evaluation device, storage medium and electronic equipment | |
| US20240153188A1 (en) | Physics-based simulation of dynamic character motion using generative artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 200245 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant after: Dayu robot Co.,Ltd. Address before: 200245 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant before: Dalu Robot Co.,Ltd. |
|
| CB02 | Change of applicant information |