WO2019218880A1 - 识别交互方法、装置、存储介质及终端设备 - Google Patents

识别交互方法、装置、存储介质及终端设备 Download PDF

Info

Publication number
WO2019218880A1
WO2019218880A1 PCT/CN2019/085460 CN2019085460W WO2019218880A1 WO 2019218880 A1 WO2019218880 A1 WO 2019218880A1 CN 2019085460 W CN2019085460 W CN 2019085460W WO 2019218880 A1 WO2019218880 A1 WO 2019218880A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
preset
posture
body part
interaction
Prior art date
Application number
PCT/CN2019/085460
Other languages
English (en)
French (fr)
Inventor
陈岩
刘耀勇
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2019218880A1 publication Critical patent/WO2019218880A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • the embodiments of the present disclosure relate to the technical field of terminal devices, for example, to an interaction method, device, storage medium, and terminal device.
  • Terminal devices include many functions that are accomplished through interaction with the user, such as an artificial intelligence assistant.
  • the general interaction method includes: acquiring information such as a user's image, sound, touch instruction or fingerprint information to determine the user's intention, and interacting according to the user's intention.
  • the terminal device is not smart enough to interact with the user, and cannot meet more needs of the user. Therefore, it is necessary to optimize the interaction function of the terminal device.
  • An identification interaction method, device, storage medium and terminal device provided by the embodiments of the present application can optimize the interaction function of the terminal device.
  • an embodiment of the present application provides a method for identifying an interaction, including:
  • Corresponding interactive content is determined according to the posture information.
  • an identification interaction device including:
  • a three-dimensional data acquisition module configured to acquire, by the recognition camera, posture three-dimensional data of the preset body part, in a case where the recognition camera captures a preset body part of the user;
  • a posture determining module configured to identify the posture three-dimensional data by a preset gesture recognition model to determine posture information of the preset body part
  • the interaction determining module is configured to determine the corresponding interactive content according to the posture information.
  • an embodiment of the present application provides a computer readable storage medium.
  • the computer readable storage medium stores a computer program, and when the program is executed by the processor, the identification interaction method as described in the embodiment of the present application is implemented.
  • an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, where the processor executes the computer program to implement the implementation of the present application.
  • a terminal device including a memory, a processor, and a computer program stored in the memory and operable on the processor, where the processor executes the computer program to implement the implementation of the present application.
  • An identification interaction scheme provided in the embodiment of the present application, by acquiring the preset body part of the user by the recognition camera, acquiring the three-dimensional data of the posture of the preset body part by using the recognition camera;
  • the gesture recognition model identifies the posture three-dimensional data to determine posture information of the preset body part; and determines corresponding interaction content according to the posture information.
  • FIG. 1 is a schematic flowchart of a method for identifying and interacting according to an embodiment of the present application
  • FIG. 2 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of initial three-dimensional data according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure
  • FIG. 7 is a structural block diagram of an identification interaction device according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of another terminal device according to an embodiment of the present disclosure.
  • FIG. 1 is a schematic flowchart of a method for identifying and interacting according to an embodiment of the present disclosure.
  • the method may be implemented by an identification interaction device, where the device may be implemented by software and/or hardware, and may be integrated into a terminal device or integrated. In other devices with an operating system installed. As shown in FIG. 1, the method includes the following steps.
  • S1100 Obtain three-dimensional posture data of the preset body part by using the recognition camera in a case where the recognition camera captures a preset body part of the user.
  • the preset body part of the user is a part of the human body that can realize different postures; for example, the preset body part may be a hand, and the hand may make a plurality of different postures.
  • the recognition camera is a camera for recognizing a preset body part of the user and capable of acquiring three-dimensional data.
  • the identification camera may be a camera on the terminal device, may be a front camera of the terminal device, and/or a rear camera.
  • at least one camera is generally provided in the terminal device, and generally includes a front camera and a rear camera. If the user who identifies the preset body part captured by the camera is the current user of the terminal device, the identification camera may be a front camera of the terminal device, so that the user can understand the interaction result through the screen of the terminal device. For another example, if the user who identifies the preset body part captured by the camera is not the current user of the terminal device, but other users, the identification camera may be the rear camera of the terminal device; the current user of the current terminal device passes the terminal device. To take pre-set body parts of other users and interact according to the postures of other users' preset body parts.
  • the image acquired by a conventional camera is generally two-dimensional data, that is, a color value (Red Green Blue (RGB) value) or a set of gray values of pixels arranged in a matrix of rows and columns.
  • the three-dimensional data further includes the depth information of the captured preset body part, that is, the distance between different spatial points on the photographed object and the camera, so the three-dimensional data can represent the spatial information of the photographed object.
  • the recognition camera may be a camera with a distance sensor, and the distance sensor may acquire the distance between different spatial points on the photographed object and the camera, so that the three-dimensional data of the photographed preset body part may be acquired.
  • the posture three-dimensional data includes: the three-dimensional data of the posture made by the predetermined body part captured by the recognition camera.
  • the posture three-dimensional data may be a set of three-dimensional data acquired when the preset body part is in a static posture.
  • the posture three-dimensional data may also be a plurality of sets of three-dimensional data acquired when the preset body part makes a dynamic posture.
  • the corresponding posture three-dimensional data is three-dimensional data of a gesture of the user's vertical thumb.
  • the gesture made by the user is a moving gesture of waving goodbye, that is, opening the palm, the palm of the hand toward the recognition camera, and performing a dynamic gesture of swinging left and right, may be multiple sets of three-dimensional data acquired within the set time; for example, if The fixed time is 3 seconds. If the three-dimensional data is acquired once every 0.1 seconds, 30 sets of three-dimensional data can be obtained, and the 30 sets of three-dimensional data can be used as the posture three-dimensional data to determine the dynamic gesture made by the user.
  • S1110 Identify the posture three-dimensional data by using a preset gesture recognition model to determine posture information of the preset body part.
  • the posture three-dimensional data is actually a set of one or more sets of data, and it is necessary to analyze the posture information of the preset body part actually corresponding to the three-dimensional data of the posture according to the set of the data.
  • the gesture recognition model may be an identification system that has been trained to determine gesture information from the three-dimensional image data, the gesture recognition model may be pre-existing in the terminal device, or pre-existing in the background server. In the case where it is necessary to recognize the posture three-dimensional data, the pre-stored gesture recognition model is called to recognize the posture three-dimensional data to determine the posture information of the actual body part.
  • the interaction content corresponding to the gesture information may be determined according to the preset mapping table; the mapping table includes a mapping relationship between the posture information and the interaction information.
  • the interactive content is information output by the terminal device to the user according to the posture information, and may be output through a screen, a speaker, and other output modules of the terminal device, so that the user can obtain the interactive content corresponding to the gesture that is made.
  • the posture information is intent information corresponding to the posture of the preset body part, and different posture information may represent different intentions of the user; for example, when the user makes an "OK" gesture, the user's intention is "agree", the terminal The device can determine the corresponding interactive content according to the user's OK gesture.
  • the mapping relationship between the specific posture information and the interactive content may be a system preset, or may be set according to a user's setting.
  • the interactive content includes at least one of audio data, picture data, and text data.
  • the corresponding audio data may be output through the speaker of the terminal device as the interactive content determined according to the gesture information.
  • the user's intention is music, which can be output through an application for playing music in the terminal device. Audio data as interactive content.
  • picture data and/or text data as interactive content determined according to the posture information through a screen of the terminal device.
  • the terminal device can output text data and/or picture through the screen. Data as interactive content.
  • the interactive content may further include: an interaction action of the augmented reality material.
  • the Augmented Reality (AR) material is a material file that is displayed on the screen of the terminal device according to the augmented reality technology.
  • Augmented reality technology is a technology that establishes a corresponding three-dimensional simulation model according to the position and angle of the camera image. Through the augmented reality technology, the user can see the virtual object that is very close to the real through the display module of the terminal device.
  • augmented reality technology such as the mobile game "Pokémon GO" released by Nintendo and the AR red envelope function in Alipay.
  • the augmented reality material may be a system preset, and the augmented reality material may be a virtual object.
  • the virtual object that enhances the display material can be a virtual simulated cat, or other virtual character.
  • the interactive action of the augmented reality material that is, the action performed by the virtual object.
  • the gesture information of the user is a gesture of beckoning
  • the virtual simulation cat can make an action looking in the direction of the screen as the interactive content determined according to the posture information of the user.
  • the interaction of the specific augmented reality material can be set according to the system preset or the user's setting, and can also be set according to the actual application.
  • the augmented reality material has stronger authenticity, and determining the interaction action of the corresponding augmented reality material according to the posture information can provide the user with more interesting interactive functions and further enrich the interactive function.
  • the embodiment of the present application can identify the posture three-dimensional data of the preset body part of the user, and determine the corresponding interactive content according to the posture three-dimensional data of the preset body part.
  • the three-dimensional data contains more information, can identify more postures of the user, and can perform different interactions according to the user's different postures, and can optimize the interactive function of the terminal device.
  • FIG. 2 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure. Based on the technical solution provided by the foregoing embodiment, a description is performed for determining corresponding interaction content according to the posture information. In an embodiment, as shown in FIG. 2, the method includes the following steps.
  • S1210 Identify the posture three-dimensional data by using a preset gesture recognition model to determine posture information of the preset body part.
  • S1220 Acquire an interactive scene image through a scene camera, and display the image on the screen of the terminal device.
  • the scene camera is a camera for acquiring an interactive scene image, and the interactive scene image is a background scene when the augmented reality material is displayed.
  • the scene camera may be the same camera as the recognition camera, and the scene camera may also be a camera different from the recognition camera, that is, the image acquired by the scene camera may include a preset body part, or may not include Preset body parts.
  • the scene camera and the recognition camera are the same camera, for example, the scene camera and the recognition camera are front cameras of the terminal device, and the image of the user's face acquired by the front camera can be used as an interaction.
  • the scene image, the front camera can also acquire the posture of the user's preset body part, and determine that the augmented reality material performs corresponding interaction in the interactive scene image.
  • the recognition camera is a front camera of the terminal device.
  • the image captured by the rear camera is the image of the room, and the image of the room is displayed as the interactive scene image on the screen of the terminal device.
  • the front camera can capture the posture of the user's preset body part, and determines that the augmented reality material performs corresponding interaction in the interactive scene image.
  • the interactive scene image is the displayed background scene of the augmented reality material, so when the augmented reality material appears, in order to make the augmented reality material closer to reality, it is necessary to determine the initial position and initial size of the augmented reality material initially displayed.
  • the augmented reality material may be initially displayed when the scene camera is turned on.
  • the interactive scene image has different objects, and the initial position of the augmented reality material displayed on the screen may be determined according to the target object included in the interaction scene graph, and according to the position of the target object in the interactive scene image.
  • the initial size of the augmented reality material is determined.
  • the target object can be a system preset or set according to the user's settings.
  • the interactive scene image is an image of a room
  • the room includes objects such as a table, a floor, and a bed.
  • the target object is a bed and the augmented reality material is a virtual simulated cat
  • the initial position of the simulated cat is determined at the end of the bed.
  • the bed is located farther away from the scene camera in the room, the initial size of the simulated cat will be smaller and it will appear to be farther away. This can make the augmented reality material closer to reality, and the user can have a better interactive experience.
  • the reaction data is reaction data corresponding to the intention of the posture information of the user, and may be reaction data of the augmented reality material corresponding to the posture information determined according to the preset mapping table.
  • the gesture information of the user is a gesture of beckoning, it indicates that the user's intention is to draw attention to the augmented reality material, and accordingly, the reaction data of the augmented reality material is an action that reacts to the user.
  • the user's gesture information is a gesture of waving goodbye, it indicates that the user's intention is that the augmented reality material is left, and accordingly, the reaction data of the augmented reality material is to leave the current interaction scene.
  • S1250 Determine an interaction of the augmented reality material according to the reaction data, an initial position, and an initial size.
  • the initial position of the augmented reality material is different from the initial size, and the interaction of performing the same reaction data is also different.
  • the interaction action is an interaction action that is actually made by the augmented reality material and displayed through the screen of the terminal device.
  • the reaction data of the simulated cat is an action of reacting to the user.
  • the action that it reacts to the user may be to make an interaction close to the user.
  • the action that it responds to the user may be to look at the user and also to make a meow.
  • the specific interaction can be determined according to system presets or actual applications.
  • the interaction of the augmented reality material is determined according to the reaction data, the initial position and the initial size, and the interaction of different augmented reality materials may be further determined.
  • execution order of the operations S1220 and S1230 is not limited to that shown in the drawing, and the operations S1220 and S1230 may be performed before the operation S1200; the operation S1220 may also be performed when the scene camera is activated.
  • the initial position and the initial size of the initial display of the augmented reality material are determined by the interactive scene image acquired by the scene camera, and the authenticity of the augmented reality material can be improved. And determining reaction data of the augmented reality material according to the posture information, and determining an interaction action of the augmented reality material according to the reaction data, an initial position, and an initial size, and combining the interaction scene and the posture of the user to determine the augmented reality material.
  • the interaction action can further enrich the interaction of the augmented reality material, increase the interest of the user to interact with the terminal device, and further optimize the interaction function of the terminal device.
  • FIG. 3 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure. Based on the technical solution provided by any of the foregoing embodiments, determining the augmented reality according to the reaction data, initial position, and initial size. The operation of the interaction of the material is explained. In an embodiment, as shown in FIG. 3, the method includes the following steps.
  • S1310 Identify the posture three-dimensional data by using a preset gesture recognition model to determine posture information of the preset body part.
  • S1320 Acquire an interactive scene image by using a scene camera, and display the image on the screen of the terminal device.
  • S1360 Move the augmented reality material from an initial position to a final position while adjusting from an initial size to a final size to determine an interaction.
  • the final position and the final size are the final position and the final size at the end of the interaction. After determining the reaction data of the augmented reality material based on the posture information, the final position is determined based on the reaction data.
  • the interactive scene image has different objects, and may determine a final object in the interactive scene image according to the reaction data, and determine a final position and a final size at the end of the augmented reality material interaction according to the final object.
  • the augmented reality material may be moved from an initial position to a final position according to a motion material corresponding to the augmented reality material, and the augmented reality material is adjusted from an initial size to a final size to determine an interaction.
  • the corresponding motion material may be different motion materials corresponding to different augmented reality materials.
  • the augmented reality material is a virtual cat
  • the corresponding motion material may be a beating motion material.
  • the specific motion material may be set according to the difference of the augmented reality material and the design of the designer, and is not limited herein.
  • the gesture information of the user is a gesture of beckoning
  • the object closest to the user in the interactive scene image is determined, and the object closest to the user is taken as the final object, and the position where the final object is located is determined as the final position, and The final size is determined based on the position of the final object. Moving the augmented reality material to the final position and resizing the displayed material, the user can see that the augmented reality material is closer to the user.
  • Embodiments of the present application can improve augmented reality by determining a final position and a final size based on the reaction data; and moving the augmented reality material from an initial position to a final position and adjusting from an initial size to a final size to determine an interaction
  • the authenticity of the material enriches the interactive action of the augmented reality material, further increasing the interest of the user to interact with the terminal device.
  • FIG. 4 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure.
  • the method includes the following steps.
  • the recognition camera is a three-dimensional (3D) camera.
  • the three-dimensional camera includes various hardware structures, and may include: an infrared sensor, a distance sensor, a lens, and the like.
  • the part depth data is a set of distance values of the spatial point included in the preset body part from the recognition camera.
  • the depth data of the part of the preset body part can be obtained by recognizing the distance sensor in the camera.
  • the portion infrared data is a set of infrared data reflected by a spatial point included in the preset body part.
  • the infrared sensor emits an infrared signal to the preset body part, and the preset body part reflects the infrared information, and the infrared sensor can realize the imaging of the preset body part according to the received reflected infrared data.
  • the location depth data includes the distance value of the spatial point included in the preset body part, so the initial three-dimensional data of the preset body part can be determined according to the part depth data.
  • points a, b, c, and d in FIG. 5 are four spatial points, and X, Y, and Z axes represent spaces, wherein the Z axis represents depth data of spatial points, and X and Y axes Indicates the coordinates of the plane position of the spatial point.
  • the depth data of point a is the largest, that is, the point a is the farthest distance from the recognition camera. It can be seen from Fig. 5 that a three-dimensional vertebral body can be formed according to the plane coordinates and depth data of the four spatial points, thereby The depth data of the location of the spatial point and the planar coordinates of the spatial point can determine the initial three-dimensional data.
  • the corresponding detail position in the initial three-dimensional data may cause data loss, so it is further necessary to correct the initial three-dimensional data according to the part infrared data.
  • S1420 Correct the initial three-dimensional data according to the part infrared data to obtain posture three-dimensional data of the preset body part.
  • the depth data of each spatial point and the infrared data are in one-to-one correspondence.
  • the initial three-dimensional data of the whole space is measured and compared according to the infrared data corresponding to the depth data of the spatial point, and then the feature points of the missing spatial points are complemented.
  • the infrared signal is an electromagnetic wave, and the human eye cannot see the infrared signal.
  • the infrared light can still propagate at night or when the environment is dark and there is no visible light, in a dark environment, the infrared data can also be generated. Clear imaging; in turn, the initial 3D data can be corrected based on the location infrared data.
  • the fitting relationship function may be established according to the depth data of the adjacent points and the infrared data, and the corresponding depth data is calculated according to the fitting relationship function and the infrared data of the missing space point, thereby obtaining the corrected posture three-dimensional.
  • Data wherein the missing spatial point is a spatial point where the depth data is missing, and the adjacent spatial point is an adjacent spatial point of the missing spatial point.
  • S1430 Identify the posture three-dimensional data by using a preset gesture recognition model to determine posture information of the preset body part.
  • the image of the preset body part that is, the acquired two-dimensional posture data of the preset body part
  • the posture of the preset body part can also be recognized by the image processing recognition technology.
  • the two-dimensional data only includes the data of the planar image, and the requirements for the light are high. If the user poses the posture of the preset body part in a dark environment, the acquired image of the planar image may not accurately recognize the accurate posture. Information, so the accuracy of 2D data is lower.
  • the embodiment of the present application determines the initial three-dimensional data of the preset body part according to the part depth data, and corrects the initial three-dimensional data according to the part infrared data to obtain the posture three-dimensional data of the preset body part.
  • the initial three-dimensional data can be corrected by the infrared data of the part to obtain the complete three-dimensional data of the posture, thereby improving the accuracy of the recognition of the posture information.
  • FIG. 6 is a schematic flowchart of another method for identifying and interacting according to an embodiment of the present disclosure.
  • the method includes the following steps. .
  • S1500 Input preset sample data into a preset classifier for training, and obtain a gesture recognition model.
  • the gesture recognition model is configured to determine corresponding posture information according to the captured three-dimensional data of the preset body part
  • the preset sample data includes sample three-dimensional data of a preset body part, and corresponding sample posture information.
  • the preset sample data may include a plurality of different sample data
  • the different sample data is: a fixed posture performed by different users, corresponding sample three-dimensional data of different users and corresponding sample posture information;
  • the fixed posture includes at least one posture.
  • the preset sample data may include a fixed plurality of gestures made by different human hands. Because different people's hands have different shapes and the details of the hands that make the same gesture are different, it is necessary to obtain three-dimensional data of the samples of different human hands, and determine the corresponding sample posture information, as the preset sample data is input to Training is performed in the preset classifier to obtain a gesture recognition model.
  • the preset classifier may be a neural network, and the preset classifier data may be input into a preset classifier for training, and the preset classifier may extract feature data of the sample three-dimensional data, because the sample three-dimensional data is labeled with the corresponding sample posture. Information, so the corresponding posture information can be determined based on the extracted feature data.
  • the obtained gesture recognition model can identify the postures made by the preset body parts of different users, and can also identify the current posture made by the user, and determine Corresponding posture information.
  • the gesture recognition model may also be modified according to the posture of the user of the terminal device to further improve the accuracy of the gesture recognition model for the recognition of the user gesture.
  • S1520 Identify the posture three-dimensional data by using a preset gesture recognition model to determine posture information of the preset body part.
  • the preset sample data is input into the preset classifier for training, and the obtained gesture recognition model can perform feature extraction on the posture three-dimensional data and classify and determine corresponding posture information, thereby improving recognition of the posture three-dimensional data. accuracy.
  • FIG. 7 is a structural block diagram of an identification interaction device according to an embodiment of the present disclosure.
  • the device may perform a recognition interaction method.
  • the device includes: a three-dimensional data acquisition module 210 configured to capture a user in a recognition camera.
  • the posture three-dimensional data of the preset body part is acquired by the recognition camera;
  • the posture determination module 211 is configured to recognize the posture three-dimensional data by a preset gesture recognition model, Determining the posture information of the preset body part;
  • the interaction determining module 212 is configured to determine the corresponding interaction content according to the posture information.
  • An identification interaction device provided in the embodiment of the present application is configured to acquire three-dimensional data of the posture of the preset body part by using the recognition camera in a case where the recognition camera captures a preset body part of the user;
  • the gesture recognition model identifies the posture three-dimensional data to determine posture information of the preset body part; and determines corresponding interaction content according to the posture information.
  • the interactive content includes: an interaction of the augmented reality material.
  • the device further includes: a scene determining module, configured to acquire an interactive scene image through the scene camera and perform display on the screen of the terminal device before determining the corresponding interactive content according to the posture information; a determining module configured to determine an initial position and an initial size of the augmented reality material displayed on the screen according to the interactive scene image; correspondingly, the interaction determining module includes: a response data determining module configured to determine an enhancement according to the posture information The reaction data of the real material; the interaction action determining module is configured to determine an interaction of the augmented reality material according to the reaction data, the initial position, and the initial size.
  • the interaction determination module includes: a final data determination unit configured to determine a final position and a final size according to the reaction data; an interaction determination unit configured to move the augmented reality material from an initial position to a final Position and adjust from initial size to final size to determine interaction.
  • the interactive content includes at least one of audio data, picture data, and text data.
  • the identification camera is a three-dimensional camera; correspondingly, the three-dimensional data acquisition module includes: a part data acquisition module, configured to acquire the depth data of the part of the preset body part by the identification camera, and the infrared part Data; an initial three-dimensional data determining module configured to determine initial three-dimensional data of the preset body part according to the part depth data; and a three-dimensional data correction module configured to correct the initial three-dimensional data according to the part infrared data, Obtaining three-dimensional data of the posture of the preset body part.
  • the three-dimensional data acquisition module includes: a part data acquisition module, configured to acquire the depth data of the part of the preset body part by the identification camera, and the infrared part Data; an initial three-dimensional data determining module configured to determine initial three-dimensional data of the preset body part according to the part depth data; and a three-dimensional data correction module configured to correct the initial three-dimensional data according to the part infrared data, Obtaining three-dimensional data of the posture of the preset
  • the apparatus further includes: a model training module, configured to input the preset sample data into the preset classifier for training before identifying the posture three-dimensional data by using the preset gesture recognition model, Obtaining a gesture recognition model, wherein the gesture recognition model is configured to determine corresponding posture information according to the captured posture three-dimensional data of the preset body part; the preset sample data includes sample three-dimensional data of the preset body part, and corresponding Sample pose information.
  • a model training module configured to input the preset sample data into the preset classifier for training before identifying the posture three-dimensional data by using the preset gesture recognition model, Obtaining a gesture recognition model, wherein the gesture recognition model is configured to determine corresponding posture information according to the captured posture three-dimensional data of the preset body part; the preset sample data includes sample three-dimensional data of the preset body part, and corresponding Sample pose information.
  • the interaction determining module 212 is configured to determine the interactive content corresponding to the gesture information according to the preset mapping table.
  • the reaction data determining module is configured to determine reaction data of the augmented reality material corresponding to the posture information according to the preset mapping table.
  • a storage medium containing computer executable instructions provided by the embodiments of the present application the computer executable instructions of the storage medium are not limited to the identification of the interaction operation as described above, and may also perform the identification interaction method provided by any embodiment of the present application. Related operations.
  • the embodiment of the present application further provides a storage medium including computer executable instructions, when executed by a computer processor, for performing an identification interaction method, the method comprising: capturing a preset of a user at a recognition camera In the case of a body part, the posture three-dimensional data of the preset body part is acquired by the recognition camera; the posture three-dimensional data is identified by a preset gesture recognition model to determine posture information of the preset body part Determining corresponding interactive content according to the posture information.
  • Storage media any type of storage device or storage device.
  • the term "storage medium” is intended to include: a mounting medium such as a Compact Disc Read-Only Memory (CD-ROM), a floppy disk or a tape device; a computer system memory or a random access memory such as a dynamic random Dynamic Random Access Memory (DRAM), Double Data Rate Random Access Memory (DDR RAM), Static Random Access Memory (SRAM), Extended Data Output Random Storage Extended Data Output Random Access Memory (EDO RAM), Rambus Random Access Memory (RAM), etc.; non-volatile memory such as flash memory, magnetic media (such as hard disk or optical storage) a register or other similar type of memory element, etc.
  • the storage medium may further comprise other types of memory or a plurality of types of memory combinations.
  • the storage medium may be located in a first computer system in which the program is executed, or may be located Second computing in a different second computer system
  • the system is connected to the first computer system via a network, such as the Internet.
  • the second computer system can provide program instructions to the first computer for execution.
  • the term "storage medium" can include being resident in different locations (eg, connected over a network) Two or more storage media in different computer systems.
  • the storage media may store program instructions (eg, program instructions implemented as a computer program) executable by one or more processors.
  • the embodiment of the present application provides a terminal device, where the identification interaction device provided by the embodiment of the present application can be integrated.
  • FIG. 8 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • the embodiment of the present application provides a terminal device 30, including a memory 31, a processor 32, and a computer stored in the memory 31 and operable on the processor. a program that implements the recognition interaction method described in the above embodiments when the processor executes the computer program.
  • the terminal device provided by the embodiment of the present application can optimize the interaction function of the terminal device.
  • FIG. 9 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • the terminal device may include: a casing (not shown in FIG. 9), a touch screen (not shown in FIG. 9), a touch button (not shown in FIG. 9), a memory 301, and a central processing unit. (Central Processing Unit, CPU) 302 (also referred to as a processor, hereinafter referred to as CPU), a circuit board (not shown in FIG. 9), and a power supply circuit (not shown in FIG. 9).
  • CPU Central Processing Unit
  • the circuit board is disposed inside a space enclosed by the casing; the CPU 302 and the memory 301 are disposed on the circuit board; and the power circuit is configured to be a plurality of circuits or devices of the terminal device
  • the memory 301 is configured to store executable program code; the CPU 302 runs a computer program corresponding to the executable program code by reading executable program code stored in the memory 301 to implement the following steps Obtaining the posture three-dimensional data of the preset body part by the recognition camera in the case that the recognition camera captures the preset body part of the user; and identifying the posture three-dimensional data by using a preset gesture recognition model, Determining posture information of the preset body part; determining corresponding interaction content according to the posture information.
  • the terminal device further includes: a peripheral interface 303, a radio frequency (RF) circuit 305, an audio circuit 306, a speaker 311, a power management chip 308, an input/output (I/O) subsystem 309, a touch screen 312, and others.
  • Input/control device 310 and external port 304 are communicated via one or more communication buses or signal lines 307.
  • terminal device 300 shown in FIG. 9 is only one example of the terminal device, and the terminal device 300 may have more or fewer components than those shown in FIG. 9, and may combine two or more. Many components, or can have different component configurations.
  • the various components shown in Figure 9 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.
  • the following describes the terminal device for implementing the identification interaction provided by the embodiment, where the terminal device takes a mobile phone as an example.
  • the memory 301 can be accessed by the CPU 302, the peripheral interface 303, etc., and the memory 301 can include a high speed random access memory, and can also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices. Or other volatile solid-state storage devices.
  • a non-volatile memory such as one or more magnetic disk storage devices, flash memory devices. Or other volatile solid-state storage devices.
  • Peripheral interface 303 which can connect the input and output peripherals of the device to CPU 302 and memory 301.
  • I/O subsystem 309 which can connect input and output peripherals on the device, such as touch screen 312 and other input/control devices 310, to peripheral interface 303.
  • I/O subsystem 309 can include display controller 3091 and one or more input controllers 3092 that are configured to control other input/control devices 310.
  • one or more input controllers 3092 receive electrical signals from other input/control devices 310 or transmit electrical signals to other input/control devices 310, and other input/control devices 310 may include physical buttons (press buttons, Rocker button, etc.), dial, slide switch, joystick, click wheel.
  • the input controller 3092 can be connected to any of the following: a keyboard, an infrared port, a Universal Serial Bus (USB) interface, and a pointing device such as a mouse.
  • USB Universal Serial Bus
  • the touch screen 312 is an input interface and an output interface between the user terminal device and the user, and displays the visual output to the user.
  • the visual output may include graphics, text, icons, videos, and the like.
  • Display controller 3091 in I/O subsystem 309 receives an electrical signal from touch screen 312 or an electrical signal to touch screen 312.
  • the touch screen 312 detects the contact on the touch screen, and the display controller 3091 converts the detected contact into an interaction with the user interface object displayed on the touch screen 312, that is, realizes human-computer interaction, and the user interface object displayed on the touch screen 312 can be operated.
  • the device may also include a light mouse, which is a touch sensitive surface that does not display a visual output, or an extension of a touch sensitive surface formed by the touch screen.
  • the RF circuit 305 is mainly configured to establish communication between the mobile phone and the wireless network (ie, the network side), and implement data reception and transmission between the mobile phone and the wireless network. For example, sending and receiving short messages, emails, and the like.
  • RF circuit 305 receives and transmits an RF signal, also referred to as an electromagnetic signal, and RF circuit 305 converts the electrical signal into an electromagnetic signal or converts the electromagnetic signal into an electrical signal, and through the electromagnetic signal and communication network And other devices to communicate.
  • RF circuitry 305 may include known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, CODER-DECoder (CODEC) chipset, Subscriber Identity Module (SIM), etc.
  • an antenna system an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, CODER-DECoder (CODEC) chipset, Subscriber Identity Module (SIM), etc.
  • CODER-DECoder CODER-DECoder
  • SIM Subscriber Identity Module
  • the audio circuit 306 is primarily configured to receive audio data from the peripheral interface 303, convert the audio data into an electrical signal, and transmit the electrical signal to the speaker 311.
  • the speaker 311 is arranged to restore the voice signal received by the handset from the wireless network via the RF circuit 305 to sound and play the sound to the user.
  • the power management chip 308 is configured to provide power and power management for the hardware connected to the CPU 302, the I/O subsystem, and the peripheral interface.
  • the terminal device provided by the embodiment of the present application can optimize the interaction function of the terminal device.
  • the identification interaction device, the storage medium, and the terminal device provided in the foregoing embodiments may perform the identification interaction method provided by any embodiment of the present application, and have corresponding functional modules and beneficial effects for performing the method.
  • the identification interaction method provided by any embodiment of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请实施例中提供的一种识别交互方法、装置、存储介质及终端设备，该方法包括：在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；根据所述姿势信息确定对应的交互内容。

Description

识别交互方法、装置、存储介质及终端设备
本公开要求在2018年05月16日提交中国专利局、申请号为201810469220.7的中国专利申请的优先权，该申请的全部内容通过引用结合在本公开中。
技术领域
本申请实施例涉及终端设备技术领域，例如涉及一种识别交互方法、装置、存储介质及终端设备。
背景技术
随着终端设备的发展，终端设备能够应用的场景以及能够实现的功能越来越多。终端设备包括很多通过和用户的交互来完成的功能，例如，人工智能助手。一般的交互方式包括：获取用户的图像、声音、触摸指令或指纹信息等信息来确定用户意图，并根据用户意图进行交互。但是终端设备在与用户进行交互上不够智能，不能满足用户更多的需求，所以需要优化终端设备的交互功能。
发明内容
本申请实施例提供的一种识别交互方法、装置、存储介质及终端设备，可以优化终端设备的交互功能。
第一方面，本申请实施例提供了一种识别交互方法，包括：
在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；
通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；
根据所述姿势信息确定对应的交互内容。
第二方面，本申请实施例提供了一种识别交互装置，包括：
三维数据获取模块，设置为在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；
姿势确定模块，设置为通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；
交互确定模块，设置为根据所述姿势信息确定对应的交互内容。
第三方面，本申请实施例提供了一种计算机可读存储介质，计算机可读存 储介质上存储有计算机程序，该程序被处理器执行时实现如本申请实施例所述的识别交互方法。
第四方面，本申请实施例提供了一种终端设备，包括存储器，处理器及存储在存储器上并可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现如本申请实施例所述的识别交互方法。
本申请实施例中提供的一种识别交互方案，通过在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；根据所述姿势信息确定对应的交互内容。通过采用上述技术方案，可以识别用户的预设身体部位的姿势三维数据，并根据预设身体部位的姿势三维数据确定对应的交互内容，用户可以做出不同的姿势来进行不同的交互，可以优化终端设备的交互功能。
附图说明
图1为本申请实施例提供的一种识别交互方法的流程示意图；
图2为本申请实施例提供的另一种识别交互方法的流程示意图；
图3为本申请实施例提供的另一种识别交互方法的流程示意图；
图4为本申请实施例提供的另一种识别交互方法的流程示意图；
图5为本申请实施例提供的一种初始三维数据的示意图；
图6为本申请实施例提供的另一种识别交互方法的流程示意图；
图7为本申请实施例提供的一种识别交互装置的结构框图；
图8为本申请实施例提供的一种终端设备的结构示意图；
图9为本申请实施例提供的另一种终端设备的结构示意图。
具体实施方式
下面结合附图并通过具体实施方式来说明本申请的技术方案。可以理解的是，此处所描述的具体实施例仅仅用于解释本申请，而非对本申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
普通的二维摄像技术已经不能满足当前的技术趋势。随着三维摄像头技术逐渐发展，越来越多的可以用于拍照的终端设备都采用了三维摄像头。相较于 二维摄像头，三维摄像头能够采集到更多的信息，本申请实施例可以通过三维摄像头识别用户的身体部位的姿势，可以根据身体部位的姿势实现更智能的交互方式。
图1为本申请实施例提供的一种识别交互方法的流程示意图，该方法可以由识别交互装置执行，其中该装置可以由软件和/或硬件实现，一般可以集成在终端设备中，也可以集成在其他安装有操作系统的设备中。如图1所示，该方法包括如下步骤。
S1100、在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据。
本实施例中，所述用户的预设身体部位为人体上可以实现不同姿势的部位；示例性地，预设身体部位可以是手部，手部可以做出多种不同的姿势。所述识别摄像头为用于识别用户的预设身体部位，并能获取三维数据的摄像头。
所述识别摄像头可以是终端设备上的摄像头，可以是终端设备的前置摄像头，和/或后置摄像头。示例性地，终端设备中一般都设置有至少一个摄像头，一般都包括前置摄像头和后置摄像头。如果识别摄像头拍摄的预设身体部位的用户是终端设备的当前使用者，所述识别摄像头可以是终端设备的前置摄像头，以便用户可以通过终端设备的屏幕了解到交互结果。再如，识别摄像头拍摄的预设身体部位的用户不是终端设备的当前使用者，而是其他用户，则所述识别摄像头可以是终端设备的后置摄像头；当前终端设备的当前使用者通过终端设备来拍摄其他用户的预设身体部位，并根据其他用户的预设身体部位的姿势来进行交互。
传统的相机进行拍摄所获取的图像一般是二维数据，即以行列矩阵规则进行排列的像素点的色彩值(红绿蓝(Red Green Blue，RGB)值)或灰度值的集合。相比二维数据，所述三维数据中还包括拍摄到的预设身体部位的深度信息，即拍摄的物体上的不同空间点与摄像头的距离，所以三维数据可以表示所拍摄的物体的空间信息。所述识别摄像头可以是带有距离传感器的摄像头，距离传感器可以获取所拍摄的物体上的不同空间点与摄像头的距离，如此可以获取到拍摄的预设身体部位的三维数据。
所述姿势三维数据包括：所述识别摄像头拍摄的所述预设身体部位所做出的姿势的三维数据。所述姿势三维数据可以是预设身体部位做静止的姿势时，所获取的一组三维数据。姿势三维数据还可以是预设身体部位做出动态的姿势 时，所获取的多组三维数据。
示例性地，如果所述预设身体部位为手部，用户做出的手势为竖大拇指的静止的姿势，则对应的姿势三维数据为用户的竖大拇指的手势的三维数据。如果用户做出的手势为挥手再见的移动的手势，即打开手掌，手心向着识别摄像头，进行左右摆手的动态的手势时，可以是在设定时间内获取的多组三维数据；例如，如果设定时间是3秒，如果每隔0.1秒获取一次三维数据，则可以得到30组三维数据，该30组三维数据作为姿势三维数据可以确定用户所做出的动态的手势。
S1110、通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息。
所述姿势三维数据实际上是一组或多组的数据的集合，需要根据数据的集合来分析识别该姿势三维数据实际上对应的预设身体部位的姿势信息。
所述姿势识别模型可以是已经训练好的用于根据三维图像数据确定姿势信息的识别系统，所述姿势识别模型可以是预存在终端设备中，或预存在后台服务器中。在需要对姿势三维数据进行识别的情况下，调用预存的姿势识别模型来识别姿势三维数据，以确定实际的身体部位的姿势信息。
S1120、根据所述姿势信息确定对应的交互内容。
在一实施例中，可以是根据预设映射表确定姿势信息对应的交互内容；所述映射表中包括姿势信息和交互信息的映射关系。所述交互内容为终端设备根据姿势信息向用户输出的信息，可以是通过终端设备的屏幕、扬声器和其他输出模块进行输出，以便用户可以得到做出的姿势对应的交互内容。
所述姿势信息为预设身体部位的姿势对应的意图信息，不同的姿势信息可以表示用户的不同意图；例如，用户做出了“OK”的手势，则表示用户的意图为“同意”，终端设备则可以根据用户的OK手势确定对应的交互内容。具体的姿势信息和交互内容的映射关系可以是系统预设，也可以根据用户的设置进行设定。
在一实施例中，所述交互内容包括：音频数据、图片数据和文本数据中的至少一种。
在一实施例中，可以通过终端设备的扬声器输出相应的音频数据作为根据所述姿势信息确定的交互内容。示例性地，用户如果做出了摇滚乐的手势，即食指和小拇指朝上，其他手指合拢的手势，则表示用户的意图为音乐，可以通 过终端设备中用于播放音乐的应用程序(application)输出音频数据作为交互内容。
还可以通过终端设备的屏幕输出图片数据和/或文本数据作为根据所述姿势信息确定的交互内容。示例性地，用户如果做出了看书的手势，即双手展开并排放置，手掌朝向用户的脸部的手势，则表示用户的意图为看书或新闻，终端设备可以通过屏幕输出文本数据和/或图片数据作为交互内容。
在一实施例中，所述交互内容还可以包括：增强现实素材的交互动作。
所述增强现实(Augmented Reality，AR)素材为根据增强现实技术建立的在终端设备的屏幕上进行显示的素材文件。增强现实技术是一种根据摄像机影像的位置及角度建立相应的三维仿真模型的技术，通过增强现实技术用户可以通过终端设备的显示模块看到十分接近真实的虚拟物体。增强现实技术的应用范围越来越多，例如，任天堂公司发行的手机游戏《精灵宝可梦GO》和支付宝中的AR红包功能等。
所述增强现实素材可以是系统预设，增强现实素材可以是虚拟物体。示例性地，增强显示素材的虚拟物体可以是一只虚拟的仿真猫，或者其他虚拟角色。
所述增强现实素材的交互动作，即虚拟物体执行的动作。示例性地，如果用户的姿势信息为招手呼唤的手势，虚拟的仿真猫可以做出看向屏幕外的方向的动作，作为根据用户的姿势信息确定的交互内容。具体的增强现实素材的交互动作可以根据系统预设或用户的设定进行设置，还可以根据实际应用进行设置。
增强现实素材具有更强的真实性，根据所述姿势信息确定对应的增强现实素材的交互动作可以给用户提供更有趣的交互功能，进一步丰富交互功能。
本申请实施例可以识别用户的预设身体部位的姿势三维数据，并根据预设身体部位的姿势三维数据确定对应的交互内容。相较于传统的二维数据，三维数据包含更多的信息，可以识别出用户更多的姿势，根据用户可以做出不同的姿势来进行不同的交互，可以优化终端设备的交互功能。
图2为本申请实施例提供的另一种识别交互方法的流程示意图，在上述实施例所提供的技术方案的基础上，对根据所述姿势信息确定对应的交互内容进行了说明。在一实施例中，如图2所示，该方法包括如下步骤。
S1200、在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据。
S1210、通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息。
具体实施方式可以参考上文的相关描述，在此不再赘述。
S1220、通过场景摄像头获取交互场景图像，并在终端设备的屏幕上进行显示。
所述场景摄像头为用于获取交互场景图像的摄像头，所述交互场景图像为增强现实素材显示时的背景场景。所述场景摄像头可以是与所述识别摄像头为同一个摄像头，所述场景摄像头还可以是与所述识别摄像头不同的摄像头，即场景摄像头获取的图像中可以包括预设身体部位，也可以不包括预设身体部位。
如果所述场景摄像头与所述识别摄像头为同一个摄像头，示例性地，所述场景摄像头与识别摄像头均为终端设备的前置摄像头，通过前置摄像头获取的用户的脸部的图像可以作为交互场景图像，前置摄像头还可以获取用户的预设身体部位的姿势，并确定增强现实素材在交互场景图像执行相应的交互动作。
如果所述场景摄像头与所述识别摄像头不是同一个摄像头，示例性地，如果所述场景摄像头为终端设备的后置摄像头，识别摄像头为终端设备的前置摄像头。相应地，后置摄像头拍摄到的为房间的图像，将房间的图像作为交互场景图像在终端设备的屏幕上进行显示。前置摄像头可以拍摄用户的预设身体部位的姿势，并确定增强现实素材在交互场景图像执行相应的交互动作。
S1230、根据所述交互场景图像确定增强现实素材在所述屏幕上显示的初始位置和初始大小。
本实施例中，交互场景图像为增强现实素材的所显示的背景场景，所以当增强现实素材出现时，为了使增强现实素材更接近真实，则需要确定增强现实素材最初显示的初始位置和初始大小。所述增强现实素材可以是开启场景摄像头时进行最初显示。
所述交互场景图像中有不同的物体，可以根据交互场景图中包括的目标物体确定增强现实素材在所述屏幕上显示的初始位置，并根据所述目标物体在所述交互场景图像中的位置确定所述增强现实素材的初始大小。目标物体可以是系统预设，或者根据用户的设定进行设置。
示例性地，如果交互场景图像为房间的图像，房间中包括桌子、地板和床等物体。如果目标物体为床，增强现实素材为虚拟的仿真猫，则将仿真猫的初始位置确定在床尾的位置。相应地，如果床位于房间内距离场景摄像头较远的 位置，则仿真猫的初始大小会比较小，看起来会同样距离较远。如此可以使增强现实素材更接近真实，用户可以有更好的互动体验。
S1240、根据所述姿势信息确定增强现实素材的反应数据。
本实施例中，所述反应数据为与用户的姿势信息的意图相对应的反应数据，可以是根据预设映射表确定姿势信息对应的增强现实素材的反应数据。示例性地，如果用户的姿势信息为招手呼唤的手势，则表示用户的意图为引起增强现实素材的注意，相应地，增强现实素材的反应数据为向用户做出反应的动作。如果用户的姿势信息为挥手再见的手势，则表示用户的意图为希望增强现实素材离开，相应地，增强现实素材的反应数据为离开当前的交互场景。
S1250、根据所述反应数据、初始位置和初始大小确定所述增强现实素材的交互动作。
增强现实素材的初始位置和初始大小的不同，执行相同的反应数据的交互动作也有所不同。所述交互动作为增强现实素材实际做出的并通过终端设备的屏幕进行显示的交互动作。
示例性地，如果用户的姿势信息为招手呼唤的手势，增强现实素材为仿真猫，则仿真猫的反应数据为向用户做出反应的动作。所述仿真猫在距离场景摄像头较远的位置的情况下，它向用户做出反应的动作可以是做出靠近用户的交互动作。而如果所述仿真猫在距离场景摄像头较近的位置的情况下，它向用户做出反应的动作可以是看向用户，还可以发出猫叫声。具体的交互动作可以根据系统预设或者实际应用进行确定。
根据所述反应数据、初始位置和初始大小确定所述增强现实素材的交互动作，可以进一步确定不同的增强现实素材的交互动作。
需要说明的是，操作S1220和操作S1230的执行顺序不限于附图中所示，操作S1220和操作S1230也可以在操作S1200之前执行；操作S1220也可以在启动场景摄像头时进行执行。
本申请实施例通过场景摄像头获取的交互场景图像确定增强现实素材的初始显示的初始位置和初始大小，可以提高增强现实素材的真实性。并根据所述姿势信息确定增强现实素材的反应数据，以及根据所述反应数据、初始位置和初始大小确定所述增强现实素材的交互动作，可以将交互场景和用户的姿势相结合确定增强现实素材的交互动作，可以进一步丰富增强现实素材的交互动作，增加用户与终端设备进行交互的趣味性，进一步还可以优化终端设备的交互功 能。
图3为本申请实施例提供的另一种识别交互方法的流程示意图，在上述任意实施例所提供的技术方案的基础上，对根据所述反应数据、初始位置和初始大小确定所述增强现实素材的交互动作的操作进行了说明。在一实施例中，如图3所示，该方法包括如下步骤。
S1300、在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据。
S1310、通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息。
S1320、通过场景摄像头获取交互场景图像，并在终端设备的屏幕上进行显示。
S1330、根据所述交互场景图像确定增强现实素材在所述屏幕上显示的初始位置和初始大小。
S1340、根据所述姿势信息确定增强现实素材的反应数据。
上述操作的具体实施方式可以参考上文的相关描述，在此不再赘述。
S1350、根据所述反应数据确定最终位置和最终大小。
S1360、将所述增强现实素材从初始位置移动至最终位置，同时从初始大小调整至最终大小，以确定交互动作。
本实施例中，所述最终位置和最终大小为交互动作结束时的最终位置和最终大小。在根据姿势信息确定增强现实素材的反应数据后，根据反应数据确定最终位置。
所述交互场景图像中有不同的物体，可以是根据反应数据确定交互场景图像中的最终物体，并根据所述最终物体确定增强现实素材交互动作结束时的最终位置和最终大小。
可以根据增强现实素材对应的动作素材，将所述增强现实素材从初始位置移动至最终位置，且将增强现实素材从初始大小调整至最终大小，以确定为交互动作。所述对应的动作素材可以是不同的增强现实素材对应的不同的动作素材，示例性地，如果增强现实素材为虚拟猫，对应的动作素材可以是跳动的动作素材。具体的动作素材可以根据增强现实素材的不同和设计人员的设计进行设置，在此不作限定。
示例性地，如果用户的姿势信息为招手呼唤的手势，则确定交互场景图像 中最接近用户的物体，并将最接近用户的物体作为最终物体，将最终物体所在的位置确定为最终位置，并根据最终物体的位置确定最终大小。将增强现实素材移动到最终位置，以及调整显示素材的大小，用户可以看到增强现实素材更接近用户。
本申请实施例通过根据所述反应数据确定最终位置和最终大小；以及将所述增强现实素材从初始位置移动至最终位置，且从初始大小调整至最终大小，以确定交互动作，可以提高增强现实素材的真实性，丰富增强现实素材的交互动作，进一步增加用户与终端设备进行交互的趣味性。
图4为本申请实施例提供的另一种识别交互方法的流程示意图，在上述任意实施例所提供的技术方案的基础上，对通过所述识别摄像头获取所述预设身体部位的姿势三维数据的操作进行了说明。在一实施例中，如图4所示，该方法包括如下步骤。
S1400、在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的部位深度数据，以及部位红外数据。
所述识别摄像头为三维(Three Dimensional，3D)摄像头，三维摄像头中包括多种硬件结构，可包括：红外传感器、距离传感器和镜头等。
所述部位深度数据为预设身体部位所包括的空间点距离识别摄像头的距离值的集合。可以通过识别摄像头中的距离传感器获取预设身体部位的部位深度数据。
所述部位红外数据为预设身体部位所包括的空间点反射的红外数据的集合。三维摄像头在拍摄时，其中红外传感器发射红外信号至预设身体部位，预设身体部位会对红外信息进行反射，红外传感器根据接收到的反射的红外数据可以实现预设身体部位的成像。
S1410、根据所述部位深度数据确定所述预设身体部位的初始三维数据。
本实施例中，部位深度数据中包括了预设身体部位所包括的空间点的距离值，所以可以根据部位深度数据确定预设身体部位的初始三维数据。示例性地，如图5所示，图5中点a、b、c和d为四个空间点，X、Y和Z轴表示空间，其中Z轴表示空间点的深度数据，X和Y轴表示空间点的平面位置坐标。其中点a的深度数据最大，也就是点a距离识别摄像头的距离最远，从图5中可以看到根据四个空间点的平面坐标和深度数据可以形成一个三维的椎体，从而根据多个空间点的部位深度数据以及空间点的平面坐标可以确定初始的三维数据。
但是如果预设身体部位的某些细节处被遮挡或者发生数据丢失的情况，则初始三维数据中对应的细节位置会出现数据缺失的问题，所以进一步需要根据部位红外数据对初始三维数据进行校正。
S1420、根据所述部位红外数据对所述初始三维数据进行校正，以得到所述预设身体部位的姿势三维数据。
本实施例中，对于预设身体部位所包括的空间点，每个空间点的深度数据和红外数据一一对应。对于数据缺失的空间点的深度数据，根据该空间点的深度数据对应的红外数据对整体的初始三维数据进行衡量和比对，进而对缺失的空间点进行特征补全。红外信号是一种电磁波，人眼无法看到红外信号，但是如果在夜晚或者环境较暗没有可见光的时候，红外光依然可以进行传播，所以在较暗的环境中，根据红外数据也可以生成较清晰的成像；进而可以根据部位红外数据来对初始三维数据进行校正。
在一实施例中，可以根据相邻点的深度数据和红外数据建立拟合关系函数，并根据拟合关系函数以及缺失空间点的部位红外数据计算对应的深度数据，进而得到校正后的姿势三维数据；其中，缺失空间点为深度数据缺失的空间点，相邻空间点为缺失空间点的相邻的空间点。
S1430、通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息。
S1440、根据所述姿势信息确定对应的交互内容。
上述操作的具体实施方式可以参考上文的相关描述，在此不再赘述。
虽然通过普通摄像头来拍摄预设身体部位的图像，即获取的预设身体部位的姿势二维数据，通过图像处理识别技术也可以识别预设身体部位所作的姿势。但是二维数据仅包括平面图像的数据，对于光线的要求较高，如果用户在较暗的环境中摆出预设身体部位的姿势，所获取的平面图像的数据中可能无法准确识别准确的姿势信息，所以二维数据的准确性较低。
本申请实施例通过根据所述部位深度数据确定所述预设身体部位的初始三维数据，根据所述部位红外数据对所述初始三维数据进行校正，以得到所述预设身体部位的姿势三维数据；在光线较暗的位置进行识别，也能通过部位红外数据对初始三维数据进行校正，得到完整的姿势三维数据，进而可以提高姿势信息的识别的精确性。
图6为本申请实施例提供的另一种识别交互方法的流程示意图，在上述任 意实施例所提供的技术方案的基础上，在一实施例中，如图6所示，该方法包括如下步骤。
S1500、将预设样本数据输入至预设分类器中进行训练，得到姿势识别模型。
本实施例中，所述姿势识别模型用于根据拍摄到的预设身体部位的姿势三维数据确定对应的姿势信息；
所述预设样本数据包括预设身体部位的样本三维数据，以及对应的样本姿势信息。
本实施例中，所述预设样本数据可以包括多个不同的样本数据，不同的样本数据为：不同用户做固定的姿势，所对应的不同用户的样本三维数据和对应的样本姿势信息；其中固定的姿势包括至少一个姿势。
示例性地，如果预设身体部位是手部，则预设样本数据可以包括不同人的手部进行做出的固定的多个手势。因为不同的人的手部的形状不同，以及做同一手势的手的细节也有差别，所以需要获取不同人的手部的样本三维数据，以及确定对应的样本姿势信息，作为预设样本数据输入至预设分类器中进行训练，以得到姿势识别模型。
所述预设分类器可以是神经网络，通过将预设样本数据输入至预设分类器中进行训练，预设分类器可以提取样本三维数据的特征数据，由于样本三维数据标注了对应的样本姿势信息，所以可以根据提取的特征数据确定对应的姿势信息。
通过预设样本数据对预设分类器进行训练后，得到的姿势识别模型可以对不同的用户的预设身体部位做出的姿势进行识别，也可以对用户做出的当前姿势进行识别，并确定对应的姿势信息。
在一实施例中，还可以根据终端设备的用户的姿势对姿势识别模型进行修正，以便进一步提高姿势识别模型对于用户姿势的识别的准确性。
S1510、在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据。
S1520、通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息。
S1530、根据所述姿势信息确定对应的交互内容。
上述操作的具体实施方式可以参考上文的相关描述，在此不再赘述。
本申请实施例通过将预设样本数据输入至预设分类器中进行训练，得到的 姿势识别模型，可以对姿势三维数据进行特征提取并进行分类确定对应的姿势信息，提高姿势三维数据的识别的准确性。
图7为本申请实施例提供的一种识别交互装置的结构框图，该装置可以执行识别交互方法，如图7所示，该装置包括：三维数据获取模块210，设置为在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；姿势确定模块211，设置为通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；交互确定模块212，设置为根据所述姿势信息确定对应的交互内容。
本申请实施例中提供的一种识别交互装置，通过在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；根据所述姿势信息确定对应的交互内容。通过采用上述技术方案，可以识别用户的预设身体部位的姿势三维数据，并根据预设身体部位的姿势三维数据确定对应的交互内容，用户可以做出不同的姿势来进行不同的交互，可以优化终端设备的交互功能。
在一实施例中，所述交互内容包括：增强现实素材的交互动作。
在一实施例中，上述装置还包括：场景确定模块，设置为在根据所述姿势信息确定对应的交互内容之前，通过场景摄像头获取交互场景图像，并在终端设备的屏幕上进行显示；素材初始确定模块，设置为根据所述交互场景图像确定增强现实素材在所述屏幕上显示的初始位置和初始大小；相应地，交互确定模块包括：反应数据确定模块，设置为根据所述姿势信息确定增强现实素材的反应数据；交互动作确定模块，设置为根据所述反应数据、初始位置和初始大小确定所述增强现实素材的交互动作。
在一实施例中，交互动作确定模块包括：最终数据确定单元，设置为根据所述反应数据确定最终位置和最终大小；交互动作确定单元，设置为将所述增强现实素材从初始位置移动至最终位置，且从初始大小调整至最终大小，以确定交互动作。
在一实施例中，所述交互内容包括：音频数据、图片数据和文本数据中的至少一种。
在一实施例中，所述识别摄像头为三维摄像头；相应地，三维数据获取模块包括：部位数据获取模块，设置为通过所述识别摄像头获取所述预设身体部 位的部位深度数据，以及部位红外数据；初始三维数据确定模块，设置为根据所述部位深度数据确定所述预设身体部位的初始三维数据；三维数据校正模块，设置为根据所述部位红外数据对所述初始三维数据进行校正，以得到所述预设身体部位的姿势三维数据。
在一实施例中，上述装置还包括：模型训练模块，设置为在通过预设的姿势识别模型对所述姿势三维数据进行识别之前，将预设样本数据输入至预设分类器中进行训练，得到姿势识别模型；其中，所述姿势识别模型用于根据拍摄到的预设身体部位的姿势三维数据确定对应的姿势信息；所述预设样本数据包括预设身体部位的样本三维数据，以及对应的样本姿势信息。
在一实施例中，交互确定模块212是设置为根据预设映射表确定所述姿势信息对应的交互内容。
在一实施例中，反应数据确定模块是设置为根据预设映射表确定所述姿势信息对应的增强现实素材的反应数据。
本申请实施例所提供的一种包含计算机可执行指令的存储介质，存储介质的计算机可执行指令不限于如上所述的识别交互操作，还可以执行本申请任意实施例所提供的识别交互方法中的相关操作。
本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行识别交互方法，该方法包括：在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；根据所述姿势信息确定对应的交互内容。
存储介质——任何类型的存储器设备或存储设备。术语“存储介质”旨在包括：安装介质，例如紧凑型光盘只读储存器(Compact Disc Read-Only Memory，CD-ROM)、软盘或磁带装置；计算机系统存储器或随机存取存储器，诸如动态随机存取存储器(Dynamic Random Access Memory，DRAM)、双倍数据速率随机存取存储器(Double Data Rate Random Access Memory，DDR RAM、静态随机存取存储器(Static Random Access Memory，SRAM)、扩展数据输出随机存取存储器(Extended Data Output Random Access Memory，EDO RAM)，兰巴斯(Rambus)随机存取存储器(Random Access Memory，RAM)等；非易失性存储器，诸如闪存、磁介质(例如硬盘或光存储)；寄存器或其它相似类型的存 储器元件等。存储介质可以还包括其它类型的存储器或多种类型的存储器组合。另外，存储介质可以位于程序在其中被执行的第一计算机系统中，或者可以位于不同的第二计算机系统中，第二计算机系统通过网络(诸如因特网)连接到第一计算机系统。第二计算机系统可以提供程序指令给第一计算机用于执行。术语“存储介质”可以包括可以驻留在不同位置中(例如在通过网络连接的不同计算机系统中)的两个或更多存储介质。存储介质可以存储可由一个或多个处理器执行的程序指令(例如程序指令实现为计算机程序)。
本申请实施例提供了一种终端设备，该终端设备中可集成本申请实施例提供的识别交互装置。
图8为本申请实施例提供的一种终端设备的结构示意图，本申请实施例提供了一种终端设备30，包括存储器31，处理器32及存储在存储器31上并可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现上述实施例所述的识别交互方法。本申请实施例提供的终端设备，可以优化终端设备的交互功能。
图9为本申请实施例提供的一种终端设备的结构示意图。如图9所示，该终端设备可以包括：壳体(图9中未示出)、触摸屏(图9中未示出)、触摸按键(图9中未示出)、存储器301、中央处理器(Central Processing Unit，CPU)302(又称处理器，以下简称CPU)、电路板(图9中未示出)和电源电路(图9中未示出)。所述电路板安置在所述壳体围成的空间内部；所述CPU302和所述存储器301设置在所述电路板上；所述电源电路，设置为为所述终端设备的多个电路或器件供电；所述存储器301，设置为存储可执行程序代码；所述CPU302通过读取所述存储器301中存储的可执行程序代码来运行与所述可执行程序代码对应的计算机程序，以实现以下步骤：在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；根据所述姿势信息确定对应的交互内容。
所述终端设备还包括：外设接口303、射频(Radio Frequency，RF)电路305、音频电路306、扬声器311、电源管理芯片308、输入/输出(I/O)子系统309、触摸屏312、其他输入/控制设备310以及外部端口304，这些部件通过一个或多个通信总线或信号线307来通信。
应该理解的是，图9所示的终端设备300仅仅是终端设备的一个范例，并 且终端设备300可以具有比图9中所示出的更多的或者更少的部件，可以组合两个或更多的部件，或者可以具有不同的部件配置。图9中所示出的多种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
下面就本实施例提供的用于实现识别交互的终端设备进行描述，该终端设备以手机为例。
存储器301，所述存储器301可以被CPU302、外设接口303等访问，所述存储器301可以包括高速随机存取存储器，还可以包括非易失性存储器，例如一个或多个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
外设接口303，所述外设接口303可以将设备的输入和输出外设连接到CPU302和存储器301。
I/O子系统309，所述I/O子系统309可以将设备上的输入输出外设，例如触摸屏312和其他输入/控制设备310，连接到外设接口303。I/O子系统309可以包括显示控制器3091和设置为控制其他输入/控制设备310的一个或多个输入控制器3092。在一实施例中，一个或多个输入控制器3092从其他输入/控制设备310接收电信号或者向其他输入/控制设备310发送电信号，其他输入/控制设备310可以包括物理按钮(按压按钮、摇臂按钮等)、拨号盘、滑动开关、操纵杆、点击滚轮。在一实施例中，输入控制器3092可以与以下任一个连接：键盘、红外端口、通用串行总线(Universal Serial Bus，USB)接口以及诸如鼠标的指示设备。
触摸屏312，所述触摸屏312是用户终端设备与用户之间的输入接口和输出接口，将可视输出显示给用户，可视输出可以包括图形、文本、图标、视频等。
I/O子系统309中的显示控制器3091从触摸屏312接收电信号或者向触摸屏312发送电信号。触摸屏312检测触摸屏上的接触，显示控制器3091将检测到的接触转换为与显示在触摸屏312上的用户界面对象的交互，即实现人机交互，显示在触摸屏312上的用户界面对象可以是运行游戏的图标、联网到相应网络的图标等。在一实施例中，设备还可以包括光鼠，光鼠是不显示可视输出的触摸敏感表面，或者是由触摸屏形成的触摸敏感表面的延伸。
RF电路305，主要设置为建立手机与无线网络(即网络侧)的通信，实现手机与无线网络的数据接收和发送。例如收发短信息、电子邮件等。在一实施例中，RF电路305接收并发送RF信号，RF信号也称为电磁信号，RF电路305 将电信号转换为电磁信号或将电磁信号转换为电信号，并且通过该电磁信号与通信网络以及其他设备进行通信。RF电路305可以包括用于执行这些功能的已知电路，RF电路305包括但不限于天线系统、RF收发机、一个或多个放大器、调谐器、一个或多个振荡器、数字信号处理器、编译码器(COder-DECoder，CODEC)芯片组、用户标识模块(Subscriber Identity Module，SIM)等等。
音频电路306，主要设置为从外设接口303接收音频数据，将该音频数据转换为电信号，并且将该电信号发送给扬声器311。
扬声器311，设置为将手机通过RF电路305从无线网络接收的语音信号，还原为声音并向用户播放该声音。
电源管理芯片308，设置为为CPU302、I/O子系统及外设接口所连接的硬件进行供电及电源管理。
本申请实施例提供的终端设备，可以优化终端设备的交互功能。
上述实施例中提供的识别交互装置、存储介质及终端设备可执行本申请任意实施例所提供的识别交互方法，具备执行该方法相应的功能模块和有益效果。未在上述实施例中描述的技术细节，可参见本申请任意实施例所提供的识别交互方法。

Claims (20)

  1. 一种识别交互方法，包括：
    在识别摄像头拍摄到用户的预设身体部位的情况下，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；
    通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；
    根据所述姿势信息确定对应的交互内容。
  2. 如权利要求1所述的方法，其中，所述交互内容包括：增强现实素材的交互动作。
  3. 如权利要求2所述的方法，在根据所述姿势信息确定对应的交互内容之前，还包括：通过场景摄像头获取交互场景图像，并在终端设备的屏幕上进行显示；根据所述交互场景图像确定增强现实素材在所述屏幕上显示的初始位置和初始大小；
    其中，根据所述姿势信息确定对应的交互内容包括：根据所述姿势信息确定增强现实素材的反应数据；根据所述反应数据、所述初始位置和所述初始大小确定所述增强现实素材的交互动作。
  4. 如权利要求3所述的方法，其中，根据所述反应数据、所述初始位置和所述初始大小确定所述增强现实素材的交互动作包括：
    根据所述反应数据确定最终位置和最终大小；
    将所述增强现实素材从所述初始位置移动至所述最终位置，且从所述初始大小调整至所述最终大小，以确定交互动作。
  5. 如权利要求1所述的方法，其中，所述交互内容包括：音频数据、图片数据和文本数据中的至少一种。
  6. 如权利要求1至5任一项所述的方法，其中，所述识别摄像头为三维摄像头；通过所述识别摄像头获取所述预设身体部位的姿势三维数据包括：
    通过所述识别摄像头获取所述预设身体部位的部位深度数据，以及部位红外数据；
    根据所述部位深度数据确定所述预设身体部位的初始三维数据；
    根据所述部位红外数据对所述初始三维数据进行校正，以得到所述预设身体部位的姿势三维数据。
  7. 如权利要求1至6任一项所述的方法，在通过预设的姿势识别模型对所述姿势三维数据进行识别之前，还包括：
    将预设样本数据输入至预设分类器中进行训练，得到所述姿势识别模型；
    其中，所述姿势识别模型用于根据拍摄到的预设身体部位的姿势三维数据确定对应的姿势信息；
    所述预设样本数据包括预设身体部位的样本三维数据，以及对应的样本姿势信息。
  8. 如权利要求1、2或5所述的方法，其中，所述根据所述姿势信息确定对应的交互内容包括：
    根据预设映射表确定所述姿势信息对应的交互内容。
  9. 如权利要求3所述的方法，其中，所述根据所述姿势信息确定增强现实素材的反应数据包括：
    根据预设映射表确定所述姿势信息对应的增强现实素材的反应数据。
  10. 一种识别交互装置，包括：
    三维数据获取模块，设置为在识别摄像头拍摄到用户的预设身体部位时，通过所述识别摄像头获取所述预设身体部位的姿势三维数据；
    姿势确定模块，设置为通过预设的姿势识别模型对所述姿势三维数据进行识别，以确定所述预设身体部位的姿势信息；
    交互确定模块，设置为根据所述姿势信息确定对应的交互内容。
  11. 如权利要求10所述的装置，其中，所述交互内容包括：增强现实素材的交互动作。
  12. 如权利要求11所述的装置，还包括：场景确定模块，设置为在根据所述姿势信息确定对应的交互内容之前，通过场景摄像头获取交互场景图像，并在终端设备的屏幕上进行显示；素材初始确定模块，设置为根据所述交互场景图像确定增强现实素材在所述屏幕上显示的初始位置和初始大小；
    其中，所述交互确定模块包括：反应数据确定模块，设置为根据所述姿势信息确定增强现实素材的反应数据；交互动作确定模块，设置为根据所述反应数据、所述初始位置和所述初始大小确定所述增强现实素材的交互动作。
  13. 如权利要求12所述的装置，其中，所述交互动作确定模块包括：
    最终数据确定单元，设置为根据所述反应数据确定最终位置和最终大小；
    交互动作确定单元，设置为将所述增强现实素材从所述初始位置移动至所述最终位置，且从所述初始大小调整至所述最终大小，以确定交互动作。
  14. 如权利要求10所述的装置，其中，所述交互内容包括：音频数据、图 片数据和文本数据中的至少一种。
  15. 如权利要求10-14任一项所述的装置，其中，所述识别摄像头为三维摄像头；
    所述三维数据获取模块包括：部位数据获取模块，设置为通过所述识别摄像头获取所述预设身体部位的部位深度数据，以及部位红外数据；初始三维数据确定模块，设置为根据所述部位深度数据确定所述预设身体部位的初始三维数据；三维数据校正模块，设置为根据所述部位红外数据对所述初始三维数据进行校正，以得到所述预设身体部位的姿势三维数据。
  16. 如权利要求10-15任一项所述的装置，还包括：模型训练模块，设置为在通过预设的姿势识别模型对所述姿势三维数据进行识别之前，将预设样本数据输入至预设分类器中进行训练，得到所述姿势识别模型；
    其中，所述姿势识别模型用于根据拍摄到的预设身体部位的姿势三维数据确定对应的姿势信息；所述预设样本数据包括预设身体部位的样本三维数据，以及对应的样本姿势信息。
  17. 如权利要求10、11或14所述的装置，其中，所述交互确定模块是设置为根据预设映射表确定所述姿势信息对应的交互内容。
  18. 如权利要求12所述的装置，其中，所述反应数据确定模块是设置为根据预设映射表确定所述姿势信息对应的增强现实素材的反应数据。
  19. 一种计算机可读存储介质，存储有计算机程序，所述程序被处理器执行时实现如权利要求1-9中任一项所述的识别交互方法。
  20. 一种终端设备，包括存储器，处理器及存储在存储器上并可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1-9任一项所述的识别交互方法。
PCT/CN2019/085460 2018-05-16 2019-05-05 识别交互方法、装置、存储介质及终端设备 WO2019218880A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810469220.7 2018-05-16
CN201810469220.7A CN108681402A (zh) 2018-05-16 2018-05-16 识别交互方法、装置、存储介质及终端设备

Publications (1)

Publication Number Publication Date
WO2019218880A1 true WO2019218880A1 (zh) 2019-11-21

Family

ID=63805560

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/085460 WO2019218880A1 (zh) 2018-05-16 2019-05-05 识别交互方法、装置、存储介质及终端设备

Country Status (2)

Country Link
CN (1) CN108681402A (zh)
WO (1) WO2019218880A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111582078A (zh) * 2020-04-23 2020-08-25 广州微盾科技股份有限公司 基于生物信息和姿势的操作方法、终端设备及存储介质
CN111784797A (zh) * 2020-06-29 2020-10-16 济南浪潮高新科技投资发展有限公司 一种基于ar的机器人物联网交互方法、装置及介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108681402A (zh) * 2018-05-16 2018-10-19 Oppo广东移动通信有限公司 识别交互方法、装置、存储介质及终端设备
CN109376669A (zh) * 2018-10-30 2019-02-22 南昌努比亚技术有限公司 智能助手的控制方法、移动终端及计算机可读存储介质
CN111414073A (zh) * 2019-01-07 2020-07-14 珠海金山办公软件有限公司 一种人机交互方法、装置、电子设备及可读存储介质
CN111949112A (zh) * 2019-05-14 2020-11-17 Oppo广东移动通信有限公司 对象交互方法及装置、系统、计算机可读介质和电子设备
CN110597390B (zh) * 2019-09-12 2022-05-20 Oppo广东移动通信有限公司 控制方法、电子装置和存储介质
CN110991304A (zh) * 2019-11-27 2020-04-10 云南电网有限责任公司电力科学研究院 基于摄像头与热感探头的交互手势采集和识别方法及装置
CN111126975A (zh) * 2019-12-11 2020-05-08 中国建设银行股份有限公司 电子红包的处理方法、装置、电子设备及可读存储介质
CN114693890A (zh) * 2020-12-31 2022-07-01 华为技术有限公司 一种增强现实交互方法及电子设备
CN113315963A (zh) * 2021-04-23 2021-08-27 深圳市洲明科技股份有限公司 扩展现实显示方法、装置、系统和存储介质
CN114020157A (zh) * 2021-11-15 2022-02-08 广州小鹏汽车科技有限公司 验车的方法、装置、车辆及存储介质
CN115919103A (zh) * 2022-10-28 2023-04-07 慕思健康睡眠股份有限公司 基于智能床垫的通话方法、智能床垫及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103971401A (zh) * 2013-02-01 2014-08-06 索尼公司 信息处理装置、终端装置以及信息处理方法
CN105353873A (zh) * 2015-11-02 2016-02-24 深圳奥比中光科技有限公司 基于三维显示的手势操控方法和系统
CN106325509A (zh) * 2016-08-19 2017-01-11 北京暴风魔镜科技有限公司 三维手势识别方法及系统
CN107622257A (zh) * 2017-10-13 2018-01-23 深圳市未来媒体技术研究院 一种神经网络训练方法及三维手势姿态估计方法
CN108646920A (zh) * 2018-05-16 2018-10-12 Oppo广东移动通信有限公司 识别交互方法、装置、存储介质及终端设备
CN108681402A (zh) * 2018-05-16 2018-10-19 Oppo广东移动通信有限公司 识别交互方法、装置、存储介质及终端设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107885317A (zh) * 2016-09-29 2018-04-06 阿里巴巴集团控股有限公司 一种基于手势的交互方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103971401A (zh) * 2013-02-01 2014-08-06 索尼公司 信息处理装置、终端装置以及信息处理方法
CN105353873A (zh) * 2015-11-02 2016-02-24 深圳奥比中光科技有限公司 基于三维显示的手势操控方法和系统
CN106325509A (zh) * 2016-08-19 2017-01-11 北京暴风魔镜科技有限公司 三维手势识别方法及系统
CN107622257A (zh) * 2017-10-13 2018-01-23 深圳市未来媒体技术研究院 一种神经网络训练方法及三维手势姿态估计方法
CN108646920A (zh) * 2018-05-16 2018-10-12 Oppo广东移动通信有限公司 识别交互方法、装置、存储介质及终端设备
CN108681402A (zh) * 2018-05-16 2018-10-19 Oppo广东移动通信有限公司 识别交互方法、装置、存储介质及终端设备

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111582078A (zh) * 2020-04-23 2020-08-25 广州微盾科技股份有限公司 基于生物信息和姿势的操作方法、终端设备及存储介质
CN111582078B (zh) * 2020-04-23 2023-11-07 广州微盾科技股份有限公司 基于生物信息和姿势的操作方法、终端设备及存储介质
CN111784797A (zh) * 2020-06-29 2020-10-16 济南浪潮高新科技投资发展有限公司 一种基于ar的机器人物联网交互方法、装置及介质

Also Published As

Publication number Publication date
CN108681402A (zh) 2018-10-19

Similar Documents

Publication Publication Date Title
WO2019218880A1 (zh) 识别交互方法、装置、存储介质及终端设备
CN110581947B (zh) 在虚拟现实内拍照
WO2019120032A1 (zh) 模型构建方法、拍照方法、装置、存储介质及终端
CN109978989B (zh) 三维人脸模型生成方法、装置、计算机设备及存储介质
US11138434B2 (en) Electronic device for providing shooting mode based on virtual character and operation method thereof
CN111726536B (zh) 视频生成方法、装置、存储介质及计算机设备
US10043308B2 (en) Image processing method and apparatus for three-dimensional reconstruction
US11636641B2 (en) Electronic device for displaying avatar corresponding to external object according to change in position of external object
US20190266775A1 (en) Electronic device displaying avatar motion-performed as per movement of facial feature point and method for operating same
JP2021524957A (ja) 画像処理方法およびその、装置、端末並びにコンピュータプログラム
CN108646920A (zh) 识别交互方法、装置、存储介质及终端设备
WO2020078319A1 (zh) 基于手势的操控方法及终端设备
US10607069B2 (en) Determining a pointing vector for gestures performed before a depth camera
CN111580652A (zh) 视频播放的控制方法、装置、增强现实设备及存储介质
US20170140215A1 (en) Gesture recognition method and virtual reality display output device
US11284020B2 (en) Apparatus and method for displaying graphic elements according to object
US20230195224A1 (en) Systems and methods for gaze-tracking
WO2019218879A1 (zh) 拍照交互方法、装置、存储介质及终端设备
US20230195301A1 (en) Text input method and apparatus based on virtual keyboard
WO2020001016A1 (zh) 运动图像生成方法、装置、电子设备及计算机可读存储介质
WO2022151687A1 (zh) 合影图像生成方法、装置、设备、存储介质、计算机程序及产品
CN111079662A (zh) 一种人物识别方法、装置、机器可读介质及设备
US11054941B2 (en) Information processing system, information processing method, and program for correcting operation direction and operation amount
CN111080589A (zh) 目标对象匹配方法、系统、设备及机器可读介质
CN107888830B (zh) 相机启动方法及装置、计算机装置和计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19802847

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19802847

Country of ref document: EP

Kind code of ref document: A1