CN111880664A - AR interaction method, electronic device and readable storage medium - Google Patents
AR interaction method, electronic device and readable storage medium Download PDFInfo
- Publication number
- CN111880664A CN111880664A CN202010776182.7A CN202010776182A CN111880664A CN 111880664 A CN111880664 A CN 111880664A CN 202010776182 A CN202010776182 A CN 202010776182A CN 111880664 A CN111880664 A CN 111880664A
- Authority
- CN
- China
- Prior art keywords
- information
- preset
- virtual image
- real
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000003993 interaction Effects 0.000 title claims abstract description 51
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 5
- 230000009471 action Effects 0.000 claims description 49
- 230000008859 change Effects 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 11
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000000875 corresponding effect Effects 0.000 description 48
- 230000001276 controlling effect Effects 0.000 description 27
- 241001455214 Acinonyx jubatus Species 0.000 description 17
- 238000005516 engineering process Methods 0.000 description 11
- 241001465754 Metazoa Species 0.000 description 8
- 230000033001 locomotion Effects 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 241000282373 Panthera pardus Species 0.000 description 3
- 230000000994 depressogenic effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 241000283973 Oryctolagus cuniculus Species 0.000 description 2
- 241001464837 Viridiplantae Species 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 206010011469 Crying Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Abstract
The application discloses an AR interaction method, electronic equipment and a readable storage medium. The method comprises the following steps: outputting at least one avatar corresponding to the scene information and/or the image information; detecting whether a preset event occurs; and presetting the virtual image according to the detection result. The application executes the preset processing on the virtual image according to the preset event, can automatically control the virtual image, and is simpler in interaction, thereby being beneficial to improving the intelligent interaction effect.
Description
Technical Field
The present disclosure relates to the field of Augmented Reality (AR), and in particular, to an AR interaction method, an electronic device, and a readable storage medium.
Background
The AR technology is a brand-new man-machine interaction technology, virtual content is applied to the real world through an intelligent terminal and a visualization technology, and the virtual content and the real world are simultaneously superposed on the same picture or space to be presented to a user. Along with the popularization of intelligent terminals, the application of the AR technology is more and more extensive, and experience can be carried out by installing AR applications on the intelligent terminals. The working principle of the AR technology is: the intelligent terminal shoots images or records videos through a camera, and then identifies target objects in the shot images or videos; tracking a target object; and acquiring an avatar associated with the target object, rendering the image frame, superposing the avatar on the target object, and finally displaying the avatar on the intelligent terminal to finish the final interaction.
At present, the AR technology is more and more widely applied to the fields of games, shopping, advertisements, education, traveling and the like, but the interaction form with the virtual image is single, a user actively implements actions such as clicking, dragging and twisting, the terminal controls the interaction with the virtual image by identifying the actions, therefore, the interaction between the real character and the virtual image is still in a feedback stage of screen clicking, the contact between the virtual image and the real character cannot be simulated and fed back, and the intelligent interaction effect is low.
The foregoing description is provided for general background information and is not admitted to be prior art.
Disclosure of Invention
In view of this, the present application provides an AR interaction method, an electronic device, and a readable storage medium, so as to solve the problem that the intelligent interaction effect of the existing AR interaction technology is low.
The application provides an AR interaction method, which comprises the following steps:
s11, outputting at least one virtual image corresponding to the scene information and/or the image information;
s12, detecting whether a preset event occurs;
and S13, presetting the virtual image according to the detection result.
Optionally, the step S11 includes:
analyzing the scene information and/or the image information;
judging whether preset characteristic information exists in the scene information and/or the image information;
if yes, outputting at least one virtual image corresponding to the preset characteristic information.
Optionally, the step of outputting at least one avatar corresponding to the preset feature information further includes:
and displaying the virtual image at a position corresponding to the preset characteristic information.
Optionally, the preset feature information includes at least one of:
plane information, object information, person information, and scene information.
Optionally, the preset event includes at least one of:
the expression changes, the action changes, and the voice information is received.
Optionally, the preset processing includes: and processing the virtual image according to a processing strategy corresponding to the preset event.
Optionally, the processing policy includes at least one of:
controlling the virtual image to move according to a preset track;
controlling the virtual image to make preset expression information and/or imitation expression change information;
and controlling the virtual image to make preset action information and/or imitation action change information.
Optionally, the step S13 further includes:
and if at least two preset events are detected, processing the virtual image simultaneously or sequentially according to the at least two preset events.
The application provides an AR interaction method, which is characterized by comprising the following steps:
s21, detecting whether a preset event occurs in the scene information and/or the image information;
s22, if yes, acquiring at least one virtual image according to at least one of a preset event, scene information and/or image information;
and S23, displaying the virtual image according to a preset rule.
Optionally, the preset event includes at least one of:
the expression changes, the action changes, and the voice information is received.
Optionally, the step S22 includes:
analyzing at least one of the preset event, the scene information and/or the image information;
judging whether preset characteristic information exists or not;
if yes, outputting at least one virtual image corresponding to the preset characteristic information.
Optionally, the preset feature information includes at least one of:
plane information, object information, character information, scene information, expression change information, action change information and voice information.
Optionally, the step of S23, including at least one of:
controlling the virtual image to move according to a preset track;
controlling the virtual image to make preset expression information and/or imitation expression change information;
and controlling the virtual image to make preset action information and/or imitation action change information.
Optionally, the step S23 further includes:
and if at least two preset events are detected, processing the virtual image simultaneously or sequentially according to the at least two preset events.
The application provides a method for interacting with an AR image, which comprises the following steps:
acquiring a real scene captured by a camera in real time, and displaying a corresponding virtual image in the real scene captured in real time;
detecting whether a real person appears in the real scene captured in real time;
when no real figure is detected, controlling the virtual figure to execute preset random operation;
when a real person is detected, detecting the contact condition of the virtual image and the real person; and controlling the virtual image to execute corresponding operation according to the contact condition.
Optionally, the contact condition includes that the avatar is not in contact with or in contact with a real person, and the controlling the avatar to perform a corresponding operation according to the contact condition includes:
when the virtual image is not in contact with the real figure, controlling the virtual image to face the real figure and executing an action according with the attribute of the virtual image; and
and when the virtual image is in contact with the real character, controlling the virtual image to execute the action of contact interaction with the real character.
Optionally, the avatar is a moveable avatar, and when the avatar is not in contact with the real person, the method further comprises: controlling the avatar to move toward the real person.
Optionally, the avatar is a non-movable avatar and is displayed with a personified expression, and the controlling of the random and corresponding operations performed by the avatar comprises altering the personified expression of the avatar.
Optionally, the controlling the avatar to perform an action of contact interaction with the real character includes:
recognizing limb actions of the real person; and
and controlling the virtual image to execute corresponding actions according to the limb actions.
Optionally, the avatar contacting the real character comprises a distance between the two being less than or equal to a predetermined threshold.
Optionally, while controlling the avatar to perform a predetermined random operation or the corresponding operation, the method includes:
generating sound, vibration and/or screen shaking that matches the operation of the avatar.
Optionally, the detecting whether a real person appears in the real scene captured in real time includes:
detecting whether a human body is identified in a preset range of the virtual image;
if not, judging that no real figure appears in the captured real scene;
and if so, judging that the real person appears in the captured real scene.
Optionally, the detecting whether a real person appears in the real scene captured in real time includes:
when a single face image appears in the real scene captured in real time, taking a person corresponding to the face image as a real figure;
and when a plurality of face images are detected to appear in the real scene captured in real time, detecting one with the largest size or the closest distance to the virtual image as a real character.
The application provides an electronic device, which comprises a memory and a processor, wherein the memory stores a program, and the program is used for being executed by the processor to execute the steps in any one of the AR interaction methods.
The application provides a readable storage medium, which stores a program, and the program is used for being executed by a processor to execute the steps in any one of the AR interaction methods.
The application executes the preset processing on the virtual image according to the preset event, can automatically control the virtual image, and is simpler in interaction, thereby being beneficial to improving the intelligent interaction effect.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of an AR interaction method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an interface for outputting an avatar according to an embodiment of the present application;
fig. 3 to 5 are schematic interface diagrams for performing preset processing on an avatar according to an embodiment of the present application;
FIG. 6 is a flowchart illustrating an AR interaction method according to a second embodiment of the present application;
FIG. 7 is a flowchart illustrating an AR interaction method according to a third embodiment of the present application;
FIG. 8 is a flowchart illustrating an AR interaction method according to a fourth embodiment of the present application;
fig. 9 is a schematic flowchart of an AR interaction method according to a fifth embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
It should be noted that, in the description, step numbers such as S11 and S12 are used for the purpose of more clearly and briefly describing the corresponding contents, and do not constitute a substantial limitation on the sequence, and those skilled in the art may perform S12 first and then S11 in specific implementation, but these should be within the protection scope of the present application.
The existing AR interaction technology is not beneficial to simulating the interaction between the feedback virtual image and the real person, so that the intelligent interaction effect is low. In view of the above, the embodiment of the application provides an AR interaction method, which performs preset processing on an avatar by detecting whether a preset event occurs, so as to simulate and feed back interaction between the avatar and a real person, and improve an intelligent interaction effect.
The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments, and not all of them. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. The following embodiments and their technical features may be combined with each other without conflict.
Fig. 1 is a schematic flowchart of an AR interaction method according to an embodiment of the present application. The main body of the method described in fig. 1 may be an electronic device, such as a Mobile Internet Devices (MID) such as a smart phone (Android phone, iOS phone, etc.), a tablet computer, a PDA (personal digital Assistant), etc., or a wearable device with AR interaction function, which is not limited in the embodiments of the present application.
Referring to FIG. 1, the AR interaction method includes the following steps S11-S13.
And S11, outputting at least one virtual image corresponding to the scene information and/or the image information.
The scene information may be a real scene captured by a camera of the electronic device in real time, including but not limited to self-shooting, game, scene arrangement, or framing of the camera during a video call.
The image information can be an image displayed on a screen of the electronic device, and the image can be captured by a camera in real time or a real scene prestored in the electronic device. The image may include at least one of a face image of a real person, a person image, and an image of another object.
Avatars are virtual content that does not exist in a real scene, such as virtual 3D animals or static plants, furniture, etc. The electronic equipment renders the virtual image of the real scene as an AR scene. Compared with a real scene, the AR scene increases the virtual image.
In one implementation, the avatar corresponding to the preset feature information is output by parsing the preset feature information in the scene information and/or the image information, that is, step S11 includes:
analyzing scene information and/or image information;
judging whether preset characteristic information exists in the scene information and/or the image information; and
if yes, outputting at least one virtual image corresponding to the preset characteristic information.
Wherein the preset feature information includes at least one of: plane information, object information, person information, and scene information. The plane information may be UI interface information of the current AR scene, for example, a type of the current real scene, such as an office scene, a home indoor scene, and the like. The object information includes, but is not limited to, position, shape, color, etc. of the object in the real scene, and the character information includes, but is not limited to, position, body movement, expression, etc. of the real character in the scene and/or image. The scene information can be information of blue sky, white cloud, mountain, sea, buildings and the like.
In a scene, such as an office scene, preset feature information obtained by analysis is staff, office tables and chairs and the like, and the output virtual image can be an animal or a green plant and the like; in another scene, for example, a home indoor scene, the preset feature information obtained by the analysis is home furniture, and the output virtual image may be a person or furniture.
The preset characteristic information and the output virtual image have a preset corresponding relation.
In an AR scenario, in one implementation, the rendered avatar may be automatically displayed directly according to a preset, i.e., default avatar, upon the AR class application just starting.
In another implementation, the preset corresponding relationship may also be a corresponding relationship between a real person and/or plane information and a virtual image, and in colloquial, when the real person appears in a real scene captured by a camera in real time, the electronic device determines the identity of the real person through a face recognition technology, for example, the identity is identified as a real person a, and at this time, the virtual image, such as a leopard, having a corresponding relationship with the real person a can be automatically rendered in the real scene; and when the camera catches real person B in real time and appears in the real scene, the electronic equipment automatically renders the virtual image which has a corresponding relation with the real person B in the real scene, such as a rabbit.
The embodiment can display the avatar at the position corresponding to the preset feature information. For example, as shown in fig. 2, an avatar "leopard" is displayed in the vicinity of the human image.
And S12, detecting whether a preset event occurs.
The preset event comprises at least one of the following: the expression changes, the action changes, and the voice information is received. For a real character, the expression information and/or the motion information of the character are/is changed, and one of the motion, the position, the expression and the like of the virtual character can be changed. And the received voice information is used as a voice control instruction, so that the virtual image can be directly controlled.
If the occurrence of the preset event is detected, executing step S13; if the occurrence of the preset event is not detected, the avatar is continuously maintained in the current state, i.e., the step S12 is continuously performed.
And S13, presetting the virtual image according to the detection result.
The preset treatment comprises the following steps: and processing the virtual image according to a processing strategy corresponding to the preset event. For example, for a preset event that the expression changes, the corresponding processing strategy is to control the virtual image to make preset expression information and/or mimic expression change information; the preset event can be the change of action, and the corresponding processing strategy can make preset action information and/or imitation action change information for the control virtual image; for a preset event that voice information is received, the corresponding processing policy may be at least one of the following:
controlling the virtual image to move according to a preset track;
controlling the virtual image to make preset expression information and/or imitation expression change information; and
and controlling the virtual image to make preset action information and/or imitation action change information.
It should be understood that the processing strategies corresponding to the preset events are only exemplary, and any preset event corresponds to one or more processing strategies described above on the premise that the implementation is possible.
For example, in one implementation, when the avatar is not in contact with a real person (i.e., a person image of the real person in an AR scene), the avatar is controlled to face the real person and an action is performed that conforms to the attributes of the avatar. And when the virtual image is contacted with the real character, the virtual image is controlled to execute the action of contact interaction with the real character.
The attribute of an avatar is understood to mean a classification of the avatar, for example, static objects such as animals, plants, furniture, vehicles such as cars, etc. The action according to its attribute can be understood as the instinctive action of the classified avatar in a state of being disturbed, such as zhao kaiwei, yakayao, rolling, etc. of cheetah, and also such as the wind-driven swinging of a flower.
The feedback made by the avatars of different attributes is different. For a movable avatar such as an animal, the electronic device may control the avatar to move toward the real person when it is not in contact with the real person. For an immovable avatar such as a flower, when it is not in contact with a real character, a first anthropomorphic expression is displayed while controlling it toward the real character, and when a person comes into contact with the avatar, it is changed to a second anthropomorphic expression, for example, when a person is not in contact with a flower, the anthropomorphic expression of a flower is a depressed expression, and when a person comes into contact with a flower, for example, a stroking expression of a flower becomes a happy or wormy expression.
When the virtual image is subjected to corresponding preset processing, the electronic equipment can generate sound, vibration and/or screen shaking matched with the preset processing of the virtual image, and the feedback of the electronic equipment is synchronously matched with the action of the virtual image, so that the real feedback interaction effect is better facilitated.
Illustratively, after starting the AR application, as shown in fig. 2, a "cheetah" avatar is rendered, and when it is detected by a human body recognition algorithm that a human body is present near the "cheetah", as shown in fig. 3, the "cheetah" automatically adjusts the posture to face or move towards the human body, and without contacting the human body, the "cheetah" can make some instinctive actions, such as a "cheetah" action, accompanied by a low roar sound. When a human body contacts with the cheetah, the cheetah can perform forward "flapping" action and can be accompanied by the hoarse sound of deterrence as shown in fig. 4, meanwhile, the electronic equipment can vibrate slightly as shown in fig. 5, the display interface applied to the AR can also perform matching special effect feedback such as slight shaking (namely screen shaking), and the like, so that the feedback experience of simulating reality is provided for a user.
In addition, in step S13, if at least two preset events are detected, the avatar is processed simultaneously or sequentially according to the at least two preset events. That is, the present embodiment can control the output of the avatar through various preset events.
The AR interaction method provided by the embodiment can automatically control the virtual image by executing the preset processing on the virtual image according to the preset event, for example, the contact between the simulation feedback virtual image and the real person, so that the interaction is simpler, and the intelligent interaction effect is favorably improved.
Based on the above embodiments, fig. 6 is a schematic flowchart of an AR interaction method according to another embodiment of the present application. Referring to FIG. 6, the AR interaction method includes the following steps S21-S23.
And S21, detecting whether the scene information and/or the image information have a preset event.
Please refer to the description of the foregoing embodiments, and details of the scene information, the image information, and the preset event are not described herein.
If the occurrence of the preset event is detected, executing step S22; if the occurrence of the preset event is not detected, the process continues to step S21.
And S22, if yes, acquiring at least one virtual image according to at least one of the preset event, the scene information and/or the image information.
In one implementation, the avatar corresponding to the preset feature information is output by parsing the preset feature information in the scene information and/or the image information, that is, step S22 includes:
judging whether preset characteristic information exists or not; and
if yes, outputting at least one virtual image corresponding to the preset characteristic information.
Wherein the preset feature information includes at least one of: plane information, object information, character information, scene information, expression change information, action change information and voice information.
The plane information may be UI interface information of the current AR scene, for example, a type of the current real scene, such as an office scene, a home indoor scene, and the like. The object information includes, but is not limited to, position, shape, color, etc. of the object in the real scene, and the character information includes, but is not limited to, position, body movement, expression, etc. of the real character in the scene and/or image. The scene information can be information of blue sky, white cloud, mountain, sea, buildings and the like.
For a real character, the expression information and/or the motion information of the character are/is changed, and one of the motion, the position, the expression and the like of the virtual character can be changed. And the received voice information is used as a voice control instruction, so that the virtual image can be directly controlled.
And S23, displaying the virtual image according to the preset rule.
Displaying the avatar according to a preset rule, including at least one of:
controlling the virtual image to move according to a preset track;
controlling the virtual image to make preset expression information and/or imitation expression change information; and
and controlling the virtual image to make preset action information and/or imitation action change information.
Specific examples of displaying the avatar according to the predetermined rule can be seen from fig. 2 to 5. When the virtual image is displayed, the electronic equipment can generate sound, vibration and/or screen shaking matched with the preset rule of the virtual image, and the feedback of the electronic equipment is synchronously matched with the action of the virtual image, so that the interactive effect is more favorably fed back really.
In addition, in step S23, if at least two preset events are detected, the avatar is processed simultaneously or sequentially according to the at least two preset events. That is, the present embodiment can control the output of the avatar through various preset events.
In the embodiment illustrated in fig. 1, the avatar may be displayed in the AR scene when the AR application is started, and in this embodiment, the avatar may not be displayed after the AR application is started, but may be displayed according to the preset rule when the preset event occurs in the real scene.
FIG. 7 is a flowchart illustrating a method for interacting with an AR character according to an embodiment of the present application. Referring to FIG. 7, the method for interacting with the AR character may include steps S31-S35.
S31: and acquiring a real scene captured by the camera in real time, and displaying a corresponding virtual image in the real scene captured in real time.
After detecting that the AR application (or the application with the AR function) is started, the terminal captures a real scene captured by the camera in real time, and renders a virtual image to be used as the AR scene. The AR scene increases the rendered avatar compared to the real scene.
Real scenes include, but are not limited to, self-timer shots, games, scene placements, or viewfinding of a camera during a video call. Avatars are virtual content that does not exist in a real scene, and may be, for example, virtual 3D animals or static plants, furniture, and the like.
In an AR scene, the rendered avatar may be automatically displayed directly according to the preset, i.e. default avatar, when the AR class application is just started. The preset may be a corresponding relationship between a scene type and an avatar, for example, the real scene is an office scene, and the added avatar may be an animal or a green plant; for another example, the real scene is a home indoor scene, and the added avatar may be a person or furniture. Further, the terminal can increase, decrease or change the default avatar according to the user instruction.
In an embodiment, the preset may also be a corresponding relationship between a real person and an avatar, and in colloquial, when the real person appears in a real scene captured by a camera in real time, the terminal effectively identifies the real person through a face recognition technology, so as to determine an identity of the real person, for example, the identity is identified as a real person a, and at this time, the avatar having a corresponding relationship with the real person a, such as a leopard, may be automatically rendered in the real scene; and when the camera catches real person B in real time and appears in the real scene, the terminal automatically renders the virtual image which has a corresponding relation with the real person B in the real scene, such as a rabbit.
In an application scenario with a requirement for confidentiality or authentication, the terminal may perform authentication on a real person according to a human face image to further determine whether the real person is a user of the AR application and whether the real person can access information of the AR application, where performing authentication on the real person according to the human face image by the terminal may include:
and the terminal sends an authentication request to the server of the AR application, wherein the authentication request comprises the captured face image and is used for requesting authentication. And the terminal receives an authentication response returned by the server of the AR application responding to the authentication request. When the identity authentication response indicates that the information corresponding to the face image exists, the terminal determines that the real person is an effective user of the AR application or determines that the identity authentication of the real person passes, and displays the face image of the real person passing the identity authentication in the AR scene. And for the real person with failed authentication, the terminal may not display the face image in the AR scene.
Further, in the real scene captured in real time, the terminal may display or output information of each real person whose identity is verified, wherein the information may include information recorded in the server, such as the name, age, and the like of the real person.
S32: and detecting whether a real person appears in a real scene captured in real time.
The terminal can recognize the real person through a face recognition technology, a human body recognition algorithm, a human body behavior recognition algorithm, a human body gait recognition algorithm, a human body posture recognition algorithm and the like.
In one embodiment, the terminal can detect whether a human body is recognized within a preset range of the virtual image, and if not, the terminal judges that no real figure appears in the captured real scene; and if so, judging that the real person appears in the captured real scene.
In practical application scenarios, various recognition algorithms are limited, and only when the terminal (or the camera) reaches a certain distance from a real person, the terminal can detect the real person based on the recognition algorithms. Or, once the distance is too far, even if the real character is displayed in the AR scene, the terminal cannot recognize the real character, and even cannot recognize the body motion of the human body.
When no real person is detected in the real scene, step S33 is executed.
When a real person is detected in the real scene, steps S34 and S35 are performed.
S33: the avatar is controlled to perform a predetermined random operation.
S34: and detecting the contact condition of the virtual image and the real person.
S35: and controlling the virtual image to execute corresponding operation according to the contact condition.
The contact condition of the virtual image and the real person can be divided into two types: one is that the avatar is not in contact with the real person, and the other is that the avatar is in contact with the real person.
Here, the contact may be understood as that the distance between the avatar and the real person in the AR scene is zero, or the distance between the avatar and the real person in the AR scene is less than or equal to a predetermined threshold, that is, the contact may be regarded as long as the distance between the avatar and the real person is close enough, so as to reduce the detection error.
Different contact situations, different operations performed by the control avatar, in particular: referring to fig. 8, when the avatar does not contact the real person, the avatar is controlled to face the real person and an action conforming to the attribute of the avatar is performed. And when the virtual image is contacted with the real character, the virtual image is controlled to execute the action of contact interaction with the real character.
The attribute of the avatar may be understood as a classification of the avatar, such as static objects like animals, plants, furniture, transportation vehicles like cars, etc. The action according to its attribute can be understood as the instinctive action of the classified avatar in a state of being disturbed, such as zhao kaiwei, yakayao, rolling, etc. of cheetah, and also such as the wind-driven swinging of a flower.
The feedback made by the avatars of different attributes is different. For a movable avatar such as an animal, the terminal may also control the avatar to move toward the real character when the avatar is not in contact with the real character. For an immovable avatar such as a flower, when it is not in contact with a real character, a first anthropomorphic expression is displayed while controlling it toward the real character, and when a person comes into contact with the avatar, it is changed to a second anthropomorphic expression, for example, when a person is not in contact with a flower, the anthropomorphic expression of a flower is a depressed expression, and when a person comes into contact with a flower, for example, a stroking expression of a flower becomes a happy or wormy expression.
When the virtual image executes corresponding operation, the terminal can generate sound, vibration and/or screen shaking matched with the operation of the virtual image, and the feedback of the terminal is synchronously matched with the action of the virtual image, so that the real feedback interaction effect is better facilitated.
Based on the above, control the virtual image according to the contact condition of reality personage and virtual image and carry out corresponding operation, this application embodiment can simulate the contact of feedback virtual image and reality personage, and interactive feedback is truer, and is interesting stronger to do not need the reality personage to click, drag, twist reverse the operation such as operation to the screen, can automatic control virtual image's operation, interactive more simply, thereby be favorable to improving intelligent interactive effect.
For example, continuing with reference to fig. 2-5, after the AR application is turned on, a "cheetah" avatar is rendered, which automatically adjusts its pose to face or move toward the person when it is detected by a human recognition algorithm that a human body is present near the "cheetah" (i.e., the avatar is not in contact with a real character), and without contact with a human body, the "cheetah" can make some instinctive actions, such as a "cheetah" motion, with a low roar sound. When a human body contacts with the cheetah, the cheetah can perform forward 'flapping' action and can be accompanied by deterrent roaring sound, meanwhile, the terminal can vibrate slightly, a display interface applied to the AR can also perform matching special effect feedback such as slight shaking (namely screen shaking) and the like, and therefore feedback experience of simulating reality is provided for a user.
In the embodiment of the application, the terminal can identify the limb actions of the real person and control the virtual image to execute corresponding actions according to the limb actions. For example, when a person does not touch a flower, the anthropomorphic expression of the flower is a depressed expression, whereas when a person touches a flower, it is detected whether the person's action is to stroke or pluck the flower, if so, the anthropomorphic expression of the flower becomes a happy or sad expression, and if pluck, the anthropomorphic expression of the flower becomes a fear or crying expression, and the terminal may issue a anthropomorphic voice "please leave the hand". For another example, if a "cheetah" is detected to repel towards the body, the "cheetah" can be moved away from contact with the body and in other directions, or stay in place and walk at random.
FIG. 9 is a flow chart illustrating a method for interacting with an AR character according to another embodiment of the present application. Referring to FIG. 9, the method for interacting with the AR image includes steps S41-S45.
S41: and acquiring a real scene captured by the camera in real time, and displaying a corresponding virtual image in the real scene captured in real time.
S42: and detecting whether a real person appears in the real scene.
S431: and when the real person is not detected in the real scene, controlling the virtual image to execute a preset random operation.
S432: and when a single face image appears in the real scene, taking a person corresponding to the face image as a real person. Steps S44 to S45 are then executed.
S433: when a plurality of face images are detected to appear in a real scene, one of the face images having the largest size or the closest distance to the avatar is detected as a real character. Steps S44 to S45 are then executed.
S44: and detecting the contact condition of the virtual image and the real person.
S45: and controlling the virtual image to execute corresponding operation according to the contact condition.
On the basis of the description of the foregoing embodiment, but unlike this, the present embodiment considers the case where there are multiple characters in the AR scene, and the terminal may select one of the characters as a real character, with which the avatar performs the aforementioned interaction only.
Of course, when detecting that a plurality of face images appear in a real scene, the terminal may regard all people as real people, and as long as the avatar contacts one of the people, the avatar may be determined to contact the real people, and the aforementioned corresponding feedback may be performed.
Or, when detecting that a plurality of face images appear in the real scene, regarding each person as a real person, wherein a plurality of real persons exist, rendering an avatar by the terminal according to the preset setting for each real person, then detecting the contact condition between each avatar and the corresponding real person, and executing the corresponding feedback according to the contact condition.
Fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 10, the electronic device 30 may be regarded as the aforementioned terminal, and includes a processor 31 and a memory 32, which are connected via a communication bus 33 for data or signal transmission.
The processor 31 is a control center of the electronic device 30, connects various parts of the entire electronic device 30 by various interfaces and lines, and performs various functions of the electronic device 30 and processes data by running or loading a program stored in the memory 32 and calling data stored in the memory 32, thereby integrally monitoring the electronic device 30.
The processor 31 loads instructions corresponding to one or more processes of the program into the memory 32 according to predetermined steps, and the processor 31 runs the program stored in the memory 32, thereby implementing the steps of any of the foregoing embodiments.
For the specific implementation manner of each step, that is, the specific content of the step executed by the processor 31 calling the program, reference may be made to the foregoing embodiments, and details are not described here.
It should be understood that, when implemented in a practical application scenario, the execution bodies of the above steps may not be the processor 31 and the memory 32, but may be implemented by other modules and units respectively, according to the device type of the electronic device 30.
It will be understood by those skilled in the art that all or part of the steps in the methods of the above embodiments may be performed by instructions or by related hardware controlled by the instructions, which may be stored in a readable storage medium and loaded and executed by a processor.
To this end, embodiments of the present application provide a readable storage medium, which stores a program, where the program can be loaded by a processor to perform one or more steps of any method for interacting with an AR character provided by embodiments of the present application.
The readable storage medium may include a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, and the like.
Since the program stored in the readable storage medium may execute one or more steps of any method for interacting with an AR character provided in the embodiments of the present application, beneficial effects that can be achieved by any method for interacting with an AR character provided in the embodiments of the present application may be achieved, for details, see the foregoing embodiments, and are not described herein again.
The present application further provides an apparatus, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the method as described above.
Embodiments of the present application also provide a computer program product, which includes computer program code, when the computer program code runs on a computer, the computer is caused to execute the method as described in the above various possible embodiments.
An embodiment of the present application further provides a chip, which includes a memory and a processor, where the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device in which the chip is installed executes the method described in the above various possible embodiments.
It should be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element, and that elements, features, or elements having the same designation in different embodiments may or may not have the same meaning as that of the other elements, and that the particular meaning will be determined by its interpretation in the particular embodiment or by its context in further embodiments.
In addition, although the terms "first, second, third, etc. are used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, depending on the context, without departing from the scope herein. The term "if" can be interpreted as "at … …" or "when … …" or "in response to a determination". Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. The terms "or" and/or "are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.
Further, although the various steps in the flowcharts herein are shown in order as indicated by the arrows, they are not necessarily performed in order as indicated by the arrows. Unless explicitly stated otherwise herein, the steps are not performed in the exact order, but may be performed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, in different orders, and may be performed alternately or at least partially with respect to other steps or sub-steps of other steps.
Although the application has been shown and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art based upon a reading and understanding of this specification and the annexed drawings. The present application includes all such modifications and variations, and is supported by the technical solutions of the foregoing embodiments. That is, the above-mentioned embodiments are only some of the embodiments of the present application, and not intended to limit the scope of the present application, and all equivalent structural changes made by using the contents of the present specification and the drawings, such as the combination of technical features between the embodiments, or the direct or indirect application to other related technical fields, are included in the scope of the present application.
Claims (16)
1. An AR interaction method, the method comprising:
s11, outputting at least one virtual image corresponding to the scene information and/or the image information;
s12, detecting whether a preset event occurs;
and S13, presetting the virtual image according to the detection result.
2. The method according to claim 1, wherein the step of S11 includes:
analyzing the scene information and/or the image information;
judging whether preset characteristic information exists in the scene information and/or the image information;
if yes, outputting at least one virtual image corresponding to the preset characteristic information.
3. The method of claim 2, wherein the step of outputting at least one avatar corresponding to the predetermined feature information further comprises:
and displaying the virtual image at a position corresponding to the preset characteristic information.
4. The method of claim 2, wherein the preset feature information comprises at least one of:
plane information, object information, person information, and scene information.
5. The method of claim 1, wherein the predetermined event comprises at least one of:
the expression changes, the action changes, and the voice information is received.
6. The method according to any one of claims 1 to 5, wherein the presetting process comprises: and processing the virtual image according to a processing strategy corresponding to the preset event.
7. The method of claim 6, wherein the processing policy comprises at least one of:
controlling the virtual image to move according to a preset track;
controlling the virtual image to make preset expression information and/or imitation expression change information;
and controlling the virtual image to make preset action information and/or imitation action change information.
8. The method according to any one of claims 1 to 5, wherein the step S13 further comprises:
and if at least two preset events are detected, processing the virtual image simultaneously or sequentially according to the at least two preset events.
9. An AR interaction method, comprising:
s21, detecting whether a preset event occurs in the scene information and/or the image information;
s22, if yes, acquiring at least one virtual image according to at least one of a preset event, scene information and/or image information;
and S23, displaying the virtual image according to a preset rule.
10. The method of claim 9, wherein the predetermined event comprises at least one of:
the expression changes, the action changes, and the voice information is received.
11. The method of claim 9, wherein the step of S22 includes:
analyzing at least one of the preset event, the scene information and/or the image information;
judging whether preset characteristic information exists or not;
if yes, outputting at least one virtual image corresponding to the preset characteristic information.
12. The method of claim 11, wherein the preset feature information comprises at least one of:
plane information, object information, character information, scene information, expression change information, action change information and voice information.
13. The method according to any one of claims 9 to 12, wherein the step S23 includes at least one of:
controlling the virtual image to move according to a preset track;
controlling the virtual image to make preset expression information and/or imitation expression change information;
and controlling the virtual image to make preset action information and/or imitation action change information.
14. The method according to any one of claims 9 to 12, wherein the step of S23 further comprises:
and if at least two preset events are detected, processing the virtual image simultaneously or sequentially according to the at least two preset events.
15. An electronic device comprising a memory and a processor, the memory storing a program for execution by the processor to perform the steps in the AR interaction method of any of claims 1 to 14.
16. A readable storage medium, in which a program is stored, the program being adapted to be executed by a processor to perform the steps of the AR interaction method as claimed in any one of claims 1 to 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010776182.7A CN111880664A (en) | 2020-08-03 | 2020-08-03 | AR interaction method, electronic device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010776182.7A CN111880664A (en) | 2020-08-03 | 2020-08-03 | AR interaction method, electronic device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111880664A true CN111880664A (en) | 2020-11-03 |
Family
ID=73211443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010776182.7A Pending CN111880664A (en) | 2020-08-03 | 2020-08-03 | AR interaction method, electronic device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111880664A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112738770A (en) * | 2020-12-07 | 2021-04-30 | 浙江合众新能源汽车有限公司 | Vehicle binding activation vehicle-mounted and mobile phone synchronous interaction mode system and application method thereof |
| CN113010140A (en) * | 2021-03-15 | 2021-06-22 | 深圳市慧鲤科技有限公司 | Sound playing method and device, electronic equipment and storage medium |
| CN113674429A (en) * | 2021-08-17 | 2021-11-19 | 北京服装学院 | Interactive experience jewelry design method capable of performing AR interaction with screen |
| CN117132743A (en) * | 2023-08-29 | 2023-11-28 | 支付宝(杭州)信息技术有限公司 | Virtual image processing method and device |
| WO2023241010A1 (en) * | 2022-06-14 | 2023-12-21 | Oppo广东移动通信有限公司 | Virtual image generation method and apparatus, electronic device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108022301A (en) * | 2017-11-23 | 2018-05-11 | 腾讯科技（上海）有限公司 | A kind of image processing method, device and storage medium |
| CN108519817A (en) * | 2018-03-26 | 2018-09-11 | 广东欧珀移动通信有限公司 | Exchange method, device, storage medium based on augmented reality and electronic equipment |
| CN110716645A (en) * | 2019-10-15 | 2020-01-21 | 北京市商汤科技开发有限公司 | Augmented reality data presentation method and device, electronic equipment and storage medium |
| CN110888532A (en) * | 2019-11-25 | 2020-03-17 | 深圳传音控股股份有限公司 | Man-machine interaction method and device, mobile terminal and computer readable storage medium |
| US20200160610A1 (en) * | 2017-07-28 | 2020-05-21 | Tencent Technology (Shenzhen) Company Limited | Information processing method and apparatus, terminal device, and computer readable storage medium |
| CN111273772A (en) * | 2020-01-17 | 2020-06-12 | 江苏艾佳家居用品有限公司 | Augmented reality interaction method and device based on slam mapping method |
| CN111339938A (en) * | 2020-02-26 | 2020-06-26 | 广州腾讯科技有限公司 | Information interaction method, device, equipment and storage medium |
-
2020
- 2020-08-03 CN CN202010776182.7A patent/CN111880664A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200160610A1 (en) * | 2017-07-28 | 2020-05-21 | Tencent Technology (Shenzhen) Company Limited | Information processing method and apparatus, terminal device, and computer readable storage medium |
| CN108022301A (en) * | 2017-11-23 | 2018-05-11 | 腾讯科技（上海）有限公司 | A kind of image processing method, device and storage medium |
| CN108519817A (en) * | 2018-03-26 | 2018-09-11 | 广东欧珀移动通信有限公司 | Exchange method, device, storage medium based on augmented reality and electronic equipment |
| CN110716645A (en) * | 2019-10-15 | 2020-01-21 | 北京市商汤科技开发有限公司 | Augmented reality data presentation method and device, electronic equipment and storage medium |
| CN110888532A (en) * | 2019-11-25 | 2020-03-17 | 深圳传音控股股份有限公司 | Man-machine interaction method and device, mobile terminal and computer readable storage medium |
| CN111273772A (en) * | 2020-01-17 | 2020-06-12 | 江苏艾佳家居用品有限公司 | Augmented reality interaction method and device based on slam mapping method |
| CN111339938A (en) * | 2020-02-26 | 2020-06-26 | 广州腾讯科技有限公司 | Information interaction method, device, equipment and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112738770A (en) * | 2020-12-07 | 2021-04-30 | 浙江合众新能源汽车有限公司 | Vehicle binding activation vehicle-mounted and mobile phone synchronous interaction mode system and application method thereof |
| CN112738770B (en) * | 2020-12-07 | 2023-04-14 | 合众新能源汽车股份有限公司 | Vehicle binding activation vehicle-mounted and mobile phone synchronous interaction mode system and application method thereof |
| CN113010140A (en) * | 2021-03-15 | 2021-06-22 | 深圳市慧鲤科技有限公司 | Sound playing method and device, electronic equipment and storage medium |
| CN113674429A (en) * | 2021-08-17 | 2021-11-19 | 北京服装学院 | Interactive experience jewelry design method capable of performing AR interaction with screen |
| CN113674429B (en) * | 2021-08-17 | 2024-02-23 | 北京服装学院 | Interactive experience jewelry design method capable of carrying out AR interaction with screen |
| WO2023241010A1 (en) * | 2022-06-14 | 2023-12-21 | Oppo广东移动通信有限公司 | Virtual image generation method and apparatus, electronic device and storage medium |
| CN117132743A (en) * | 2023-08-29 | 2023-11-28 | 支付宝(杭州)信息技术有限公司 | Virtual image processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111880664A (en) | AR interaction method, electronic device and readable storage medium | |
| US11182615B2 (en) | Method and apparatus, and storage medium for image data processing on real object and virtual object | |
| Betancourt et al. | The evolution of first person vision methods: A survey | |
| JP7137804B2 (en) | Method and system for gesture-based interaction | |
| US20180088677A1 (en) | Performing operations based on gestures | |
| US20210191690A1 (en) | Virtual Reality Device Control Method And Apparatus, And Virtual Reality Device And System | |
| CN108942919B (en) | Interaction method and system based on virtual human | |
| CN109254650B (en) | Man-machine interaction method and device | |
| JP7268071B2 (en) | Virtual avatar generation method and generation device | |
| US20230021533A1 (en) | Method and apparatus for generating video with 3d effect, method and apparatus for playing video with 3d effect, and device | |
| CN113867531A (en) | Interaction method, device, equipment and computer readable storage medium | |
| CN111142967B (en) | Augmented reality display method and device, electronic equipment and storage medium | |
| CN113610953A (en) | Information processing method and device and computer readable storage medium | |
| CN113327311B (en) | Virtual character-based display method, device, equipment and storage medium | |
| CN112637692B (en) | Interaction method, device and equipment | |
| Verma et al. | Digital assistant with augmented reality | |
| CN112686990A (en) | Three-dimensional model display method and device, storage medium and computer equipment | |
| CN114979789A (en) | Video display method and device and readable storage medium | |
| CN109800652A (en) | Character translation method and device | |
| CN113112613B (en) | Model display method and device, electronic equipment and storage medium | |
| US20240037832A1 (en) | Metaverse system | |
| CN111627097B (en) | Virtual scene display method and device | |
| US20220118358A1 (en) | Computer-readable recording medium, and image generation system | |
| CN116977510A (en) | Data processing method, device, computer equipment and readable storage medium | |
| CN117636386A (en) | Method, device, equipment and medium for detecting hand key points |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |