CN113934297A - Interaction method and device based on augmented reality, electronic equipment and medium - Google Patents
Interaction method and device based on augmented reality, electronic equipment and medium Download PDFInfo
- Publication number
- CN113934297A CN113934297A CN202111193452.2A CN202111193452A CN113934297A CN 113934297 A CN113934297 A CN 113934297A CN 202111193452 A CN202111193452 A CN 202111193452A CN 113934297 A CN113934297 A CN 113934297A
- Authority
- CN
- China
- Prior art keywords
- recognized
- image
- dimensional model
- target
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Abstract
The embodiment of the invention discloses an interaction method, an interaction device, electronic equipment and a medium based on augmented reality. Wherein, the method comprises the following steps: acquiring an image to be recognized from any plane of a geometric entity through image acquisition equipment on a terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model; determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model; and displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen. The realization carries out the model show of augmented reality technique through the geometric solid, lets the user carry out the interaction with virtual object through the entity perception, ensures that the picture on virtual three-dimensional model and the geometric solid corresponds each other, improves the model show precision, promotes user experience.
Description
Technical Field
Embodiments of the present invention relate to augmented reality technologies, and in particular, to an interaction method and apparatus, an electronic device, and a medium based on augmented reality.
Background
Augmented Reality (AR) technology is a technology that increases the user's perception of the real world using computer simulation information. The method superimposes a simulated three-dimensional model, a scene and system information generated by a computer in a real scene, and enhances the perception of a user to the real world through multi-element information fusion.
In the current augmented reality technology, when the image of the virtual object is generated, the correct image of the virtual object is difficult to find out quickly, the display precision and efficiency of the virtual object are low, and the interaction between the user and the virtual object depends on the operation of the user on the touch screen, so that the AR experience of the user is influenced.
Disclosure of Invention
Embodiments of the present invention provide an interaction method, an interaction device, an electronic device, and a medium based on augmented reality, so as to improve interaction efficiency of an AR technology and improve AR interaction experience of a user.
In a first aspect, an embodiment of the present invention provides an interaction method based on augmented reality, where the method includes:
acquiring an image to be recognized from any plane of a geometric entity through image acquisition equipment on a terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model;
and displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
In a second aspect, an embodiment of the present invention further provides an interaction device based on augmented reality, where the device includes:
the object determination module is used for acquiring an image to be recognized from any plane of the geometric solid through image acquisition equipment on the terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
the model determining module is used for determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model;
and the model display module is used for displaying the target three-dimensional model as an avatar at the image object to be identified in the terminal screen.
In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the augmented reality-based interaction method according to any embodiment of the present invention.
In a fourth aspect, the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the augmented reality based interaction method according to any embodiment of the present invention.
According to the embodiment of the invention, the image acquisition equipment arranged on the terminal acquires the image on the geometric entity to obtain the image to be identified. And performing feature extraction on the image to be recognized according to a pre-trained neural network model to determine the object to be recognized. And determining the target three-dimensional model according to the pre-stored association relationship between the object and the three-dimensional model. And displaying the target three-dimensional model, wherein the target three-dimensional model is an AR virtual image, and the display position is consistent with the position of the object to be recognized in the screen. The problems that in the prior art, target three-dimensional models are not searched timely and accurately are solved through the preset neural network model, and the efficiency and the precision of AR interaction are improved. The user can interact with the virtual object through the geometric entity, the display position of the virtual target three-dimensional model is adjusted, and the interaction experience of the user is improved.
Drawings
Fig. 1 is a schematic flowchart of an interaction method based on augmented reality according to a first embodiment of the present invention;
fig. 2 is a schematic flowchart of an interaction method based on augmented reality according to a second embodiment of the present invention;
fig. 3 is a block diagram of an augmented reality-based interaction device according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an augmented reality-based interaction device in a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a schematic flowchart of an interaction method based on augmented reality according to an embodiment of the present invention, where the embodiment is applicable to a case where an AR technology is used to interact with a virtual object, and the method can be executed by an interaction device based on augmented reality. As shown in fig. 1, the method specifically includes the following steps:
and step 110, acquiring an image to be recognized from any plane of the geometric solid through image acquisition equipment on the terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model.
The terminal can be a mobile terminal such as a mobile phone or a computer, and the terminal is provided with an image acquisition device, for example, a camera can be installed. The geometric solid may be a geometric structure composed of planes, for example, a tetrahedron or a square, each plane of the geometric solid may be provided with a picture of the object to be recognized, and the object to be recognized in each plane picture may be different. For example, the object to be identified in the picture may be an animal, a plant, a table, a chair, or the like. In this embodiment, the material of the geometric solid is not specifically limited, for example, the material of the geometric solid may be wood, magnet, plastic, or acrylic, and the geometric solid may be solid or hollow. The material of the picture containing the object to be identified is not particularly limited, for example, the material of the picture can be paper, magnet, plastic or acrylic, etc., and the picture can be pasted or printed on a plane and can also be arranged on the plane through a card insertion or magnetic attraction.
The image acquisition equipment on the terminal can shoot or scan the external environment, the user can hold the geometric solid by hand, the picture on any plane of the geometric solid faces the terminal, and the image acquisition equipment acquires the picture on the plane to obtain the image to be identified. The image to be recognized may include a picture in a plane of the geometric entity and an environment where the geometric entity is located, angles between the geometric entity and the terminal are different, and contents of the image to be recognized are different. The user can hold the geometric solid and the terminal for alignment, so that the picture on the plane of the geometric solid occupies a larger range in the image to be recognized, and the recognition of the object to be recognized on the picture is facilitated. After the image to be recognized is obtained, the image to be recognized is input into a pre-trained object recognition model, and an object to be recognized on the image to be recognized is determined. The object recognition model may be a neural network model for recognizing objects in the image. For example, a picture of a puppy is input into the object recognition model, and the content in the picture is output as the puppy.
In this embodiment, optionally, determining an object to be recognized on an image to be recognized based on a pre-trained object recognition model includes: extracting the characteristics of the object to be recognized from the image to be recognized according to a pre-trained object recognition model; and outputting the category of the object to be recognized on the image to be recognized according to the characteristics of the object to be recognized.
Specifically, the object recognition model may be a pre-trained neural network model, and when the object recognition model is trained, a plurality of pictures to be trained may be pre-collected as a picture set, and for each object, pictures at various angles may be collected. Inputting the picture to be trained into the constructed object recognition model, and determining the object characteristics in the picture to be trained through the characteristic extraction processes such as convolution, pooling and the like. And outputting the class probability of the object according to the object characteristics, and selecting the class with high probability as the object class in the picture. And comparing the output object type with a predetermined correct object type to determine whether the output result is correct. If the model is correct, the model training is finished; and if not, continuing training according to a preset loss function until the training is finished. After the training of the object recognition model is completed, the image to be recognized is input into the object recognition model, feature extraction is carried out on the object to be recognized in the image to be recognized, the feature of the object to be recognized is obtained, and the object to be recognized is the object in the picture of the geometric solid plane. And determining the probability of the object to be recognized belonging to each object category according to the characteristics of the object to be recognized, and taking the category with the highest probability as the category of the object to be recognized, namely determining the object to be recognized. The beneficial effect who sets up like this lies in, trains neural network model in advance, can treat discernment object and carry out the feature extraction, does not restrict the terminal to the scanning angle of the picture on the geometric solid plane, as long as can discern the object feature on the picture can, improves object recognition accuracy, and then improves the show precision of virtual image.
And step 120, determining a target three-dimensional model of the object to be recognized according to the preset incidence relation between the object and the three-dimensional model.
The three-dimensional model is an virtual image used for AR display, the object can be displayed in a three-dimensional mode, and a user can view the physical image of the object in 360 degrees through the virtual three-dimensional model. For example, the three-dimensional model may be a virtual stereo model of a dinosaur. And after the object to be recognized is determined, determining a target three-dimensional model of the object to be recognized according to the incidence relation between the object and the three-dimensional model. For example, an association relationship between a table and a three-dimensional model of the table may be stored, and when it is determined that the object to be recognized is the table, the three-dimensional model of the table is searched from a plurality of pre-stored three-dimensional models according to a preset association relationship as the target three-dimensional model.
And step 130, displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
After the target three-dimensional model is determined, the target three-dimensional model is displayed in a screen of the terminal as an avatar of the object to be recognized, and the display position of the target three-dimensional model in the screen can be consistent with the position of the picture of the object to be recognized in the screen. For example, the target three-dimensional model may be displayed above the picture of the object to be recognized in the screen, that is, the target three-dimensional model is made to stand on the picture of the object to be recognized, so that the viewing experience of the user is improved.
In this embodiment, optionally, displaying the target three-dimensional model as an avatar at a position of the object to be recognized in the terminal screen includes: determining the current angle of an object to be identified in a terminal screen; and taking the target three-dimensional model as an avatar, and displaying the target three-dimensional model at the position of the object to be recognized in the terminal screen at the current angle.
Specifically, when the target three-dimensional model is subjected to AR display, a current angle of an object to be recognized in a terminal screen may be determined, where the current angle may be a relative angle between a plane of a geometric entity scanned by the image acquisition device and the image acquisition device, that is, the current angle is a relative angle between a picture on the scanned plane and the image acquisition device. For example, if the current angle of the object to be recognized in the picture is facing the image capturing device while standing, that is, the user holds the geometric entity by hand, and the plane of the geometric entity is perpendicular to the ground, and the user holds the terminal by hand, and the terminal is also at an angle perpendicular to the ground, then the current angle of the object to be recognized faces the image capturing device, and the object to be recognized in the picture can be facing the image capturing device while standing. If the user holds the terminal or the geometric entity to tilt, the current angle can be changed accordingly. After the current angle is determined, the target three-dimensional model is displayed at the current angle, for example, the target three-dimensional model may be stood on a picture of the object to be recognized in a forward direction, facing the image acquisition device. And if the picture to be identified in the picture is upside down, the target three-dimensional model can be displayed in an inverted mode. The beneficial effect that sets up like this lies in, can confirm the show angle of avatar according to the handheld operation of user to the geometric solid, makes the avatar show according to the angle that the user needs, promotes user experience. In the process of AR display, a user can hold the geometric solid by hand to rotate or tilt the angle, and the current angle is changed. The current angle of the object to be recognized is determined in real time, and the display angle of the virtual image can be changed in real time according to the current angle, for example, the user holds the geometric entity by hand to rotate the object to be recognized on the picture, and then the virtual image of the target three-dimensional model rotates along with the rotation. The user can comprehensively check the target three-dimensional model without operating on the terminal, the problem that the interaction between the user and the virtual object depends on the operation of the user on the touch screen is solved, the interaction between the user and the virtual object is realized through the perception of the entity, the virtual three-dimensional model is ensured to correspond to the picture on the geometric entity, and the display precision of the AR model is improved.
In this embodiment, optionally, after displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen, the method further includes: responding to a three-dimensional model adjusting instruction sent by a user on a terminal screen, and determining a target angle and a target size of an avatar of a target three-dimensional model; and adjusting the virtual image of the target three-dimensional model according to the target angle and the target size, and displaying the virtual image at the position of the object to be recognized in the terminal screen.
Specifically, the user can adjust the display angle of the virtual image by adjusting the geometric entity, and can also operate the terminal screen to adjust the virtual image. The user can send out a three-dimensional model adjusting instruction on the terminal screen, and the three-dimensional model adjusting instruction can comprise instructions of rotating, enlarging, reducing and viewing information, and for example, the user can perform a zooming operation on the screen by sliding a finger. And responding to a three-dimensional model adjusting instruction sent by a user, adjusting the virtual image of the target three-dimensional model, and determining the target angle and the target size of the virtual image. For example, when the user zooms in or out of the avatar, the zoomed size is used as the target size. Or, rotating the virtual image on the screen, wherein the rotated angle is the target angle. And adjusting the virtual image according to the target angle and the target size, displaying the adjusted virtual image on a screen, wherein the position of the adjusted virtual image can still correspond to the position of the object to be recognized in the picture in the screen. Namely, the terminal interaction interface can generate information of operation data, such as button click, 3D model enlargement, reduction, rotation, movement, menu bar interaction, and the like, and update and render the scene in real time according to the user's operation and camera sensing. The beneficial effect who sets up like this lies in, and the user can adjust the AR model through geometry entity and terminal, improves the interactive flexibility of AR and interactive real-time, and the user of being convenient for looks over the model, promotes user's interactive experience.
According to the technical scheme of the embodiment, the image acquisition equipment installed on the terminal acquires the image on the geometric entity to obtain the image to be identified. And extracting the characteristics of the picture to be recognized according to a pre-trained neural network model, and determining the object to be recognized. And determining the target three-dimensional model according to the pre-stored association relationship between the object and the three-dimensional model. And displaying the target three-dimensional model, wherein the target three-dimensional model is an AR virtual image, and the display position is consistent with the position of the object to be recognized in the screen. The problems that in the prior art, target three-dimensional models are not searched timely and accurately are solved through the preset neural network model, and the efficiency and the precision of AR interaction are improved. The user can interact with the virtual object through the geometric entity, the display position of the virtual target three-dimensional model is adjusted, and the interaction experience of the user is improved.
Example two
Fig. 2 is a schematic flow chart of an interaction method based on augmented reality according to a second embodiment of the present invention, and the present embodiment is further optimized based on the above embodiments. As shown in fig. 2, the method specifically includes the following steps:
and step 210, acquiring an image to be identified from any plane of the geometric solid through an image acquisition device on the terminal.
And step 220, judging whether the image to be recognized exists in a preset candidate image, if so, determining a target three-dimensional model of the image to be recognized according to the incidence relation between the preset candidate image and the three-dimensional model.
The method comprises the steps of obtaining an image to be identified, wherein the content in the image to be identified can contain an object to be identified. To perform AR virtual display on an object to be recognized, a virtual three-dimensional model corresponding to the object to be recognized needs to be searched. And pre-storing the incidence relation between different candidate images and the three-dimensional model, and searching whether the image to be identified exists in the candidate images after the image to be identified is obtained. For example, if the content in the image to be recognized is a front image of a dog, and the candidate image also has a front image of the dog, it is determined that the image to be recognized exists in the preset candidate image.
In the embodiment, feature extraction is not required to be performed on the content of the image to be recognized through a neural network model for feature extraction, and only whether a candidate image consistent with the image to be recognized exists is searched. And if so, determining the three-dimensional model of the corresponding candidate image as the target three-dimensional model. If not, the virtual display of the object to be recognized cannot be performed. When the image to be identified is obtained through the image acquisition equipment, a user can hold the terminal by hand to face the plane of the geometric solid in the forward direction, so that the image acquisition equipment can accurately acquire the picture on the plane of the geometric solid, the problem that the angle inclination cannot find out the corresponding candidate image is avoided, and the determination precision of the target three-dimensional model is improved. After determining the target three-dimensional model, the user can rotate or tilt the geometric entity again to adjust the virtual image.
And step 230, displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
According to the embodiment of the invention, the image acquisition equipment arranged on the terminal acquires the image on the geometric entity to obtain the image to be identified. And determining the target three-dimensional model according to the pre-stored incidence relation between the candidate image and the three-dimensional model, so that the determination efficiency of the target three-dimensional model is improved. And displaying the target three-dimensional model, wherein the target three-dimensional model is an AR virtual image, and the display position is consistent with the position of the object to be recognized in the screen. The picture of the object to be recognized is arranged on the geometric solid, so that the interestingness and convenience of AR display are improved, the user can interact with the virtual object through the geometric solid, the display position of the virtual target three-dimensional model is adjusted, and the interaction experience of the user is improved.
EXAMPLE III
Fig. 3 is a block diagram of an interaction apparatus based on augmented reality according to a third embodiment of the present invention, which is capable of executing an interaction method based on augmented reality according to any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 3, the apparatus specifically includes:
an object determining module 301, configured to acquire an image to be recognized from any plane of a geometric entity through an image acquisition device on a terminal, and determine an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
the model determining module 302 is configured to determine a target three-dimensional model of the object to be recognized according to a preset association relationship between the object and the three-dimensional model;
and the model display module 303 is configured to display the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
Optionally, the object determining module 301 includes:
the characteristic extraction unit is used for extracting the characteristics of the object to be recognized from the image to be recognized according to a pre-trained object recognition model;
and the object type determining unit is used for outputting the type of the object to be recognized on the image to be recognized according to the characteristics of the object to be recognized.
Optionally, the geometric entity is a geometric structure composed of a plane on which a picture of the object to be recognized is disposed.
Optionally, the apparatus further comprises:
the image recognition system comprises a picture judgment module, a geometric entity recognition module and a recognition module, wherein the picture judgment module is used for judging whether an image to be recognized exists in a preset candidate image or not after the image to be recognized is acquired from any plane of the geometric entity through image acquisition equipment on a terminal;
and the model searching module is used for determining the target three-dimensional model of the image to be recognized according to the preset incidence relation between the candidate image and the three-dimensional model if the target three-dimensional model is the target three-dimensional model.
Optionally, the model display module 303 is specifically configured to:
determining the current angle of the object to be identified in the terminal screen;
and taking the target three-dimensional model as an avatar, and displaying the current angle at the position of the object to be recognized in the terminal screen.
Optionally, the apparatus further comprises:
the model adjustment determining module is used for responding to a three-dimensional model adjustment instruction sent by a user on a terminal screen after the target three-dimensional model is displayed as an avatar at the position of an object to be recognized in the terminal screen, and determining a target angle and a target size of the avatar of the target three-dimensional model;
and the model adjusting and displaying module is used for adjusting the virtual image of the target three-dimensional model according to the target angle and the target size and displaying the virtual image at the position of the object to be identified in the terminal screen.
According to the embodiment of the invention, the image acquisition equipment arranged on the terminal acquires the image on the geometric entity to obtain the image to be identified. And extracting the characteristics of the picture to be recognized according to a pre-trained neural network model, and determining the object to be recognized. And determining the target three-dimensional model according to the pre-stored association relationship between the object and the three-dimensional model. And displaying the target three-dimensional model, wherein the target three-dimensional model is an AR virtual image, and the display position is consistent with the position of the object to be recognized in the screen. The problems that in the prior art, target three-dimensional models are not searched timely and accurately are solved through the preset neural network model, and the efficiency and the precision of AR interaction are improved. The user can interact with the virtual object through the geometric entity, the display position of the virtual target three-dimensional model is adjusted, and the interaction experience of the user is improved.
Example four
Fig. 4 is a schematic structural diagram of an interaction device based on augmented reality according to a fourth embodiment of the present invention. An augmented reality-based interaction device is an electronic device and fig. 4 shows a block diagram of an exemplary electronic device 400 suitable for use in implementing embodiments of the present invention. The electronic device 400 shown in fig. 4 is only an example and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
As shown in fig. 4, electronic device 400 is embodied in the form of a general purpose computing device. The components of electronic device 400 may include, but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 that couples the various system components (including the system memory 402 and the processing unit 401).
The system memory 402 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)404 and/or cache memory 405. The electronic device 400 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 406 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 403 by one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 407 generally perform the functions and/or methods of the described embodiments of the invention.
The electronic device 400 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), with one or more devices that enable a user to interact with the electronic device 400, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 400 to communicate with one or more other computing devices. Such communication may be through input/output (I/O) interface 411. Also, the electronic device 400 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 412. As shown in FIG. 4, the network adapter 412 communicates with the other modules of the electronic device 400 over the bus 403. It should be appreciated that although not shown in FIG. 4, other hardware and/or software modules may be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 401 executes various functional applications and data processing by running the program stored in the system memory 402, for example, to implement an augmented reality-based interaction method provided by the embodiment of the present invention, including:
acquiring an image to be recognized from any plane of a geometric entity through image acquisition equipment on a terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model;
and displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
EXAMPLE five
The fifth embodiment of the present invention further provides a storage medium containing computer-executable instructions, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method for implementing an augmented reality-based interaction method provided in the fifth embodiment of the present invention includes:
acquiring an image to be recognized from any plane of a geometric entity through image acquisition equipment on a terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model;
and displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. An augmented reality-based interaction method, comprising:
acquiring an image to be recognized from any plane of a geometric entity through image acquisition equipment on a terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model;
and displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
2. The method of claim 1, wherein determining the object to be recognized on the image to be recognized based on a pre-trained object recognition model comprises:
extracting the characteristics of the object to be recognized from the image to be recognized according to a pre-trained object recognition model;
and outputting the category of the object to be recognized on the image to be recognized according to the characteristics of the object to be recognized.
3. Method according to claim 1, characterized in that the geometrical entity is a geometrical structure consisting of a plane on which a picture of the object to be recognized is arranged.
4. The method according to claim 1, wherein after acquiring the image to be identified from any plane of the geometric solid by an image acquisition device on the terminal, the method further comprises:
judging whether the image to be identified exists in a preset candidate image or not;
and if so, determining the target three-dimensional model of the image to be recognized according to the preset incidence relation between the candidate image and the three-dimensional model.
5. The method of claim 1, wherein displaying the target three-dimensional model as an avatar at a location of an object to be recognized in a terminal screen comprises:
determining the current angle of the object to be identified in the terminal screen;
and taking the target three-dimensional model as an avatar, and displaying the current angle at the position of the object to be recognized in the terminal screen.
6. The method according to claim 1, further comprising, after displaying the target three-dimensional model as an avatar at a position of an object to be recognized in a terminal screen:
responding to a three-dimensional model adjusting instruction sent by a user on a terminal screen, and determining a target angle and a target size of an avatar of the target three-dimensional model;
and adjusting the virtual image of the target three-dimensional model according to the target angle and the target size, and displaying the virtual image at the position of the object to be recognized in the terminal screen.
7. An augmented reality-based interaction device, comprising:
the object determination module is used for acquiring an image to be recognized from any plane of the geometric solid through image acquisition equipment on the terminal, and determining an object to be recognized on the image to be recognized based on a pre-trained object recognition model;
the model determining module is used for determining a target three-dimensional model of the object to be recognized according to a preset incidence relation between the object and the three-dimensional model;
and the model display module is used for displaying the target three-dimensional model as an avatar at the position of the object to be recognized in the terminal screen.
8. The apparatus of claim 7, wherein the object determination module comprises:
the characteristic extraction unit is used for extracting the characteristics of the object to be recognized from the image to be recognized according to a pre-trained object recognition model;
and the category output unit is used for outputting the category of the object to be recognized on the image to be recognized according to the characteristics of the object to be recognized.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the augmented reality based interaction method according to any one of claims 1-6 when executing the program.
10. A storage medium containing computer-executable instructions for performing the augmented reality based interaction method of any one of claims 1-6 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111193452.2A CN113934297A (en) | 2021-10-13 | 2021-10-13 | Interaction method and device based on augmented reality, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111193452.2A CN113934297A (en) | 2021-10-13 | 2021-10-13 | Interaction method and device based on augmented reality, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113934297A true CN113934297A (en) | 2022-01-14 |
Family
ID=79278753
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111193452.2A Pending CN113934297A (en) | 2021-10-13 | 2021-10-13 | Interaction method and device based on augmented reality, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113934297A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023246189A1 (en) * | 2022-06-24 | 2023-12-28 | 如你所视(北京)科技有限公司 | Image information display method and device |
| CN117412019A (en) * | 2023-12-14 | 2024-01-16 | 深圳市欧冠微电子科技有限公司 | Aerial imaging and virtual reality dynamic backlight determination method and device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701863A (en) * | 2016-01-11 | 2016-06-22 | 华为技术有限公司 | Image processing method and device |
| CN109685905A (en) * | 2017-10-18 | 2019-04-26 | 深圳市掌网科技股份有限公司 | Cell planning method and system based on augmented reality |
| US20190340649A1 (en) * | 2018-05-07 | 2019-11-07 | Adobe Inc. | Generating and providing augmented reality representations of recommended products based on style compatibility in relation to real-world surroundings |
| CN111626183A (en) * | 2020-05-25 | 2020-09-04 | 深圳市商汤科技有限公司 | Target object display method and device, electronic equipment and storage medium |
| CN111640200A (en) * | 2020-06-10 | 2020-09-08 | 浙江商汤科技开发有限公司 | AR scene special effect generation method and device |
| CN111680570A (en) * | 2020-05-13 | 2020-09-18 | 珠海格力电器股份有限公司 | Augmented reality image data processing method, device, equipment and storage medium |
| CN111970557A (en) * | 2020-09-01 | 2020-11-20 | 深圳市慧鲤科技有限公司 | Image display method, image display device, electronic device, and storage medium |
-
2021
- 2021-10-13 CN CN202111193452.2A patent/CN113934297A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701863A (en) * | 2016-01-11 | 2016-06-22 | 华为技术有限公司 | Image processing method and device |
| CN109685905A (en) * | 2017-10-18 | 2019-04-26 | 深圳市掌网科技股份有限公司 | Cell planning method and system based on augmented reality |
| US20190340649A1 (en) * | 2018-05-07 | 2019-11-07 | Adobe Inc. | Generating and providing augmented reality representations of recommended products based on style compatibility in relation to real-world surroundings |
| CN111680570A (en) * | 2020-05-13 | 2020-09-18 | 珠海格力电器股份有限公司 | Augmented reality image data processing method, device, equipment and storage medium |
| CN111626183A (en) * | 2020-05-25 | 2020-09-04 | 深圳市商汤科技有限公司 | Target object display method and device, electronic equipment and storage medium |
| CN111640200A (en) * | 2020-06-10 | 2020-09-08 | 浙江商汤科技开发有限公司 | AR scene special effect generation method and device |
| CN111970557A (en) * | 2020-09-01 | 2020-11-20 | 深圳市慧鲤科技有限公司 | Image display method, image display device, electronic device, and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023246189A1 (en) * | 2022-06-24 | 2023-12-28 | 如你所视(北京)科技有限公司 | Image information display method and device |
| CN117412019A (en) * | 2023-12-14 | 2024-01-16 | 深圳市欧冠微电子科技有限公司 | Aerial imaging and virtual reality dynamic backlight determination method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105981368B (en) | Picture composition and position guidance in an imaging device | |
| US11947729B2 (en) | Gesture recognition method and device, gesture control method and device and virtual reality apparatus | |
| CN108895981B (en) | Three-dimensional measurement method, device, server and storage medium | |
| US10249089B2 (en) | System and method for representing remote participants to a meeting | |
| US20140002443A1 (en) | Augmented reality interface | |
| US11893702B2 (en) | Virtual object processing method and apparatus, and storage medium and electronic device | |
| CN107656962B (en) | Panoramic display method in electronic map, server and computer readable medium | |
| CN113934297A (en) | Interaction method and device based on augmented reality, electronic equipment and medium | |
| US20200042777A1 (en) | Method, apparatus and device for determining an object, and storage medium for the same | |
| US11556605B2 (en) | Search method, device and storage medium | |
| US11372518B2 (en) | Systems and methods for augmented or mixed reality writing | |
| CN108597034B (en) | Method and apparatus for generating information | |
| CN111882634A (en) | Image rendering method, device and equipment and storage medium | |
| WO2020034981A1 (en) | Method for generating encoded information and method for recognizing encoded information | |
| CN112766027A (en) | Image processing method, device, equipment and storage medium | |
| CN111340960B (en) | Image modeling method and device, storage medium and electronic equipment | |
| US11922568B2 (en) | Finite aperture omni-directional stereo light transport | |
| CN106919260B (en) | Webpage operation method and device | |
| CN108874141B (en) | Somatosensory browsing method and device | |
| CN109461203B (en) | Gesture three-dimensional image generation method and device, computer equipment and storage medium | |
| CN108536830B (en) | Picture dynamic search method, device, equipment, server and storage medium | |
| US11562538B2 (en) | Method and system for providing a user interface for a 3D environment | |
| CN114089836B (en) | Labeling method, terminal, server and storage medium | |
| CN109857244B (en) | Gesture recognition method and device, terminal equipment, storage medium and VR glasses | |
| CN108920598B (en) | Panorama browsing method and device, terminal equipment, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |