WO2019184889A1

WO2019184889A1 - Method and apparatus for adjusting augmented reality model, storage medium, and electronic device

Info

Publication number: WO2019184889A1
Application number: PCT/CN2019/079588
Authority: WO
Inventors: 王健; 谭筱; 蓝和; 邹奎
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-03-26
Filing date: 2019-03-25
Publication date: 2019-10-03
Also published as: CN108537889A

Abstract

Disclosed is a method for adjusting an augmented reality model, comprising: obtaining an image corresponding to a scene of an electronic device, and creating a spatial coordinate system according to the image; adding an augmented reality virtual model into the scene; obtaining a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system; and adjusting the augmented reality virtual model according to the positional relationship. The present application also provides an apparatus for adjusting an augmented reality model, a storage medium, and an electronic device.

Description

Method, device, storage medium and electronic device for adjusting augmented reality model

This application claims the priority of the Chinese patent application filed on March 26, 2018, the Chinese Patent Office, the application number is 201810252914.5, and the application name is "adjustment method, device, storage medium and electronic device of the augmented reality model". This is incorporated herein by reference.

Technical field

The present application relates to the field of electronic devices, and in particular, to a method, an apparatus, a storage medium, and an electronic device for adjusting an augmented reality model.

Background technique

AR (Augmented Reality) is a new technology that integrates real world information and virtual world information "seamlessly". It is an entity information (visual information) that is difficult to experience in a certain time and space of the real world. , sound, taste, touch, etc.) through computer and other scientific techniques, after simulation, the real environment and virtual objects are superimposed in real time on the same picture or space, and the virtual information is applied to the real world, being sensed by humans. Perceived to achieve a sensory experience that transcends reality. Augmented reality technology not only displays the information of the real world, but also displays the virtual information at the same time, and the two kinds of information are charged and superimposed.

Application content

The embodiment of the present application provides an adjustment method, device, storage medium, and electronic device for an augmented reality model, which can fuse the augmented reality virtual model with a real scene, and can greatly improve the accuracy of the fusion.

In a first aspect, an embodiment of the present application provides an adjustment method of an augmented reality model, including:

Obtaining an image corresponding to the current scene of the electronic device, and establishing a spatial coordinate system according to the image;

Adding an augmented reality virtual model to the scene;

Acquiring a positional relationship between the augmented reality virtual model and the electronic device according to the space coordinate system;

The augmented reality virtual model is adjusted according to the positional relationship.

In a second aspect, the embodiment of the present application further provides an apparatus for adjusting an augmented reality model, including: an image acquiring module, an adding module, a position acquiring module, and an adjusting module;

The image acquisition module is configured to acquire an image corresponding to a scene where the electronic device is currently located, and establish a spatial coordinate system according to the image;

The adding module is configured to add an augmented reality virtual model to the scene;

The location obtaining module is configured to acquire a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system;

The adjusting module is configured to adjust the augmented reality virtual model according to the positional relationship.

In a third aspect, the embodiment of the present application further provides a storage medium, where a computer program is stored thereon, and the computer program is executed by the processor to implement the step of adjusting the augmented reality model.

In a fourth aspect, an embodiment of the present application further provides an electronic device, including a processor and a memory, where the memory stores a plurality of instructions, and the processor loads the instructions in the memory to perform the following steps:

Adding an augmented reality virtual model to the scene;

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present application. Other drawings can also be obtained from those skilled in the art based on these drawings without paying any creative effort.

FIG. 1 is a schematic flowchart of a method for adjusting an augmented reality model according to an embodiment of the present application.

FIG. 2 is a schematic flowchart of selecting an augmented reality virtual model according to an embodiment of the present application.

FIG. 3 is another schematic flowchart of a method for adjusting an augmented reality model according to an embodiment of the present application.

FIG. 4 is a schematic structural diagram of an apparatus for adjusting an augmented reality model according to an embodiment of the present application.

FIG. 5 is another schematic structural diagram of an apparatus for adjusting an augmented reality model according to an embodiment of the present application.

FIG. 6 is another schematic structural diagram of an electronic device according to an embodiment of the present application.

FIG. 7 is another schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

Referring to the drawings, wherein like reference numerals represent the same components, the principles of the present application are illustrated by the implementation in a suitable computing environment. The following description is based on the specific embodiments of the present invention as illustrated, and should not be construed as limiting the specific embodiments that are not described herein.

In the following description, specific embodiments of the present application will be described with reference to the steps and symbols executed by one or more computers, unless otherwise stated. Thus, these steps and operations will be referred to several times by a computer, and the computer execution referred to herein includes the operation of a computer processing unit that represents an electronic signal in data in a structured version. This operation converts the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise altered in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the above text, which is not intended to be a limitation, and those skilled in the art will appreciate that the various steps and operations described below can also be implemented in hardware.

The principles of the present application operate using many other general purpose or special purpose computing, communication environments, or configurations. Examples of well-known computing systems, environments, and configurations suitable for use in the present application may include, but are not limited to, hand-held phones, personal computers, servers, multi-processor systems, microcomputer-based systems, mainframe computers, and A distributed computing environment, including any of the above systems or devices.

The details will be described separately below.

The embodiment will be described from the perspective of an adjustment device of an augmented reality model, and the device may be integrated into an electronic device, and the electronic device may be a network-enabled electronic device such as a mobile internet device (such as a smart phone or a tablet computer). device.

An embodiment of the present application provides an adjustment method of an augmented reality model, including:

Adding an augmented reality virtual model to the scene;

In an embodiment, the positional relationship includes a distance;

And adjusting the augmented reality virtual model according to the location relationship, including:

The size of the augmented reality virtual model is adjusted according to a distance between the augmented reality virtual model and the electronic device.

In an embodiment, the positional relationship includes a relative angle;

Adjusting the orientation of the augmented reality virtual model according to a relative angle between the augmented reality virtual model and the electronic device.

In an embodiment, the adding an augmented reality virtual model in the scene includes:

Obtaining a target object in the scene, and generating a corresponding augmented reality virtual model according to the target object;

Extracting coordinate information of the target object in the space coordinate system;

Adding the augmented reality virtual model according to the coordinate information.

In an embodiment, after the augmented reality virtual model is adjusted according to the location relationship, the method further includes:

Deeply merging the adjusted augmented reality virtual model with the scene;

The augmented reality picture after the depth blending is displayed.

In an embodiment, establishing a spatial coordinate system according to the image comprises:

Establishing a three-dimensional model of the current scene according to the image;

A spatial coordinate system is established in the three-dimensional model.

In an embodiment, acquiring a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system, including:

Obtaining first coordinate information of the augmented reality virtual model in the space coordinate system;

Acquiring the shooting posture information of the electronic device according to the image, and calculating second coordinate information of the electronic device in the space coordinate system according to the shooting posture information;

And generating a positional relationship between the augmented reality virtual model and the electronic device according to the first coordinate information and the second coordinate information.

Referring first to FIG. 1 , FIG. 1 is a schematic flowchart of a method for adjusting an augmented reality model according to an embodiment of the present application, including the following steps:

Step S101: Acquire an image corresponding to a scene where the electronic device is currently located, and establish a spatial coordinate system according to the image.

In an embodiment, the current scene may be image captured by the camera to acquire a scene image. The above scene may include a scene that a user, an animal, or a landscape wants to capture.

In this embodiment, the camera acquires the picture for the picture taken by the current scene. The format of the image can be Bmp, jpg or other formats.

Further, after acquiring the scene image, the image may also be pre-processed, and the pre-processing may include noise reduction processing and smoothing processing.

In an embodiment, after acquiring the scene image corresponding to the current scene, the three-dimensional model of the current scene may also be established according to the scene image. The basic process of modeling can be: data acquisition and processing (training site data, device size data, etc.), professional 3D software modeling (such as: Pro/E, 3DMax modeling), visual editing software correction (such as: Cosmo Wrolds corrects errors, adds materials and interactions, and text editing software is perfect (eg, using Vrmlpad to optimize structure, reduce code, compress files).

In an embodiment, after the three-dimensional model is established, the three-dimensional model may be image-rendered, and the image is finally rendered to conform to the stage of the 3D scene. Rendering has a variety of software, such as: each CG software comes with a rendering engine, as well as such as RenderMan. A three-dimensional scene image is obtained, and then the three-dimensional scene image is projected onto a display screen of the augmented reality device, and the user can view the three-dimensional scene image by watching the display screen of the augmented reality device.

After establishing the three-dimensional model of the current scene, a spatial coordinate system can be established in the three-dimensional model, for example, using the Marker-less technique to determine the initial coordinate plane, and initializing the coordinate system with the true scale.

Step S102, adding an augmented reality virtual model to the scene.

In an embodiment, the electronic device may generate a corresponding augmented reality virtual model according to the current scene, so as to combine the acquired virtual model with the recognized real scene based on the augmented reality technology to generate a scene image of the augmented reality. Optionally, the augmented reality virtual model may also be selected by the user. As shown in FIG. 2, after the electronic device acquires the current scene, the preset virtual number is automatically recommended according to the scenario, and the user can pass the user. Click the way to select the target virtual model and add the augmented reality virtual model to the scene.

The augmented reality virtual model may have multiple expressions, such as text, image or video, etc., so that various types of virtual models may be combined with the recognized real scenes in order to be enhanced. Realistic scene picture. For example, the augmented reality virtual model can be superimposed in the real scene in the form of text and projected on the display screen of the electronic device for display. Then, the image information of the augmented reality virtual model can be superimposed on the real scene in combination with the audio information, and projected on the display screen of the electronic device for display, and the audio information can be played through the speaker of the electronic device.

In one embodiment, merging the virtual model with the real scene requires omnidirectional alignment of the computer generated virtual model with the user's real environment. The purpose of applying augmented reality is to combine virtual environments with real environments to make them look like a whole. If there is no exact match, the virtual model will appear to float above the real environment, so accurate 3D matching is critical for AR.

The above three-dimensional matching may include various methods such as a reference point method, a surface based method, a template method, an indefinite standard, and the like. The reference point method is taken as an example to describe the embodiment of the present invention. The method is to manually place some reference points in a real environment, and implement three-dimensional matching of the system according to the method of matching the reference points. These reference points can be LEDs or specific landmarks. Their position in the real environment is known, can be identified by image processing, and then the transformation matrix of different spaces is calculated, and finally the matching is done by least squares method. This method simplifies the calculation and is accurate.

Step S103: Acquire a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system.

In an embodiment, the coordinate information of the augmented reality virtual model may be directly acquired in a spatial coordinate system of the current scene. Further, since the body of the electronic device does not exist in the current scene, acquiring the coordinate information of the electronic device requires calculation, for example, acquiring the shooting posture information of the electronic device according to the image capturing the current scene, and the shooting posture information may include shooting. Height and direction information, and then calculating coordinate information in the coordinate system corresponding to the electronic device in the current scene according to the shooting posture information.

In an embodiment, the obtaining the positional relationship between the augmented reality virtual model and the electronic device may also be obtained by acquiring the depth information of the virtual model. For example, after adding the augmented reality virtual model to the current scene, determining to add the scene to the scene The location of the virtual model is then obtained by a depth of field camera in the electronic device to obtain a positional relationship between the location and the electronic device. The location information may include distance information and relative angle information.

Step S104, adjusting the augmented reality virtual model according to the positional relationship.

In an embodiment, the location relationship may include a distance between the augmented reality virtual model and the electronic device. Because in the prior art, the above virtual model often does not change with the distance from the electronic device, which reduces the authenticity. Therefore, the embodiment may change the size of the virtual model according to the distance between the augmented reality virtual model and the electronic device, that is, the step of adjusting the augmented reality virtual model according to the positional relationship may include:

The size of the augmented reality virtual model is adjusted according to the distance between the augmented reality virtual model and the electronic device.

In an embodiment, the positional relationship may further include a relative angle between the augmented reality virtual model and the electronic device. In this embodiment, the orientation of the virtual model can also be adjusted in consideration of the fact that the virtual model presents different ways in different positions of the electronic device. Such as cartoon character models, etc., can make the virtual character always face the electronic device, improving the fun and authenticity. That is, the step of adjusting the augmented reality virtual model according to the positional relationship may further include:

The orientation of the augmented reality virtual model is adjusted according to the relative angle between the augmented reality virtual model and the electronic device.

In the embodiment of the present invention, the electronic device may be any device having a shooting function, such as a mobile phone, a tablet personal computer, a laptop computer, and a personal digital assistant (PDA). , Mobile Internet Device (MID) or Wearable Device.

It can be seen that the embodiment of the present application can acquire an image corresponding to the current scene of the electronic device, and establish a spatial coordinate system according to the image, add an augmented reality virtual model to the scene, and obtain an augmented reality virtual model and the electronic device according to the spatial coordinate system. The positional relationship adjusts the augmented reality virtual model according to the positional relationship. The application can fuse the augmented reality virtual model with the real scene, and adjust the augmented reality virtual model according to the positional relationship, thereby greatly improving the accuracy of the fusion.

According to the description of the previous embodiment, the adjustment method of the augmented reality model of the present application will be further explained below.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of another method for adjusting an augmented reality model according to an embodiment of the present application, including the following steps:

Step S201: Acquire an image corresponding to the current scene of the electronic device, and establish a spatial coordinate system according to the image.

In an embodiment, the current scene may be image-collected by the camera to acquire a scene image. The above scene may include a scene that a user, an animal, or a landscape wants to capture.

Further, after establishing the three-dimensional model of the current scene, a spatial coordinate system can be established in the three-dimensional model, for example, using the Marker-less technique to determine the initial coordinate plane, and initializing the coordinate system with the true scale.

Step S202: Acquire a target object in the scene, and generate a corresponding augmented reality virtual model according to the target object.

The target object in the above scenario may be automatically selected by the electronic device or manually selected by the user. For example, the user may click on the object in the current scene in the finder frame to determine the target object.

After the target object is determined, an augmented reality virtual model corresponding to the target object is generated, wherein the augmented reality virtual model may include a plurality of different types of virtual models, such as text information, image information, and/or audio information.

Step S203, extracting coordinate information of the target object in the space coordinate system.

Step S204, adding an augmented reality virtual model according to the coordinate information.

In an embodiment, different virtual models in the same type of virtual model may be sequentially combined with the current scene according to different types of the augmented reality virtual model, and the corresponding other types of virtual models are superimposed and displayed together. To generate augmented reality scenes.

For example, the virtual model can be superimposed in the real scene in the form of text and projected onto the display screen of the augmented reality device for display. Then, the image information of the virtual model may be superimposed on the real scene in combination with the audio information, and projected on the display screen of the augmented reality device for display, and the audio information may be played by the speaker of the augmented reality device; wherein the image The information can be a virtual three-dimensional image and fits it with the real scene to achieve seamless display, and the text information is displayed in a semi-transparent manner next to the intersection of the virtual three-dimensional image and the real scene for related description. Finally, the virtual model content is projected onto the display screen of the augmented reality device by means of text information, image information and audio information.

Step S205: Acquire a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system.

Step S206, adjusting the augmented reality virtual model according to the positional relationship.

In an embodiment, the location relationship may include a distance between the augmented reality virtual model and the electronic device, so the embodiment may change the size of the virtual model according to the distance between the augmented reality virtual model and the electronic device.

In an embodiment, the positional relationship may further include a relative angle between the augmented reality virtual model and the electronic device. Therefore, the embodiment may be based on the relative angle between the augmented reality virtual model and the electronic device, and the augmented reality virtual model. Adjust towards orientation.

Step S207, deeply integrating the adjusted augmented reality virtual model with the scene.

Optionally, the embodiment may further obtain depth depth data by using a depth camera, and then perform depth fusion according to the virtual object distance data obtained in the three-dimensional engine, thereby achieving an occlusion effect, so that the real object and the virtual model are directly strong. The sense of interaction. The application is robust and enables real objects and virtual data to occlude interactions.

Step S208, displaying the augmented reality picture after the depth fusion.

In the embodiment of the present invention, the electronic device may be any device capable of shooting, such as a mobile phone, a tablet personal computer, a laptop computer, or a personal digital assistant (PDA). , Mobile Internet Device (MID) or Wearable Device.

It can be seen that the embodiment of the present application can acquire an image corresponding to the current scene of the electronic device, and establish a spatial coordinate system according to the image, acquire a target object in the scene, and generate a corresponding augmented reality virtual model according to the target object, in the space coordinate system. The coordinate information of the target object is extracted, the augmented reality virtual model is added according to the coordinate information, the positional relationship between the augmented reality virtual model and the electronic device is obtained according to the spatial coordinate system, and the augmented reality virtual model is adjusted according to the positional relationship, and the adjusted The augmented reality virtual model is deeply integrated with the scene to display the augmented reality picture after deep fusion. The embodiment of the present application can fuse the augmented reality virtual model with the real scene, and adjust the augmented reality virtual model according to the positional relationship, thereby greatly improving the accuracy of the fusion.

In order to facilitate the implementation of the method for adjusting the augmented reality model provided by the embodiment of the present application, the embodiment of the present application further provides an apparatus for adjusting the method based on the augmented reality model described above. The meaning of the noun is the same as that of the augmented reality model described above. For specific implementation details, refer to the description in the method embodiment.

Referring to FIG. 4 , FIG. 4 is a schematic structural diagram of an apparatus for adjusting an augmented reality model according to an embodiment of the present disclosure. The apparatus for adjusting the augmented reality model includes: an image acquiring module 301, an adding module 302, and a location acquiring module 303. Adjustment module 304;

The image obtaining module 301 is configured to acquire an image corresponding to a scene where the electronic device is currently located, and establish a spatial coordinate system according to the image;

The adding module 302 is configured to add an augmented reality virtual model to the scene;

The location obtaining module 303 is configured to acquire a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system;

The adjustment module 304 is configured to adjust the augmented reality virtual model according to the positional relationship.

In an embodiment, the positional relationship includes a distance;

The adjusting module is specifically configured to adjust a size of the augmented reality virtual model according to a distance between the augmented reality virtual model and the electronic device.

In an embodiment, the positional relationship includes a relative angle;

The adjusting module is specifically configured to adjust an orientation of the augmented reality virtual model according to a relative angle between the augmented reality virtual model and the electronic device.

In an embodiment, as shown in FIG. 5, the adding module 302 may specifically include: an obtaining submodule 3021, an extracting submodule 3022, and an adding submodule 3023;

The acquiring sub-module 3021 is configured to acquire a target object in the scene, and generate a corresponding augmented reality virtual model according to the target object;

The extraction submodule 3022 is configured to extract coordinate information of the target object in a spatial coordinate system;

The adding submodule 3023 is configured to add an augmented reality virtual model according to the coordinate information.

In an embodiment, the augmented reality model adjustment device 30 may further include: a fusion module 305 and a display module 306;

The fusion module 305 is configured to deeply integrate the adjusted augmented reality virtual model with the scene;

The display module 306 is configured to display the augmented reality picture after the depth fusion.

It can be seen that the adjusting device 30 of the augmented reality model provided by the embodiment of the present application can acquire an image corresponding to the current scene of the electronic device, and establish a spatial coordinate system according to the image, add an augmented reality virtual model to the scene, and obtain according to the spatial coordinate system. The positional relationship between the augmented reality virtual model and the electronic device is adjusted, and the augmented reality virtual model is adjusted according to the positional relationship. The application can fuse the augmented reality virtual model with the real scene, and adjust the augmented reality virtual model according to the positional relationship, thereby greatly improving the accuracy of the fusion.

The application further provides a storage medium on which a computer program is stored, wherein the computer program is executed by the processor to implement an adjustment method of the augmented reality model provided by the method embodiment.

The application further provides an electronic device comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor loads instructions in the memory for performing the following steps:

Adding an augmented reality virtual model to the scene;

In an embodiment, the positional relationship includes a distance, and when the augmented reality virtual model is adjusted according to the positional relationship, the processor is configured to perform the following steps:

And adjusting a size of the augmented reality virtual model according to a distance between the augmented reality virtual model and the electronic device.

In an embodiment, the positional relationship includes a relative angle, and when the augmented reality virtual model is adjusted according to the positional relationship, the processor is configured to perform the following steps:

In an embodiment, when the augmented reality virtual model is added to the scene, the processor is configured to perform the following steps:

In an embodiment, after adjusting the augmented reality virtual model according to the positional relationship, the processor is further configured to perform the following steps:

Deeply merging the adjusted augmented reality virtual model with the scene;

The augmented reality picture after the depth blending is displayed.

In an embodiment, when the spatial coordinate system is established according to the image, the processor is configured to perform the following steps:

A spatial coordinate system is established in the three-dimensional model.

In an embodiment, when acquiring the positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system, the processor is configured to perform the following steps:

In another embodiment of the present application, an electronic device is further provided, and the electronic device may be a device such as a smart phone or a tablet computer. As shown in FIG. 6, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

The processor 401 is a control center of the electronic device 400, which connects various parts of the entire electronic device using various interfaces and lines, executes the electronic by running or loading an application stored in the memory 402, and calling data stored in the memory 402. The various functions and processing data of the device enable overall monitoring of the electronic device.

In this embodiment, the processor 401 in the electronic device 400 loads the instructions corresponding to the process of one or more applications into the memory 402 according to the following steps, and is stored and stored in the memory 402 by the processor 401. In the application, thus implementing various functions:

Adding an augmented reality virtual model to the scene;

Please refer to FIG. 7. FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 500 can include a radio frequency (RF) circuit 501, a memory 502 including one or more computer readable storage media, an input unit 503, a display unit 504, a sensor 504, an audio circuit 506, and wireless fidelity ( WiFi, Wireless Fidelity module 507, processor 508 including one or more processing cores, and power supply 509 and the like. It will be understood by those skilled in the art that the electronic device structure illustrated in FIG. 7 does not constitute a limitation on the electronic device, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.

The radio frequency circuit 501 can be used for transmitting and receiving information, or receiving and transmitting signals during a call. Specifically, after receiving the downlink information of the base station, the radio network is processed by one or more processors 508; in addition, the data related to the uplink is sent to the base station. . Generally, the radio frequency circuit 501 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, and a low noise amplifier (LNA, Low Noise Amplifier), duplexer, etc. In addition, the radio frequency circuit 501 can also communicate with the network and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), and Code Division Multiple Access (CDMA). Code Division Multiple Access), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), e-mail, Short Messaging Service (SMS), and the like.

Memory 502 can be used to store applications and data. The application stored in the memory 502 contains executable code. Applications can form various functional modules. The processor 508 executes various functional applications and data processing by running an application stored in the memory 502. The memory 502 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to Data created by the use of electronic devices (such as audio data, phone books, etc.). Moreover, memory 502 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, memory 502 may also include a memory controller to provide access to memory 502 by processor 508 and input unit 503.

The input unit 503 can be configured to receive input numbers, character information or user characteristic information (such as fingerprints), and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. In particular, in one particular embodiment, input unit 503 can include a touch-sensitive surface as well as other input devices. Touch-sensitive surfaces, also known as touch screens or trackpads, collect touch operations on or near the user (such as the user using a finger, stylus, etc., any suitable object or accessory on a touch-sensitive surface or touch-sensitive Operation near the surface), and drive the corresponding connecting device according to a preset program. Alternatively, the touch sensitive surface may include two parts of a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor 508 is provided and can receive commands from the processor 508 and execute them.

Display unit 504 can be used to display information entered by the user or information provided to the user, as well as various graphical user interfaces of the electronic device, which can be composed of graphics, text, icons, video, and any combination thereof. The display unit 504 can include a display panel. Optionally, the display panel can be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch-sensitive surface can cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 508 to determine the type of the touch event, and then the processor 508 displays the type according to the type of the touch event. A corresponding visual output is provided on the panel. Although in FIG. 7, the touch-sensitive surface and display panel are implemented as two separate components to perform input and input functions, in some embodiments, the touch-sensitive surface can be integrated with the display panel to implement input and output functions.

The electronic device can also include at least one type of sensor 505, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may close the display panel when the electronic device moves to the ear, and/or Backlighting. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for the electronic device can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, not here Let me repeat.

The audio circuit 506 can provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 506 can convert the received audio data into an electrical signal, which is transmitted to a speaker, and converted into a sound signal output by the speaker. On the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 506 and converted into The audio data is processed by the audio data output processor 508, transmitted via the RF circuit 501 to, for example, another electronic device, or the audio data is output to the memory 502 for further processing. The audio circuit 506 may also include an earbud jack to provide communication of the peripheral earphones with the electronic device.

Wireless Fidelity (WiFi) is a short-range wireless transmission technology. The wireless device can help users send and receive e-mail, browse web pages and access streaming media through the wireless fidelity module 507, which provides users with wireless broadband Internet access. Although FIG. 7 shows the wireless fidelity module 507, it can be understood that it does not belong to the essential configuration of the electronic device, and may be omitted as needed within the scope of not changing the essence of the invention.

The processor 508 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, executes the electronic device by running or executing an application stored in the memory 502, and calling data stored in the memory 502. The various functions and processing of data to provide overall monitoring of the electronic device. Optionally, the processor 508 may include one or more processing cores; preferably, the processor 508 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like. The modem processor primarily handles wireless communications. It will be appreciated that the above described modem processor may also not be integrated into the processor 508.

The electronic device also includes a power source 509 (such as a battery) that supplies power to the various components. Preferably, the power source can be logically coupled to the processor 508 through the power management system to manage functions such as charging, discharging, and power management through the power management system. The power supply 509 may also include any one or more of a DC or AC power source, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

Although not shown in FIG. 7, the electronic device may further include a camera, a Bluetooth module, and the like, and details are not described herein again.

For the specific implementation, the foregoing modules may be implemented as a separate entity, or may be implemented in any combination, and may be implemented as the same or a plurality of entities. For the specific implementation of the foregoing modules, refer to the foregoing method embodiments, and details are not described herein.

It should be noted that those skilled in the art can understand that all or part of the steps of the foregoing embodiments can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium, such as Stored in a memory of the electronic device and executed by at least one processor within the electronic device, the flow of an embodiment of an adjustment method such as an augmented reality model may be included during execution. The storage medium may include: a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

The method, device, storage medium and electronic device for adjusting an augmented reality model provided by the embodiments of the present application are described in detail. The functional modules may be integrated into one processing chip, or each module may exist physically separately. Two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. The principles and implementations of the present application are described in the following by using specific examples. The description of the above embodiments is only for helping to understand the method of the present application and its core ideas. Meanwhile, for those skilled in the art, according to the present application, There is a change in the scope of the present invention and the scope of application, and the contents of this specification should not be construed as limiting the present application.

Claims

An adjustment method of an augmented reality model, comprising the following steps:

Obtaining an image corresponding to the current scene of the electronic device, and establishing a spatial coordinate system according to the image;

Adding an augmented reality virtual model to the scene;

Acquiring a positional relationship between the augmented reality virtual model and the electronic device according to the space coordinate system;

The augmented reality virtual model is adjusted according to the positional relationship.
The method of adjusting an augmented reality model according to claim 1, wherein the positional relationship comprises a distance;

And adjusting the augmented reality virtual model according to the location relationship, including:

And adjusting a size of the augmented reality virtual model according to a distance between the augmented reality virtual model and the electronic device.
The method of adjusting an augmented reality model according to claim 1, wherein the positional relationship comprises a relative angle;

And adjusting the augmented reality virtual model according to the location relationship, including:

Adjusting the orientation of the augmented reality virtual model according to a relative angle between the augmented reality virtual model and the electronic device.
The method for adjusting an augmented reality model according to claim 1, wherein the adding an augmented reality virtual model to the scene comprises:

Obtaining a target object in the scene, and generating a corresponding augmented reality virtual model according to the target object;

Extracting coordinate information of the target object in the space coordinate system;

Adding the augmented reality virtual model according to the coordinate information.
The method of adjusting an augmented reality model according to claim 1, wherein after the augmented reality virtual model is adjusted according to the positional relationship, the method further comprises:

Deeply merging the adjusted augmented reality virtual model with the scene;

The augmented reality picture after the depth blending is displayed.
The method for adjusting an augmented reality model according to claim 1, wherein the establishing a spatial coordinate system according to the image comprises:

Establishing a three-dimensional model of the current scene according to the image;

A spatial coordinate system is established in the three-dimensional model.
The method for adjusting an augmented reality model according to claim 1, wherein the obtaining a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system comprises:

Obtaining first coordinate information of the augmented reality virtual model in the space coordinate system;

Acquiring the shooting posture information of the electronic device according to the image, and calculating second coordinate information of the electronic device in the space coordinate system according to the shooting posture information;

And generating a positional relationship between the augmented reality virtual model and the electronic device according to the first coordinate information and the second coordinate information.
An apparatus for adjusting an augmented reality model, wherein the apparatus comprises: an image acquisition module, an addition module, a position acquisition module, and an adjustment module;

The image acquisition module is configured to acquire an image corresponding to a scene where the electronic device is currently located, and establish a spatial coordinate system according to the image;

The adding module is configured to add an augmented reality virtual model to the scene;

The location obtaining module is configured to acquire a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system;

The adjusting module is configured to adjust the augmented reality virtual model according to the positional relationship.
The adjustment device of the augmented reality model according to claim 8, wherein the positional relationship includes a distance;

The adjusting module is specifically configured to adjust a size of the augmented reality virtual model according to a distance between the augmented reality virtual model and the electronic device.
The adjustment device of the augmented reality model according to claim 8, wherein the positional relationship includes a relative angle;

The adjusting module is specifically configured to adjust an orientation of the augmented reality virtual model according to a relative angle between the augmented reality virtual model and the electronic device.
The apparatus for adjusting an augmented reality model according to claim 8, wherein the adding module comprises: an obtaining submodule, an extracting submodule, and adding a submodule;

The acquiring sub-module is configured to acquire a target object in the scene, and generate a corresponding augmented reality virtual model according to the target object;

The extraction submodule is configured to extract coordinate information of the target object in the space coordinate system;

And the adding submodule is configured to add the augmented reality virtual model according to the coordinate information.
The apparatus for adjusting an augmented reality model according to claim 8, wherein the apparatus further comprises: a fusion module and a display module;

The fusion module is configured to deeply integrate the adjusted augmented reality virtual model with the scene;

The display module is configured to display the augmented reality picture after the depth fusion.
A storage medium having stored thereon a computer program, wherein the computer program is executed by a processor to perform the steps of the method of any of claims 1-7.
An electronic device includes a processor and a memory, the memory storing a plurality of instructions, wherein the processor loads instructions in the memory for performing the following steps:

Obtaining an image corresponding to the current scene of the electronic device, and establishing a spatial coordinate system according to the image;

Adding an augmented reality virtual model to the scene;

Acquiring a positional relationship between the augmented reality virtual model and the electronic device according to the space coordinate system;

The augmented reality virtual model is adjusted according to the positional relationship.
The electronic device according to claim 14, wherein the positional relationship comprises a distance, and when the augmented reality virtual model is adjusted according to the positional relationship, the processor is configured to perform the following steps:

And adjusting a size of the augmented reality virtual model according to a distance between the augmented reality virtual model and the electronic device.
The electronic device according to claim 14, wherein the positional relationship comprises a relative angle, and when the augmented reality virtual model is adjusted according to the positional relationship, the processor is configured to perform the following steps:

Adjusting the orientation of the augmented reality virtual model according to a relative angle between the augmented reality virtual model and the electronic device.
The electronic device of claim 14, wherein when the augmented reality virtual model is added to the scene, the processor is configured to perform the following steps:

Obtaining a target object in the scene, and generating a corresponding augmented reality virtual model according to the target object;

Extracting coordinate information of the target object in the space coordinate system;

Adding the augmented reality virtual model according to the coordinate information.
The electronic device of claim 14, wherein the processor is further configured to perform the following steps after adjusting the augmented reality virtual model according to the location relationship:

Deeply merging the adjusted augmented reality virtual model with the scene;

The augmented reality picture after the depth blending is displayed.
The electronic device of claim 14, wherein the processor is configured to perform the following steps when establishing a spatial coordinate system based on the image:

Establishing a three-dimensional model of the current scene according to the image;

A spatial coordinate system is established in the three-dimensional model.
The electronic device according to claim 14, wherein the processor is configured to perform the following steps when acquiring a positional relationship between the augmented reality virtual model and the electronic device according to the spatial coordinate system:

Obtaining first coordinate information of the augmented reality virtual model in the space coordinate system;

Acquiring the shooting posture information of the electronic device according to the image, and calculating second coordinate information of the electronic device in the space coordinate system according to the shooting posture information;

And generating a positional relationship between the augmented reality virtual model and the electronic device according to the first coordinate information and the second coordinate information.