CN111833459B - Image processing method and device, electronic equipment and storage medium - Google Patents
Image processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111833459B CN111833459B CN202010662796.2A CN202010662796A CN111833459B CN 111833459 B CN111833459 B CN 111833459B CN 202010662796 A CN202010662796 A CN 202010662796A CN 111833459 B CN111833459 B CN 111833459B
- Authority
- CN
- China
- Prior art keywords
- image
- augmented reality
- reality model
- depth
- set object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 33
- 230000003190 augmentative effect Effects 0.000 claims abstract description 179
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 25
- 230000008569 process Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 238000009877 rendering Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 17
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the disclosure discloses an image processing method, an image processing device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object; acquiring a preset model motion trail, wherein the model motion trail is used for indicating position information and depth information of the augmented reality model in the first image of each frame; and according to the position information and the depth information, superposing the augmented reality model on the first image to obtain a second image, and displaying the second image. According to the embodiment of the disclosure, the augmented reality model can be realized by taking the set object as a background, the effect of movement according to the movement track of the model is enriched, the image display effect is enriched, the problem that the display effect is single in the current shooting scene is solved, and a novel playing method is provided to promote the user experience.
Description
Technical Field
The embodiment of the disclosure relates to computer technology, in particular to an image processing method, an image processing device, electronic equipment and a storage medium.
Background
AR (Augmented Reality ) technology is a technology that can combine real environment and virtual information to enable the display of an AR model and superimposed images on a real world image on a smart terminal screen.
At present, video shooting through an intelligent terminal only carries out image recording on a shot object, but the intelligent terminal with an augmented reality function can only provide some simple application scenes, such as background replacement, sticker addition and the like, the display effect is relatively single, and the requirement of a user for pursuing novel playing methods cannot be met.
Disclosure of Invention
The embodiment of the disclosure provides an image processing method, an image processing device, electronic equipment and a storage medium, which can enrich the display effect of a shot image.
In a first aspect, an embodiment of the present disclosure provides an image processing method, including:
Acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object;
Acquiring a preset model motion trail, wherein the model motion trail is used for indicating position information and depth information of the augmented reality model in the first image of each frame;
and according to the position information and the depth information, superposing the augmented reality model on the first image to obtain a second image, and displaying the second image.
In a second aspect, an embodiment of the present disclosure further provides an image processing apparatus, including:
The model acquisition module is used for acquiring a first image, identifying a set object in the first image and acquiring an augmented reality model corresponding to the set object;
The track acquisition module is used for acquiring a preset model motion track, wherein the model motion track is used for indicating the position information and the depth information of the augmented reality model in the first image of each frame;
And the image superposition module is used for superposing the augmented reality model on the first image to obtain a second image according to the position information and the depth information, and displaying the second image.
In a third aspect, embodiments of the present disclosure further provide an electronic device, including:
One or more processors;
a memory for storing one or more programs,
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the image processing methods as provided by any of the embodiments of the present disclosure.
In a fourth aspect, the embodiments of the present disclosure further provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements an image processing method as provided by any of the embodiments of the present disclosure.
The embodiment of the disclosure provides an image processing method, an image processing device, electronic equipment and a storage medium, wherein a set object in a first image is identified, an augmented reality model corresponding to the set object and a preset model motion track are obtained, the augmented reality model is overlapped to the first image according to position information and depth information in the model motion track to obtain a second image, and the second image is displayed. Because the depth of field of the pixel points of the augmented reality model and the depth of field of the pixel points in the first image are considered in the superposition process of the augmented reality model and the image, the effect that the augmented reality model moves by taking a set object as a background according to the motion track of the model can be realized, the image display effect is enriched, the problem that the display effect in the current shooting scene is single is solved, and a novel playing method is provided to promote the user experience.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure;
FIG. 2 is a flow chart of a model overlaying method in an image processing method according to an embodiment of the present disclosure;
FIG. 3 is a flow chart of another image processing method provided by an embodiment of the present disclosure;
FIG. 4 is a flow chart of yet another image processing method provided by an embodiment of the present disclosure;
fig. 5 is a block diagram of an image processing apparatus provided in an embodiment of the present disclosure;
Fig. 6 is a block diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
Fig. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, which may be performed by an image processing apparatus, which may be implemented by software and/or hardware, and which is typically provided in an electronic device. As shown in fig. 1, the method includes:
Step 110, acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object.
It should be noted that, the electronic device in the embodiments of the present disclosure may include a mobile terminal such as a smart phone, a notebook computer, a PDA (personal digital assistant), a PAD (tablet computer), and the like, and a fixed terminal such as a desktop computer, and the like.
In an embodiment of the disclosure, the first image may be an image about the real world acquired by a camera of the electronic device. For example, a plurality of frames of original images photographed by a smart phone are taken as the first image.
A model (which may be a three-dimensional model or a two-dimensional model, not limited herein) is previously created for some objects in the real world, and as an augmented reality model, an object having an augmented reality model may be referred to as a set object. In the embodiment of the disclosure, a 3D model is created for a landmark building in advance and is used as an augmented reality model corresponding to the landmark building. The augmented reality model may be built for different objects according to actual needs, and the embodiments of the present disclosure do not limit the types of objects.
Illustratively, the first image is acquired at a set period for the duration of the photographing event; identifying the first image, and judging whether the first image contains a setting object according to the identification result; if yes, obtaining an augmented reality model corresponding to the set object. In an embodiment of the present disclosure, the step of obtaining the augmented reality model corresponding to the set object includes, but is not limited to: and acquiring the augmented reality model corresponding to the set object by a resource library of the client. Or the intelligent terminal requests the server for the augmented reality model corresponding to the set object. For example, a resource library is built in a client downloaded by the intelligent terminal, the resource library comprises a plurality of common augmented reality models, and when a new resource exists in a server, the server can send an update notification to the client, so that the client is reminded of updating the built-in resource library. Alternatively, if the user needs to download new resources, the resources in the download list may be ordered according to the user's usage preferences to preferentially display resources that meet the user's usage preferences.
In one exemplary embodiment, after the client identifies the set object in the first image, the client sends a model request to the server to obtain, by the server, an augmented reality model corresponding to the set object. Optionally, the downloaded augmented reality model may be cached locally for the next use.
The setting period is an empirical value set in advance, and the setting periods in different shooting scenes may be the same or different. The shooting scene may be a sunrise scene, a cloudy day scene, a sunny day scene, a daytime scene, a darklight scene, or the like, and the embodiments of the present disclosure are not particularly limited.
Step 120, obtaining a preset model motion trail.
It should be noted that, the model motion trail is used to indicate the position information and depth information of the augmented reality model in the first image of each frame. In the embodiment of the disclosure, the position information may be coordinate information of a pixel point in the augmented reality model, or other information representing the position of the pixel point in the augmented reality model. The depth information may be depth information of pixels in the augmented reality model, or other information indicating whether the pixels in the augmented reality model are foreground pixels or background pixels.
The model motion trail is preset, and after the augmented reality model is built, the model motion trail is associated with the built augmented reality model. In order to be able to display the augmented reality model on the display screen of the electronic device, the augmented reality model may be processed based on a specific transformation matrix, which may be transformed from a model coordinate system to screen coordinates. The method is adopted to process the augmented reality model corresponding to each position on the preset model motion trail, and position information and depth information of the augmented reality model in the first image of each frame are obtained. After the first image is captured by the camera, the conversion matrix processing is performed to complete the conversion from the world coordinates to the screen coordinates.
Optionally, when the client acquires the augmented reality model from the server, the client may download data corresponding to a model motion trail associated with the augmented reality model and store the data in a built-in resource library. Or the identification information of the model motion trail associated with the augmented reality model is stored in a built-in resource library of the client, so that when the augmented reality model is required to be used, the data corresponding to the model motion trail is acquired by the server according to the identification information.
Exemplary, according to the obtained augmented reality model, determining a model motion trail corresponding to the augmented reality model, and obtaining data corresponding to the model motion trail.
And 130, superposing the augmented reality model on the first image to obtain a second image according to the position information and the depth information.
In an exemplary embodiment, according to position information and depth information included in a model motion track, a superposition area of an augmented reality model corresponding to each frame of a first image is determined, the augmented reality model is respectively superposed on the superposition area, multiple frames of second images are obtained, and the second images are displayed according to a set sequence. The set order may be an acquisition order of the first image. Or the order of generation of the second image. Or may be in other custom orders, embodiments of the disclosure not being specifically limited.
Fig. 2 is a flowchart of a model stacking method in an image processing method according to an embodiment of the present disclosure. As shown in fig. 2, the model stacking method includes:
and 131, determining a superposition area corresponding to the augmented reality model in the first image of each frame according to the position information.
For example, for each frame of the first image, the augmented reality model may be different for locations in the first image as known from the model motion trajectories. Coordinate information of each pixel point in the augmented reality model is obtained, and a superposition area of the augmented reality model in the first image of each frame is determined according to the coordinate information.
And 132, determining a depth-of-field relationship between the pixel point of the set object in the first image of each frame and the pixel point of the augmented reality model according to the depth information.
For example, depth information of a pixel point of a setting target is acquired for a first image of an arbitrary frame. And determining a target pixel point with the same coordinate as the pixel point of the first image in the pixel point of the augmented reality model, acquiring depth information of the target pixel point, and calculating a depth difference value of the target pixel point and the pixel point with the same coordinate of the set object to obtain a depth relation between the pixel point of the set object and the target pixel point in the current first image.
And 133, superposing the pixel points of the augmented reality model to the superposition area according to the depth of field relationship to obtain a second image.
For example, for any one frame of the first image, according to the depth difference, the augmented reality model is superimposed on the superimposed area, so as to obtain a frame of the second image. And the same mode is adopted for the first images of other frames to obtain a plurality of frames of second images.
According to the embodiment of the disclosure, the set object in the first image is identified, the augmented reality model corresponding to the set object and the preset model motion trail are obtained, and the augmented reality model is superimposed on the first image according to the position information and the depth information in the model motion trail to obtain the second image. Because the depth of field of the pixel points of the augmented reality model and the depth of field of the pixel points in the first image are considered in the superposition process of the augmented reality model and the image, the effect that the augmented reality model moves by taking a set object as a background according to the motion track of the model can be realized, the image display effect is enriched, the problem that the display effect in the current shooting scene is single is solved, and a novel playing method is provided to promote the user experience.
Fig. 3 is a flowchart of another image processing method according to an embodiment of the present disclosure, as shown in fig. 3, including:
step 310, acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object.
Step 320, obtaining a preset model motion trail.
And 330, determining a superposition area corresponding to the augmented reality model in the first image of each frame according to the position information.
And 340, determining a depth of field relationship between the pixel point of the set object in the first image and the pixel point of the augmented reality model according to the depth information.
And 350, adding the pixel points of the augmented reality model to the superposition area according to the depth of field relationship.
Step 360, determining a projection area of the augmented reality model on the surface of the set object.
Illustratively, a projected area of the augmented reality model on the surface of the set object is determined based on the coordinate information.
And 370, adjusting the pixel points of the set object in the projection area according to the depth information of the target pixel points of the augmented reality model to obtain a second image.
For example, the depth of field of the pixel points of the set object in the projection area may be adjusted according to the depth of field information of the template pixel points of the augmented reality model, so as to obtain a second image, so that the surface of the set object in the projection area follows the augmented reality model to deform. If the augmented reality model is projected from the setting object, the surface of the setting object is correspondingly deformed in the process that the augmented reality model is far away from the surface of the setting object. If the augmented reality model is retracted into the setting object, the surface of the setting object is deformed correspondingly during the process that the augmented reality model approaches the surface of the setting object.
Step 380, rendering the second image to a display interface, and displaying the motion process of the augmented reality model taking the set object as a background.
For example, multiple frames of second images are sequentially rendered to a display interface, and a video of a motion process of an augmented reality model with a set object as a background can be displayed.
In the embodiment of the disclosure, a projection area of an augmented reality model on the surface of a set object is determined by adding pixel points of the augmented reality model to a superposition area; and adjusting the pixel points of the set object in the projection area according to the depth information of the target pixel points of the augmented reality model to obtain a second image so that the surface of the set object in the projection area deforms along with the augmented reality model, thereby providing a novel display special effect.
Fig. 4 is a flowchart of still another image processing method according to an embodiment of the present disclosure, as shown in fig. 4, including:
Step 401, acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object.
Step 402, obtaining a preset model motion trail.
Step 403, determining a superposition area corresponding to the augmented reality model in the first image of each frame according to the position information.
And 404, determining the depth of field relationship between the pixel point of the set object in the first image of each frame and the pixel point of the augmented reality model according to the depth information.
Step 405, judging whether the depth relation satisfies the set condition, if not, executing step 406, otherwise, executing step 407.
The depth difference value is compared with a set threshold value, and whether the depth relation meets the set condition is judged according to the comparison result. The setting condition may be that if the depth of field difference is greater than a setting threshold, it is determined that the depth of field relationship satisfies the setting condition; if the depth of field difference is less than or equal to the set threshold, it is determined that the depth of field relationship does not satisfy the set condition.
The set threshold may be set according to an actual application scenario. In the embodiment of the present disclosure, the set threshold may be zero, that is, when at least one surface of the augmented reality model protrudes from the set object, the pixel points of the overlapping area in the first image are replaced with the pixel points corresponding to the augmented reality model, so as to present the effect that the augmented reality model covers the overlapping area in the first image.
Step 406, setting the pixel points of the augmented reality model corresponding to the superposition area to be transparent, superposing the pixel points of the augmented reality model to the superposition area to obtain a second image, and then executing step 410.
And step 407, replacing the pixel points of the superposition area in the first image by the pixel points of the augmented reality model corresponding to the superposition area to obtain a second image.
Step 408, determining a projection area of the augmented reality model on the surface of the set object.
For example, the projected area of the augmented reality model on the surface of the set object is determined in real time during the duration of the shooting event.
And 409, obtaining texture information of the projection area to render the augmented reality model according to the texture information.
For example, for each newly determined projection area, texture information of a set object corresponding to the projection area is acquired in real time, so as to render an augmented reality model according to the texture information acquired in real time.
Step 410, rendering the second image to a display interface, and displaying the motion process of the augmented reality model with the set object as a background.
In the embodiment of the disclosure, the texture information of the set object corresponding to the projection area is obtained by determining the projection area of the augmented reality model on the surface of the set object in real time, so as to render the augmented reality model according to the texture information, render and display the second image, so as to present the effect that the augmented reality model is a part of the set object, takes the set object as the background and moves according to the movement track of the model.
In an exemplary embodiment, the step 408 may be replaced with: upon detection of a texture acquisition event, a projected area of the augmented reality model on the surface of the set object is determined. It should be noted that, there are various ways to trigger the texture acquiring event, and embodiments of the present disclosure are not limited in particular. For example, a texture fetch event is triggered when the algorithm runs stable. Or triggering a texture acquisition event when the frame loss rate over a period of time is less than a set threshold. Or trigger a texture fetch event upon detection of a shoot stabilization. Or triggering a texture fetch event upon detection of a user set operation. Wherein the set threshold may be a system default value. The setting operation may be a user clicking on a screen, shaking the electronic device, or a prescribed gesture, etc. According to the embodiment of the disclosure, when the texture acquisition event is detected, the projection area of the augmented reality model on the surface of the set object is determined, and the texture information of the projection area is acquired, so that the augmented reality model is rendered according to the texture information, and the problem that the rendering effect is influenced by factors such as unstable algorithm or unstable shooting can be avoided. Meanwhile, a new interactive function can be provided for the user to select what texture is adopted to render the augmented reality model.
Fig. 5 is a block diagram of an image processing apparatus provided in an embodiment of the present disclosure. The apparatus may be implemented by software and/or hardware, and is typically integrated in an electronic device, enriching the display effect of a photographed image by performing the image processing method of the embodiments of the present disclosure. As shown in fig. 5, the apparatus includes:
the model obtaining module 510 is configured to obtain a first image, identify a set object in the first image, and obtain an augmented reality model corresponding to the set object;
The track acquisition module 520 is configured to acquire a preset model motion track, where the model motion track is used to indicate position information and depth information of the augmented reality model in the first image of each frame;
and the image superposition module 530 is configured to superimpose the augmented reality model on the first image according to the position information and the depth information to obtain a second image, and display the second image.
The image processing device provided in the embodiments of the present disclosure is configured to implement an image processing method, and the implementation principle and technical effects of the image processing device are similar to those of the image processing method, and are not described herein again.
Fig. 6 is a block diagram of an electronic device according to an embodiment of the present disclosure. Referring now to fig. 6, a schematic diagram of an electronic device 600 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, magnetic tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 601.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object;
Acquiring a preset model motion trail, wherein the model motion trail is used for indicating position information and depth information of the augmented reality model in the first image of each frame;
and according to the position information and the depth information, superposing the augmented reality model on the first image to obtain a second image, and displaying the second image.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented in software or hardware. The name of a module does not in some cases define the module itself.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, in which a first image is acquired, a set object in the first image is identified, and an augmented reality model corresponding to the set object is acquired, including:
acquiring a first image according to a set period in the duration of a shooting event;
Identifying the first image, and judging whether the first image contains a set object according to an identification result;
If yes, obtaining an augmented reality model corresponding to the set object.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, wherein the superimposing the augmented reality model to the first image according to the position information and the depth information to obtain a second image includes:
determining a superposition area corresponding to the augmented reality model in each frame of the first image according to the position information;
According to the depth information, determining a depth-of-field relationship between the pixel point of the set object in the first image of each frame and the pixel point of the augmented reality model;
and according to the depth of field relationship, overlapping the pixel points of the augmented reality model to the overlapping area to obtain a second image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, wherein the determining, according to the depth information, a depth-of-field relationship between a pixel point of the set object and a pixel point of the augmented reality model in each frame of the first image includes:
obtaining depth information of pixel points of the set object for the first image of any frame;
determining a target pixel point in the pixel points of the augmented reality model, wherein the target pixel point and the pixel point of the set object have the same coordinates;
and obtaining depth of field information of the target pixel point, and calculating a depth of field difference value of the target pixel point and the pixel point with the same coordinates of the set object to obtain a depth of field relation between the pixel point of the set object and the target pixel point in the current first image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, wherein the overlaying the pixel points of the augmented reality model to the overlaying region according to the depth of field relationship, to obtain a second image, includes:
Adding the pixel points of the augmented reality model to the superposition area according to the depth of field relation;
Determining a projection area of the augmented reality model on the surface of the set object;
And adjusting the depth of field of the pixel point of the set object in the projection area according to the depth of field information of the target pixel point to obtain a second image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, wherein the overlaying the pixel points of the augmented reality model to the overlaying region according to the depth of field relationship, to obtain a second image, includes:
Judging whether the depth of field relation meets a set condition or not;
if yes, pixel points of the superposition area in the first image are replaced by pixel points of the augmented reality model corresponding to the superposition area, and a second image is obtained;
Otherwise, setting the pixel points of the augmented reality model corresponding to the superposition area to be transparent, and superposing the pixel points of the augmented reality model to the superposition area to obtain a second image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, wherein the determining whether the depth of field relationship satisfies a set condition includes:
if the depth of field difference is larger than a set threshold, determining that the depth of field relation meets a set condition;
And if the depth of field difference value is smaller than or equal to a set threshold value, determining that the depth of field relation does not meet a set condition.
According to one or more embodiments of the present disclosure, there is provided an image processing method, which further includes, after the step of overlaying the augmented reality model to the first image to obtain a second image:
Determining a projection area of the augmented reality model on the surface of the set object;
texture information of the projection area is acquired, and the augmented reality model is rendered according to the texture information.
According to one or more embodiments of the present disclosure, there is provided an image processing method, which further includes, after the step of overlaying the augmented reality model to the first image to obtain a second image:
When a texture acquisition event is detected, determining a projection area of the augmented reality model on the surface of the set object;
texture information of the projection area is acquired, and the augmented reality model is rendered according to the texture information.
In accordance with one or more embodiments of the present disclosure, the present disclosure provides an image processing method, wherein,
And rendering the second image to a display interface, and displaying the motion process of the augmented reality model taking the set object as a background.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, wherein the model acquisition module is specifically configured to:
acquiring a first image according to a set period in the duration of a shooting event;
Identifying the first image, and judging whether the first image contains a set object according to an identification result;
If yes, obtaining an augmented reality model corresponding to the set object.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, wherein the image superimposition module includes:
The region determining submodule is used for determining a superposition region corresponding to the augmented reality model in the first image of each frame according to the position information;
The relation determining submodule is used for determining the depth-of-field relation between the pixel point of the set object in each frame of the first image and the pixel point of the augmented reality model according to the depth information;
And the model superposition sub-module is used for superposing the pixel points of the augmented reality model to the superposition area according to the depth of field relationship to obtain a second image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, wherein the relationship determination submodule is specifically configured to:
obtaining depth information of pixel points of the set object for the first image of any frame;
determining a target pixel point in the pixel points of the augmented reality model, wherein the target pixel point and the pixel point of the set object have the same coordinates;
and obtaining depth of field information of the target pixel point, and calculating a depth of field difference value of the target pixel point and the pixel point with the same coordinates of the set object to obtain a depth of field relation between the pixel point of the set object and the target pixel point in the current first image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, wherein the model overlaying sub-module is specifically configured to:
Adding the pixel points of the augmented reality model to the superposition area according to the depth of field relation;
Determining a projection area of the augmented reality model on the surface of the set object;
And adjusting the depth of field of the pixel point of the set object in the projection area according to the depth of field information of the target pixel point to obtain a second image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, wherein the model overlaying sub-module is specifically configured to:
Judging whether the depth of field relation meets a set condition or not;
if yes, pixel points of the superposition area in the first image are replaced by pixel points of the augmented reality model corresponding to the superposition area, and a second image is obtained;
Otherwise, setting the pixel points of the augmented reality model corresponding to the superposition area to be transparent, and superposing the pixel points of the augmented reality model to the superposition area to obtain a second image.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, wherein the determining whether the depth of field relationship satisfies a set condition includes:
if the depth of field difference is larger than a set threshold, determining that the depth of field relation meets a set condition;
And if the depth of field difference value is smaller than or equal to a set threshold value, determining that the depth of field relation does not meet a set condition.
According to one or more embodiments of the present disclosure, there is provided an image processing apparatus, the apparatus further comprising:
The first projection area determining module is used for determining a projection area of the augmented reality model on the surface of the set object after the step of superposing the augmented reality model on the first image to obtain a second image;
And the first texture acquisition module is used for acquiring texture information of the projection area so as to render the augmented reality model according to the texture information.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, the apparatus further comprising:
a second projection area determining module, configured to determine a projection area of the augmented reality model on the surface of the set object when a texture acquisition event is detected after the step of overlaying the augmented reality model on the first image to obtain a second image;
and the second texture acquisition module is used for acquiring texture information of the projection area so as to render the augmented reality model according to the texture information.
According to one or more embodiments of the present disclosure, the present disclosure provides an image processing apparatus, the image superimposition module further configured to:
and rendering the second image to a display interface, and displaying the motion process of the augmented reality model taking the set object as a background.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims.
Claims (11)
1. An image processing method, comprising:
Acquiring a first image, identifying a set object in the first image, and acquiring an augmented reality model corresponding to the set object;
Acquiring a preset model motion trail, wherein the model motion trail is used for indicating position information and depth information of the augmented reality model in the first image of each frame;
according to the position information and the depth information, the augmented reality model is overlapped to the first image to obtain a second image, and the second image is displayed;
the step of overlaying the augmented reality model to the first image to obtain a second image according to the position information and the depth information includes:
determining a superposition area corresponding to the augmented reality model in each frame of the first image according to the position information;
According to the depth information, determining a depth-of-field relationship between the pixel point of the set object in the first image of each frame and the pixel point of the augmented reality model;
According to the depth of field relationship, pixel points of the augmented reality model are overlapped to the overlapped area, and a second image is obtained;
The determining, according to the depth information, a depth of field relationship between the pixel point of the set object and the pixel point of the augmented reality model in each frame of the first image includes:
obtaining depth information of pixel points of the set object for the first image of any frame;
determining a target pixel point in the pixel points of the augmented reality model, wherein the target pixel point and the pixel point of the set object have the same coordinates;
and obtaining depth of field information of the target pixel point, and calculating a depth of field difference value of the target pixel point and the pixel point with the same coordinates of the set object to obtain a depth of field relation between the pixel point of the set object and the target pixel point in the current first image.
2. The method of claim 1, wherein the acquiring the first image, identifying a set object in the first image, and acquiring the augmented reality model corresponding to the set object, comprises:
acquiring a first image according to a set period in the duration of a shooting event;
Identifying the first image, and judging whether the first image contains a set object according to an identification result;
If yes, obtaining an augmented reality model corresponding to the set object.
3. The method according to claim 1, wherein the overlaying the pixel points of the augmented reality model to the overlaying region according to the depth of field relationship to obtain a second image includes:
Adding the pixel points of the augmented reality model to the superposition area according to the depth of field relation;
Determining a projection area of the augmented reality model on the surface of the set object;
And adjusting the depth of field of the pixel point of the set object in the projection area according to the depth of field information of the target pixel point to obtain a second image.
4. The method according to claim 1, wherein the overlaying the pixel points of the augmented reality model to the overlaying region according to the depth of field relationship to obtain a second image includes:
Judging whether the depth of field relation meets a set condition or not;
if yes, pixel points of the superposition area in the first image are replaced by pixel points of the augmented reality model corresponding to the superposition area, and a second image is obtained;
Otherwise, setting the pixel points of the augmented reality model corresponding to the superposition area to be transparent, and superposing the pixel points of the augmented reality model to the superposition area to obtain a second image.
5. The method of claim 4, wherein determining whether the depth of field relationship satisfies a set condition comprises:
if the depth of field difference is larger than a set threshold, determining that the depth of field relation meets a set condition;
And if the depth of field difference value is smaller than or equal to a set threshold value, determining that the depth of field relation does not meet a set condition.
6. The method of claim 1, further comprising, after overlaying the augmented reality model to the first image to obtain a second image:
Determining a projection area of the augmented reality model on the surface of the set object;
texture information of the projection area is acquired, and the augmented reality model is rendered according to the texture information.
7. The method of claim 1, further comprising, after overlaying the augmented reality model to the first image to obtain a second image:
When a texture acquisition event is detected, determining a projection area of the augmented reality model on the surface of the set object;
texture information of the projection area is acquired, and the augmented reality model is rendered according to the texture information.
8. The method of any of claims 1 to 7, wherein the displaying the second image comprises:
and rendering the second image to a display interface, and displaying the motion process of the augmented reality model taking the set object as a background.
9. An image processing apparatus, comprising:
The model acquisition module is used for acquiring a first image, identifying a set object in the first image and acquiring an augmented reality model corresponding to the set object;
The track acquisition module is used for acquiring a preset model motion track, wherein the model motion track is used for indicating the position information and the depth information of the augmented reality model in the first image of each frame;
The image superposition module is used for superposing the augmented reality model on the first image to obtain a second image according to the position information and the depth information, and displaying the second image;
the image superposition module is further configured to:
determining a superposition area corresponding to the augmented reality model in each frame of the first image according to the position information;
According to the depth information, determining a depth-of-field relationship between the pixel point of the set object in the first image of each frame and the pixel point of the augmented reality model;
According to the depth of field relationship, pixel points of the augmented reality model are overlapped to the overlapped area, and a second image is obtained;
the image superposition module is further configured to:
obtaining depth information of pixel points of the set object for the first image of any frame;
determining a target pixel point in the pixel points of the augmented reality model, wherein the target pixel point and the pixel point of the set object have the same coordinates;
and obtaining depth of field information of the target pixel point, and calculating a depth of field difference value of the target pixel point and the pixel point with the same coordinates of the set object to obtain a depth of field relation between the pixel point of the set object and the target pixel point in the current first image.
10. An electronic device, the electronic device comprising:
One or more processors;
a memory for storing one or more programs,
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the image processing method of any of claims 1-8.
11. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the image processing method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010662796.2A CN111833459B (en) | 2020-07-10 | 2020-07-10 | Image processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010662796.2A CN111833459B (en) | 2020-07-10 | 2020-07-10 | Image processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111833459A CN111833459A (en) | 2020-10-27 |
| CN111833459B true CN111833459B (en) | 2024-04-26 |
Family
ID=72900408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010662796.2A Active CN111833459B (en) | 2020-07-10 | 2020-07-10 | Image processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111833459B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022120533A1 (en) * | 2020-12-07 | 2022-06-16 | 深圳市大疆创新科技有限公司 | Motion trajectory display system and method, and storage medium |
| CN112672185B (en) * | 2020-12-18 | 2023-07-07 | 脸萌有限公司 | Augmented reality-based display method, device, equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106308946A (en) * | 2016-08-17 | 2017-01-11 | 清华大学 | Augmented reality device applied to stereotactic surgical robot and method of augmented reality device |
| CN106774937A (en) * | 2017-01-13 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | Image interactive method and its device in a kind of augmented reality |
| CN107564089A (en) * | 2017-08-10 | 2018-01-09 | 腾讯科技（深圳）有限公司 | Three dimensional image processing method, device, storage medium and computer equipment |
| CN107680164A (en) * | 2016-08-01 | 2018-02-09 | 中兴通讯股份有限公司 | A kind of virtual objects scale adjusting method and device |
| CN109427096A (en) * | 2017-08-29 | 2019-03-05 | 深圳市掌网科技股份有限公司 | A kind of automatic guide method and system based on augmented reality |
| CN110162258A (en) * | 2018-07-03 | 2019-08-23 | 腾讯数码（天津）有限公司 | The processing method and processing device of individual scene image |
| CN110889890A (en) * | 2019-11-29 | 2020-03-17 | 深圳市商汤科技有限公司 | Image processing method and device, processor, electronic device and storage medium |
-
2020
- 2020-07-10 CN CN202010662796.2A patent/CN111833459B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107680164A (en) * | 2016-08-01 | 2018-02-09 | 中兴通讯股份有限公司 | A kind of virtual objects scale adjusting method and device |
| CN106308946A (en) * | 2016-08-17 | 2017-01-11 | 清华大学 | Augmented reality device applied to stereotactic surgical robot and method of augmented reality device |
| CN106774937A (en) * | 2017-01-13 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | Image interactive method and its device in a kind of augmented reality |
| CN107564089A (en) * | 2017-08-10 | 2018-01-09 | 腾讯科技（深圳）有限公司 | Three dimensional image processing method, device, storage medium and computer equipment |
| CN109427096A (en) * | 2017-08-29 | 2019-03-05 | 深圳市掌网科技股份有限公司 | A kind of automatic guide method and system based on augmented reality |
| CN110162258A (en) * | 2018-07-03 | 2019-08-23 | 腾讯数码（天津）有限公司 | The processing method and processing device of individual scene image |
| CN110889890A (en) * | 2019-11-29 | 2020-03-17 | 深圳市商汤科技有限公司 | Image processing method and device, processor, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111833459A (en) | 2020-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111242881B (en) | Method, device, storage medium and electronic equipment for displaying special effects | |
| CN111833461B (en) | Method and device for realizing special effect of image, electronic equipment and storage medium | |
| CN110796664B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN112051961A (en) | Virtual interaction method and device, electronic equipment and computer readable storage medium | |
| CN111833459B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN112348748A (en) | Image special effect processing method and device, electronic equipment and computer readable storage medium | |
| CN112396676B (en) | Image processing method, apparatus, electronic device, and computer-readable storage medium | |
| CN114531553B (en) | Method, device, electronic equipment and storage medium for generating special effect video | |
| CN116934577A (en) | Method, device, equipment and medium for generating style image | |
| CN111818265B (en) | Interaction method and device based on augmented reality model, electronic equipment and medium | |
| CN112351221B (en) | Image special effect processing method, device, electronic equipment and computer readable storage medium | |
| CN116527993A (en) | Video processing method, apparatus, electronic device, storage medium and program product | |
| CN112492230B (en) | Video processing method and device, readable medium and electronic equipment | |
| CN114049403A (en) | Multi-angle three-dimensional face reconstruction method and device and storage medium | |
| CN112070903A (en) | Virtual object display method and device, electronic equipment and computer storage medium | |
| CN113837918A (en) | Method and device for realizing rendering isolation by multiple processes | |
| CN112668474B (en) | Plane generation method and device, storage medium and electronic equipment | |
| KR102534449B1 (en) | Image processing method, device, electronic device and computer readable storage medium | |
| CN112395826B (en) | Text special effect processing method and device | |
| CN111105345B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN116360661A (en) | Special effect processing method and device, electronic equipment and storage medium | |
| CN116152046A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN115731570A (en) | Image recognition method and device and electronic equipment | |
| CN117152385A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN115619918A (en) | Image rendering method, device and equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |