CN111862344B - Image processing method, apparatus and storage medium - Google Patents
Image processing method, apparatus and storage medium Download PDFInfo
- Publication number
- CN111862344B CN111862344B CN202010693146.4A CN202010693146A CN111862344B CN 111862344 B CN111862344 B CN 111862344B CN 202010693146 A CN202010693146 A CN 202010693146A CN 111862344 B CN111862344 B CN 111862344B
- Authority
- CN
- China
- Prior art keywords
- illumination information
- target
- reflection
- virtual object
- terminal equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 238000005286 illumination Methods 0.000 claims abstract description 156
- 238000009877 rendering Methods 0.000 claims abstract description 99
- 230000000694 effects Effects 0.000 claims abstract description 84
- 238000000034 method Methods 0.000 claims abstract description 50
- 239000000463 material Substances 0.000 claims description 34
- 239000013598 vector Substances 0.000 claims description 32
- 238000013507 mapping Methods 0.000 claims description 12
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- 230000003068 static effect Effects 0.000 abstract description 13
- 238000010586 diagram Methods 0.000 description 13
- 238000005070 sampling Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Image Generation (AREA)
Abstract
The application relates to an image processing method, an image processing device and a storage medium. The method comprises the following steps: acquiring illumination information corresponding to the current position of the terminal equipment, wherein the illumination information reflects the reflection effect of the reflection surface of the current position on ambient light; and rendering a reflecting surface of the target three-dimensional virtual object by utilizing the illumination information. The reflection surface of the target three-dimensional virtual object rendered by the illumination information also shows the reflection effect on the ambient light of the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, and storage medium.
Background
Rendering techniques of three-dimensional Virtual objects are widely used in AR (Augmented Reality ), VR (Virtual Reality), games, and other scenes.
In practical application, the processor renders the three-dimensional virtual object according to preset rendering parameters, and displays the rendered three-dimensional virtual object through the display screen.
The rendering effect presented by the rendering mode is required to be lifted.
Disclosure of Invention
In order to improve the rendering effect obtained by the rendering mode, an embodiment of the application provides an image processing method, an image processing device and a storage medium.
In a first aspect, an embodiment of the present application provides an image processing method, applied to a terminal device, where the method includes:
acquiring illumination information corresponding to the current position of the terminal equipment, wherein the illumination information reflects the reflection effect of the reflection surface of the current position on ambient light;
and rendering the reflecting surface of the target three-dimensional virtual object by utilizing the illumination information.
According to the method provided by the embodiment of the application, the target three-dimensional virtual object is rendered by combining the current position. Specifically, illumination information corresponding to the current position is obtained, and the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is a material capturing map.
The material capturing mapping is adopted as illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is improved.
Optionally, rendering the reflective surface of the target three-dimensional virtual object using the illumination information may be implemented by: obtaining a reflection vector of each pixel point of the reflection surface of the target three-dimensional virtual object; determining corresponding reference pixel points of reflection vectors of all pixel points in the material capturing mapping; and respectively utilizing the pixel values of the reference pixel points corresponding to the reflection vectors of the pixel points to modify the pixel values of the pixel points.
And the reflection vector is utilized to sample the reference pixel point of the material capturing problem diagram, so that the rendering effect can be improved.
Optionally, the reflection mode of the reflection surface in the material capturing map is specular reflection.
For the reflection mode of specular reflection, the rendering effect of sampling the reference pixel point of the material capturing problem diagram by using the reflection vector is more prominent.
On the basis of any of the method embodiments, after the reflection surface of the target three-dimensional virtual object is rendered by using the illumination information, the target three-dimensional virtual object may be further superimposed in an image acquired in real time by an image acquisition module of the terminal device.
In a second aspect, an embodiment of the present application provides a terminal device, including:
the illumination information acquisition module is used for acquiring illumination information corresponding to the current position of the terminal equipment, and the illumination information reflects the reflection effect of the reflection surface of the current position on the ambient light;
and the reflecting surface rendering module is used for rendering the reflecting surface of the target three-dimensional virtual object by utilizing the illumination information.
The terminal equipment provided by the embodiment of the application is used for rendering the target three-dimensional virtual object in combination with the current position. Specifically, illumination information corresponding to the current position is obtained, and the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is a material capturing map.
The material capturing mapping is adopted as illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is improved.
Optionally, the reflective surface rendering module is configured to: obtaining a reflection vector of each pixel point of the reflection surface of the target three-dimensional virtual object; determining corresponding reference pixel points of reflection vectors of all pixel points in the material capturing mapping; and respectively utilizing the pixel values of the reference pixel points corresponding to the reflection vectors of the pixel points to modify the pixel values of the pixel points.
And the reflection vector is utilized to sample the reference pixel point of the material capturing problem diagram, so that the rendering effect can be improved.
Optionally, the reflection mode of the reflection surface in the material capturing map is specular reflection.
For the reflection mode of specular reflection, the rendering effect of sampling the reference pixel point of the material capturing problem diagram by using the reflection vector is more prominent.
On the basis of any of the above embodiments of the terminal device, the terminal device may further include an image synthesis module configured to superimpose the target three-dimensional virtual object on the image acquired by the image acquisition module of the terminal device in real time after the reflective surface rendering module renders the reflective surface of the target three-dimensional virtual object using the illumination information.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory;
the memory is used for storing a program for executing any method embodiment of the first aspect;
the processor is configured to execute a program stored in the memory.
The electronic device provided by the embodiment of the application is used for rendering the target three-dimensional virtual object in combination with the current position. Specifically, illumination information corresponding to the current position is obtained, and the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium storing a program for implementing any of the method embodiments of the first aspect.
The program stored in the computer readable storage medium provided by the embodiment of the application is combined with the current position to render the target three-dimensional virtual object. Specifically, illumination information corresponding to the current position is obtained, and the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
In a fifth aspect, an embodiment of the present application provides an image processing method, where the method is applied to a server device, and includes:
acquiring position information sent by terminal equipment;
acquiring illumination information corresponding to the position information, wherein the illumination information reflects the reflection effect of the reflection surface of a target position on ambient light, and the target position is indicated by the position information;
and sending the illumination information to the terminal equipment so that the terminal equipment can render the reflecting surface of the target three-dimensional virtual object by utilizing the illumination information.
According to the method provided by the embodiment of the application, the illumination information corresponding to the target position reported by the terminal equipment is sent to the terminal equipment, so that the terminal equipment can render the target three-dimensional virtual object by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. Therefore, the method provided by the embodiment of the application can make the environment where the target three-dimensional virtual object presented by the terminal equipment and the current position thereof are located more uniform, so that the rendering effect is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is a material capturing map.
The material capturing mapping is adopted as illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is improved.
In a sixth aspect, an embodiment of the present application provides a server device, where the server device includes:
the position information acquisition module is used for acquiring position information sent by the terminal equipment;
the illumination information acquisition module is used for acquiring illumination information corresponding to the position information, the illumination information reflects the reflection effect of the reflection surface of the target position on the ambient light, and the target position is the position indicated by the position information;
and the illumination information sending module is used for sending the illumination information to the terminal equipment so that the terminal equipment can render the reflecting surface of the target three-dimensional virtual object by utilizing the illumination information.
The server device provided by the embodiment of the application sends the illumination information corresponding to the target position reported by the terminal device to the terminal device, so that the terminal device renders the target three-dimensional virtual object by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. Therefore, the server device provided by the embodiment of the application can make the environment where the target three-dimensional virtual object presented by the terminal device and the current position thereof are located more uniform, so that the rendering effect of the target three-dimensional virtual object is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is a material capturing map.
The material capturing mapping is adopted as illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is improved.
In a seventh aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory;
the memory is used for storing a program for executing any method embodiment of the fifth aspect;
the processor is configured to execute a program stored in the memory.
According to the electronic device provided by the embodiment of the application, the illumination information corresponding to the target position reported by the terminal device is sent to the terminal device, so that the terminal device can render the target three-dimensional virtual object by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. Therefore, the electronic device provided by the embodiment of the application can make the environment where the target three-dimensional virtual object presented by the terminal device and the current position thereof are located more uniform, so that the rendering effect of the electronic device is more vivid than that obtained by the static rendering mode.
In an eighth aspect, an embodiment of the present invention provides a computer readable storage medium storing a program for implementing any of the method embodiments described above.
The program stored in the computer readable storage medium provided by the embodiment of the application sends illumination information corresponding to the target position reported by the terminal equipment to the terminal equipment, so that the terminal equipment can render the target three-dimensional virtual object by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. Therefore, the program can make the environment where the target three-dimensional virtual object presented by the terminal equipment and the current position thereof are located more uniform, so that the rendering effect of the target three-dimensional virtual object is more vivid than that obtained by the static rendering mode.
Drawings
FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of sampling with reflection vectors according to one embodiment of the present application;
FIG. 3 is a flowchart of an image processing method according to another embodiment of the present application;
fig. 4 is a block diagram of a terminal device according to an embodiment of the present application;
fig. 5 is a block diagram of a server device according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a server device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In some of the processes described in the specification and claims of this application and the above figures, a number of operations occurring in a particular order are included, but it should be understood that these processes may include more or less operations, and that these operations may be performed sequentially or in parallel.
An embodiment of the present application provides an image processing method, applied to a terminal device, as shown in fig. 1, where the method includes:
step 101, obtaining illumination information corresponding to the current position of the terminal equipment, wherein the illumination information reflects the reflection effect of the reflection surface of the current position on the ambient light.
And 102, rendering the reflecting surface of the target three-dimensional virtual object by utilizing the illumination information.
Through the rendering of step 102, the reflective surface of the target three-dimensional virtual object presents a reflective effect on the ambient light at the current location.
Wherein, the reflecting surface refers to a surface which reflects ambient light. The method for determining the reflecting surface of the target three-dimensional virtual object is not limited in the embodiment. In one implementation, the surface to be presented by the target three-dimensional virtual object is taken as a reflecting surface, and in another implementation, the reflecting surfaces at different viewing angles are predefined.
The ambient light may be light emitted by a light source in the environment (such as sunlight, lamplight, etc.), or light reflected by a mirror in the environment.
According to the method provided by the embodiment of the application, the target three-dimensional virtual object is rendered by combining the current position. Specifically, illumination information corresponding to the current position is obtained, and the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
In practical applications, there are various implementation manners for obtaining illumination information corresponding to a current position by a terminal device.
One implementation is as follows: the terminal equipment reports the position information to the server equipment, the position information indicates the current position of the terminal equipment, and the illumination information corresponding to the current position returned by the server equipment is received.
In order to obtain illumination information which is more matched with the current situation, the terminal equipment can also report the azimuth information of an image acquisition module (such as a camera) of the terminal equipment.
In order to obtain illumination information more matched with the target three-dimensional virtual object, the terminal equipment can report the material of the reflecting surface of the target three-dimensional virtual object or the reflecting mode corresponding to the reflecting surface.
In the embodiment of the application, the reflection mode includes specular reflection and diffuse reflection.
Another implementation is: the terminal equipment determines illumination information corresponding to the current position according to the image acquired by the image acquisition module (such as a camera) in real time. Because the image acquired by the image acquisition module in real time reflects the environment of the current position, illumination information can be obtained by processing the image. Specifically, a reference object in the image can be identified, and the reference object can be a building, a person, an animal, a vehicle, a plant, etc., and in practical application, which objects can be used as the reference objects can be determined according to the needs. Then, the illumination information can be determined according to the surface light intensity distribution information of each reference object, specifically, an illumination information model can be trained in advance, and the images of each reference object are input into the illumination information model to obtain illumination information; the illumination information can also be determined according to the shadow shape and direction of each reference object and the surface light intensity of each reference object, specifically, an illumination information model can be trained in advance, and the images of each reference object and the shadow thereof are input into the illumination information model to obtain the illumination information; the target reference object can be selected from the reference objects, the image of the target reference object is used as illumination information, or the image of the target reference object is processed according to a preset rule and then used as illumination information, in practical application, the standard for selecting the target reference object can be determined according to needs or experience, and in practical application, the image processing strategy of the target reference object can be determined according to needs or experience.
The embodiment of the application does not limit the specific content of the illumination information, and in practical application, the specific content can be determined according to needs.
In one implementation, the illumination information may consist of parameters indicating information of the type of reflection of the reflecting surface, the shape of the reflecting surface, the direction of the ambient light, the intensity of the ambient light, etc. After the terminal device obtains the parameters, it may, but is not limited to, use a PBR (physical based rendering) technology to render the reflective surface of the target three-dimensional virtual object using the parameters.
In another implementation manner, the illumination information may be a material capturing (matcap) map, and the matcap map is used as the illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is increased.
If the matcap map is used as illumination information, the terminal device can render the reflective surface of the target three-dimensional virtual object according to the matcap map.
In one implementation, the matcap map may be sampled using a normal vector of the reflective surface of the target three-dimensional virtual object at each pixel point to obtain a reference pixel point on the matcap map, and the pixel value of the pixel point on the reflective surface of the three-dimensional virtual object is adjusted using the pixel value of the reference pixel point. The embodiments of the present application are not limited to the adjustment manner, and for example, the pixel value of the reference pixel point may be weighted or multiplied with the pixel value of the pixel point on the reflective surface.
In another implementation manner, the matcap map may be sampled by using reflection vectors of the reflection surface of the target three-dimensional virtual object at each pixel point to obtain a reference pixel point on the matcap map, and the pixel value of the pixel point on the reflection surface of the three-dimensional virtual object is adjusted by using the pixel value of the reference pixel point. Specifically, obtaining a reflection vector of each pixel point of a reflection surface of a target three-dimensional virtual object; determining corresponding reference pixel points of reflection vectors of all pixel points in the material capturing mapping; and respectively utilizing the pixel values of the reference pixel points corresponding to the reflection vectors of the pixel points to modify the pixel values of the pixel points.
And the reflection vector is utilized to sample the reference pixel point of the material capturing problem diagram, so that the rendering effect can be improved.
The inventor finds that when a plane with a large area exists on the reflecting surface, if the matcap map is sampled by adopting a normal vector, the normal vector of each point of the plane is the same, so that the same pixel point in the matcap map is sampled, and the rendering reflection effect is poor. The above-mentioned problem also exists when the curvature of the reflecting surface is small, which is particularly pronounced for specular reflection. And the problem can be avoided by adopting the reflection vector for sampling, so that a better reflection effect is realized.
Optionally, the reflection mode of the reflection surface in the material capturing map is specular reflection.
For the reflection mode of specular reflection, the rendering effect of sampling the reference pixel point of the material capturing problem diagram by using the reflection vector is more prominent.
The method provided by the embodiment of the application can be applied to an AR scene, and then after the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information, the target three-dimensional virtual object can be superimposed in the image acquired by the image acquisition module of the terminal equipment in real time.
The implementation principle of sampling the matcap map by using the reflection vector is as follows:
in the camera space, the cross section passing through the sphere center of the target sphere is assumed to be a circle with a radius value of 1, wherein the target sphere is a sphere in the matcap map.
Then, the interface parallel to the line of sight is shown in fig. 2. Wherein,is spherical normal +.>Is incident ray, < >>Is->About->Is a reflection line of the above. G is the point where the straight line CD intersects the xy plane.
The sampling being based on a known quantityDetermine->
Wherein the reflection vector of the target three-dimensional virtual objectIncident vector->Normal vector->The incident vector is parallel to the direction of the line of sight, and in the camera spaceIn turn, the line of sight direction is parallel to the camera normal.
Because ofAnd->About->Symmetrical and->Thus (S)>And->Collinear.
And also hasIs a sphere normal, thus +.>Collinear, with a cross-sectional circle radius of 1, < >>Thus, the first and second substrates are bonded together,
parallel to the camera normal, therefore->
Is->Projection in the xy plane, thus +.>
The coordinates of the last matcap map for sampling are
And taking the pixel point on the coordinates as a reference pixel point.
The embodiment of the application also provides an image processing method, which is applied to the server device, as shown in fig. 3, and includes:
step 301, obtaining position information sent by a terminal device.
Optionally, the azimuth information of the image acquisition module sent by the terminal equipment and/or the reflection mode are also received.
Step 302, obtaining illumination information corresponding to the position information, wherein the illumination information reflects the reflection effect of the reflection surface of the target position on the ambient light, and the target position is the position indicated by the position information.
In practical application, a plurality of illumination information may be pre-stored, each illumination information corresponds to one position information, the position information sent by the terminal device is compared with the position information corresponding to each illumination information, and the illumination information corresponding to the closest position information is selected.
Furthermore, each piece of illumination information also corresponds to a reflection mode, so that candidate illumination information can be selected from a plurality of pieces of illumination information according to the reflection mode sent by the terminal equipment, and then the position information sent by the terminal equipment is compared with the position information corresponding to each piece of candidate illumination information, and illumination information corresponding to the closest position information is selected.
Furthermore, each piece of illumination information corresponds to azimuth information, and then candidate illumination information can be selected from a plurality of pieces of illumination information according to the azimuth information sent by the terminal equipment, and then the position information sent by the terminal equipment is compared with the position information corresponding to each piece of candidate illumination information, and illumination information corresponding to the nearest position information is selected.
And 303, transmitting the illumination information to the terminal equipment so that the terminal equipment can render the reflecting surface of the target three-dimensional virtual object by using the illumination information.
According to the method provided by the embodiment of the application, the illumination information corresponding to the target position reported by the terminal equipment is sent to the terminal equipment, so that the terminal equipment can render the target three-dimensional virtual object by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. Therefore, the method provided by the embodiment of the application can make the environment where the target three-dimensional virtual object presented by the terminal equipment and the current position thereof are located more uniform, so that the rendering effect is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is a material capturing map.
The material capturing mapping is adopted as illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is improved.
The method provided in the embodiment of the present application will be described in detail below by taking an AR scene as an example.
The terminal equipment sends a request message to the server-side equipment, wherein the request message carries position information and orientation information of the camera.
And the server equipment acquires the current weather information of the target position indicated by the position information according to the position information carried in the request message.
And the server side equipment also acquires a corresponding matcap map according to the position information and the azimuth information carried in the request message and the current time, wherein the matcap map reflects the reflection effect of the reflection surface of the target position indicated by the position information under the current ambient light.
And the server side equipment sends the current weather information and the matcap map to the terminal equipment.
The terminal equipment selects a weather rendering effect according to the current weather information, and renders the image acquired by the camera in real time by utilizing the selected weather rendering effect.
And the terminal equipment renders the reflecting surface of the target three-dimensional virtual object according to the matcap map, and synthesizes the rendered target three-dimensional virtual object with the image acquired by the camera in real time.
Based on the same inventive concept as the method shown in fig. 1, the embodiment of the present application further provides a terminal device, as shown in fig. 4, including:
the illumination information acquisition module 401 is configured to acquire illumination information corresponding to a current position of the terminal device, where the illumination information reflects a reflection effect of a reflection surface of the current position on ambient light;
a reflective surface rendering module 402 for rendering a reflective surface of the target three-dimensional virtual object using the illumination information.
The terminal equipment provided by the embodiment of the application is used for rendering the target three-dimensional virtual object in combination with the current position. Specifically, illumination information corresponding to the current position is obtained, and the reflection surface of the target three-dimensional virtual object is rendered by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. The three-dimensional virtual object presented by the rendering mode is more unified with the environment where the current position is located, so that the rendering effect is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is an environmental ball map.
By adopting the environment ball map as the environment information, the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed can be improved.
Optionally, the environmental ball map is a material capturing map.
By adopting the material capturing mapping, the reflection effect rendering process of the target three-dimensional virtual object can be further simplified, and the rendering speed is further improved.
Optionally, the reflective surface rendering module is configured to: obtaining a reflection vector of each pixel point of the reflection surface of the target three-dimensional virtual object; determining corresponding reference pixel points of reflection vectors of all pixel points in the material capturing mapping; and respectively utilizing the pixel values of the reference pixel points corresponding to the reflection vectors of the pixel points to modify the pixel values of the pixel points.
And the reflection vector is utilized to sample the reference pixel point of the material capturing problem diagram, so that the rendering effect can be improved.
Optionally, the reflection mode of the reflection surface in the material capturing map is specular reflection.
For the reflection mode of specular reflection, the rendering effect of sampling the reference pixel point of the material capturing problem diagram by using the reflection vector is more prominent.
On the basis of any of the above embodiments of the terminal device, the terminal device may further include an image synthesis module configured to superimpose the target three-dimensional virtual object on the image acquired by the image acquisition module of the terminal device in real time after the reflective surface rendering module renders the reflective surface of the target three-dimensional virtual object using the illumination information.
Based on the same inventive concept as the method shown in fig. 3, an embodiment of the present application further provides a server device, as shown in fig. 5, including:
a location information obtaining module 501, configured to obtain location information sent by a terminal device;
the illumination information obtaining module 502 is configured to obtain illumination information corresponding to the position information, where the illumination information reflects a reflection effect of a reflection surface of a target position on ambient light, and the target position is a position indicated by the position information;
and the illumination information sending module 503 is configured to send the illumination information to the terminal device, so that the terminal device renders the reflective surface of the target three-dimensional virtual object by using the illumination information.
The server device provided by the embodiment of the application sends the illumination information corresponding to the target position reported by the terminal device to the terminal device, so that the terminal device renders the target three-dimensional virtual object by utilizing the illumination information. Because the illumination information reflects the reflection effect of the reflection surface at the current position on the ambient light, the reflection surface of the target three-dimensional virtual object rendered by using the illumination information also shows the reflection effect on the ambient light at the current position. Therefore, the server device provided by the embodiment of the application can make the environment where the target three-dimensional virtual object presented by the terminal device and the current position thereof are located more uniform, so that the rendering effect of the target three-dimensional virtual object is more vivid than that obtained by the static rendering mode.
Optionally, the illumination information is a material capturing map.
The material capturing mapping is adopted as illumination information, so that the reflection effect rendering process of the target three-dimensional virtual object can be simplified, and the rendering speed is improved.
Based on the same inventive concept as the method shown in fig. 1, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a program for executing any of the method embodiments described above;
the processor is configured to execute a program stored in the memory.
The electronic device may be a mobile terminal such as a smart phone, a head-mounted device, and the like, and correspondingly, as shown in fig. 6, the mobile terminal provided in the embodiment of the application not only includes a processor and a memory, but also includes an input device (such as a touch screen, a camera, a microphone, and the like), an output device (such as a display screen, a speaker, and the like), a communication module, and a power module.
The memory, the input device, the output device, the communication module and the power module are connected with the processor through serial ports, buses or USB interfaces. Wherein, for a single processor mobile terminal, the processor is a CPU (central processing unit); for the dual-processor mobile terminal, the processor comprises a main processor and a slave processor, the main processor executes an application program to realize the method provided by the embodiment of the invention, and if communication with the outside is needed, the slave processor controls the communication module to cooperate for realizing; for a terminal device including a GPU (graphics processor) and a CPU, the processor refers to the GPU and the CPU, and the methods provided in the embodiments of the present application are implemented by the GPU and the CPU in cooperation or by the CPU alone.
The memory of the mobile terminal may include, but is not limited to: flash memory, RAM (random access memory), ROM (read only memory), etc. The RAM is mainly used for storing programs and data when the mobile terminal is running, and the programs to be executed or the data to be processed must be loaded into the RAM first. The ROM is mainly used for checking the configuration condition of an operating system of the terminal equipment and providing most basic input/output (I/O) programs. Flash memory is a long-life non-volatile (that still holds stored data information in the event of a power failure) memory in which data is deleted not in units of individual bytes but in units of fixed blocks. Since the Flash memory can still store data when power is off, it is generally used to store setting information, such as user setting information for a mobile terminal.
It should be noted that for different mobile terminals (single processor smartphones, dual processor smartphones, smart wearable devices, tablet computers, etc.), it is possible to include more or less hardware structures than the mobile terminal shown in fig. 6, but it is within the scope of the present disclosure to include a memory and a processor and be able to implement the functions of the above-described method embodiments.
Based on the same inventive concept as the method shown in fig. 3, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a program for executing any of the method embodiments described above;
the processor is configured to execute a program stored in the memory.
The electronic device may also be a server device. As shown in fig. 7, the server device at least includes: the power supply is logically connected with the central processing unit through the power supply management system, so that the functions of managing charging, discharging, managing power consumption and the like are realized through the power supply management system; the main board provides an electric connection path between the components, each component transmits electric signals through the main board, and each component is arranged on the main board in a plugging, welding and other modes; a central processing unit mainly comprising an arithmetic logic unit (ALU, arithmetic Logic Unit), a Program Counter (PC), a register set, a Cache (Cache) and a Bus (Bus) interface for transmitting data, control and status; the internal memory, the magnetic disk (i.e. the memory storing the program for executing the method according to each method embodiment), the input unit (including a mouse, a keyboard, a touch panel, etc.), the display unit (including a display screen and a display card), and the network card exchange data with the central processing unit through the bus.
Based on the same inventive concept as the method, the embodiments of the present application provide a computer-readable storage medium storing a program for implementing any of the method embodiments described above.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The embodiments described above are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
Claims (9)
1. An image processing method, characterized by being applied to a terminal device, the method comprising:
acquiring an image acquired by an image acquisition module of the terminal equipment, determining illumination information corresponding to the current position of the terminal equipment according to the image, and acquiring weather information corresponding to the current position of the terminal equipment, wherein the illumination information reflects the reflection effect of a reflection surface of the current position on ambient light;
rendering a reflective surface of the target three-dimensional virtual object using the illumination information;
and determining a weather rendering effect according to the weather information, rendering the image by using the weather rendering effect, and superposing the target three-dimensional virtual object in the image.
2. The method of claim 1, wherein the illumination information is a texture capture map.
3. The method of claim 2, wherein rendering the reflective surface of the target three-dimensional virtual object using the illumination information comprises:
obtaining a reflection vector of each pixel point of the reflection surface of the target three-dimensional virtual object;
determining corresponding reference pixel points of the reflection vector of each pixel point in the material capturing mapping;
and modifying the pixel value of each pixel point by using the pixel value of the reference pixel point corresponding to the reflection vector of each pixel point.
4. A method according to claim 3, wherein the reflective surface in the material capture map is specularly reflective.
5. An image processing method, applied to a server device, comprising:
acquiring position information sent by terminal equipment;
acquiring weather information corresponding to the position information;
the weather information is sent to the terminal equipment, so that the terminal equipment determines weather rendering effect according to the weather information, the weather rendering effect is utilized to render an image acquired by an image acquisition module of the terminal equipment, the terminal equipment determines illumination information corresponding to the current position of the terminal equipment according to the image, the illumination information reflects the reflection effect of the reflection surface of the current position on ambient light, the reflection surface of a target three-dimensional virtual object is rendered according to the illumination information, and the target three-dimensional virtual object is overlapped in the image.
6. A terminal device, comprising:
the illumination information acquisition module is used for acquiring an image acquired by the image acquisition module of the terminal equipment, determining illumination information corresponding to the current position of the terminal equipment according to the image, and acquiring weather information corresponding to the current position of the terminal equipment, wherein the illumination information reflects the reflection effect of the reflection surface of the current position on ambient light;
the reflecting surface rendering module is used for rendering the reflecting surface of the target three-dimensional virtual object by utilizing the illumination information;
and the image synthesis module is used for determining weather rendering effects according to the weather information, rendering the image by utilizing the weather rendering effects, and superposing the target three-dimensional virtual object in the image.
7. A server device, comprising:
the position information acquisition module is used for acquiring position information sent by the terminal equipment;
the illumination information acquisition module is used for acquiring weather information corresponding to the position information;
the illumination information sending module is used for sending the weather information to the terminal equipment, so that the terminal equipment determines weather rendering effect according to the weather information, renders an image acquired by the image acquisition module of the terminal equipment by utilizing the weather rendering effect, and enables the terminal equipment to determine illumination information corresponding to the current position of the terminal equipment according to the image, the illumination information reflects the reflection effect of the reflection surface of the current position on ambient light, renders the reflection surface of the target three-dimensional virtual object by utilizing the illumination information, and superimposes the target three-dimensional virtual object in the image.
8. An electronic device comprising a processor and a memory;
the memory is used for storing a program for executing the method of any one of claims 1 to 5;
the processor is configured to execute a program stored in the memory.
9. A computer-readable storage medium, characterized in that a program for executing the method of any one of claims 1 to 5 is stored.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010693146.4A CN111862344B (en) | 2020-07-17 | 2020-07-17 | Image processing method, apparatus and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010693146.4A CN111862344B (en) | 2020-07-17 | 2020-07-17 | Image processing method, apparatus and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111862344A CN111862344A (en) | 2020-10-30 |
| CN111862344B true CN111862344B (en) | 2024-03-08 |
Family
ID=73001988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010693146.4A Active CN111862344B (en) | 2020-07-17 | 2020-07-17 | Image processing method, apparatus and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111862344B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112489179B (en) * | 2020-12-15 | 2023-10-03 | 网易（杭州）网络有限公司 | Target model processing method and device, storage medium and computer equipment |
| CN112686988A (en) * | 2020-12-31 | 2021-04-20 | 北京北信源软件股份有限公司 | Three-dimensional modeling method, three-dimensional modeling device, electronic equipment and storage medium |
| CN113793402B (en) * | 2021-08-10 | 2023-12-26 | 北京达佳互联信息技术有限公司 | Image rendering method and device, electronic equipment and storage medium |
| CN113989473B (en) * | 2021-12-23 | 2022-08-12 | 北京天图万境科技有限公司 | Method and device for relighting |
| CN114419240B (en) * | 2022-04-01 | 2022-06-17 | 腾讯科技（深圳）有限公司 | Illumination rendering method and device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103003847A (en) * | 2010-05-16 | 2013-03-27 | 诺基亚公司 | Method and apparatus for rendering a location-based user interface |
| CN105321201A (en) * | 2015-11-26 | 2016-02-10 | 郭小虎 | Cloud light baking-based mobile terminal 3D real-time rendering method and system |
| CN107808409A (en) * | 2016-09-07 | 2018-03-16 | 中兴通讯股份有限公司 | The method, device and mobile terminal of illumination render are carried out in a kind of augmented reality |
| CN108305328A (en) * | 2018-02-08 | 2018-07-20 | 网易（杭州）网络有限公司 | Dummy object rendering intent, system, medium and computing device |
| CN111127623A (en) * | 2019-12-25 | 2020-05-08 | 上海米哈游天命科技有限公司 | Model rendering method and device, storage medium and terminal |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109427088B (en) * | 2017-08-18 | 2023-02-03 | 腾讯科技（深圳）有限公司 | Rendering method for simulating illumination and terminal |
-
2020
- 2020-07-17 CN CN202010693146.4A patent/CN111862344B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103003847A (en) * | 2010-05-16 | 2013-03-27 | 诺基亚公司 | Method and apparatus for rendering a location-based user interface |
| CN105321201A (en) * | 2015-11-26 | 2016-02-10 | 郭小虎 | Cloud light baking-based mobile terminal 3D real-time rendering method and system |
| CN107808409A (en) * | 2016-09-07 | 2018-03-16 | 中兴通讯股份有限公司 | The method, device and mobile terminal of illumination render are carried out in a kind of augmented reality |
| CN108305328A (en) * | 2018-02-08 | 2018-07-20 | 网易（杭州）网络有限公司 | Dummy object rendering intent, system, medium and computing device |
| CN111127623A (en) * | 2019-12-25 | 2020-05-08 | 上海米哈游天命科技有限公司 | Model rendering method and device, storage medium and terminal |
Non-Patent Citations (1)
| Title |
|---|
| MatCap Shader 改进：解决平面渲染和环境反射问题;Richard Pan;《知乎（https://zhuanlan.zhihu.com/p/79040521）》;第1-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111862344A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111862344B (en) | Image processing method, apparatus and storage medium | |
| US11636641B2 (en) | Electronic device for displaying avatar corresponding to external object according to change in position of external object | |
| US10930056B2 (en) | Electronic device for generating images having rendering qualities differing by view vector | |
| CN112933599B (en) | Three-dimensional model rendering method, device, equipment and storage medium | |
| CN110544289B (en) | Utilizing inter-frame coherence in a mid-ordering architecture | |
| US10249089B2 (en) | System and method for representing remote participants to a meeting | |
| WO2020192608A1 (en) | Graphics rendering method and apparatus, and computer readable storage medium | |
| CN113052947B (en) | Rendering method, rendering device, electronic equipment and storage medium | |
| US11563887B2 (en) | Method for controlling synchronization of plurality of image sensors and electronic device for implementing same | |
| CN110853128B (en) | Virtual object display method and device, computer equipment and storage medium | |
| CN114745598A (en) | Video data display method and device, electronic equipment and storage medium | |
| CN115797535B (en) | Texture mapping method and related device for three-dimensional model | |
| CN113384880A (en) | Virtual scene display method and device, computer equipment and storage medium | |
| CN108492339B (en) | Method and device for acquiring resource compression packet, electronic equipment and storage medium | |
| CN112884873B (en) | Method, device, equipment and medium for rendering virtual object in virtual environment | |
| CN112802206B (en) | Roaming view generation method, device, equipment and storage medium | |
| US10754498B2 (en) | Hybrid image rendering system | |
| US11636566B2 (en) | Computing system and methods of operating the computing system | |
| US20170186218A1 (en) | Method for loading 360 degree images, a loading module and mobile terminal | |
| CN112037339B (en) | Image processing method, apparatus and storage medium | |
| US8755819B1 (en) | Device location determination using images | |
| US20230090732A1 (en) | System and method for real-time ray tracing in a 3d environment | |
| CN117582661A (en) | Virtual model rendering method, device, medium and equipment | |
| CN117631810A (en) | Operation processing method, device, equipment and medium based on virtual reality space | |
| CN114926250A (en) | Order processing method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: Tiktok vision (Beijing) Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |