WO2021249091A1 - 图像处理方法、装置、计算机存储介质及电子设备 - Google Patents

图像处理方法、装置、计算机存储介质及电子设备 Download PDF

Info

Publication number
WO2021249091A1
WO2021249091A1 PCT/CN2021/093232 CN2021093232W WO2021249091A1 WO 2021249091 A1 WO2021249091 A1 WO 2021249091A1 CN 2021093232 W CN2021093232 W CN 2021093232W WO 2021249091 A1 WO2021249091 A1 WO 2021249091A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
shadow
texel
average
information
Prior art date
Application number
PCT/CN2021/093232
Other languages
English (en)
French (fr)
Inventor
李奇
黄谦
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP21822842.7A priority Critical patent/EP4044122A4/en
Priority to JP2022554220A priority patent/JP7390497B2/ja
Priority to KR1020227014210A priority patent/KR102611066B1/ko
Publication of WO2021249091A1 publication Critical patent/WO2021249091A1/zh
Priority to US17/705,169 priority patent/US11776202B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • G06T15/60—Shadow generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • G06T15/80—Shading
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2215/00—Indexing scheme for image rendering
    • G06T2215/12—Shadow map, environment map
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • This application relates to the field of image processing technology, in particular to the shadow removal technology in pictures.
  • 3D model modeling technology based on photogrammetry is usually used for modeling.
  • 3D model modeling technology based on photogrammetry requires the use of multiple photos for 3D modeling and the use of pixels from multiple photos
  • the color information is used to restore the diffuse reflection color information of the object, so the color map of the generated model must have the shadow in the original photo.
  • the shadows in the diffuse reflection map include the shadows caused by direct light and the shadows caused by ambient light.
  • tools such as Agisoft Texture Delighting are usually used to remove the shadows caused by direct light
  • tools such as Unity De-Lightingtool are used to remove non-direct light.
  • these de-illuminated shadow schemes generally have problems such as loss of texture details, hue changes, color overflow, and color blocks. As a result, the effect of the model generated from the diffuse map after the shadow is removed is not realistic and cannot be applied to any In a scene with lighting conditions.
  • the embodiments of the present application provide an image processing method, an image processing device, a computer-readable storage medium, and an electronic device, which can improve the removal efficiency and quality of shadows in diffuse reflection maps at least to a certain extent, and further improve the results based on the removal of shadows.
  • the image quality of the model generated by the diffuse map can improve the removal efficiency and quality of shadows in diffuse reflection maps at least to a certain extent, and further improve the results based on the removal of shadows.
  • an image processing method executed by an electronic device including: obtaining a diffuse reflection map, and obtaining shadow texels in a shadow area of the diffuse reflection map;
  • the spatial coordinate information of the pixel is queried in the average color look-up table, and the average brightness difference corresponding to the shadow texel is determined according to the query result;
  • the restoration is determined according to the average brightness difference and the color information corresponding to the shadow texel Color information, and restore the color information of the shadow texels according to the restored color information.
  • an image processing device including: an acquisition module for acquiring a diffuse reflection map, and acquiring shadow texels in a shadow area in the diffuse reflection map; and a calculation module for Query in the average color look-up table according to the spatial coordinate information of the shadow texel, and determine the average brightness difference corresponding to the shadow texel according to the query result; the restoration module is used to determine the average brightness difference corresponding to the shadow texel according to the average brightness difference and the The color information corresponding to the shadow texel is determined to restore the color information, and the color information of the shadow texel is restored according to the restored color information.
  • a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the image processing method as described in the above-mentioned embodiment is implemented.
  • an electronic device including: one or more processors; a storage device, the storage device is used to store one or more programs, when the one or more programs are When the one or more processors are executed, the one or more processors are caused to execute the image processing method described in the foregoing embodiment.
  • a computer program product including instructions, which when run on a computer, cause the computer to execute the image processing method described in the above-mentioned embodiments.
  • the diffuse reflection map is first obtained, and the shadow texels in the shadow area in the diffuse reflection map are obtained; then the average color lookup table is performed according to the spatial coordinate information of the shadow texels Query, and determine the average brightness difference corresponding to the shadow texel according to the query result; finally determine the restoration color information according to the average brightness difference and the color information of the shadow texel, and restore the color information of the shadow texel according to the restoration color information.
  • the technical solution of the application improves the efficiency and quality of color reproduction; on the other hand, it does not affect the color of the texels in the non-shaded area of the texture, avoiding the loss of texture details, hue changes, color overflow, and color blocks, etc., and ensures The generated model can be applied to scenes with any lighting conditions, further improving the picture effect and user experience.
  • FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present disclosure can be applied;
  • 2A-2B schematically show a schematic diagram of an interface for processing model texture in related technologies
  • Fig. 3 schematically shows a flow chart of an image processing method according to an embodiment of the present disclosure
  • Fig. 4 schematically shows a schematic flow chart of determining shadow texels according to an embodiment of the present disclosure
  • FIG. 5 schematically shows a flow chart of obtaining a bright surface average color lookup table and a dark surface average color lookup table according to an embodiment of the present disclosure
  • FIG. 6 schematically shows a flow chart of obtaining a bright surface color lookup table and a dark surface color lookup table according to an embodiment of the present disclosure
  • FIG. 7 schematically shows a flow chart of constructing a bright surface average color lookup table and a dark surface average color lookup table according to an embodiment of the present disclosure
  • Fig. 8 schematically shows a flow chart of forming an average color look-up table according to an embodiment of the present disclosure
  • FIG. 9 schematically shows a flow chart of calculating the average color of the bright part and the average color of the dark part according to an embodiment of the present disclosure
  • Fig. 10 schematically shows a flow chart of removing shadows caused by non-direct light illumination according to an embodiment of the present disclosure
  • FIG. 11 schematically shows a schematic diagram of an interface for removing shadows in a model according to the image processing method of the present disclosure according to an embodiment of the present disclosure
  • 12A-12C schematically show a schematic diagram of the interface before and after the removal of indirect light shadows in a model according to an embodiment of the present disclosure
  • Fig. 13 schematically shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure
  • Fig. 14 schematically shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure
  • FIG. 15 shows a schematic structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure.
  • Fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.
  • the system architecture 100 includes a terminal device 101, a network 102, and a server 103.
  • the terminal device 101 can be any electronic device with a display screen and a shooting unit, such as a tablet computer, a notebook computer, a desktop computer, and a smart phone. Electronic devices such as mobile phones, cameras, and video cameras; the network 102 is used to provide a communication link between the terminal device 101 and the server 103.
  • the network 102 may include various connection types, such as wired communication links, wireless communication links, etc. ;
  • the server 103 can be an independent server or a server cluster composed of multiple servers.
  • terminal device 101 the network 102, and the server 103 in FIG. 1 are merely illustrative. According to actual needs, there can be any number of terminal devices 101, networks 102, and servers 103 in the system architecture.
  • the user uses the photographing unit of the terminal device 101 to take multi-angle photographs of objects in real space, and then transfer the photographed multiple photographs to a device equipped with a model rendering engine (such as server 103) Perform three-dimensional modeling.
  • a model rendering engine such as server 103
  • the pixel color information of multiple photos is used to restore the diffuse reflection color information of the object. Due to the effect of direct light and/or indirect light, there are usually shadows in the photo. It is necessary to adjust the light type and angle. If the color information in the photo is used directly, there will be shadows in the final generated model where there should be no shadows, which will cause the model to be distorted and cannot be used in scenes with arbitrary lighting conditions.
  • the terminal device 101 can process the rigid model corresponding to the model through the rendering engine, and export the texture map after transcribing the texture map.
  • the texture map includes the diffuse Reflection map, position map, normal map, shadow map and ambient light occlusion map. After the texture map is obtained, the texture map can be sent to the server 103 via the network 102, so that the server 103 restores the color of the shadow texels in the diffuse map.
  • the server 103 After the server 103 receives the texture map, it can determine the restored color information according to the color information corresponding to the shadow texels in the diffuse reflection map and the average brightness difference, and update the color information of the shadow texels in the diffuse reflection map to the restored color Information can remove the shadows in the image. Further, the terminal device 101 may also send photos to the server 103 via the network 102, and perform three-dimensional modeling based on the photos through the rendering engine carried in the server 103, and obtain texture maps formed during the modeling process.
  • the average brightness difference corresponding to the shadow texel can be obtained by different methods, and the shadow texel is restored according to the average brightness difference and the color information of the shadow texel.
  • One method is to collect the light color information and dark color information of each texel in the diffuse reflection map, and form the light color lookup table and the dark color lookup table according to the light color information and the dark color information, and further , According to the transparency value of each pixel in the bright-side color look-up table and dark-side color look-up table, the bright-side color information in the bright-side color look-up table can be converted into the light-side average color, and the dark side in the dark-side color look-up table The color information is converted into the average color of the dark side to form the average light color lookup table and the average dark color lookup table; at the same time, the light side color lookup table and the dark side color lookup table can be down-sampled at multiple levels to obtain the bright side average.
  • Color map atlas and dark surface average color map atlas are respectively searched for the non-zero light surface average corresponding to the spatial coordinate information Color information and dark surface average color information.
  • the corresponding average color information is not queried, you can query in the bright surface average color map set and/or dark surface average color map set according to the spatial coordinate information, for each shadow texel There must be at least one light surface average color information and dark surface average color information corresponding to it.
  • the restored color information can be determined, and the color of the shadow texels can be restored based on the restored color information.
  • Another method to obtain the average brightness difference is to extract the edges in the shadow map through the edge extraction algorithm, and use the edge point as the center point to determine the bright area and the dark area in the diffuse map; then collect the bright part of the bright area The average color and the average color of the dark part of the dark area, and the average brightness difference is determined according to the average color of the bright part and the average color of the dark part.
  • the average brightness difference can be stored in a four-dimensional lookup table to form an average color lookup table; further, the average color can be The lookup table performs multiple downsampling to obtain the average color map set; finally, according to the spatial position coordinates of the shadow texel in the diffuse reflection map and the brightness information of the dark part, query the corresponding shadow texel in the average color lookup table or the average color map set.
  • the average brightness difference of, and the restored color information is determined according to the average brightness difference and the color information of the shadow texel, and then the color of the shadow texel is restored according to the restored color information.
  • the technical solution of the embodiment of the application improves the efficiency and quality of color reproduction; on the other hand, it does not affect the color of the texels in the non-shaded area of the texture map, and avoids problems such as loss of texture details, hue changes, color overflow, and color blocks. , To ensure that the model generated according to the texture can be applied to the scene of any lighting conditions, further improving the picture effect and user experience.
  • the image processing method provided by the embodiment of the present application can be executed by a server, and accordingly, the image processing device can be set in the server. However, in other embodiments of the present application, the image processing method provided in the embodiments of the present application may also be executed by a terminal device.
  • FIGS 2A-2B show a schematic diagram of the model texture obtained by processing the model texture.
  • Figure 2A shows the original model texture without direct light illumination. It can be seen that there are shadows on the backlit surface of the object (as shown in the figure).
  • Figure 2B shows the texture of the model after using the Agisoft Texture Delighting tool to remove direct light illumination.
  • the embodiments of the present application provide an image processing method that can remove the shadows in the diffuse reflection map and ensure that the model generated based on the diffuse reflection map after removing the shadows can be used in Use it in a scene with any lighting conditions.
  • FIG. 3 schematically shows a flowchart of an image processing method according to an embodiment of the present disclosure.
  • the method may be executed by a server, and the server is the server 103 shown in FIG. 1.
  • the image processing method includes at least step S310 to step S330, which are described in detail as follows:
  • step S310 a diffuse reflection map is acquired, and shadow texels located in a shadow area in the diffuse reflection map are acquired.
  • a 3D model modeling technology based on photogrammetry can be used to perform three-dimensional modeling based on the multiple photos.
  • the 3D model information with vertex color is the point cloud information containing color information.
  • graphics software can be used to map texture maps to generate texture maps; specifically, the 3D graphics interface of any programmable pipeline can be called to render the model on the rendering target in a standard rasterization method, and the model can be rendered through the vertex shader
  • the vertex coordinates of is set to the UV coordinates of the texture map to realize the transliteration of the texture map; the UV coordinates are the abbreviation of the UV texture map coordinates.
  • All image files are a two-dimensional plane, the horizontal direction is U, and the vertical direction is V, through this flat, two-dimensional UV coordinate system can locate any pixel on the image.
  • Texture maps are maps used to express information such as diffuse reflections, normals, and object spatial positions on the surface of the model. Each map is mapped and stored in the form of the model's UV coordinates.
  • the texture maps include diffuse maps and position maps. , Normal map, shadow map, and further, can also include ambient light occlusion map and environment normal map.
  • transcribing textures take diffuse textures, position maps, and normal maps as examples.
  • the GPU After calculating color information, object spatial position, and normal information in the vertex shader, the GPU performs linear interpolation during rasterization and rendering Go to the rendering target of the corresponding diffuse map, position map, and normal map, and export the diffuse map, position map, and normal map through graphics software for use.
  • the direction of the direct light can be specified in the graphics software in two ways.
  • the first way can be by specifying the origin and projection point on the model, the graphics software can calculate the direction of the direct light according to the information of the origin and projection point; the second way can be to measure the longitude, latitude and time of shooting with an instrument to make the graphics
  • the software can calculate the direct light direction according to the latitude, longitude and time.
  • the space coordinates of the object where the pixel is located can be obtained in the pixel shader, and the coordinates corresponding to the depth information bitmap can be obtained by linear transformation processing on the space coordinates, and the information in the depth information bitmap can be obtained and
  • the shadow map can be obtained by comparing with the preset depth value.
  • the shadow map is used to express whether the surface of the model can be directly illuminated by a specific light source.
  • the shadow map can use different values to determine whether the texels are shadow texels and non-shadow texels.
  • a shadow map can be a map composed of 0 and 1, where 0 represents non-shadow texels and 1 represents shadow texels. Similar to diffuse maps, position maps, and normal maps, shadow maps are also transcribed and stored in UV coordinates.
  • the three-dimensional model modeling technology based on photogrammetry when used for three-dimensional modeling, whether it is direct light generated by a light source with a fixed light emission direction such as the sun and a lighting lamp, or a non-fixed light source Due to the effect of indirect light emitted by the light source, there will be shadows in the photographs taken, and the shadows produced by direct light are more common.
  • the shadow removal is achieved by restoring the color of the shadow texels located in the shadow area in the diffuse reflection map.
  • the position map, normal map, and shadow map corresponding to the diffuse map can be acquired.
  • the shadow texels in the shadow area of the diffuse map need to be determined.
  • the color can be restored in a targeted manner to remove the shadows in the image.
  • Figure 4 shows a schematic diagram of the process of determining shadow texels.
  • step S401 the first UV coordinates of the texels corresponding to the shadow area in the shadow map are obtained; in step S402, the first UV coordinates are Match with the UV coordinates of the texels in the diffuse reflection map to obtain the texels corresponding to the first UV coordinates in the diffuse reflection map as shadow texels.
  • step S401 take the shadow map with 0/1 representing the shadow value in the above embodiment as an example, where the texels with the shadow value of 1 are the texels located in the shadow area, and these texels are obtained from the shadow map
  • the UV coordinate of is the first UV coordinate. Since the diffuse map and shadow map are mapped and stored by UV coordinates, the texels corresponding to the same UV coordinate in the diffuse map and shadow map correspond to the same face in the model, and then after the first UV coordinate is obtained, you can Find the texel corresponding to the first UV coordinate in the diffuse reflection map according to the first UV coordinate, and the texel is the shadow texel located in the shadow area in the diffuse reflection map.
  • step S320 a query is performed in the average color look-up table according to the spatial coordinate information of the shadow texel, and the average brightness difference corresponding to the shadow texel is determined according to the query result.
  • the spatial coordinate information of each shadow texel can be obtained, and the corresponding corresponding color look-up table can be searched according to the spatial coordinate information of the shadow texel. Average color information, and determine the average brightness difference corresponding to the shadow texel according to the query result, and then restore the color of the shadow texel according to the average brightness difference.
  • two different methods can be used to remove shadows caused by direct light illumination.
  • the specific content of the average color look-up table involved is also different.
  • the average color lookup table involved includes a bright surface average color lookup table and a dark surface average color lookup table. Next, how to obtain the bright surface average color lookup table and the dark surface The surface average color lookup table will explain in detail.
  • two lookup tables can be set: a bright surface lookup table and a dark surface lookup table.
  • the bright surface lookup table is used to store the spatial coordinates of the non-shadow texels in the diffuse reflection map.
  • Information and color information the dark surface lookup table is used to store the spatial coordinate information and color information of the shadow texels in the diffuse reflection map. It is worth noting that the lookup table (Lookup table, LUT for short) can be queried using address information A data structure, by entering an address to look up the table, you can find the content corresponding to the address.
  • FIG. 5 shows a schematic diagram of the process of obtaining the bright surface average color lookup table and the dark surface average color lookup table. As shown in FIG. 5, the process includes at least steps S501-S502, specifically:
  • step S501 each texel in the diffuse reflection map is traversed, and the color information of each texel is added to the bright surface lookup table and the dark surface lookup table according to the shadow information corresponding to each texel in the shadow map to form Bright side color lookup table and dark side color lookup table.
  • each texel in the diffuse reflection map in order to store the color information of all texels in the diffuse reflection map in the bright surface lookup table and the dark surface lookup table, respectively, each texel in the diffuse reflection map can be traversed and its The color information is accumulated to the pixel bit corresponding to the spatial coordinate information of the texel in the corresponding look-up table.
  • both the bright surface lookup table and the dark surface lookup table are five-dimensional lookup tables, and the five dimensions include the three coordinate dimensions corresponding to the texels in the diffuse reflection map in the spatial rectangular coordinate system.
  • the position information and normal information of the texel in the spatial rectangular coordinate system can be in the vertex shader It is calculated and determined according to the vertex coordinates of each triangle face of the model, and is transferred to the position map and the normal map when the map is transferred.
  • the spatial coordinate information of each texel can be obtained according to the UV coordinates of each texel.
  • the spatial coordinate information is the same as the dimension of the lookup table, including position coordinates (X, Y, Z) and normal coordinates (longitude, latitude); at the same time, the shadow information of each texel can be obtained according to the UV coordinates of each texel;
  • the shadow information of the texel determines whether the look-up table storing the color information of the texel is a bright-side look-up table or a dark-side look-up table, and determines the pixel location for storing the color information in the determined look-up table according to the spatial coordinate information of the texel; finally; Just add the color information of the texel to the corresponding pixel position.
  • the color information stored in the lookup table is a color vector containing four channels of R, G, B, and A
  • the transparency value of the alpha channel is 0, it means that the texel is not mapped on the model.
  • the value is 1, it means that the texel is mapped on the model.
  • the alpha channel value of the collected texel should be 1, that is to say, the
  • Fig. 6 shows a schematic diagram of the process of obtaining a bright-side color look-up table and a dark-side color look-up table.
  • the target texel is determined according to the transparency value in the color information of each texel, and the The second UV coordinates of the target texel; in step S602, the target shadow information corresponding to the target texel is determined in the shadow map according to the second UV coordinates, and the target shadow information is in the bright surface lookup table and the dark surface lookup table according to the target shadow information Determine the target lookup table; in step S603, determine the target location information and target normal information in the location map and the normal map respectively according to the second UV coordinates; in step S604, according to the target location information and target normal information in the target The target pixel is determined in the lookup table, and the color information of the target texel is accumulated on the target pixel.
  • step S601 the target texel is the texel with a transparency value of 1 in the color information; in step S602, according to the second UV coordinates, the shadow map can be queried to obtain the target shadow information of the target texel.
  • the target lookup table for storing the color information of the texel can be determined according to the target shadow information, that is, when the target shadow information is that the target texel is a shadow texel, the dark surface lookup table is The target lookup table; when the target shadow information is that the target texel is a non-shadow texel, the bright surface lookup table is used as the target lookup table; in step S603, similar to the above-mentioned embodiment, the target texel can be set according to the second UV coordinates The position map and normal map determine the corresponding target position information and target normal information.
  • the target position information is the position coordinate of the target texel in the spatial rectangular coordinate system
  • the target normal information is the corresponding target texel.
  • the latitude and longitude of the normal; in step S604, the pixel corresponding to it can be determined in the bright or dark look-up table according to the target position information and the target normal information, and then the color information of the target texel can be accumulated to the pixel superior.
  • the color information is still accumulated in the form of a (R, G, B, A) vector. It is worth noting that the color value of each pixel in the bright side look-up table and the dark side look-up table are all zero at the beginning.
  • step S502 according to the transparency value of the pixel in the bright-side color look-up table, the color information in the bright-side color look-up table is converted to obtain the bright-side average color look-up table; according to the transparency value of the pixel in the dark-side color look-up table , Convert the color information in the dark surface color lookup table to obtain the dark surface average color lookup table.
  • the color information of each texel in the diffuse reflection map to the bright surface lookup table and the dark surface lookup table to form the bright surface color lookup table and the dark surface color lookup table
  • they can be based on
  • the transparency value of the color information in the bright surface color lookup table and the transparency value of the color information in the dark surface color lookup table are converted to the color information of the corresponding lookup table to form the bright surface average color lookup table and the dark surface average color lookup table.
  • FIG. 7 shows a schematic diagram of the process of constructing a bright-side average color look-up table and a dark-side average color look-up table.
  • step S701 the first sum of the transparency values of all pixels in the bright-side color look-up table is obtained.
  • the second sum value of the transparency values of all pixels in the dark surface color look-up table is obtained;
  • step S702 the color information of each pixel in the bright surface color look-up table is divided by the first sum value to obtain the bright surface Average color lookup table:
  • step S703 the color information of each pixel in the dark surface color lookup table is divided by the second sum value to obtain the dark surface average color lookup table.
  • the bright surface average color map set includes multiple bright surface average color maps of different sizes
  • the dark surface average color map set includes multiple The average color map of the dark side of different sizes.
  • the color information of texel A will be stored in the dark surface color lookup table, and the color information of texel B will be stored in bright In the surface color look-up table, that is, if the pixel corresponding to the spatial coordinate information of texel A in the bright surface color look-up table does not have color information, the dark surface color look-up table corresponds to the spatial coordinate information of texel B There is no color information in the pixel, but in order to obtain the average color of the bright surface corresponding to texel A or the average color of the dark surface corresponding to texel B, it is necessary to downsample the bright surface color lookup table and the dark surface color lookup table When down-sampling, the color information of a larger area will be processed, and down-sampling is essentially the process of averaging the color information, so by down-sampling the bright surface
  • the dark surface color look-up table can be down-sampled to compare it with the non-shaded texel B
  • the color information of adjacent shadow texels is counted in, and the average color of the dark surface corresponding to the spatial coordinate information of the non-shaded texel B is obtained.
  • the down-sampling process can also be understood as a convolution process.
  • a five-dimensional convolution check with a side length of 2 pixels can be used to compare the average color of the N-1 level.
  • the color information of each pixel within the coverage of the convolution kernel can be directly added as the new color information .
  • the corresponding pattern can be obtained from the spatial coordinate information of the texel.
  • the average color of the bright side and the average color of the dark side corresponding to the element Since the purpose of this application is to remove the shadows in the diffuse reflection map, only the shadow texels in the diffuse reflection map need to be color restored.
  • the space coordinate information of the shadow texel can be obtained first; then, according to the space coordinate information of the shadow texel, look up and The spatial coordinate information of the shadow texel corresponds to the dark surface average color information whose value is not zero; and according to the spatial coordinate information of the shadow texel, the light surface average color map set can be from the low-level bright surface average color map to the high-level bright surface average
  • the color map is queried step by step to obtain the bright surface average color information corresponding to the spatial coordinate information and the value is not zero, and then the average brightness difference is determined according to the bright surface average color information and the dark surface average color information.
  • the dark surface average color information when the corresponding dark surface average color information is determined in the dark surface average color lookup table according to the spatial coordinate information of the shadow texels, the dark surface average color information can be subjected to multi-dimensional linear interpolation processing, To reduce the color gradient of the image and make the color smoother.
  • multi-dimensional linear interpolation processing on the dark surface average color lookup table, the pixels corresponding to the spatial coordinate information of the shadow texels in the dark surface average color lookup table can be used as the original sampling point, and sampling is performed in each dimension , To obtain multiple sampling points including the original sampling point.
  • the dark surface average color look-up table is a five-dimensional look-up table
  • 32 (25) sampling points can be obtained; then according to multiple sampling points in five dimensions Interpolate in turn, calculate the average color shift of the interpolation point, and use the volume of the hypercube with the average color shift as the weight, where the total volume of the hypercube is 1; finally, the average color shift is based on the volume of the hypercube
  • the quantity weighted summation can obtain the dark surface average color information obtained by multi-dimensional linear interpolation processing.
  • the average brightness difference is determined according to the average color information of the bright surface and the average color information of the dark surface
  • the average color information of the bright surface and the average color information of the dark surface contain (R, G, B, A )
  • the color information of the four-channel color value and only need to restore the color value of the R, G, and B three channels when the color is restored, so the bright surface RGB vector can be obtained from the bright surface average color information, and at the same time from the dark surface
  • the dark side RGB vector is obtained from the average color information, and then the bright side RGB vector is divided by the dark side RGB vector to obtain the average brightness difference.
  • the average color lookup table involved is a lookup table formed according to the average brightness difference. Next, how to obtain the average color lookup table will be described in detail.
  • the shadow map and the diffuse reflection map may be processed to obtain an average color lookup table.
  • Fig. 8 shows a schematic diagram of the process of forming an average color look-up table. As shown in Fig. 8, the process at least includes steps S801-S803, specifically:
  • step S801 the edge texels in the shadow map are extracted, and the center texels are determined in the diffuse reflection map according to the edge texels.
  • the shadow map can be edge extracted based on the edge extraction algorithm. Specifically, when the edge of the shadow map is extracted, the shadow map can be convolved by the horizontal direction template and the vertical direction template. To get edge texels. Among them, the horizontal direction template is used to detect horizontal edges based on the texels in the shadow map, and the vertical direction template is used to detect vertical edges based on the texels in the shadow map. After obtaining the horizontal edge detection results and vertical edge detection corresponding to the same texel After the result, the detection value can be obtained by taking the maximum of the two or adding the two together. Finally, the detection value is compared with the preset threshold. If the detection value is greater than or equal to the preset threshold, the texel is Edge texels.
  • the UV coordinates of the edge texels can be obtained as the third UV coordinates, since both the diffuse map and the shadow map are mapped and stored through the UV coordinates Therefore, according to the third UV coordinate, the texel corresponding to the third UV coordinate can be determined in the diffuse reflection map, and the texel is used as the center texel for subsequent average color collection.
  • step S802 the target texture area is determined according to the central texel and the preset range, and the bright part area and the dark part area in the target texture area are determined according to the shadow map.
  • an area within a preset range centered on the central texel can be determined in the diffuse reflection map as the target map area.
  • the preset range may be set according to actual needs, for example, it may be set to 64 ⁇ 64, 80 ⁇ 80, etc., which is not specifically limited in the embodiment of the present application.
  • the bright area and the dark area can be determined in the target map area according to the position information of the shadow texels and non-shadow texels in the shadow map.
  • the fourth UV of each texel in the target map area can be obtained first. Coordinate, and then determine the shadow information of each texel in the target texture area according to the fourth UV coordinate of each texel and the shadow map.
  • the target texture area is the texel in the bright area
  • the shadow information of the texel corresponding to the UV coordinate is that the texel is a shadow texel
  • the target texture area is the texel in the dark area, that is to say, the area formed by the texels with the shadow information as non-shadow texels and adjacent spatial coordinate information as the bright area
  • the shadow information is regarded as the dark area.
  • the mapping of texels in the texture map is not necessarily continuous, there may be close to the texture, but the mapped surface is away from the Is very far away, so in the embodiment of this application, we can also judge whether the position map and normal are continuous according to the position information and normal information of each texel in the target texture area, and filter the discontinuous texels . This can improve the continuity of texels in the bright area and the dark area, and improve the accuracy of the calculation.
  • step S803 calculate the average color of the bright part corresponding to the bright part and the average color of the dark part corresponding to the dark part respectively, determine the average brightness difference according to the average color of the bright part and the average color of the dark part, and add the average brightness difference to the average color Lookup table.
  • the average color of the bright area can be determined according to the color information of each texel in the bright area, and at the same time according to each texel in the dark area. Determine the average color of the dark part from the color information.
  • Figure 9 shows a schematic flow chart of calculating the average color of the bright part and the average color of the dark part.
  • step S902 add the color information of all the texels in the dark part area to obtain the dark part color
  • step S903 divide the light color information by the third sum value to obtain the average color of the light part, and The dark part color information is divided by the fourth sum value to obtain the average dark part color.
  • the bright part RGB vector in the bright part average color and the dark part RGB vector in the dark part average color are obtained, and then the bright part RGB vector and the dark part RGB vector are obtained. Divide the vector to get the average brightness difference.
  • the average brightness difference corresponding to each center texel can be added to the average color lookup table.
  • the average color lookup table is a four-dimensional lookup table, and the four dimensions are the centers of the diffuse reflection map. The spatial position coordinates (X, Y, Z) of the object corresponding to the texel and the average color value of the average color of the dark part corresponding to the central texel.
  • the average brightness difference corresponding to the central texel is accumulated to the average color look-up table, it is based on The spatial position coordinates of the object corresponding to the center texel and the color average value of the corresponding dark part average color are determined in the average color look-up table, and then the average brightness difference corresponding to the center texel is added to the pixel.
  • the color average value of the average color of the dark part is the average value of the color values of the R, G, and B three channels in the average color of the dark part.
  • the average color look-up table can provide data support for the subsequent calculation of the restored color information, but in the average color look-up table, there is not a corresponding average brightness difference corresponding to each shadow texel.
  • Multi-level down-sampling is performed on the average color look-up table to obtain a corresponding average color map set, which includes a plurality of average color maps of different sizes.
  • the process of down-sampling the average color look-up table is the same as the method of down-sampling the bright-side color look-up table and the dark-side color look-up table in the foregoing embodiment, and will not be repeated here.
  • the spatial position of each shadow texel can be obtained according to the position map
  • the color average of the average color of the dark part corresponding to each shadow texel is obtained, and then the average brightness difference corresponding to the shadow texel is found in the average color look-up table according to the space position coordinate and the color average of the average color of the dark part.
  • the average color map is searched from the low-level average color map to the high-level average color map in the average color map set according to the spatial position coordinates and the average color of the dark part, until Find the corresponding average brightness difference, where the size of the low-level average color map is larger than the high-level average color map, and the highest-level average color map can be a 1 ⁇ 1 ⁇ 1 ⁇ 1 matrix.
  • the average color information when querying the average color look-up table based on the spatial position coordinates of the shadow texels and the average color of the average color of the dark part, can be multi-dimensional linear interpolation to improve color smoothness. , Reduce the color gradient of the image. Similar to the method of performing multi-dimensional linear interpolation processing on the dark surface average color look-up table in the foregoing embodiment, when performing multi-dimensional linear interpolation processing on the average color look-up table, the spatial position coordinates of the shadow texels in the average color look-up table may be used. The pixel corresponding to the color average of the average color of the dark part is the original sampling point, and sampling is performed in each dimension to obtain multiple sampling points including the original sampling point.
  • the average color lookup table is a four-dimensional lookup table, then 16 (24) sampling points can be obtained; then according to multiple sampling points in the four dimensions, the interpolation is carried out in sequence, the average brightness difference offset of the interpolation point is calculated, and the volume of the hypercube corresponding to the average brightness difference offset As a weight, the total volume of the hypercube is 1; finally, the average color shift is weighted and summed according to the volume of the hypercube to obtain the average brightness difference information obtained by multi-dimensional linear interpolation.
  • step S330 the restored color information is determined according to the average brightness difference and the color information corresponding to the shadow texel, and the color information of the shadow texel is restored according to the restored color information.
  • the average brightness difference can be determined according to the average brightness difference.
  • the color information of the shadow texels for color restoration can be multiplied by the RGB vector in the color information of the shadow texel to obtain the restored color information, and the color restoration of the shadow texel can be realized based on the restored color information.
  • the boundary between the shadow texel and the non-shadow texel in the diffuse reflection map that is, the boundary between light and dark
  • the blurring processing may be performed by methods such as Gaussian filtering, median filtering, bilateral filtering, and mean filtering.
  • the average brightness difference can be determined according to the average brightness information of the light surface and the average color information of the dark surface corresponding to the shadow texel, and the color information of the shadow texel is restored according to the average brightness difference, Or calculate the average brightness difference based on the average brightness of the center texels and the average colors of the dark regions corresponding to the edge texels in the shadow map in the diffuse map, and restore the color information of the shadow texels based on the average brightness difference.
  • It can effectively remove the shadows caused by direct light, while avoiding the loss of texture details, hue changes, color overflow and color blocks in other places in the image, making the model's visual effect better, and it can be used in scenes with any lighting conditions For example, it can be widely used in real-time rendering applications such as games and 3D interactive experiences.
  • Fig. 11 shows a schematic diagram of an interface for removing shadows in a model according to the image processing method of the present disclosure. As shown in Fig. 11, compared with Figs. Higher quality.
  • the first method for removing shadows caused by direct light illumination requires collecting the average color of each texel in the diffuse reflection map, and then according to the average color information of the bright surface corresponding to the shadow texel and The dark surface average color information determines the restoration of the color information, and restores the color of the shadow texels according to the restored color information.
  • the first shadow removal method has a better effect on the shadow removal in the diffuse map with less color
  • the second method The method of removing the shadows caused by direct light illumination is to collect the average color of the bright area and the dark area corresponding to the central texel in the diffuse reflection map, and then determine the restoration according to the average color of the bright area and the average color of the dark area Color information, and restore the color of the shadow texels according to the restored color information, so the second shadow removal method has a better effect on the shadow removal in the diffuse map with more colors.
  • direct light illumination can produce shadows
  • indirect light illumination can also produce shadows
  • various textures corresponding to the model can be exported through the rendering engine.
  • the light source in the photo is ambient light
  • it can also export ambient light occlusion map (AO mapping) and environment normal map (Bent normal mapping).
  • AO mapping ambient light occlusion map
  • Bent normal mapping environment normal map
  • the surface normal vector is a necessary parameter of the geometric model
  • the environment normal is the new vector obtained after modifying the original normal. It points to an average direction of the current pixel that is not occluded by other objects (or geometric voxels). That is, the main direction of the incoming light.
  • the environment normal can be used for two purposes: to change the original normal, and to record the main direction of light, so that when sampling the environment map, the sampled light can be calculated from a more optimized direction; Instead of the original normals for normal rendering, you can get a soft shadow effect similar to ambient occlusion to a certain extent with only the normals.
  • FIG. 10 shows a schematic flow chart of removing shadows caused by non-direct light illumination. As shown in FIG. 10, the flow at least includes steps S1010-S1040, specifically:
  • step S1010 a first average color lookup table is constructed according to the normal information of each texel in the diffuse reflection map.
  • a lookup table can be constructed using the latitude and longitude of the normal as the coordinates, and then the color information of each texel in the diffuse reflection map is accumulated to the pixel corresponding to each texel in the lookup table to form the first A color lookup table. Specifically, first, the fifth UV coordinates of each texel of the diffuse reflection map can be obtained. Since both the diffuse reflection map and the normal map are stored through UV coordinate mapping, the corresponding fifth UV coordinates can be determined in the normal map according to the fifth UV coordinates.
  • the target pixel can be determined in the first color look-up table according to the longitude and latitude in the obtained normal information; then it is judged whether the transparency value in the color information of the texel with the fifth UV coordinate is 1, when When the transparency value is 1, the color information of the texel is accumulated on the target pixel, when the transparency value is not 1, the color information of the texel is not processed; finally the first color search containing the color information of the texel is collected The transparency value of each pixel in the table, the transparency value of each pixel is added to obtain the fifth sum value, and the color information of each pixel in the first color lookup table is divided by the fifth sum value to obtain the first average color lookup surface.
  • step S1020 the overall average bright area color information corresponding to the diffuse reflection map is determined according to the first average color lookup table, and the intermediate result texture is obtained according to the first average color lookup table.
  • each pixel in the first average color lookup table can be traversed to calculate the degree to which each pixel is close to 0.7 brightness, and then each pixel can be close to 0.7 brightness.
  • the degree is used as the weight, and the pixels in the first average color look-up table are processed to obtain the overall average bright area color information corresponding to the diffuse reflection map.
  • ⁇ is the weight
  • Abs is the absolute value
  • Lumin(RGB) is the brightness value of the RGB vector of each pixel in the first average color look-up table.
  • the brightness value of the RGB vector can be obtained by converting the RGB color space to the YIQ color space, where Y is the brightness of the color, that is, the brightness.
  • Y is the brightness of the color
  • the color information of each pixel can be multiplied with its corresponding weight to obtain the weighted color information;
  • the weighted color information corresponding to all pixels is added and averaged to obtain the overall average bright area color information.
  • the overall average bright area color information can be calculated according to formula (2), which is specifically:
  • Q is the overall average color information of the bright area
  • i is any pixel with color information in the first average color lookup table
  • m is the number of pixels with color information in the first average color lookup table
  • ⁇ is the same as each pixel The corresponding weight.
  • the weight corresponding to each pixel is close to 0.7 brightness of each pixel.
  • the weight value can make the weight of the color value of the desired brightness higher, so as to achieve the goal of approaching the desired average brightness as a whole. In turn, the restoration of shadows caused by indirect light illumination is realized.
  • each texel in the diffuse reflection map it is also possible to traverse each texel in the diffuse reflection map, and determine the intermediate result texture according to the normal information of each texel and the first average color look-up table.
  • the intermediate result texture is composed of the diffuse reflection map.
  • Each texel corresponds to the processed average color information. Since only the average color information of the texels whose transparency value is 1 is stored in the first average color look-up table, there is no corresponding average color information in the first average color look-up table for other texels whose transparency value is not 1.
  • the first average color look-up table needs to be down-sampled multiple times to obtain the first average color map set.
  • the first average color map set contains multiple first average color maps of different sizes, and the corresponding diffuse reflection can be guaranteed by downsampling.
  • Each texel in the texture map has a corresponding first average color.
  • the intermediate result map When determining the intermediate result map, first obtain the sixth UV coordinates of each texel in the diffuse reflection map, and then find the corresponding normal information in the normal map according to the sixth UV coordinates, and then use the normal information in the first The corresponding average color information is obtained in the average color look-up table. If the average color information corresponding to the normal information cannot be obtained in the first average color look-up table, the normal information will be ranked first in the first average color map set according to the normal information. The average color map searches the advanced first average color map until the average color information corresponding to the normal information is obtained; finally, the average color information corresponding to each texel can be processed according to the obtained average color information to obtain the processed average color information .
  • the first RGB vector in the average color information corresponding to the texel and the second RGB vector in the color information corresponding to the texel in the diffuse map can be obtained first , And then calculate according to formula (3) to obtain the processed average color information, and then form an intermediate result map according to the processed average color information.
  • the formula (3) is specifically:
  • P is the processed color information
  • V1 is the first RGB vector
  • V2 is the second RGB vector.
  • step S1030 a second average color lookup table is constructed according to the intermediate result map and the ambient light occlusion map.
  • a look-up table can be formed using ambient light shading information and environmental normal information as coordinates, and the color information of each texel in the intermediate result map is weighted and added to the look-up table to form the first Two average color lookup table.
  • the ambient light shading information and the ambient normal information are both floating-point numbers of a channel value. That is to say, the second average color lookup table is also a two-dimensional lookup table.
  • the first average color lookup table and the second average color lookup table may have the same size, for example, all are set to 64 ⁇ 32, etc., of course, they may also be set to different sizes, which is not specifically limited in the embodiment of the present application.
  • each texel in the intermediate result map is traversed, and the corresponding ambient light occlusion value in the intermediate result map is obtained according to the UV coordinates of each texel, and then The ambient light shading value is used as the weight, and the intermediate result corresponding to the corresponding texel is weighted.
  • the transparency value in the weighted intermediate result corresponding to each texel can be obtained, and then the transparency values corresponding to all texels are added to obtain the first Six sum values, and then divide the weighted intermediate result corresponding to each texel by the sixth sum value to obtain the average color information corresponding to each texel; finally accumulate the average color information corresponding to each texel to
  • the second average color lookup table can be obtained from the corresponding pixels in the lookup table.
  • the second average color look-up table may also be down-sampled multiple times to obtain a second average color map set.
  • the second average color map set includes a plurality of second average colors of different sizes.
  • the method of down-sampling the second average color look-up table is the same as the method of down-sampling the bright-side color look-up table or the dark-side color look-up table, and will not be repeated here.
  • step S1040 each texel in the intermediate result map is traversed, the restored color information is determined according to the color information of each texel in the second average color look-up table, and the color of each texel is restored according to the restored color information.
  • the texels in the intermediate result map are traversed, and the UV coordinates of each intermediate result texel in the intermediate result map are obtained, and the UV coordinates are performed in the ambient light shading map and the environment normal map according to the UV coordinates Query to obtain the ambient light shading information and ambient normal information corresponding to each intermediate result texel. Then query the second average color look-up table according to the ambient light shading information and environment normal information of each intermediate result texel to obtain the corresponding second average color information.
  • the ambient light occlusion information and the environment normal information can be searched from the low-level second average color map to the advanced second average color map in the second average color map set, Until the second average color information corresponding to the intermediate result texel is obtained.
  • the restored color after obtaining the second average color information corresponding to the intermediate result texel, the restored color can be determined according to the second average color information, the color information of the intermediate result texel, and the overall average bright area color information information.
  • the restored color information first obtain the RGB vector in the color information of the intermediate result texel, and then calculate the restored color information according to formula (4).
  • the specific formula (4) is as follows:
  • C is the restored color information
  • L is the second average color information
  • V3 is the RGB vector in the color information of the intermediate result texel
  • Q is the overall average bright area color information.
  • the intermediate result texture is obtained by processing the texels in the diffuse reflection map
  • the intermediate result texels correspond to the texels in the diffuse reflection map, so the intermediate result texels are obtained
  • the restored color information corresponding to each texel can be used to replace the color information of each texel in the diffuse reflection map, so as to remove the shadows caused by the indirect light in the diffuse reflection map.
  • Figures 12A-12C show the schematic diagrams of the interface before and after the indirect light shadows in the model are removed.
  • the texture of the model after the De-Lighting tool removes the shadow, in which the color of the moss on the stone is distorted; and after the shadow in the model is removed by the image processing method in the embodiment of this application, the color of the moss is not distorted, and the image quality is better. High, as shown in Figure 12C.
  • the image processing can be performed only according to the method of removing the shadow caused by the direct light illumination. Can.
  • the image processing method in the embodiment of the present application uses two methods to remove the shadows caused by direct light illumination in the diffuse reflection map, and on the other hand, uses one method to remove the shadows caused by the non-direct light illumination in the diffuse reflection map. Both effectively avoid the problems of loss of texture details, hue changes, color overflow, and color blocks that exist when using known tools to remove shadows in the prior art, and improve the quality of diffuse textures, thereby making diffuse textures covered
  • the model can be used in scenes with any lighting conditions, and will not cause problems such as poor image quality and low user experience due to the existence of shadows.
  • Fig. 13 schematically shows a block diagram of an image processing apparatus according to an embodiment of the present application.
  • the device can be used to execute the corresponding steps in the method provided in the embodiments of the present application.
  • the image processing device 1300 includes: an acquisition module 1301, a calculation module 1302, and a restoration module 1303.
  • the obtaining module 1301 is used to obtain the diffuse reflection map and the shadow texels in the shadow area in the diffuse reflection map;
  • the calculation module 1302 is used to look up the average color in the average color according to the spatial coordinate information of the shadow texels And determine the average brightness difference corresponding to the shadow texel according to the query result;
  • the restoration module 1303 is configured to determine and restore the color information according to the average brightness difference and the color information corresponding to the shadow texel, and The color information of the shadow texel is restored according to the restored color information.
  • the acquiring module 1301 is further configured to acquire a position map, a normal map, and a shadow map corresponding to the diffuse reflection map.
  • the acquisition module 1301 is specifically configured to: acquire the first UV coordinates of the texels corresponding to the shadow area in the shadow map; and compare the first UV coordinates with the diffuse reflection map The UV coordinates of the texels are matched to obtain the texels matching the first UV coordinates in the diffuse reflection map as the shadow texels.
  • the average color lookup table includes a bright surface average color lookup table and a dark surface average color lookup table;
  • the image processing device 1300 further includes: a color lookup table forming module for obtaining In the diffuse reflection map before the shadow texels in the shadow area, each texel in the diffuse reflection map is traversed, and according to the shadow information corresponding to each texel in the shadow map, the The color information is accumulated into the light-side look-up table and the dark-side look-up table to form the light-side color look-up table and the dark-side color look-up table; the average color look-up table forming module is used to look up the transparency of the pixels in the bright-side color table.
  • the color information in the bright surface color lookup table is converted to obtain the bright surface average color lookup table; according to the transparency value of the pixel in the dark surface color lookup table, the dark surface color lookup table The color information in is converted to obtain the dark surface average color lookup table.
  • the bright surface look-up table and the dark surface look-up table are both five-dimensional look-up tables, and the five-dimensional look-up table includes the texels in the diffuse reflection map in the spatial rectangular coordinate system. The three coordinate dimensions in and the longitude and latitude of the normal corresponding to the texel.
  • the average color look-up table forming module is specifically configured to: obtain the first sum value of the transparency values of all pixels in the bright-side color look-up table, and at the same time obtain the dark-side color look-up table The second sum value of the transparency values of all pixels in the bright surface; divide the color information of each pixel in the bright surface color look-up table by the first sum value to obtain the bright surface average color look-up table; The color information of each pixel in the dark surface color lookup table is divided by the second sum value to obtain the dark surface average color lookup table.
  • the color look-up table forming module includes: a first coordinate acquisition unit, configured to determine a target texel according to the transparency value in the color information of each texel, and obtain the target texel The second UV coordinate of the pixel; the shadow information query unit is used to determine the target shadow information corresponding to the target texel in the shadow map according to the second UV coordinate, and according to the target shadow information in the The bright surface lookup table and the dark surface lookup table determine the target lookup table; the position information query unit is configured to determine the target position information and the target in the position map and the normal map according to the second UV coordinates.
  • a color accumulation unit for determining a target pixel in the target lookup table according to the target position information and the target normal information, and accumulating the color information of the target texel to the target pixel To form the bright surface color lookup table and the dark surface color lookup table.
  • the first coordinate acquisition unit is specifically configured to: use a texel with a transparency value of 1 in the color information as the target texel.
  • the target position information is the position coordinates of the target texel in a spatial rectangular coordinate system
  • the target normal information is the longitude and latitude of the normal corresponding to the target texel
  • the target shadow information is used to indicate whether the target texel is the shadow texel.
  • the shadow information query unit is configured to query the shadow map according to the second UV coordinates to obtain the target shadow information, and determine the target shadow information Type; when the target shadow information indicates that the target texel is a shadow texel, use the dark surface lookup table as the target lookup table; when the target shadow information indicates that the target texel is a non-shadow texel , Using the bright surface lookup table as the target lookup table.
  • the image processing device 1300 further includes: a texture atlas generation module, configured to perform multi-level down-sampling on the bright surface color lookup table and the dark surface color lookup table to obtain The bright surface average color map set and the dark surface average color map set, wherein the bright surface average color map set includes a plurality of bright surface average color maps of different sizes, and the dark surface average color map set includes a plurality of dark surface average color maps of different sizes.
  • a texture atlas generation module configured to perform multi-level down-sampling on the bright surface color lookup table and the dark surface color lookup table to obtain The bright surface average color map set and the dark surface average color map set, wherein the bright surface average color map set includes a plurality of bright surface average color maps of different sizes, and the dark surface average color map set includes a plurality of dark surface average color maps of different sizes.
  • Surface average color map configured to perform multi-level down-sampling on the bright surface color lookup table and the dark surface color lookup table to obtain The bright surface average color map set
  • the calculation module 1302 includes: a dark surface average color acquisition unit, configured to determine in the dark surface average color look-up table according to the spatial coordinate information of the shadow texels The dark surface average color information corresponding to the spatial coordinate information and the value is not zero; the bright surface average color acquisition unit is used to, according to the spatial coordinate information, in the bright surface average color map set, from the low-level bright surface average color map to the high-level The bright surface average color map performs a step-by-step query to obtain the bright surface average color information corresponding to the spatial coordinate information and the value is not zero; the average brightness difference calculation unit is used for calculating the average brightness difference according to the bright surface average color information and the The dark surface average color information determines the average brightness difference.
  • a dark surface average color acquisition unit configured to determine in the dark surface average color look-up table according to the spatial coordinate information of the shadow texels The dark surface average color information corresponding to the spatial coordinate information and the value is not zero
  • the bright surface average color acquisition unit is used to, according to the spatial coordinate information, in
  • the calculation module 1302 is further configured to perform multi-dimensional linear interpolation processing on the dark surface average color lookup table when determining the dark surface average color information.
  • the average brightness difference calculation unit is specifically configured to: obtain the bright surface RGB vector in the bright surface average color information and the dark surface RGB vector in the dark surface average color information; The bright surface RGB vector is divided by the dark surface RGB vector to obtain the average brightness difference.
  • the restoration module 1303 is specifically configured to: multiply the average brightness difference and the RGB vector in the color information corresponding to the shadow texel to obtain the restored color information.
  • the image processing device 1300 further includes: a center texel determining module, configured to extract edge texels in the shadow map, and use the edge texels in the diffuse reflection map according to the Determine the central texel in the middle; the bright and dark area determination module is used to determine the target texture area according to the central texel and the preset range, and determine the bright area and the dark area in the target texture area according to the shadow map; The average brightness difference calculation module is used to calculate the average color of the bright part corresponding to the bright part area and the average color of the dark part corresponding to the dark part area respectively, and determine the average brightness according to the average color of the bright part and the average color of the dark part And add the average brightness difference to the average color look-up table.
  • a center texel determining module configured to extract edge texels in the shadow map, and use the edge texels in the diffuse reflection map according to the Determine the central texel in the middle
  • the bright and dark area determination module is used to determine the target texture area according to the
  • the central texel determination module is specifically configured to: perform neighborhood convolution on the shadow map through a horizontal template and a vertical template to obtain the edge texels; The third UV coordinates of the edge texels are searched in the diffuse reflection map to obtain the center texels according to the third UV coordinates.
  • the bright and dark area determining module is specifically configured to: obtain the fourth UV coordinates of each texel in the target texture area, and determine according to the fourth UV coordinates and the shadow map The shadow information corresponding to each texel in the target map area; the area formed by the texels whose shadow information is non-shadow texels and the spatial coordinate information is adjacent is used as the bright area, and the shadow information is shadow texels And the area formed by the texels adjacent to the spatial coordinate information is used as the dark area.
  • the average brightness difference calculation module is specifically configured to: add the color information of all texels in the bright part area to obtain the bright part color information, and at the same time, add the color information of all the texels in the bright part area.
  • the transparency values in the color information of the texels are added to obtain the third sum value; the color information of all texels in the dark area is added to obtain the dark color information, and the color information of all texels in the dark area is added to Add the transparency values of to obtain a fourth sum value; divide the bright part color information and the third sum value to obtain the bright part average color, and combine the dark part color information with the fourth sum value Divide by to obtain the average color of the dark part.
  • the average brightness difference calculation module is specifically configured to: obtain the bright part RGB vector in the bright part average color and the dark part RGB vector in the dark part average color; The RGB vector is divided by the dark part RGB vector to obtain the average brightness difference.
  • the average color lookup table is a four-dimensional lookup table, including the three dimensions of the spatial position coordinates of the center texel and the color average value of the average color of the dark part.
  • the calculation module 1302 includes: an information acquisition unit, configured to determine the spatial position coordinates of the shadow texel according to the position map, and at the same time acquire the average color of the dark part corresponding to the shadow texel The color average value of; an information search unit for querying the corresponding average brightness difference in the average color look-up table according to the spatial position coordinates of the shadow texels and the color average of the average color of the dark part.
  • the information search unit is specifically configured to: query the average color lookup table for whether there is a corresponding average color according to the spatial position coordinates of the shadow texels and the color average value of the average color of the dark part. Brightness difference; if not present, according to the spatial position coordinates of the shadow texels and the average color of the average color of the dark part, query the corresponding average brightness difference in the average color map set corresponding to the average color lookup table, where the average
  • the color map atlas includes a plurality of maps formed by multi-level down-sampling the average color look-up table.
  • the information search unit is further configured to perform multi-dimensional linear interpolation processing on the average color look-up table when querying the average brightness difference.
  • the restoration module 1303 is configured to multiply the average brightness difference and the RGB vector in the color information corresponding to the shadow texel to obtain the restored color information.
  • the image processing device 1300 further includes: a boundary blur module for blurring the boundary between shadow texels and non-shadow texels in the diffuse reflection map.
  • Fig. 14 schematically shows a block diagram of an image processing apparatus according to an embodiment of the present application.
  • the device can be used to execute the corresponding steps in the method provided in the embodiments of the present application.
  • the image processing apparatus 1400 includes: a first lookup table construction module 1401, an intermediate result map acquisition module 1402, a second lookup table construction module 1403, and a color restoration module 1404.
  • the first look-up table construction module 1401 is used to construct a first average color look-up table according to the normal information of each texel in the diffuse reflection map; Describes the overall average bright area color information corresponding to the diffuse reflection map, and obtains the intermediate result texture according to the first average color lookup table; the second lookup table building module is used to construct the second average color lookup based on the intermediate result texture and the ambient light occlusion map Table; color restoration module, used to traverse each texel in the intermediate result map, determine the restored color information according to the color information of each texel in the second average color lookup table, and perform the color of each texel according to the restored color information reduction.
  • FIG. 15 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
  • the computer system 1500 includes a central processing unit (Central Processing Unit, CPU) 1501, which can be loaded into a random system according to a program stored in a read-only memory (Read-Only Memory, ROM) 1502 or from a storage part 1508. Access to the program in the memory (Random Access Memory, RAM) 1503 to execute various appropriate actions and processing to implement the image labeling method described in the foregoing embodiment. In RAM 1503, various programs and data required for system operation are also stored.
  • the CPU 1501, ROM 1502, and RAM 1503 are connected to each other through a bus 1504.
  • An input/output (Input/Output, I/O) interface 1505 is also connected to the bus 1504.
  • the following components are connected to the I/O interface 1505: the input part 1506 including keyboard, mouse, etc.; including the output part 1507 such as cathode ray tube (Cathode Ray Tube, CRT), liquid crystal display (LCD), and speakers 1507 ; A storage part 1508 including a hard disk, etc.; and a communication part 1509 including a network interface card such as a LAN (Local Area Network) card and a modem.
  • the communication section 1509 performs communication processing via a network such as the Internet.
  • the driver 1510 is also connected to the I/O interface 1505 as needed.
  • a removable medium 1511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as required, so that the computer program read therefrom is installed into the storage portion 1508 as required.
  • the process described below with reference to the flowchart can be implemented as a computer software program.
  • the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication part 1509, and/or installed from the removable medium 1511.
  • CPU central processing unit
  • the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • Computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Erasable Programmable Read Only Memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable of the above The combination.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein.
  • This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable medium may send, propagate or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
  • this application also provides a computer-readable medium.
  • the computer-readable medium may be included in the image processing device described in the above-mentioned embodiments; or may exist alone without being integrated into the electronic device. In the device.
  • the foregoing computer-readable medium carries one or more programs, and when the foregoing one or more programs are executed by an electronic device, the electronic device realizes the method described in the foregoing embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Generation (AREA)

Abstract

一种图像处理方法、装置、计算机存储介质及电子设备，涉及图像处理领域。该方法包括：获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素（S310）；根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差（S320）；根据所述平均亮度差和所述阴影纹素对应的颜色信息确定还原颜色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原（S330）。该方法能够有效去除漫反射贴图中的阴影，使得模型能够在任意光照条件的场景中使用。

Description

图像处理方法、装置、计算机存储介质及电子设备
本申请要求于2020年06月10日提交中国专利局、申请号为2020105244737、申请名称为“图像处理方法、装置、计算机存储介质及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，具体涉及图片中的阴影去除技术。
背景技术
在美术设计时，通常会采用基于摄影测量法的三维模型建模技术进行建模，基于摄影测量法的三维模型建模技术需要使用多张照片来进行三维建模，并使用多张照片的像素颜色信息来还原物件漫反射颜色信息，因此生成的模型的颜色贴图中必然带有原始照片中的阴影。
为了在任意光照条件的场景中使用生成的模型，必须在尽可能多的保留原始颜色信息的前提下，把漫反射贴图中的阴影去除。漫反射贴图中的阴影包括直射光作用产生的阴影和环境光作用产生的阴影，目前通常采用诸如Agisoft Texture Delighting等工具去除直射光作用产生的阴影，采用诸如Unity De-Lightingtool等工具去除非直射光作用产生的阴影，但是这些去光照阴影的方案普遍存在贴图细节丢失、色相改变、色彩溢出和出现色块等问题，导致根据去除阴影后的漫反射贴图生成的模型效果不逼真、无法适用于任意光照条件的场景中。
发明内容
本申请实施例提供了一种图像处理方法、图像处理装置、计算机可读存储介质和电子设备，至少在一定程度上可以提高漫反射贴图中阴影的去除效率和去除质量，进而提高根据去除阴影后的漫反射贴图生成的模型的图像质量。
根据本申请实施例的一个方面，提供了一种图像处理方法，由电子设备执行，包括：获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素；根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差；根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原。
根据本申请实施例的一个方面，提供了一种图像处理装置，包括：获取模块，用于获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素；计算模块，用于根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差；还原模块，用于根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜 色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原。
根据本申请实施例的一个方面，提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现如上述实施例所述的图像处理方法。
根据本申请实施例的一个方面，提供了一种电子设备，包括：一个或多个处理器；存储装置，所述存储装置用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器执行如上述实施例中所述的图像处理方法。
根据本申请实施例的一个方面，提供了一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行如上述实施例中所述的图像处理方法。
在本申请的一些实施例所提供的技术方案中，首先获取漫反射贴图，并获取漫反射贴图中位于阴影区域的阴影纹素；接着根据阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与阴影纹素对应的平均亮度差；最后根据平均亮度差和阴影纹素的颜色信息确定还原颜色信息，并根据还原颜色信息对阴影纹素的颜色信息进行还原。本申请技术方案一方面提高了颜色还原的效率和质量；另一方面不影响贴图中非阴影区域纹素的颜色，避免了贴图细节丢失、色相改变、色彩溢出和出现色块等问题，保证了生成的模型可以适用于任意光照条件的场景中，进一步提高了画面效果和用户体验。
附图说明
图1示出了可以应用本公开实施例的技术方案的示例性系统架构的示意图；
图2A-2B示意性示出了相关技术中的对模型纹理进行处理的界面示意图；
图3示意性示出了根据本公开的一个实施例的图像处理方法的流程示意图；
图4示意性示出了根据本公开的一个实施例的确定阴影纹素的流程示意图；
图5示意性示出了根据本公开的一个实施例的获取亮面平均色查找表和暗面平均色查找表的流程示意图；
图6示意性示出了根据本公开的一个实施例的获取亮面颜色查找表和暗面颜色查找表的流程示意图；
图7示意性示出了根据本公开的一个实施例的构建亮面平均色查找表和暗面平均色查找表的流程示意图；
图8示意性示出了根据本公开的一个实施例的形成平均色查找表的流程示意图；
图9示意性示出了根据本公开的一个实施例的计算亮部平均色和暗部平均色的流程示意图；
图10示意性示出了根据本公开的一个实施例的去除非直射光光照产生的阴影的流程示意图；
图11示意性示出了根据本公开的一个实施例的根据本公开的图像处理方法去除模型中的阴影的界面示意图；
图12A-12C示意性示出了根据本公开的一个实施例的模型中的非直射光阴影去除前和去除后的界面示意图；
图13示意性示出了根据本公开的一个实施例的图像处理装置的框图；
图14示意性示出了根据本公开的一个实施例的图像处理装置的框图；
图15示出了适于用来实现本公开实施例的电子设备的计算机系统的结构示意图。
具体实施方式
图1示出了可以应用本申请实施例的技术方案的示例性系统架构的示意图。
如图1所示，系统架构100包括终端设备101、网络102和服务器103，其中，终端设备101可以是任意的具有显示屏幕以及拍摄单元的电子设备，诸如平板电脑、笔记本电脑、台式计算机、智能手机、照相机、摄像机等电子设备；网络102用以在终端设备101和服务器103之间提供通信链路的介质，网络102可以包括各种连接类型，例如有线通信链路、无线通信链路等等；服务器103可以是独立的服务器，也可以是由多个服务器组成的服务器集群。
应该理解，图1中的终端设备101、网络102、服务器103的数目仅仅是示意性的。根据实际需要，该系统架构中可以具有任意数目的终端设备101、网络102、服务器103。
在本申请的一个实施例中，用户通过终端设备101的拍摄单元对实际空间的物体进行多角度拍摄，然后将拍摄得到的多张照片传送至搭载有模型渲染引擎的设备(如服务器103)中进行三维建模。在三维建模的过程中，通过多张照片的像素颜色信息来还原物件漫反射颜色信息，由于直射光和/或非直射光的作用，照片中通常会存在阴影，而在建模时根据实际需要会调整光照类型和角度，如果直接使用照片中的颜色信息，会使最终生成的模型中在不该有阴影的位置存在阴影，导致模型失真并且不能在任意光照条件的场景中使用。
在本申请实施例中，用户通过终端设备101拍摄到照片后，终端设备101可以通过渲染引擎对与模型对应的刚模进行处理，在转写纹理贴图后将纹理贴图导出，该纹理贴图包括漫反射贴图、位置贴图、法线贴图、阴影贴图和环境光遮蔽贴图。在获取到纹理贴图后，可以将纹理贴图通过网络102发送 至服务器103，以使服务器103对漫反射贴图中阴影纹素的颜色进行还原。服务器103在接收到纹理贴图后，可以根据漫反射贴图中的阴影纹素所对应的颜色信息和平均亮度差，确定还原颜色信息，通过将漫反射贴图中阴影纹素的颜色信息更新为还原颜色信息即可去除图像中的阴影。进一步地，终端设备101还可以通过网络102将照片发送至服务器103，通过服务器103中搭载的渲染引擎基于这些照片进行三维建模，并获取建模过程中形成的纹理贴图。
在本申请的一个实施例中，可以通过不同的方法获取与阴影纹素对应的平均亮度差，并根据平均亮度差和阴影纹素的颜色信息对阴影纹素进行还原。
一种方法为，采集漫反射贴图中各个纹素的亮面颜色信息和暗面颜色信息，根据亮面颜色信息和暗面颜色信息分别形成亮面颜色查找表和暗面颜色查找表，进一步地，可以根据亮面颜色查找表和暗面颜色查找表中各像素的透明值，将亮面颜色查找表中的亮面颜色信息转换为亮面平均色，将暗面颜色查找表中的暗面颜色信息转换为暗面平均色，形成亮面平均色查找表和暗面平均色查找表；同时可以对亮面颜色查找表和暗面颜色查找表分别进行多级下采样，以获取亮面平均色贴图集和暗面平均色贴图集。然后，根据漫反射贴图中每个阴影纹素的空间坐标信息，在亮面平均色查找表和暗面平均色查找表中分别查找与该空间坐标信息对应的、且不为零的亮面平均色信息和暗面平均色信息，对于未查询到对应平均色信息的情况，可以根据空间坐标信息在亮面平均色贴图集和/或暗面平均色贴图集中进行查询，对于每个阴影纹素必然存在与之对应的至少一个亮面平均色信息和暗面平均色信息。最后根据亮面平均色信息、暗面平均色信息和阴影纹素的颜色信息即可确定还原颜色信息，并基于该还原颜色信息对阴影纹素的颜色进行还原。
另一种获取平均亮度差的方法为，通过边缘提取算法提取阴影贴图中的边缘，并以边缘点为中心点在漫反射贴图中确定亮部区域和暗部区域；然后采集亮部区域的亮部平均色和暗部区域的暗部平均色，并根据亮部平均色和暗部平均色确定平均亮度差，该平均亮度差可被存储于四维查找表中形成平均色查找表；进一步地，可以对平均色查找表进行多次下采样，获取平均色贴图集；最后，可以根据漫反射贴图中阴影纹素的空间位置坐标和暗部亮度信息，在平均色查找表或平均色贴图集中查询与阴影纹素对应的平均亮度差，并根据平均亮度差和阴影纹素的颜色信息确定还原颜色信息，进而根据该还原颜色信息对阴影纹素的颜色进行还原。
本申请实施例的技术方案一方面提高了颜色还原的效率和质量；另一方面不影响贴图中非阴影区域纹素的颜色，避免了贴图细节丢失、色相改变、色彩溢出和出现色块等问题，保证了根据贴图生成的模型可以适用于任意光照条件的场景中，进一步提高了画面效果和用户体验。
需要说明的是，本申请实施例所提供的图像处理方法可由服务器执行， 相应地，图像处理装置可设置于服务器中。但是，在本申请的其它实施例中，也可以由终端设备执行本申请实施例所提供的图像处理方法。
在本领域的相关技术中，为了去除直射光光照，通常采用工具Agisoft Texture Delighting进行处理，但是通常会存在贴图细节丢失、色相改变、色彩溢出和出现色块等问题。图2A-2B示出了对模型纹理进行处理得到的模型纹理示意图，图2A示出了未经去直射光光照的原始模型纹理，可以看出在物体的背光面处存在阴影(如图中的椭圆标识所示)；图2B示出了使用Agisoft Texture Delighting工具去除直射光光照后的模型纹理，虽然经过该工具处理后能够减弱阴影的程度，但是还是存在残留的阴影(如图中的椭圆标识所示)，阴影部分的纹理有一块块的色块，同时锥形柱的色相和色彩也较原始照片中的色相和色彩发生了改变。
鉴于相关技术中存在的问题，本申请实施例提供了一种图像处理方法，该图像处理方法能够对漫反射贴图中的阴影进行去除，并且保证基于去除阴影后的漫反射贴图生成的模型可在任意的光照条件的场景中使用。
图3示意性示出了根据本公开的一个实施例的图像处理方法的流程图，该方法可由服务器执行，该服务器为图1中所示的服务器103。参照图3所示，该图像处理方法至少包括步骤S310至步骤S330，详细介绍如下：
在步骤S310中，获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素。
在本申请的一个实施例中，在获取关于同一场景不同角度的多张照片后，可以采用基于摄影测量法的3D模型建模技术根据多张照片进行三维建模。在建模的过程中，首先可以根据照片获取对应的带有顶点色的三维模型信息，该带有顶点色的三维模型信息为包含颜色信息的点云信息，在获取包含颜色信息的点云信息后，可以采用图形软件进行纹理贴图映射以生成纹理贴图；具体地，可以调用任意可编程管线的三维图形接口，以标准光栅化的方式将模型渲染在渲染目标上，并通过顶点着色器将模型的顶点坐标设置为纹理贴图映射的UV坐标，实现纹理贴图的转写；其中UV坐标是UV纹理贴图坐标的简称，所有的图像文件都是二维的一个平面，水平方向是U，垂直方向是V，通过这个平面的、二维的UV坐标系可以定位图像上的任意一个像素。
纹理贴图为用于表达模型表面漫反射、法线、物件空间位置等信息的贴图，各个贴图以模型的UV坐标形式映射存储，在本申请的实施例中，纹理贴图包括漫反射贴图、位置贴图、法线贴图、阴影贴图，进一步地，还可以包括环境光遮蔽贴图和环境法线贴图。在转写贴图时，以漫反射贴图、位置贴图和法线贴图为例，可以在顶点着色器中计算颜色信息、物件空间位置、法线信息后，由GPU在光栅化时进行线性插值，渲染到对应的漫反射贴图、位置贴图、法线贴图的渲染目标上，并通过图形软件将漫反射贴图、位置贴图和法线贴图导出待用。
进一步地，对于直射光，还可以在图形软件中指定直射光方向，以光源为视点，通过正交投影渲染模型得到深度信息位图，其中在图形软件中指定直射光方向可以通过两种方式实现，第一种方式可以通过在模型上指定原点和投影点，图形软件根据原点和投影点的信息可以计算得到直射光的方向；第二种方式可以是通过仪器测量拍摄经纬度和时间，以使图形软件根据经纬度和时间能够计算得到直射光方向。
在获取深度信息位图后，可以在像素着色器中获取像素所在物件的空间坐标，通过对空间坐标进行线性变换处理以变换得到深度信息位图对应的坐标，获取深度信息位图中的信息并与预设深度值进行比较即可得到阴影贴图，该阴影贴图用于表达模型表面是否能被特定光源直接照射到，阴影贴图中可以以不同的数值对纹素是否为阴影纹素和非阴影纹素进行表示，例如阴影贴图可以是由0和1组成的贴图，其中0代表非阴影纹素，1代表阴影纹素。与漫反射贴图、位置贴图、法线贴图类似，阴影贴图也以UV坐标转写存储。
在本申请的一个实施例中，在采用基于摄影测量法的三维模型建模技术进行三维建模时，无论是太阳、照明灯等具有固定的光发射方向的光源产生的直射光，还是非固定光源发射的非直射光的作用，拍摄的照片中都会存在阴影，其中以直射光产生的阴影更为普遍。为了使模型能够在任意光照条件的场景中都能使用，因此有必要对各类光产生的阴影进行去除。在本申请的实施例中，通过对漫反射贴图中位于阴影区域的阴影纹素的颜色进行还原实现阴影的去除。接下来对如何还原直射光光照产生的阴影纹素的颜色进行详细说明。
在本申请的一个实施例中，在获取漫反射贴图的同时，可以获取到与漫反射贴图对应的位置贴图、法线贴图和阴影贴图，首先需要确定漫反射贴图中位于阴影区域的阴影纹素，然后才能有针对性的进行颜色还原，去除图像中的阴影。
图4示出了确定阴影纹素的流程示意图，如图4所示，在步骤S401中，获取阴影贴图中对应阴影区域的纹素的第一UV坐标；在步骤S402中，将第一UV坐标与漫反射贴图中纹素的UV坐标进行匹配，以获取漫反射贴图中与第一UV坐标对应的纹素作为阴影纹素。
其中，在步骤S401中，以上述实施例中以0/1表示阴影值的阴影贴图为例，其中阴影值为1的纹素即为位于阴影区域的纹素，从阴影贴图中获取这些纹素的UV坐标即为第一UV坐标。由于漫反射贴图和阴影贴图是通过UV坐标进行映射存储的，因此漫反射贴图和阴影贴图中与同一UV坐标对应的纹素对应模型中的同一面，进而在获取到第一UV坐标后，可以根据该第一UV坐标查找漫反射贴图中对应于第一UV坐标的纹素，该纹素即为漫反射贴图中位于阴影区域的阴影纹素。
在步骤S320中，根据所述阴影纹素的空间坐标信息在平均色查找表中进 行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差。
在本申请的一个实施例中，在确定漫反射贴图中的阴影纹素后，可以获取各个阴影纹素的空间坐标信息，并根据阴影纹素的空间坐标信息在平均色查找表中查找对应的平均色信息，并根据查询结果确定与阴影纹素对应的平均亮度差，进而根据平均亮度差对阴影纹素的颜色进行还原。
在本申请的一个实施例中，可以通过两种不同的方法去除直射光光照产生的阴影，对于各个去除方法，所涉及的平均色查找表的具体内容也不同。
对于第一种去除直射光光照产生的阴影的方法，其中所涉及的平均色查找表包括亮面平均色查找表和暗面平均色查找表，接下来对如何获取亮面平均色查找表和暗面平均色查找表进行详细说明。
在本申请的一个实施例中，在步骤S310之前，可以设置两个查找表：亮面查找表和暗面查找表，其中亮面查找表用于存储漫反射贴图中非阴影纹素的空间坐标信息和颜色信息，暗面查找表用于存储漫反射贴图中阴影纹素的空间坐标信息和颜色信息，值得说明的是，查找表(Look up table，简称LUT)是可以使用地址信息进行查询的一种数据结构，通过输入一个地址进行查表，可以找到地址对应的内容。
图5示出了获取亮面平均色查找表和暗面平均色查找表的流程示意图，如图5所示，该流程至少包括步骤S501-S502，具体的：
在步骤S501中，遍历漫反射贴图中的各个纹素，根据阴影贴图中与各纹素对应的阴影信息，将各纹素的颜色信息累加至亮面查找表和暗面查找表中，以形成亮面颜色查找表和暗面颜色查找表。
在本申请的一个实施例中，为了将漫反射贴图中所有纹素的颜色信息分别存储在亮面查找表和暗面查找表中，可以对漫反射贴图中的各个纹素进行遍历并将其颜色信息累加至相应的查找表中与该纹素的空间坐标信息对应的像素位上。在本申请的实施例中，亮面查找表和暗面查找表均为五维查找表，其中五个维度包括漫反射贴图中的纹素在空间直角坐标系中所对应的三个坐标维度，包括X轴坐标、Y轴坐标和Z轴坐标，同时还包括各纹素对应的法线的经度和维度，其中纹素在空间直角坐标系中的位置信息和法线信息可以在顶点着色器中根据模型各个三角形面的顶点坐标进行计算确定，并在转写贴图的时候转写至位置贴图和法线贴图中。
在本申请的一个实施例中，在将漫反射贴图中各纹素的颜色信息累加至亮面查找表或暗面查找表时，首先可以根据各纹素的UV坐标获取其空间坐标信息，该空间坐标信息与查找表的维度相同，也包括位置坐标(X,Y,Z)和法线坐标(经度，维度)；同时可以根据各纹素的UV坐标获取各纹素的阴影信息；然后根据纹素的阴影信息确定存放该纹素颜色信息的查找表是亮面查找表还是暗面查找表，并根据纹素的空间坐标信息在确定好的查找表中确定存储颜色信息的像素位置；最后将纹素的颜色信息累加至对应的像素位置 上即可。
由于查找表中存放的颜色信息为包含R、G、B、A四个通道的颜色向量，当α通道的透明值为0时，说明该纹素并未映射在模型上，当α通道的透明值为1时，说明该纹素映射在模型上，同时考虑到采集的模型只能是不透明的物件，因此所采集的纹素的α通道值应为1，也就是说，在将纹素的颜色信息累加至查找表时，只需将透明值为1的纹素的颜色信息累加即可，这样可以节省计算量，提高数据处理效率。
图6示出了获取亮面颜色查找表和暗面颜色查找表的流程示意图，如图6所示，在步骤S601中，根据各纹素的颜色信息中的透明值确定目标纹素，并获取目标纹素的第二UV坐标；在步骤S602中，根据第二UV坐标在阴影贴图中确定与目标纹素对应的目标阴影信息，并根据目标阴影信息在亮面查找表和暗面查找表中确定目标查找表；在步骤S603中，根据第二UV坐标分别在位置贴图和法线贴图中确定目标位置信息和目标法线信息；在步骤S604中，根据目标位置信息和目标法线信息在目标查找表中确定目标像素，并将目标纹素的颜色信息累加至目标像素上。
其中，在步骤S601中，该目标纹素即为颜色信息中透明值为1的纹素；在步骤S602中，根据第二UV坐标可以在阴影贴图中进行查询获取目标纹素的目标阴影信息，在确定目标阴影信息的类型后，可以根据目标阴影信息确定用于存储该纹素的颜色信息的目标查找表，即当目标阴影信息为目标纹素是阴影纹素时，将暗面查找表作为目标查找表；当目标阴影信息为目标纹素是非阴影纹素时，将亮面查找表作为目标查找表；在步骤S603中，与上述实施例类似，可以根据目标纹素的第二UV坐标在位置贴图和法线贴图中确定与其对应的目标位置信息和目标法线信息，该目标位置信息即为目标纹素在空间直角坐标系中的位置坐标，目标法线信息即为目标纹素对应的法线的经纬度；在步骤S604中，可以根据目标位置信息和目标法线信息在亮面查找表或暗面查找表中确定与其对应的像素，进而可以将目标纹素的颜色信息累加至该像素上。在累加颜色信息时，仍将颜色信息以(R,G,B,A)向量的形式进行累加。值得说明的是，亮面查找表和暗面查找表在初始时，各个像素中的颜色值均为零。
在步骤S502中，根据亮面颜色查找表中像素的透明值，对亮面颜色查找表中的颜色信息进行转换，以获取亮面平均色查找表；根据暗面颜色查找表中像素的透明值，对暗面颜色查找表中的颜色信息进行转换，以获取暗面平均色查找表。
在本申请的一个实施例中，在将漫反射贴图中各个纹素的颜色信息累加至亮面查找表和暗面查找表中形成亮面颜色查找表和暗面颜色查找表之后，可以分别根据亮面颜色查找表中颜色信息的透明值和暗面颜色查找表中颜色信息的透明值，对对应的查找表的颜色信息进行转换，形成亮面平均色查找 表和暗面平均色查找表。
图7示出了构建亮面平均色查找表和暗面平均色查找表的流程示意图，如图7所示，在步骤S701中，获取亮面颜色查找表中所有像素的透明值的第一和值，同时获取暗面颜色查找表中所有像素的透明值的第二和值；在步骤S702中，将亮面颜色查找表中各个像素的颜色信息与第一和值相除，以获取亮面平均色查找表；在步骤S703中，将暗面颜色查找表中各个像素的颜色信息与第二和值相除，以获取暗面平均色查找表。
在本申请的一个实施例中，由于需要获取与阴影纹素对应的亮面平均色和暗面平均色以计算平均亮度差，而对于同一纹素，不一定能同时在亮面平均色查找表和暗面平均色查找表中找到对应的亮面平均色和暗面平均色，因此在获取亮面颜色查找表和暗面颜色查找表之后，可以对亮面颜色查找表和暗面颜色查找表分别进行多级下采样，以获取亮面平均色贴图集和暗面平均色贴图集，其中亮面平均色贴图集包括多个大小不同的亮面平均色贴图，暗面平均色贴图集包括多个大小不同的暗面平均色贴图。
举例而言，纹素A为阴影纹素，纹素B为非阴影纹素，那么纹素A的颜色信息将被存储于暗面颜色查找表中，纹素B的颜色信息将被存储于亮面颜色查找表中，也就是说，若亮面颜色查找表中与纹素A的空间坐标信息对应的像素中不存在颜色信息，暗面颜色查找表中与纹素B的空间坐标信息对应的像素中不存在颜色信息，但是为了获取与纹素A对应的亮面平均色或者获取与纹素B对应的暗面平均色，就需要对亮面颜色查找表和暗面颜色查找表进行下采样，下采样的时候会对更大区域的颜色信息进行处理，并且下采样实质上是对颜色信息进行平均处理的过程，因此通过对亮面颜色查找表进行下采样就会将与阴影纹素A相邻的非阴影纹素的颜色信息统计进来，进而得到与阴影纹素A的空间坐标信息对应的亮面平均色，同时通过对暗面颜色查找表进行下采样可以将与非阴影纹素B相邻的阴影纹素的颜色信息统计进来，进而得到与非阴影纹素B的空间坐标信息对应的暗面平均色。
在本申请的一个实施例中，下采样的过程还可以理解为卷积过程，在本申请实施例中，可以以边长为2个像素的五维卷积核对第N-1级的平均色贴图进行卷积处理以获取第N级的平均色贴图，直至得到最小平均色贴图，其中N为正整数，并且当N＝1时，第0级的平均色贴图为亮面颜色查找表或暗面颜色查找表。在本申请的实施例中，在对亮面颜色查找表或暗面颜色查找表进行多级下采样时，可以直接将卷积核覆盖范围内各个像素的颜色信息相加，作为新的颜色信息。
在本申请的一个实施例中，在获取亮面平均色查找表、暗面平均色查找表、亮面平均色贴图和暗面平均色贴图后，可以根据纹素的空间坐标信息从中获取与纹素对应的亮面平均色和暗面平均色。由于本申请的目的是要去除漫反射贴图中的阴影，因此只需对漫反射贴图中的阴影纹素进行颜色还原即 可。
在查找与阴影纹素对应的亮面平均色和暗面平均色时，首先可以获取阴影纹素的空间坐标信息；然后根据阴影纹素的空间坐标信息，在暗面平均色查找表中查找与阴影纹素的空间坐标信息对应且值不为零的暗面平均色信息；并且可以根据阴影纹素的空间坐标信息，在亮面平均色贴图集中从低级亮面平均色贴图到高级亮面平均色贴图进行逐级查询，以获取与该空间坐标信息对应且值不为零的亮面平均色信息，进而根据亮面平均色信息和暗面平均色信息确定平均亮度差。
在本申请的一个实施例中，在根据阴影纹素的空间坐标信息在暗面平均色查找表中确定与其对应的暗面平均色信息时，可以对暗面平均色信息进行多维线性插值处理，以降低图像颜色梯度，使颜色更平滑。在对暗面平均色查找表进行多维线性插值处理时，具体可以以暗面平均色查找表中与阴影纹素的空间坐标信息所对应的像素为原采样点，并在每个维度上进行采样，以获取包括原采样点在内的多个采样点，例如暗面平均色查找表为五维查找表，则可以获取32(25)个采样点；然后根据多个采样点在五个维度上依次进行插值，计算插值点的平均色偏移量，并将平均色偏移量的超立方体的体积作为权重，其中超立方体的体积总量为1；最后根据超立方体的体积对平均色偏移量加权求和，即可获取经多维线性插值处理所得到的暗面平均色信息。
在本申请的一个实施例中，在根据亮面平均色信息和暗面平均色信息确定平均亮度差后，由于亮面平均色信息和暗面平均色信息为包含(R,G,B,A)四通道颜色值的颜色信息，而在颜色还原时只需对R、G、B三通道的颜色值进行还原，因此可以先从亮面平均色信息中获取亮面RGB向量，同时从暗面平均色信息中获取暗面RGB向量，然后将亮面RGB向量与暗面RGB向量相除，即可得到平均亮度差。
对于第二种去除直射光光照产生的阴影的方法，其中所涉及的平均色查找表为根据平均亮度差形成的查找表，接下来对如何获取平均色查找表进行详细说明。
在本申请的一个实施例中，在步骤S310之前，可以对阴影贴图和漫反射贴图进行处理以获取平均色查找表。图8示出了形成平均色查找表的流程示意图，如图8所示，该流程至少包括步骤S801-S803，具体地：
在步骤S801中，提取阴影贴图中的边缘纹素，并根据边缘纹素在漫反射贴图中确定中心纹素。
在本申请的一个实施例中，可以基于边缘提取算法对阴影贴图进行边缘提取，具体地，在提取阴影贴图的边缘时，可以通过水平方向模板和垂直方向模板对阴影贴图进行邻域卷积，以获取边缘纹素。其中，水平方向模板用于基于阴影贴图中的纹素检测水平边缘，垂直方向模板用于基于阴影贴图中的纹素检测垂直边缘，在获取到对应同一纹素的水平边缘检测结果和垂直边 缘检测结果后，可以通过取二者中的最大值或者将二者相加，以获取检测值，最后将该检测值与预设阈值进行比较，若检测值大于或等于预设阈值则说明纹素为边缘纹素。
在本申请的一个实施例中，在获取阴影贴图中的边缘纹素后，可以获取边缘纹素的UV坐标作为第三UV坐标，由于漫反射贴图和阴影贴图都是通过UV坐标进行映射存储的，因此根据第三UV坐标可以在漫反射贴图中确定与第三UV坐标对应的纹素，并将该纹素作为中心纹素以进行后续的平均色的采集。
在步骤S802中，根据所述中心纹素和预设范围确定目标贴图区域，并根据阴影贴图确定目标贴图区域中的亮部区域和暗部区域。
在本申请的一个实施例中，在确定中心纹素后，可以在漫反射贴图中确定以中心纹素为中心的预设范围内的区域为目标贴图区域。该预设范围可以是根据实际需要设置的，例如可以设置为64×64、80×80等等，本申请实施例对此不作具体限定。
进一步地，可以根据阴影贴图中的阴影纹素和非阴影纹素的位置信息在目标贴图区域中确定亮部区域和暗部区域，具体地，首先可以获取目标贴图区域中各个纹素的第四UV坐标，然后根据各纹素的第四UV坐标和阴影贴图，确定目标贴图区域中各个纹素的阴影信息，若与该第四UV坐标对应的纹素的阴影信息为纹素为非阴影纹素时，则目标贴图区域中该第四UV坐标对应的纹素为亮部区域中的纹素，若与该UV坐标对应的纹素的阴影信息为纹素为阴影纹素时，则目标贴图区域中该UV坐标对应的纹素为暗部区域中的纹素，也就是说，将阴影信息为非阴影纹素且空间坐标信息相邻的纹素所形成的区域作为亮部区域，并将阴影信息为阴影纹素且空间坐标信息相邻的纹素所形成的区域作为暗部区域。
在本申请的一个实施例中，在确定目标贴图区域中的亮部区域和暗部区域时，由于贴图中纹素的映射不一定是连续的，可能会有在贴图上临近，但是映射的表面离的非常远的情况，因此在本申请的实施例中还可以根据目标贴图区域中各纹素的位置信息和法线信息，判断位置映射和法线是否连续，并对非连续的纹素进行过滤。这样可以提高亮部区域和暗部区域中纹素的连续性，提高计算的准确度。
在步骤S803中，分别计算与亮部区域对应的亮部平均色和与暗部区域对应的暗部平均色，根据亮部平均色和暗部平均色确定平均亮度差，并将平均亮度差累加至平均色查找表中。
在本申请的一个实施例中，在获取目标贴图区域中的亮部区域和暗部区域后，可以根据亮部区域中各纹素的颜色信息确定亮部平均色，同时根据暗部区域中各纹素的颜色信息确定暗部平均色，图9示出了计算亮部平均色和暗部平均色的流程示意图，如图9所示：在步骤S901中，将亮部区域中所有 纹素的颜色信息相加以获取亮部颜色信息，同时将亮部区域中所有纹素的颜色信息中的透明值相加以获取第三和值；在步骤S902中，将暗部区域中所有纹素的颜色信息相加以获取暗部颜色信息，同时将暗部区域中所有纹素的颜色信息中的透明值相加以获取第四和值；在步骤S903中，将亮部颜色信息与第三和值相除以获取亮部平均色，并将暗部颜色信息与第四和值相除以获取暗部平均色。
在本申请的一个实施例中，获取亮部平均色和暗部平均色之后，获取亮部平均色中的亮部RGB向量和暗部平均色中的暗部RGB向量，然后将亮部RGB向量与暗部RGB向量相除，即可获取平均亮度差。在获取平均亮度差之后，可以将各个中心纹素对应的平均亮度差累加至平均色查找表中，其中平均色查找表为一个四维查找表，其中的四个维度分别为漫反射贴图中各中心纹素所对应物件的空间位置坐标(X,Y,Z)和中心纹素对应的暗部平均色的颜色均值，而在将中心纹素对应的平均亮度差累加至平均色查找表时，则根据中心纹素所对应物件的空间位置坐标和其对应的暗部平均色的颜色均值在平均色查找表中确定对应的像素，然后将该中心纹素对应的平均亮度差累加至该像素上即可。其中，暗部平均色的颜色均值为暗部平均色中R、G、B三通道颜色值的平均值。
在本申请的一个实施例中，该平均色查找表可为后续计算还原颜色信息提供数据支持，但是在平均色查找表中并不是对应每个阴影纹素都存在对应的平均亮度差，因此可以对平均色查找表进行多级下采样以获取与其对应的平均色贴图集，该平均色贴图集包括多个大小不同的平均色贴图。其中对平均色查找表进行下采样的过程与上述实施例中对亮面颜色查找表和暗面颜色查找表进行下采样的方法相同，在此不再赘述。
在本申请的一个实施例中，为了对漫反射贴图中的阴影纹素的颜色信息进行还原，可以在确定漫反射贴图中所有的阴影纹素后，根据位置贴图获取各阴影纹素的空间位置坐标，同时获取各阴影纹素对应的暗部平均色的颜色均值，然后根据空间位置坐标和暗部平均色的颜色均值在平均色查找表中查找与阴影纹素对应的平均亮度差。当平均色查找表中不存在与阴影纹素对应的平均亮度差时，则根据空间位置坐标和暗部平均色的颜色均值在平均色贴图集中从低级平均色贴图向高级平均色贴图的查找，直至找到与其对应的平均亮度差，其中低级的平均色贴图的大小大于高级的平均色贴图，最高级的平均色贴图可以是1×1×1×1的矩阵。
在本申请的一个实施例中，在根据阴影纹素的空间位置坐标和暗部平均色的颜色均值在平均色查找表中进行查询时，可以对平均色信息进行多维线性插值，以提高颜色的平滑性，减小图像颜色的梯度。与上述实施例中对暗面平均色查找表进行多维线性插值处理的方法类似，在对平均色查找表进行多维线性插值处理时，具体可以以平均色查找表中与阴影纹素的空间位置坐 标和暗部平均色的颜色均值所对应的像素为原采样点，并在每个维度上进行采样，以获取包括原采样点在内的多个采样点，例如平均色查找表为四维查找表，则可以获取16(24)个采样点；然后根据多个采样点在四个维度上依次进行插值，计算插值点的平均亮度差偏移量，并将平均亮度差偏移量对应的超立方体的体积作为权重，其中超立方体的体积总量为1；最后根据超立方体的体积对平均色偏移量加权求和即可获取经多维线性插值处理所得到的平均亮度差信息。
在步骤S330中，根据所述平均亮度差和所述阴影纹素对应的颜色信息确定还原颜色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原。
在本申请的一个实施例中，无论是第一种去除直射光光照的方法还是第二种去除直射光光照的方法，在获取与阴影纹素对应的平均亮度差之后，都可以根据平均亮度差和阴影纹素的颜色信息进行颜色还原。具体地，可以将平均亮度差与阴影纹素的颜色信息中的RGB向量相乘，以获取还原颜色信息，根据该还原颜色信息即可实现对阴影纹素的颜色还原。
在本申请的一个实施例中，在根据上述实施例的方法对阴影纹素的颜色进行还原后，还可以对漫反射贴图中阴影纹素和非阴影纹素之间的交界处，即明暗交界处，进行模糊化处理，以消除接缝处颜色不一致的问题。作为示例性说明，本申请实施例中可以通过高斯滤波、中值滤波、双边滤波、均值滤波等方法进行模糊化处理。
本申请实施例中的图像处理方法，通过根据与阴影纹素对应的亮面平均色信息和暗面平均色信息可以确定平均亮度差，并根据平均亮度差对阴影纹素的颜色信息进行还原，或者根据漫反射贴图中与阴影贴图中的边缘纹素对应的中心纹素所对应的亮部平均色和暗部平均色计算平均亮度差，并根据平均亮度差对阴影纹素的颜色信息进行还原，能够有效去除直射光光照产生的阴影，同时避免了图像中其它地方出现贴图细节丢失、色相改变、色彩溢出和出现色块等问题，使得模型的视觉效果更好，且能够在任意光照条件的场景中使用，例如可广泛应用于游戏、3D交互式体验等实时渲染应用中，
另外，便于从现实场景中采集和制作模型素材，使得用户体验更真实、更好。图11示出了根据本公开的图像处理方法去除模型中的阴影的界面示意图，如图11所示，与图2A-2B相比，根据本申请的图像处理方法去除阴影的效果更好，图像质量更高。
在本申请的一个实施例中，第一种去除直射光光照产生的阴影的方法需要针对漫反射贴图中每一个纹素进行平均色的采集，进而根据阴影纹素对应的亮面平均色信息和暗面平均色信息确定还原颜色信息，并根据还原颜色信息对阴影纹素的颜色进行还原，因此第一种阴影去除方法对于颜色较少的漫反射贴图中的阴影去除效果更好；第二种去除直射光光照产生的阴影的方法 针对与漫反射贴图中的中心纹素所对应的亮部区域和暗部区域的平均色进行采集，进而根据亮部区域的平均色和暗部区域的平均色确定还原颜色信息，并根据还原颜色信息对阴影纹素的颜色进行还原，因此第二种阴影去除方法对于颜色较多的漫反射贴图中的阴影去除效果更好。
在本申请的一个实施例中，直射光光照可以产生阴影，非直射光光照也可以产生阴影，接下来对如何去除因非直射光光照产生的阴影进行详细说明。
在本申请的一个实施例中，通过渲染引擎根据多张照片进行建模时，能够通过渲染引擎导出与模型对应的多种贴图，当照片中的光源为环境光时，除了能够通过渲染引擎导出与模型对应的漫反射贴图、法线贴图、位置贴图之外，还能导出环境光遮蔽贴图(AO mapping)和环境法线贴图(Bent normal mapping)。表面法线向量是几何模型的一个必要参数，而环境法线就是对原始的法线进行修改之后得到的新向量，它指向了当前像素一个不被其它物体(或几何体元)遮挡的平均方向，也即光线传入的主要方向。环境法线一般来说可以有两种用途：用来改变原始的法线，以及记录光线的主要通过方向，这样在做环境贴图的采样时可以从更优化的方向来计算采样的光线；用来代替原始的法线做正常的渲染，这样可以在只有法线的情况下在一定程度上得到类似于环境光遮蔽的软阴影效果。
在本申请的一个实施例中，图10示出了去除非直射光光照产生的阴影的流程示意图，如图10所示，该流程至少包括步骤S1010-S1040，具体地：
在步骤S1010中，根据漫反射贴图中各纹素的法线信息构建第一平均色查找表。
在本申请的一个实施例中，可以以法线的经纬度为坐标构建查找表，然后将漫反射贴图中各纹素的颜色信息累加至查找表中与各纹素对应的像素上，以形成第一颜色查找表。具体地，首先可以获取漫反射贴图各个纹素的第五UV坐标，由于漫反射贴图和法线贴图均通过UV坐标映射存储，因此根据该第五UV坐标可以在法线贴图中确定与其对应的法线信息；接着根据获取的法线信息中的经度和维度在第一颜色查找表中可以确定目标像素；然后判断具有第五UV坐标的纹素的颜色信息中的透明值是否为1，当透明值为1时，将该纹素的颜色信息累加至目标像素上，当透明值不为1时，则不对该纹素的颜色信息进行处理；最后采集包含纹素颜色信息的第一颜色查找表中各像素的透明值，将各像素的透明值相加以获取第五和值，并将第一颜色查找表中各像素的颜色信息与第五和值相除，以获取第一平均色查找表。
在步骤S1020中，根据第一平均色查找表确定与所述漫反射贴图对应的整体平均亮区域颜色信息，并根据第一平均色查找表获取中间结果贴图。
在本申请的一个实施例中，在获取第一平均色查找表后，可以遍历第一平均色查找表中的各个像素，计算各个像素接近0.7亮度的程度，接着可以将各个像素接近0.7亮度的程度作为权重，对第一平均色查找表中的像素进 行处理，以获取与漫反射贴图对应的整体平均亮区域颜色信息。在计算各个像素接近0.7亮度的程度时，可以根据公式(1)进行计算，具体地：
ω＝(1.0-Abs(Lumin(RGB)-0.7)×2) 3       (1)
其中，ω为权重，Abs为取绝对值，Lumin(RGB)为第一平均色查找表中各个像素的RGB向量的亮度值。
在本申请的一个实施例中，RGB向量的亮度值可以通过将RGB颜色空间转换为YIQ色彩空间来获得，其中的Y即为颜色的明视度，即亮度。在计算RGB向量的亮度值时，可以根据一定的转换比例对RGB三通道的颜色值进行转换，具体地，Y＝0.299R+0.587G+0.114B。
在本申请的一个实施例中，在获取第一平均色查找表中各个像素所对应的权重之后，可以将各个像素的颜色信息和与其对应的权重相乘，以获取加权颜色信息；接着可以将所有像素对应的加权颜色信息进行加和求平均，以获取整体平均亮区域颜色信息。整体平均亮区域颜色信息可以根据公式(2)计算得到，具体为：
Figure PCTCN2021093232-appb-000001
其中，Q为整体平均亮区域颜色信息，i为第一平均色查找表中存在颜色信息的任一像素，m为第一平均色查找表中存在颜色信息的像素的数量，ω为与各像素对应的权重。
在本申请的一个实施例中，与各个像素对应的权重为各个像素接近0.7亮度的程度，通过权重值可以让期望亮度的颜色值的权重更高，以达到整体向期望平均亮度接近的目的，进而实现对非直射光光照产生的阴影的恢复。
在本申请的一个实施例中，还可以遍历漫反射贴图中的各个纹素，根据各个纹素的法线信息和第一平均色查找表确定中间结果贴图，该中间结果贴图由与漫反射贴图中各个纹素所对应的经处理的平均色信息构成。由于第一平均色查找表中只保存有透明值为1的纹素的平均色信息，对于其它透明值不为1的纹素在第一平均色查找表中不存在对应的平均色信息，因此需要对第一平均色查找表进行多次下采样，以获取第一平均色贴图集，该第一平均色贴图集包含多个大小不同的第一平均色贴图，通过下采样可以保证对应漫反射贴图中的每一个纹素，都有与之对应的第一平均色。
在确定中间结果贴图时，首先获取漫反射贴图中各个纹素的第六UV坐标，然后根据第六UV坐标在法线贴图中查找与其对应的法线信息，接着根据该法线信息在第一平均色查找表中获取对应的平均色信息，如果在第一平均色查找表中无法获取与法线信息对应的平均色信息，则根据该法线信息在第一平均色贴图集中由低级第一平均色贴图向高级第一平均色贴图进行查找，直至获取与该法线信息对应的平均色信息；最后根据获取的与各个纹素对应的平均色信息可以进行处理，得到经处理的平均色信息。在对与各个纹 素对应的平均色信息进行处理时，可以先获取与纹素对应的平均色信息中的第一RGB向量和漫反射贴图中与纹素对应的颜色信息中的第二RGB向量，然后根据公式(3)进行计算得到处理后的平均色信息，进而根据处理后的平均色信息可以形成中间结果贴图。公式(3)具体为：
P＝(1.0f/V 1)×V 2   (3)
其中，P为经处理后的颜色信息，V1为第一RGB向量，V2为第二RGB向量。
在步骤S1030中，根据中间结果贴图和环境光遮蔽贴图构建第二平均色查找表。
在本申请的一个实施例中，可以以环境光遮蔽信息和环境法线信息为坐标形成查找表，并将中间结果贴图中各纹素的颜色信息经加权处理后累加到该查找表中形成第二平均色查找表。其中环境光遮蔽信息和环境法线信息均为一个通道值的浮点数，也就是说，第二平均色查找表也是一个二维的查找表，进一步地，第一平均色查找表和第二平均色查找表可以具有相同的大小，例如均设置为64×32等等，当然也可以设置为不同的大小，本申请实施例对此不作具体限定。
在本申请的一个实施例中，在形成第二平均色查找表时，遍历中间结果贴图中的各个纹素，根据各个纹素的UV坐标获取中间结果贴图中与其对应的环境光遮蔽值，然后将该环境光遮蔽值作为权重，对与相应纹素对应的中间结果进行加权处理。在获取中间结果贴图中各个纹素对应的加权处理后的中间结果后，可以获取各纹素对应的加权处理后的中间结果中的透明值，然后将所有纹素对应的透明值相加得到第六和值，接着将各个纹素对应的加权处理后的中间结果与第六和值相除，以获取与各个纹素对应的平均色信息；最后将与各个纹素对应的平均色信息累加至查找表中对应的像素中即可获取第二平均色查找表。
在本申请的一个实施例中，还可以对第二平均色查找表进行多次下采样，以获取第二平均色贴图集，该第二平均色贴图集包含多个大小不同的第二平均色贴图，其中对第二平均色查找表进行下采样的方法与对亮面颜色查找表或暗面颜色查找表进行下采样的方法相同，在此不再赘述。
在步骤S1040中，遍历中间结果贴图中的各个纹素，根据各个纹素在第二平均色查找表中的颜色信息确定还原颜色信息，并根据还原颜色信息对各个纹素的颜色进行还原。
在本申请的一个实施例中，对中间结果贴图中的纹素进行遍历，获取中间结果贴图中各中间结果纹素的UV坐标，根据该UV坐标在环境光遮蔽贴图和环境法线贴图中进行查询，以获取与各中间结果纹素对应的环境光遮蔽信息和环境法线信息。接着根据各中间结果纹素的环境光遮蔽信息和环境法线信息在第二平均色查找表中进行查询，以获取对应的第二平均色信息，如 果在第二平均色查找表中不存在对应的第二平均色信息，那么可以根据中间结果纹素的环境光遮蔽信息和环境法线信息在第二平均色贴图集中从低级的第二平均色贴图到高级的第二平均色贴图进行查找，直至获取与中间结果纹素对应的第二平均色信息。
在本申请的一个实施例中，在获取与中间结果纹素对应的第二平均色信息后，可以根据第二平均色信息、中间结果纹素的颜色信息和整体平均亮区域颜色信息确定还原颜色信息。在确定还原颜色信息时，首先获取中间结果纹素的颜色信息中的RGB向量，然后根据公式(4)计算得到还原颜色信息，公式(4)具体如下所示：
C＝(1.0f/L)×V 3×Q      (4)
其中，C为还原颜色信息，L为第二平均色信息，V3为中间结果纹素的颜色信息中的RGB向量，Q为整体平均亮区域颜色信息。
在本申请的一个实施例中，由于中间结果贴图是通过对漫反射贴图中的纹素进行处理得到的，那么中间结果纹素与漫反射贴图中的纹素对应，因此在获取中间结果纹素对应的还原颜色信息后，可以采用各纹素对应的还原颜色信息替换漫反射贴图中各纹素的颜色信息，进而可以去除漫反射贴图中由非直射光光照所产生的阴影。
图12A-12C示出了模型中的非直射光阴影去除前和去除后的界面示意图，如图12A所示，图像为未经去除阴影的原始模型纹理；如图12B所示，图像为经Unity De-Lighting tool工具去除阴影后的模型纹理，其中的石头上的青苔颜色有失真；而经过本申请实施例中的图像处理方法对模型中的阴影进行去除后，青苔颜色没有失真，图像质量较高，如图12C所示。
值得说明的是，对于同时存在直射光和非直射光的情况，由于直射光的作用较强，非直射光的作用较弱，因此可以仅根据去除直射光光照产生的阴影的方法进行图像处理即可。
本申请实施例中的图像处理方法，一方面通过两种方法去除漫反射贴图中由于直射光光照产生的阴影，另一方面通过一种方法去除漫反射贴图中由于非直射光光照产生的阴影，均有效地避免了现有技术中采用已知工具去除阴影时存在的贴图细节丢失、色相改变、色彩溢出和出现色块等问题，提高了漫反射贴图的质量，进而使得覆盖有漫反射贴图的模型可以在任意光照条件的场景中使用，而不会因为阴影的存在导致图像质量差、用户体验低等问题。
以下介绍本申请的装置实施例，可以用于执行本申请上述实施例中的图像处理方法。对于本申请装置实施例中未披露的细节，请参照本申请上述的图像处理方法的实施例。
图13示意性示出了根据本申请的一个实施例的图像处理装置的框图。该装置可以用于执行本申请实施例提供的方法中的相应步骤。
参照图13所示，根据本申请的一个实施例的图像处理装置1300，包括：获取模块1301、计算模块1302和还原模块1303。
其中，获取模块1301，用于获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素；计算模块1302，用于根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差；还原模块1303，用于根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原。
在本申请的一个实施例中，所述获取模块1301还用于：获取与所述漫反射贴图对应的位置贴图、法线贴图和阴影贴图。
在本申请的一个实施例中，所述获取模块1301具体用于：获取所述阴影贴图中对应阴影区域的纹素的第一UV坐标；将所述第一UV坐标与所述漫反射贴图中纹素的UV坐标进行匹配，以获取所述漫反射贴图中与所述第一UV坐标匹配的纹素作为所述阴影纹素。
在本申请的一个实施例中，所述平均色查找表包括亮面平均色查找表和暗面平均色查找表；所述图像处理装置1300还包括：颜色查找表形成模块，用于在获取所述漫反射贴图中位于阴影区域的阴影纹素之前，遍历所述漫反射贴图中的各个纹素，根据所述阴影贴图中与各所述纹素对应的阴影信息，将各所述纹素的颜色信息累加至亮面查找表和暗面查找表中，以形成亮面颜色查找表和暗面颜色查找表；平均色查找表形成模块，用于根据所述亮面颜色查找表中像素的透明值，对所述亮面颜色查找表中的颜色信息进行转换，以获取所述亮面平均色查找表；根据所述暗面颜色查找表中像素的透明值，对所述暗面颜色查找表中的颜色信息进行转换，以获取所述暗面平均色查找表。
在本申请的一个实施例中，所述亮面查找表和所述暗面查找表均为五维查找表，所述五维查找表包括所述漫反射贴图中的纹素在空间直角坐标系中的三个坐标维度以及纹素所对应的法线的经度和维度。
在本申请的一个实施例中，所述平均色查找表形成模块具体用于：获取所述亮面颜色查找表中所有像素的透明值的第一和值，同时获取所述暗面颜色查找表中所有像素的透明值的第二和值；将所述亮面颜色查找表中各个像素的颜色信息与所述第一和值相除，以获取所述亮面平均色查找表；将所述暗面颜色查找表中各个像素的颜色信息与所述第二和值相除，以获取所述暗面平均色查找表。
在本申请的一个实施例中，所述颜色查找表形成模块包括：第一坐标获取单元，用于根据各所述纹素的颜色信息中的透明值确定目标纹素，并获取所述目标纹素的第二UV坐标；阴影信息查询单元，用于根据所述第二UV坐标在所述阴影贴图中确定与所述目标纹素对应的目标阴影信息，并根据所 述目标阴影信息在所述亮面查找表和所述暗面查找表中确定目标查找表；位置信息查询单元，用于根据所述第二UV坐标分别在所述位置贴图和所述法线贴图中确定目标位置信息和目标法线信息；颜色累加单元，用于根据所述目标位置信息和所述目标法线信息在所述目标查找表中确定目标像素，并将所述目标纹素的颜色信息累加至所述目标像素上，以形成所述亮面颜色查找表和所述暗面颜色查找表。
在本申请的一个实施例中，所述第一坐标获取单元具体用于：将颜色信息中透明值为1的纹素作为所述目标纹素。
在本申请的一个实施例中，所述目标位置信息为所述目标纹素在空间直角坐标系中的位置坐标，所述目标法线信息为所述目标纹素对应的法线的经度和维度，所述目标阴影信息用于表示所述目标纹素是否为所述阴影纹素。
在本申请的一个实施例中，所述阴影信息查询单元配置为：根据所述第二UV坐标在所述阴影贴图中进行查询，以获取所述目标阴影信息，并判断所述目标阴影信息的类型；所述目标阴影信息表示所述目标纹素是阴影纹素时，将所述暗面查找表作为所述目标查找表；当所述目标阴影信息表示所述目标纹素是非阴影纹素时，将所述亮面查找表作为所述目标查找表。
在本申请的一个实施例中，所述图像处理装置1300还包括：贴图集生成模块，用于对所述亮面颜色查找表和所述暗面颜色查找表分别进行多级下采样，以获取亮面平均色贴图集和暗面平均色贴图集，其中所述亮面平均色贴图集包括多个大小不同的亮面平均色贴图，所述暗面平均色贴图集包括多个大小不同的暗面平均色贴图。
在本申请的一个实施例中，所述计算模块1302包括：暗面平均色获取单元，用于根据所述阴影纹素的空间坐标信息，在所述暗面平均色查找表中确定与所述空间坐标信息对应且值不为零的暗面平均色信息；亮面平均色获取单元，用于根据所述空间坐标信息，在所述亮面平均色贴图集中从低级亮面平均色贴图到高级亮面平均色贴图进行逐级查询，以获取与所述空间坐标信息对应且值不为零的亮面平均色信息；平均亮度差计算单元，用于根据所述亮面平均色信息与所述暗面平均色信息确定所述平均亮度差。
在本申请的一个实施例中，所述计算模块1302还用于：在确定所述暗面平均色信息时，对所述暗面平均色查找表进行多维线性插值处理。
在本申请的一个实施例中，所述平均亮度差计算单元具体用于：获取所述亮面平均色信息中的亮面RGB向量和所述暗面平均色信息中的暗面RGB向量；将所述亮面RGB向量与所述暗面RGB向量相除，以获取所述平均亮度差。
在本申请的一个实施例中，所述还原模块1303具体用于：将所述平均亮度差和所述阴影纹素对应的颜色信息中的RGB向量相乘，以获取所述还原颜色信息。
在本申请的一个实施例中，所述图像处理装置1300还包括：中心纹素确定模块，用于提取所述阴影贴图中的边缘纹素，并根据所述边缘纹素在所述漫反射贴图中确定中心纹素；亮暗区域确定模块，用于根据所述中心纹素和预设范围确定目标贴图区域，并根据所述阴影贴图确定所述目标贴图区域中的亮部区域和暗部区域；平均亮度差计算模块，用于分别计算与所述亮部区域对应的亮部平均色和与所述暗部区域对应的暗部平均色，根据所述亮部平均色和所述暗部平均色确定平均亮度差，并将所述平均亮度差累加至所述平均色查找表中。
在本申请的一个实施例中，所述中心纹素确定模块具体用于：通过水平方向模板和垂直方向模板对所述阴影贴图进行邻域卷积，以获取所述边缘纹素；获取所述边缘纹素的第三UV坐标，根据所述第三UV坐标，在所述漫反射贴图中进行查找以获取所述中心纹素。
在本申请的一个实施例中，所述亮暗区域确定模块具体用于：获取所述目标贴图区域中各个纹素的第四UV坐标，根据所述第四UV坐标和所述阴影贴图，确定所述目标贴图区域中各个纹素对应的阴影信息；将阴影信息为非阴影纹素且空间坐标信息相邻的纹素所形成的区域作为所述亮部区域，并将阴影信息为阴影纹素且空间坐标信息相邻的纹素所形成的区域作为所述暗部区域。
在本申请的一个实施例中，所述平均亮度差计算模块具体用于：将所述亮部区域中所有纹素的颜色信息相加以获取亮部颜色信息，同时将所述亮部区域中所有纹素的颜色信息中的透明值相加以获取第三和值；将所述暗部区域中所有纹素的颜色信息相加以获取暗部颜色信息，同时将所述暗部区域中所有纹素的颜色信息中的透明值相加以获取第四和值；将所述亮部颜色信息与所述第三和值相除以获取所述亮部平均色，并将所述暗部颜色信息与所述第四和值相除以获取所述暗部平均色。
在本申请的一个实施例中，所述平均亮度差计算模块具体用于：获取所述亮部平均色中的亮部RGB向量和所述暗部平均色中的暗部RGB向量；将所述亮部RGB向量与所述暗部RGB向量相除，以获取所述平均亮度差。
在本申请的一个实施例中，所述平均色查找表为四维查找表，包括所述中心纹素的空间位置坐标的三个维度和暗部平均色的颜色均值。
在本申请的一个实施例中，所述计算模块1302包括：信息获取单元，用于根据所述位置贴图确定所述阴影纹素的空间位置坐标，同时获取所述阴影纹素对应的暗部平均色的颜色均值；信息查找单元，用于根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在所述平均色查找表中查询对应的平均亮度差。
在本申请的一个实施例中，所述信息查找单元具体用于：根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在所述平均色查找表中查询 是否存在对应的平均亮度差；若不存在，根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在与所述平均色查找表对应的平均色贴图集中查询对应的平均亮度差，其中所述平均色贴图集包括通过对所述平均色查找表进行多级下采样形成的多个贴图。
在本申请的一个实施例中，所述信息查找单元还用于：在查询所述平均亮度差时，对所述平均色查找表进行多维线性插值处理。
在本申请的一个实施例中，所述还原模块1303配置为：将所述平均亮度差和所述阴影纹素对应的颜色信息中的RGB向量相乘，以获取所述还原颜色信息。
在本申请的一个实施例中，所述图像处理装置1300还包括：交界模糊模块，用于对所述漫反射贴图中阴影纹素和非阴影纹素之间的交界处进行模糊化处理。
图14示意性示出了根据本申请的一个实施例的图像处理装置的框图。该装置可以用于执行本申请实施例提供的方法中的相应步骤。
参照图14所示，根据本申请的一个实施例的图像处理装置1400，包括：第一查找表构建模块1401、中间结果贴图获取模块1402、第二查找表构建模块1403和颜色还原模块1404。
其中，第一查找表构建模块1401，用于根据漫反射贴图中各纹素的法线信息构建第一平均色查找表；中间结果贴图获取模块，用于根据第一平均色查找表确定与所述漫反射贴图对应的整体平均亮区域颜色信息，并根据第一平均色查找表获取中间结果贴图；第二查找表构建模块，用于根据中间结果贴图和环境光遮蔽贴图构建第二平均色查找表；颜色还原模块，用于遍历中间结果贴图中的各个纹素，根据各个纹素在第二平均色查找表中的颜色信息确定还原颜色信息，并根据还原颜色信息对各个纹素的颜色进行还原。
图15示出了适于用来实现本申请实施例的电子设备的计算机系统的结构示意图。
需要说明的是，图15示出的电子设备的计算机系统1500仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图15所示，计算机系统1500包括中央处理单元(Central Processing Unit，CPU)1501，其可以根据存储在只读存储器(Read-Only Memory，ROM)1502中的程序或者从存储部分1508加载到随机访问存储器(Random Access Memory，RAM)1503中的程序而执行各种适当的动作和处理，实现上述实施例中所述的图像标注方法。在RAM 1503中，还存储有系统操作所需的各种程序和数据。CPU 1501、ROM 1502以及RAM 1503通过总线1504彼此相连。输入/输出(Input/Output，I/O)接口1505也连接至总线1504。
以下部件连接至I/O接口1505：包括键盘、鼠标等的输入部分1506；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal  Display，LCD)等以及扬声器等的输出部分1507；包括硬盘等的存储部分1508；以及包括诸如LAN(Local Area Network，局域网)卡、调制解调器等的网络接口卡的通信部分1509。通信部分1509经由诸如因特网的网络执行通信处理。驱动器1510也根据需要连接至I/O接口1505。可拆卸介质1511，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1510上，以便于从其上读出的计算机程序根据需要被安装入存储部分1508。
特别地，根据本申请的实施例，下文参考流程图描述的过程可以被实现为计算机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分1509从网络上被下载和安装，和/或从可拆卸介质1511被安装。在该计算机程序被中央处理单元(CPU)1501执行时，执行本申请的系统中限定的各种功能。
需要说明的是，本申请实施例所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、有线等等，或者上述的任意合适的组合。
作为另一方面，本申请还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的图像处理装置中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被一个该电子设备执行时，使得该电子设备实现上述实施例中所述的方法。

Claims (30)

  1. 一种图像处理方法，由电子设备执行，包括：
    获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素；
    根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差；
    根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原。
  2. 根据权利要求1所述的图像处理方法，在所述获取漫反射贴图时，所述方法还包括：
    获取与所述漫反射贴图对应的位置贴图、法线贴图和阴影贴图。
  3. 根据权利要求2所述的图像处理方法，所述获取漫反射贴图中位于阴影区域的阴影纹素，包括：
    获取所述阴影贴图中对应阴影区域的纹素的第一UV坐标；
    将所述第一UV坐标与所述漫反射贴图中纹素的UV坐标进行匹配，以获取所述漫反射贴图中与所述第一UV坐标匹配的纹素作为所述阴影纹素。
  4. 根据权利要求2所述的图像处理方法，所述平均色查找表包括亮面平均色查找表和暗面平均色查找表；在获取所述漫反射贴图中位于阴影区域的阴影纹素之前，所述方法还包括：
    遍历所述漫反射贴图中的各个纹素，根据所述阴影贴图中与各所述纹素对应的阴影信息，将各所述纹素的颜色信息累加至亮面查找表和暗面查找表中，以形成亮面颜色查找表和暗面颜色查找表；
    根据所述亮面颜色查找表中像素的透明值，对所述亮面颜色查找表中的颜色信息进行转换，以获取所述亮面平均色查找表；根据所述暗面颜色查找表中像素的透明值，对所述暗面颜色查找表中的颜色信息进行转换，以获取所述暗面平均色查找表。
  5. 根据权利要求4所述的图像处理方法，所述亮面查找表和所述暗面查找表均为五维查找表，所述五维查找表包括所述漫反射贴图中的纹素在空间直角坐标系中的三个坐标维度以及纹素所对应的法线的经度和维度。
  6. 根据权利要求4所述的图像处理方法，所述根据所述亮面颜色查找表中像素的透明值，对所述亮面颜色查找表中的颜色信息进行转换，以获取所述亮面平均色查找表；根据所述暗面颜色查找表中像素的透明值，对所述暗面颜色查找表中的颜色信息进行转换，以获取所述暗面平均色查找表，包括：
    获取所述亮面颜色查找表中所有像素的透明值的第一和值，同时获取所述暗面颜色查找表中所有像素的透明值的第二和值；
    将所述亮面颜色查找表中各个像素的颜色信息与所述第一和值相除，以获取所述亮面平均色查找表；
    将所述暗面颜色查找表中各个像素的颜色信息与所述第二和值相除，以 获取所述暗面平均色查找表。
  7. 根据权利要求4所述的图像处理方法，所述根据各所述纹素的阴影信息，将各所述纹素的颜色向量累加至所述亮面查找表和所述暗面查找表中，以形成亮面颜色查找表和暗面颜色查找表，包括：
    根据各所述纹素的颜色信息中的透明值确定目标纹素，并获取所述目标纹素的第二UV坐标；
    根据所述第二UV坐标在所述阴影贴图中确定与所述目标纹素对应的目标阴影信息，并根据所述目标阴影信息在所述亮面查找表和所述暗面查找表中确定目标查找表；
    根据所述第二UV坐标分别在所述位置贴图和所述法线贴图中确定目标位置信息和目标法线信息；
    根据所述目标位置信息和所述目标法线信息在所述目标查找表中确定目标像素，并将所述目标纹素的颜色信息累加至所述目标像素上，以形成所述亮面颜色查找表和所述暗面颜色查找表。
  8. 根据权利要求7所述的图像处理方法，所述目标位置信息为所述目标纹素在空间直角坐标系中的位置坐标，所述目标法线信息为所述目标纹素对应的法线的经度和维度，所述目标阴影信息用于表示所述目标纹素是否为所述阴影纹素。
  9. 根据权利要求8所述的图像处理方法，所述根据所述第二UV坐标在所述阴影贴图中确定与所述目标纹素对应的目标阴影信息，并根据所述目标阴影信息在所述亮面查找表和所述暗面查找表确定目标查找表，包括：
    根据所述第二UV坐标在所述阴影贴图中进行查询，以获取所述目标阴影信息，并判断所述目标阴影信息的类型；
    当所述目标阴影信息表示所述目标纹素是阴影纹素时，将所述暗面查找表作为所述目标查找表；
    当所述目标阴影信息表示所述目标纹素是非阴影纹素时，将所述亮面查找表作为所述目标查找表。
  10. 根据权利要求4所述的图像处理方法，在获取所述亮面颜色查找表和所述暗面颜色查找表之后，所述方法还包括：
    对所述亮面颜色查找表和所述暗面颜色查找表分别进行多级下采样，以获取亮面平均色贴图集和暗面平均色贴图集，其中所述亮面平均色贴图集包括多个大小不同的亮面平均色贴图，所述暗面平均色贴图集包括多个大小不同的暗面平均色贴图。
  11. 根据权利要求10所述的图像处理方法，所述根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差，包括：
    根据所述阴影纹素的空间坐标信息，在所述暗面平均色查找表中确定与 所述空间坐标信息对应且值不为零的暗面平均色信息；
    根据所述空间坐标信息，在所述亮面平均色贴图集中从低级亮面平均色贴图到高级亮面平均色贴图进行逐级查询，以获取与所述空间坐标信息对应且值不为零的亮面平均色信息；
    根据所述亮面平均色信息与所述暗面平均色信息确定所述平均亮度差。
  12. 根据权利要求11所述的图像处理方法，所述方法还包括：
    在确定所述暗面平均色信息时，对所述暗面平均色查找表进行多维线性插值处理。
  13. 根据权利要求11所述的图像处理方法，所述根据所述亮面平均色信息和所述暗面平均色信息确定所述平均亮度差，包括：
    获取所述亮面平均色信息中的亮面RGB向量和所述暗面平均色信息中的暗面RGB向量；
    将所述亮面RGB向量与所述暗面RGB向量相除，以获取所述平均亮度差。
  14. 根据权利要求1或13所述的图像处理方法，所述根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜色信息，包括：
    将所述平均亮度差和所述阴影纹素对应的颜色信息中的RGB向量相乘，以获取所述还原颜色信息。
  15. 根据权利要求2所述的图像处理方法，在所述获取所述漫反射贴图中位于阴影区域的阴影纹素之前，所述方法还包括：
    提取所述阴影贴图中的边缘纹素，并根据所述边缘纹素在所述漫反射贴图中确定中心纹素；
    根据所述中心纹素和预设范围确定目标贴图区域，并根据所述阴影贴图确定所述目标贴图区域中的亮部区域和暗部区域；
    分别计算与所述亮部区域对应的亮部平均色和与所述暗部区域对应的暗部平均色，根据所述亮部平均色和所述暗部平均色确定平均亮度差，并将所述平均亮度差累加至所述平均色查找表中。
  16. 根据权利要求15所述的图像处理方法，所述提取所述阴影贴图中的边缘纹素，并根据所述边缘纹素在所述漫反射贴图中确定中心纹素，包括：
    通过水平方向模板和垂直方向模板对所述阴影贴图进行邻域卷积，以获取所述边缘纹素；
    获取所述边缘纹素的第三UV坐标，根据所述第三UV坐标在所述漫反射贴图中进行查找，以获取所述中心纹素。
  17. 根据权利要求15所述的图像处理方法，所述根据所述阴影贴图确定所述目标贴图区域中的亮部区域和暗部区域，包括：
    获取所述目标贴图区域中各个纹素的第四UV坐标，根据所述第四UV坐标和所述阴影贴图，确定所述目标贴图区域中各个纹素对应的阴影信息；
    将阴影信息为非阴影纹素且空间坐标信息相邻的纹素所形成的区域作为所述亮部区域，并将阴影信息为阴影纹素且空间坐标信息相邻的纹素所形成的区域作为所述暗部区域。
  18. 根据权利要求15所述的图像处理方法，所述分别计算与所述亮部区域对应的亮部平均色和与所述暗部区域对应的暗部平均色，包括：
    将所述亮部区域中所有纹素的颜色信息相加以获取亮部颜色信息，同时将所述亮部区域中所有纹素的颜色信息中的透明值相加以获取第三和值；
    将所述暗部区域中所有纹素的颜色信息相加以获取暗部颜色信息，同时将所述暗部区域中所有纹素的颜色信息中的透明值相加以获取第四和值；
    将所述亮部颜色信息与所述第三和值相除以获取所述亮部平均色，并将所述暗部颜色信息与所述第四和值相除以获取所述暗部平均色。
  19. 根据权利要求15所述的图像处理方法，所述根据所述亮部平均色和所述暗部平均色确定平均亮度差，包括：
    获取所述亮部平均色中的亮部RGB向量和所述暗部平均色中的暗部RGB向量；
    将所述亮部RGB向量与所述暗部RGB向量相除，以获取所述平均亮度差。
  20. 根据权利要求19所述的图像处理方法，所述平均色查找表为四维查找表，包括所述中心纹素的空间位置坐标的三个维度和暗部平均色的颜色均值。
  21. 根据权利要求20所述的图像处理方法，所述根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差，包括：
    根据所述位置贴图确定所述阴影纹素的空间位置坐标，同时获取所述阴影纹素对应的暗部平均色的颜色均值；
    根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在所述平均色查找表中查询对应的平均亮度差。
  22. 根据权利要求21所述的图像处理方法，所述根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在所述平均色查找表中查询对应的平均亮度差，包括：
    根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在所述平均色查找表中查询是否存在对应的平均亮度差；
    若不存在，根据所述阴影纹素的空间位置坐标和暗部平均色的颜色均值，在与所述平均色查找表对应的平均色贴图集中查询对应的平均亮度差，其中所述平均色贴图集包括通过对所述平均色查找表进行多级下采样形成的多个贴图。
  23. 根据权利要求22所述的图像处理方法，所述方法还包括：
    在查询所述平均亮度差时，对所述平均色查找表进行多维线性插值处理。
  24. 根据权利要求21或22所述的图像处理方法，所述根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜色信息，包括：
    将所述平均亮度差和所述阴影纹素对应的颜色信息中的RGB向量相乘，以获取所述还原颜色信息。
  25. 根据权利要求1所述的图像处理方法，所述方法还包括：
    对所述漫反射贴图中阴影纹素和非阴影纹素之间的交界处进行模糊化处理。
  26. 根据权利要求1所述的图像处理方法，所述方法还包括：
    根据所述漫反射贴图中各纹素的法线信息构建第一平均色查找表；
    根据所述第一平均色查找表确定与所述漫反射贴图对应的整体平均亮区域颜色信息，并根据所述第一平均色查找表获取中间结果贴图；
    根据所述中间结果贴图和环境光遮蔽贴图构建第二平均色查找表；
    遍历所述中间结果贴图中的各个纹素，根据各个纹素在所述第二平均色查找表中的颜色信息确定还原颜色信息，并根据还原颜色信息对各个纹素的颜色进行还原。
  27. 一种图像处理装置，包括：
    获取模块，用于获取漫反射贴图，并获取所述漫反射贴图中位于阴影区域的阴影纹素；
    计算模块，用于根据所述阴影纹素的空间坐标信息在平均色查找表中进行查询，并根据查询结果确定与所述阴影纹素对应的平均亮度差；
    还原模块，用于根据所述平均亮度差和所述阴影纹素对应的颜色信息，确定还原颜色信息，并根据所述还原颜色信息对所述阴影纹素的颜色信息进行还原。
  28. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至26中任一项所述的图像处理方法。
  29. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，所述存储装置用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1至26中任一项所述的图像处理方法。
  30. 一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行权利要求1至26中任一项所述的图像处理方法。
PCT/CN2021/093232 2020-06-10 2021-05-12 图像处理方法、装置、计算机存储介质及电子设备 WO2021249091A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP21822842.7A EP4044122A4 (en) 2020-06-10 2021-05-12 IMAGE PROCESSING METHOD AND DEVICE, COMPUTER STORAGE MEDIUM AND ELECTRONIC DEVICE
JP2022554220A JP7390497B2 (ja) 2020-06-10 2021-05-12 画像処理方法、装置、コンピュータプログラム、および電子デバイス
KR1020227014210A KR102611066B1 (ko) 2020-06-10 2021-05-12 이미지 처리 방법 및 장치, 컴퓨터 저장 매체, 그리고 전자 기기
US17/705,169 US11776202B2 (en) 2020-06-10 2022-03-25 Image processing method and apparatus, computer storage medium, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010524473.7A CN111612882B (zh) 2020-06-10 2020-06-10 图像处理方法、装置、计算机存储介质及电子设备
CN202010524473.7 2020-06-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/705,169 Continuation US11776202B2 (en) 2020-06-10 2022-03-25 Image processing method and apparatus, computer storage medium, and electronic device

Publications (1)

Publication Number Publication Date
WO2021249091A1 true WO2021249091A1 (zh) 2021-12-16

Family

ID=72196278

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/093232 WO2021249091A1 (zh) 2020-06-10 2021-05-12 图像处理方法、装置、计算机存储介质及电子设备

Country Status (6)

Country Link
US (1) US11776202B2 (zh)
EP (1) EP4044122A4 (zh)
JP (1) JP7390497B2 (zh)
KR (1) KR102611066B1 (zh)
CN (1) CN111612882B (zh)
WO (1) WO2021249091A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111612882B (zh) * 2020-06-10 2023-04-07 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机存储介质及电子设备
CN112419459B (zh) * 2020-10-20 2023-09-08 上海哔哩哔哩科技有限公司 烘焙模型ao贴图的方法、装置、计算机设备和存储介质
CN112381705B (zh) * 2021-01-14 2021-03-26 奥特酷智能科技(南京)有限公司 在Unity3d中快速点云绘制的方法
CN113487716A (zh) * 2021-07-23 2021-10-08 杭州群核信息技术有限公司 瓷砖铺贴渲染方法、装置、存储介质和计算机程序产品
CN113570583B (zh) * 2021-07-30 2024-06-07 北京市商汤科技开发有限公司 图像处理方法及装置、电子设备和存储介质
CN113538549B (zh) * 2021-08-31 2023-12-22 广州光锥元信息科技有限公司 在处理图像时保留图像质感纹理的方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108124079A (zh) * 2016-11-29 2018-06-05 精工爱普生株式会社 阴影校正装置、电子设备和阴影校正方法
EP3352138A1 (en) * 2017-01-24 2018-07-25 Thomson Licensing Method and apparatus for processing a 3d scene
CN110998669A (zh) * 2017-08-08 2020-04-10 索尼公司 图像处理装置和方法
CN111612882A (zh) * 2020-06-10 2020-09-01 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机存储介质及电子设备

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3943973B2 (ja) * 2002-03-20 2007-07-11 キヤノン株式会社 画像処理装置およびその方法
CN1319028C (zh) * 2004-05-20 2007-05-30 明基电通股份有限公司 图像修正系统及方法
WO2010087162A1 (ja) * 2009-01-27 2010-08-05 日本電気株式会社 カラー画像処理方法、カラー画像処理装置および記録媒体
KR101605770B1 (ko) 2009-07-28 2016-03-23 삼성전자주식회사 영상 처리 방법 및 장치
US8294794B2 (en) * 2010-07-06 2012-10-23 GM Global Technology Operations LLC Shadow removal in an image captured by a vehicle-based camera for clear path detection
EP2650843A4 (en) 2010-12-09 2018-03-28 Samsung Electronics Co., Ltd. Image processor, lighting processor and method therefor
US10699477B2 (en) * 2018-03-21 2020-06-30 Zoox, Inc. Generating maps without shadows
JP7104575B2 (ja) * 2018-06-29 2022-07-21 キヤノン株式会社 画像処理装置、制御方法、及びプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108124079A (zh) * 2016-11-29 2018-06-05 精工爱普生株式会社 阴影校正装置、电子设备和阴影校正方法
EP3352138A1 (en) * 2017-01-24 2018-07-25 Thomson Licensing Method and apparatus for processing a 3d scene
CN110998669A (zh) * 2017-08-08 2020-04-10 索尼公司 图像处理装置和方法
CN111612882A (zh) * 2020-06-10 2020-09-01 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机存储介质及电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LIU YONGGANG: "Summary of Practical Experiences in Normal Mapping: Case Study of 3D Modeling of Changzhou Biji Lane during Qing Dynasty", ART SCIENCE AND TECHNOLOGY, 15 May 2016 (2016-05-15), pages 13 - 14, XP055879315 *

Also Published As

Publication number Publication date
US20220215618A1 (en) 2022-07-07
US11776202B2 (en) 2023-10-03
CN111612882B (zh) 2023-04-07
JP2023520841A (ja) 2023-05-22
KR102611066B1 (ko) 2023-12-06
KR20220070506A (ko) 2022-05-31
EP4044122A4 (en) 2023-05-10
EP4044122A1 (en) 2022-08-17
JP7390497B2 (ja) 2023-12-01
CN111612882A (zh) 2020-09-01

Similar Documents

Publication Publication Date Title
WO2021249091A1 (zh) 图像处理方法、装置、计算机存储介质及电子设备
RU2324978C2 (ru) Системы и способы для обеспечения управляемой дискретизации текстуры
CN108986195B (zh) 一种结合环境映射和全局光照渲染的单镜头混合现实实现方法
US10049486B2 (en) Sparse rasterization
CN112116692B (zh) 模型渲染方法、装置、设备
CN108805971B (zh) 一种环境光遮蔽方法
CN114373056A (zh) 一种三维重建方法、装置、终端设备及存储介质
US9224233B2 (en) Blending 3D model textures by image projection
CN115375820B (zh) 顶点环境光遮蔽值的确定方法、应用方法、装置及设备
CN112652046A (zh) 游戏画面的生成方法、装置、设备及存储介质
CN111724313A (zh) 一种阴影贴图生成方法与装置
CN115810101A (zh) 三维模型风格化方法、装置、电子设备及存储介质
CN114549374A (zh) 对使用蒙特卡罗绘制进行绘制的图像去噪
CN112891946A (zh) 游戏场景生成方法和装置、可读存储介质、电子设备
CN110517348B (zh) 基于图像前景分割的目标物体三维点云重建方法
CN116310036A (zh) 场景渲染方法、装置、设备、计算机可读存储介质及产品
WO2022088927A1 (zh) 基于图像的光效处理方法、装置、设备及存储介质
WO2023193613A1 (zh) 高光渲染方法、装置、介质及电子设备
CN112509108A (zh) 基于gpu的顶点环境光遮蔽的生成方法以及图像渲染方法
CN111739074A (zh) 一种场景多点光源渲染方法及装置
CN115375847B (zh) 材质恢复方法、三维模型的生成方法和模型的训练方法
CN111127588B (zh) 基于DirectX的大数据量参数曲线回放方法
CN114820904A (zh) 支持光照的伪室内渲染方法、装置、介质和设备
CN112465692A (zh) 图像处理方法、装置、设备及存储介质
CN115690175A (zh) 图像处理方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21822842

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20227014210

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021822842

Country of ref document: EP

Effective date: 20220426

ENP Entry into the national phase

Ref document number: 2022554220

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE