CN114972466A

CN114972466A - Image processing method, image processing device, electronic equipment and readable storage medium

Info

Publication number: CN114972466A
Application number: CN202210750126.5A
Authority: CN
Inventors: 郭桦
Original assignee: Vivo Mobile Communication Hangzhou Co Ltd
Current assignee: Vivo Mobile Communication Hangzhou Co Ltd
Priority date: 2022-06-29
Filing date: 2022-06-29
Publication date: 2022-08-30
Also published as: WO2024002086A1

Abstract

The application discloses an image processing method, an image processing device, electronic equipment and a readable storage medium, and belongs to the field of information processing. The method comprises the following steps: analyzing the acquired first image, and determining depth information of the first image, first environment light source information and position information of an object in the first image; according to the first environment light source information, carrying out illumination elimination processing on the first three-dimensional scene graph to obtain a second three-dimensional scene graph, wherein the first three-dimensional scene graph is obtained based on depth information construction; determining light source parameter information according to the depth information, the first environment light source information and the position information; performing simulated illumination processing on the second three-dimensional scene graph according to the light source parameter information to obtain a third three-dimensional scene graph; and generating a target image or a target video based on the third stereo scene graph.

Description

Image processing method, image processing device, electronic equipment and readable storage medium

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method and apparatus, an electronic device, and a readable storage medium.

Background

With the development of image processing technology, more and more users use electronic equipment to take pictures and create the pictures obtained by shooting. In recent years, the way of reconstructing the two-dimensional image obtained by shooting to obtain the three-dimensional image is very popular. In general, a rendering engine is used, and illumination debugging needs to be performed on virtual three-dimensional scene information. Such a processing method requires the user to adjust the information related to the light, and is very cumbersome and difficult for the ordinary user.

Disclosure of Invention

An object of the embodiments of the present application is to provide an image processing method, an image processing apparatus, an electronic device, and a readable storage medium, which can solve the problems in the prior art that adjusting light in a stereoscopic scene corresponding to an image is very tedious and difficult.

In a first aspect, an embodiment of the present application provides an image processing method, including:

analyzing the acquired first image, and determining depth information of the first image, first environment light source information and position information of an object in the first image;

according to the first environment light source information, carrying out illumination elimination processing on the first three-dimensional scene graph to obtain a second three-dimensional scene graph, wherein the first three-dimensional scene graph is obtained based on depth information construction;

determining light source parameter information according to the depth information, the first environment light source information and the position information;

performing simulated illumination processing on the second three-dimensional scene graph according to the light source parameter information to obtain a third three-dimensional scene graph;

and generating a target image or a target video based on the third stereo scene graph.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

the analysis module is used for analyzing the acquired first image and determining depth information of the first image, first environment light source information and position information of an object in the first image;

the elimination module is used for carrying out illumination elimination processing on the first three-dimensional scene graph according to the first environment light source information to obtain a second three-dimensional scene graph, and the first three-dimensional scene graph is constructed and obtained on the basis of the depth information;

the determining module is used for determining light source parameter information according to the depth information, the first environment light source information and the position information;

the simulated illumination module is used for performing simulated illumination processing on the second three-dimensional scene graph according to the light source parameter information to obtain a third three-dimensional scene graph;

and the generating module is used for generating a target image or a target video based on the third stereo scene graph.

In a third aspect, embodiments of the present application provide an electronic device, which includes a processor and a memory, where the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the method according to the first aspect.

In a fourth aspect, embodiments of the present application provide a readable storage medium, on which a program or instructions are stored, which when executed by a processor implement the steps of the method according to the first aspect.

In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the method according to the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product, stored on a storage medium, for execution by at least one processor to implement the method according to the first aspect.

In the embodiment of the application, the depth information of the first image, the first ambient light source information and the position information of the object in the first image are determined by analyzing the acquired first image; according to the first environment light source information, the first three-dimensional scene graph constructed based on the depth information is subjected to illumination elimination processing to obtain a second three-dimensional scene graph, the illumination information of the first image can be automatically eliminated, and the light source parameter information is adaptively reconstructed according to the depth information, the first environment light source information and the position information. And finally, based on the third stereo scene graph, a real, natural and high-adaptation-degree lighted target image or target video of the object can be generated.

Drawings

Fig. 1 is a schematic view of a stereoscopic scene provided in an embodiment of the present application;

fig. 2 is a flowchart of an image processing method provided in an embodiment of the present application;

FIG. 3 is a schematic diagram of an image processing process provided by an embodiment of the present application;

fig. 4 is a block diagram of an image processing apparatus according to an embodiment of the present application;

fig. 5 is one of the hardware configuration diagrams of the electronic device according to the embodiment of the present application;

fig. 6 is a second schematic diagram of a hardware structure of the electronic device according to the embodiment of the present application.

Detailed Description

The technical solutions of the embodiments of the present application will be described below clearly with reference to the drawings of the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments that can be derived from the embodiments of the present application by one of ordinary skill in the art are intended to be within the scope of the present application.

The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.

The image processing method provided by the embodiment of the present application can be applied to at least the following application scenarios, which are described below.

With the development of image processing technology, more and more users use electronic devices to take pictures and often create pictures. Currently, three-dimensional scene information can be reconstructed based on planar images, as shown in fig. 1. The three-dimensional scene information reconstructed based on the first image is generally polished by placing light sources with different brightness and different materials in a virtual world by using a rendering engine. Such a processing method requires a user to adjust information such as a light source position, light source brightness, and light source material by himself, and is very complicated and difficult for a general user.

For the problems in the related art, embodiments of the present application provide an image processing method and apparatus, an electronic device, and a storage medium, which can solve the problems in the related art that adjusting light in a stereoscopic scene corresponding to an image is very tedious and difficult.

The image processing method provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

Fig. 2 is a flowchart of an image processing method according to an embodiment of the present application.

As shown in fig. 2, the image processing method may include steps 210 to 250, and the method is applied to an image processing apparatus, and specifically as follows:

step 210, analyzing the acquired first image, and determining depth information of the first image, first ambient light source information, and position information of the object in the first image.

Specifically, the acquired first image may be analyzed through a deep learning network, and the depth information, the first ambient light source information, and the position information of the object in the first image are determined.

The depth information is used for representing the distance between each object in the image and the camera. The first ambient light source information is ambient light source information when the first image is obtained by shooting, that is, original ambient light source information of the first image. The position information of the object in the first image may specifically be coordinates of the object in the first image.

Before step 210, the method may further include:

receiving selection input of a user on a first image in the album; in response to a selection input, a first image is acquired.

In a possible embodiment, step 210 may specifically include the following steps:

analyzing the first image to determine depth information;

extracting position information from the first image according to the depth information;

first ambient light source information is determined from the location information.

The step of analyzing the first image may specifically include the following steps: and inputting the first image into a deep learning network to obtain depth information.

The deep learning network specifically comprises a high-precision depth estimation module, a main body distinguishing module, a background complementing module and a dynamic visual angle rendering module. The deep learning network may specifically output: a background compensation map, position information of the object, depth information, and position information of the camera. The background compensation image is an image obtained by removing the object from the first image.

According to the depth information, the position information is extracted from the first image, and the object with the closest distance to the camera in the objects in the first image can be determined as the related object.

The step of extracting the position information of the object from the first image according to the depth information may specifically include the following steps:

extracting a first object image from the first image according to the depth information;

carrying out binarization processing on the first object image to obtain a second object image;

filtering the first object image to obtain a third object image;

combining the second object image and the third object image to obtain a fourth object image;

from the fourth object image, position information of the object is extracted.

In order to obtain more accurate position information of an object, the obtained depth information needs to be processed and analyzed, and a first object image is extracted from a first image according to the depth information, wherein the object can be a main body closest to a camera.

Then, the first object image is binarized to obtain a second object image. The image area corresponding to the object in the second object image may be a white pixel value, and the background area may be a black pixel value. The background area may be an image area of the first object image other than an image area corresponding to the object.

And simultaneously, carrying out multiple times of filtering processing on the first object image to obtain a smoother edge, extracting edge lines of the first object image at the moment, and determining the extracted image comprising the edge lines as a third object image.

Finally, the second object image and the third object image are combined, which is equivalent to combining the image area corresponding to the object and the edge line corresponding to the object to obtain a fourth object image, and the fourth object image comprises the position information of the complete object, so that the position information of the object can be extracted from the fourth object image.

The analyzing the obtained first image to determine the depth information, the first ambient light source information, and the position information of the object in the first image may specifically include the following steps:

analyzing the acquired first image, and determining depth information and position information of an object in the first image;

constructing and obtaining first three-dimensional scene information based on the depth information;

first ambient light source information is determined from the first stereoscopic scene information.

The first stereoscopic scene information may be an obj file, and the obj file is in a three-dimensional model file format.

The step of determining the first ambient light source information according to the position information may specifically include the following steps:

determining brightness information of each surface of the object from the first image according to the position information of the object;

and calculating the brightness information based on the light reflection law to obtain first environment light source information.

In combination with the determined position information of the object, a pixel value of each surface of the object may be determined from the first stereoscopic scene information, and then luminance information of each surface of the object may be determined based on the pixel value of each surface of the object.

And then calculating the brightness information based on the light reflection law to obtain first ambient light source information.

The original light source corresponding to the first ambient light source information is irradiated on the object and reflected, and the brightness information is the reflected light, so that the incident light, namely the first ambient light source information, can be deduced based on the light reflection law and the reflected light.

Among them, reflection is an optical phenomenon. This refers to the phenomenon that when light propagates to different substances, the propagation direction is changed at the interface and the light returns to the original substance. Light is reflected off surfaces of water, glass, and many other objects. The phenomenon when light changes direction of propagation at the interface of two substances and returns to the original substance is called light reflection.

The reflection law of light includes: the reflection angle is equal to the incident angle, and the angle between the incident light and the plane is equal to the angle between the reflected light and the plane. The reflected light and the incident light are positioned on two sides of the normal. The reflected light, the incident light and the normal are all in the same plane.

Here, the luminance information of each surface of the object is determined from the first image based on the position information of the object, and the luminance information is calculated based on the law of reflection of light, so that the first ambient light source information can be obtained quickly and accurately in combination with the information in the first image and the law of nature.

In addition, the step of calculating the luminance information based on the light reflection law to obtain the first ambient light source information may further include:

and converting the first image into a gray image to obtain the gray value of the gray image and the RGB value of the first image, and fitting the environmental light source intensity information through the RGB brightness value and the gray value of the whole image. The ambient light source intensity information may be RGB channel information.

The RGB color scheme is a color standard in the industry, and various colors are obtained by changing three color channels of red (R), green (G), and blue (B) and superimposing the three color channels on each other, where RGB represents the colors of red, green, and blue. The proportion of the three channels in each pixel point is 100 percent.

The channel information of RGB may be an expression of the material of the three-dimensional object. For example, fur and glass have different light reflection effects, so that channel information calculation proportion for calculating RGB can be provided to express the effect of the original light source on the object.

The channel information of RGB corresponding to each pixel point is determined by the gray value of the gray image of the first image and the RGB value of the first image.

And step 220, according to the first environment light source information, performing illumination elimination processing on the first three-dimensional scene graph to obtain a second three-dimensional scene graph, wherein the first three-dimensional scene graph is constructed on the basis of the depth information.

Wherein the first ambient light source information may include: the intensity of the original light source and the irradiation direction of the original light source are used for eliminating the illumination information in the first three-dimensional scene information in the step, and the whole three-dimensional scene is placed under the scene without illumination to obtain second three-dimensional scene information.

In one possible embodiment, step 220 includes:

generating illumination information according to the first environment light source information;

and eliminating illumination information from the first stereo scene image to obtain a second stereo scene image.

And generating illumination information according to the first environment light source information, wherein the related illumination information can be specifically a pixel value used for representing illumination, and the illumination pixel value is eliminated from the first stereoscopic scene image to obtain a second stereoscopic scene image.

As shown in fig. 3, in this step, illumination information brought by the first ambient light source information in the first image is removed to obtain second stereoscopic scene information.

And step 230, determining light source parameter information according to the depth information, the first environment light source information and the position information. In a possible embodiment, the light source parameter information includes a light source position and second ambient light source information, step 230, including:

determining the position of the light source according to the position information;

according to the depth information, determining the position information of a target area closest to an original camera from the position information of the object, wherein the original camera is a camera for shooting a first image;

and determining second environment light source information according to the position information of the target area and the first environment light source information.

Determining the light source position according to the position information may specifically include: determining the position information of the original camera according to the position information of the object; the light source position is determined from the position information of the original camera and the position information of the object. The specific calculation method is as follows: the position information of the original camera is calculated from the intersection of the distance lines between the object and the background. The light source position is obtained by reflection calculation of the position information of the original camera and the position information of the object.

From the position information of the object, position information of a target area closest to the original camera is determined based on the depth information. The target region may be a critical part in the subject.

For example, since the object is a person and the hand area of the person is closest to the original camera, the hand area can be determined as the target area, and the position information of the target area can be determined from the position information of the object.

The second ambient light source information is determined according to the position information of the target area and the first ambient light source information, and specifically, the second ambient light source information is obtained by adaptively adjusting the first ambient light source information according to the position information of the target area, for example, adjusting the illumination intensity, the light hue, and the like.

And 240, performing simulated illumination processing on the second three-dimensional scene graph according to the light source parameter information to obtain a third three-dimensional scene graph.

And performing polishing treatment on the second three-dimensional scene information again according to the light source parameter information to obtain third three-dimensional scene information, performing remediation treatment on an image with a poor effect caused by a poor light and shadow effect in a user album, and guiding a user to perform virtual shooting on the third three-dimensional scene information after the third three-dimensional scene information is obtained to obtain a target image or a target video.

Based on the step of determining the RGB channel information according to the first image in step 210, the first image may be specifically converted into a gray scale image, so as to obtain a gray scale value of the gray scale image and an RGB value of the first image, and the ambient light source intensity information is fitted through the RGB brightness value and the gray scale value of the full map. The ambient light source intensity information may be RGB channel information.

The light source parameter information may further include: RGB channel information, so in step 240, the method may specifically include: generating reconstructed illumination information according to the second environment light source information and the RGB channel information;

and carrying out polishing treatment on the second three-dimensional scene information according to the light source position and the reconstructed illumination information to obtain third three-dimensional scene information.

Here, it is considered that the reflection effects of objects of different materials on light are different, so the proportions of different channels in each pixel point can be distributed through RGB channel information, reconstructed illumination information can be generated according to the second environment light source information and the RGB channel information, the reconstructed illumination information can express the effect of a light source on objects of different materials, the second three-dimensional scene information is subjected to illumination processing according to the position of the light source and the reconstructed illumination information, and the third three-dimensional scene information which is real and natural can be obtained.

In one possible embodiment, step 240 includes:

and according to the light source position and the second environment light source information, performing simulated illumination processing on the second three-dimensional scene graph to obtain a third three-dimensional scene graph.

And combining the position information of the object, simultaneously referring to the second environment light source information, taking the target area of the object as the center, controlling the second three-dimensional scene information, and performing lighting transformation according to the light source position of the second environment light source information.

And step 250, generating a target image or a target video based on the third stereo scene graph.

In a possible embodiment, step 250 may specifically include:

receiving an input of a user, the input being used to control a shooting direction of the virtual camera;

responding to the input, and carrying out simulated shooting on the third three-dimensional scene graph based on the shooting direction to obtain a target image; alternatively, the first and second electrodes may be,

determining a plurality of photographing directions according to the position information of the object;

performing surrounding shooting on the object in the third stereoscopic scene image based on a plurality of shooting directions to obtain a plurality of video frames;

based on the plurality of video frames, a target video is generated.

In one aspect, a user input may be received, and the user input may be a direction of a sliding movement of a screen of the electronic device by the user, or a moving operation of the electronic device by the user.

Under the condition of the movement operation of the electronic equipment by the user, the gyroscope parameter of the electronic equipment can be determined according to the input of the user, the shooting direction of the virtual camera is controlled according to the gyroscope parameter, and the reconstructed light source is made to move aiming at the target area of the object.

The gyroscope is an angular motion detection device which uses a momentum moment sensitive shell of a high-speed revolving body to rotate around one or two axes which are orthogonal to a rotation axis relative to an inertia space. Angular motion detection devices made using other principles are also known as gyroscopes, which serve the same function.

Here, the shooting direction of the virtual camera may be controlled by a user input, and since the second ambient light source information in the third stereoscopic scene information is determined, when the shooting direction of the camera changes, different directions of the object may be irradiated to the fixed second ambient light source information, so that the whole of the camera has more stereoscopic impression. Meanwhile, the interaction between the user and the electronic equipment can be combined to create a more real and more immersive three-dimensional experience. And performing simulated shooting on the third three-dimensional scene graph based on the shooting direction, so as to obtain a real and natural target image.

On the other hand, a plurality of shooting directions may be determined from the position information of the subject; performing surrounding shooting on the object in the third stereoscopic scene image based on a plurality of shooting directions to obtain a plurality of video frames; based on the plurality of video frames, a target video is generated.

When the user does not interact, the object can be subjected to 360-degree surrounding rendering in the virtual third stereo scene information, and finally a 360-degree surrounding lighting video effect is obtained.

The step of generating the target video includes determining a plurality of shooting directions according to the position information of the object, performing surround shooting on the object based on the plurality of shooting directions to obtain a plurality of video frames, and generating the target video according to the plurality of video frames. Through based on a plurality of shooting directions, surround the shooting to the object in the third three-dimensional scene information, can promote the third dimension dynamic sense of reality, bring more interesting creation experience for the user.

In the image processing method provided by the embodiment of the application, the execution main body can be an image processing device. In the embodiment of the present application, an image processing apparatus is taken as an example to execute an image processing method, and the image processing apparatus provided in the embodiment of the present application is described.

Fig. 4 is a block diagram of an image processing apparatus according to an embodiment of the present application, where the apparatus 400 includes:

the analysis module 410 is configured to analyze the acquired first image, and determine depth information of the first image, first ambient light source information, and position information of the object in the first image.

And the eliminating module 420 is configured to perform illumination elimination processing on the first stereoscopic scene image according to the first ambient light source information to obtain a second stereoscopic scene image, where the first stereoscopic scene image is constructed based on the depth information.

The determining module 430 is configured to determine light source parameter information according to the depth information, the first ambient light source information, and the position information.

And the simulated illumination module 440 is configured to perform simulated illumination processing on the second stereoscopic scene image according to the light source parameter information to obtain a third stereoscopic scene image.

And a generating module 450, configured to generate a target image or a target video based on the third stereoscopic scene graph.

In a possible embodiment, the analysis module 410 is specifically configured to:

the first image is analyzed to determine depth information.

An analysis module 410, comprising:

and the extraction module is used for extracting the position information from the first image according to the depth information.

And the first determining module is used for determining the first ambient light source information according to the position information.

In a possible embodiment, the extraction module is specifically configured to:

filtering the first object image to obtain a third object image;

from the fourth object image, position information is extracted.

In a possible embodiment, the first determining module is specifically configured to:

determining brightness information of each surface of the object from the first image according to the position information;

In a possible embodiment, the elimination module 420 is specifically configured to:

In a possible embodiment, the light source parameter information includes a light source position and second ambient light source information, and the determining module 430 is specifically configured to:

according to the depth information, determining the position information of a target area closest to an original camera from the position information, wherein the original camera is a camera for shooting a first image;

In a possible embodiment, the analog illumination module 440 is specifically configured to:

In a possible embodiment, the generating module 450 is specifically configured to:

based on the plurality of video frames, a target video is generated.

In the embodiment of the application, the depth information of the first image, the first ambient light source information and the position information of the object in the first image are determined by analyzing the acquired first image; according to the first environment light source information, the first three-dimensional scene graph constructed based on the depth information is subjected to illumination elimination processing to obtain a second three-dimensional scene graph, the illumination information in the first image can be automatically eliminated, and the light source parameter information is adaptively reconstructed according to the depth information, the first environment light source information and the position information. And finally, based on the third stereo scene graph, a real, natural and high-adaptation-degree lighted target image or target video of the object can be generated.

The image processing apparatus in the embodiment of the present application may be an electronic device, and may also be a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be a device other than a terminal. The electronic Device may be, for example, a Mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic Device, a Mobile Internet Device (MID), an Augmented Reality (AR)/Virtual Reality (VR) Device, a robot, a wearable Device, an ultra-Mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and may also be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a Television (TV), a teller machine, a self-service machine, and the like, and the embodiments of the present application are not particularly limited.

The image processing apparatus according to the embodiment of the present application may be an apparatus having an action system. The action system may be an Android (Android) action system, an ios action system, or other possible action systems, and the embodiment of the present application is not particularly limited.

The image processing apparatus provided in the embodiment of the present application can implement each process implemented by the foregoing method embodiment, and is not described here again to avoid repetition.

Optionally, as shown in fig. 5, an electronic device 510 is further provided in the embodiment of the present application, and includes a processor 511, a memory 512, and a program or an instruction stored in the memory 512 and executable on the processor 511, where the program or the instruction is executed by the processor 511 to implement each step of any one of the above embodiments of the image processing method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

It should be noted that the electronic device of the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.

Fig. 6 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 600 includes, but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, a processor 610, and the like.

Those skilled in the art will appreciate that the electronic device 600 may further comprise a power source (e.g., a battery) for supplying power to the various components, and the power source may be logically connected to the processor 610 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The electronic device structure shown in fig. 6 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is omitted here.

The processor 610 is configured to analyze the acquired first image, and determine depth information of the first image, first ambient light source information, and position information of the object in the first image.

The processor 610 is further configured to perform illumination elimination processing on the first stereoscopic scene image according to the first ambient light source information to obtain a second stereoscopic scene image, where the first stereoscopic scene image is constructed based on the depth information.

The processor 610 is further configured to determine light source parameter information according to the depth information, the first ambient light source information, and the position information.

The processor 610 is further configured to perform simulated illumination processing on the second stereoscopic scene image according to the light source parameter information, so as to obtain a third stereoscopic scene image.

And the processor 610 is further configured to generate a target image or a target video based on the third stereoscopic scene graph.

Optionally, the processor 610 is further configured to analyze the first image to determine depth information;

position information is extracted from the first image based on the depth information.

Optionally, the processor 610 is further configured to extract a first object image from the first image according to the depth information;

filtering the first object image to obtain a third object image;

from the fourth object image, position information is extracted.

Optionally, the processor 610 is further configured to determine brightness information of each surface of the object from the first image according to the position information;

Optionally, the processor 610 is further configured to generate illumination information according to the first ambient light source information;

and eliminating the illumination information from the first stereo scene graph to obtain a second stereo scene graph.

Optionally, the processor 610 is further configured to determine a light source position according to the position information;

Optionally, the processor 610 is further configured to perform simulated illumination processing on the second stereoscopic scene graph according to the light source position and the second ambient light source information, so as to obtain a third stereoscopic scene graph.

Optionally, the processor 610 is further configured to receive an input from a user, the input being used to control a shooting direction of the virtual camera;

responding to the input, and carrying out simulated shooting on the third three-dimensional scene graph based on the shooting direction to obtain a target image; alternatively, the first and second liquid crystal display panels may be,

determining a plurality of shooting directions according to the position information of the object;

based on the plurality of video frames, a target video is generated.

It is to be understood that, in the embodiment of the present application, the input Unit 604 may include a Graphics Processing Unit (GPU) 6041 and a microphone 6042, and the Graphics processor 6041 processes image data of a still picture or a video image obtained by an image capturing apparatus (such as a camera) in a video image capturing mode or an image capturing mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. Touch panel 6071, also known as a touch screen. The touch panel 6071 may include two portions of a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, and are not described in further detail herein. The memory 609 may be used to store software programs as well as various data including, but not limited to, applications and action systems. The processor 610 may integrate an application processor, which primarily handles motion systems, user pages, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 610.

The memory 609 may be used to store software programs as well as various data. The memory 609 may mainly include a first storage area storing a program or an instruction and a second storage area storing data, wherein the first storage area may store an operating system, an application program or an instruction (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 609 may include volatile memory or nonvolatile memory, or the memory x09 may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. The volatile Memory may be a Random Access Memory (RAM), a Static Random Access Memory (Static RAM, SRAM), a Dynamic Random Access Memory (Dynamic RAM, DRAM), a Synchronous Dynamic Random Access Memory (Synchronous DRAM, SDRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (Double Data Rate SDRAM, ddr SDRAM), an Enhanced Synchronous SDRAM (ESDRAM), a Synchronous Link DRAM (SLDRAM), and a Direct Memory bus RAM (DRRAM). The memory 609 in the embodiments of the subject application include, but are not limited to, these and any other suitable types of memory.

Processor 610 may include one or more processing units; optionally, the processor 610 integrates an application processor, which mainly handles operations related to the operating system, user interface, application programs, etc., and a modem processor, which mainly handles wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 610.

The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the embodiment of the image processing method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read only memory ROM, a random access memory RAM, a magnetic or optical disk, and the like.

The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the embodiment of the image processing method, and can achieve the same technical effect, and the details are not repeated here to avoid repetition.

It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.

Embodiments of the present application provide a computer program product, where the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the processes of the foregoing embodiments of the image processing method, and achieve the same technical effects, and in order to avoid repetition, details are not repeated here.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element identified by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.

While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. An image processing method, characterized in that the method comprises:

analyzing the acquired first image, and determining depth information of the first image, first ambient light source information and position information of an object in the first image;

according to the first environment light source information, carrying out illumination elimination processing on a first three-dimensional scene graph to obtain a second three-dimensional scene graph, wherein the first three-dimensional scene graph is obtained based on the depth information;

2. The method of claim 1, wherein analyzing the acquired first image to determine depth information of the first image, first ambient light source information, and position information of an object in the first image comprises:

analyzing the first image to determine the depth information;

extracting the position information from the first image according to the depth information;

and determining the first ambient light source information according to the position information.

3. The method of claim 2, wherein extracting the location information from the first image according to the depth information comprises:

filtering the first object image to obtain a third object image;

extracting the position information from the fourth object image.

4. The method of claim 2, wherein determining the first ambient light source information from the location information comprises:

and calculating the brightness information based on a light reflection law to obtain the first ambient light source information.

5. The method of claim 1, wherein the performing the illumination elimination process on the first stereoscopic scene graph according to the first ambient light source information to obtain a second stereoscopic scene graph comprises:

and eliminating the illumination information from the first stereo scene graph to obtain the second stereo scene graph.

6. The method of claim 1, wherein the light source parameter information comprises a light source location and second ambient light source information, and wherein determining light source parameter information based on the depth information, the first ambient light source information, and the location information comprises:

according to the depth information, determining the position information of a target area closest to an original camera from the position information, wherein the original camera is a camera for shooting the first image;

and determining the second environment light source information according to the position information of the target area and the first environment light source information.

7. The method according to claim 6, wherein the performing simulated lighting processing on the second stereoscopic scene graph according to the light source parameter to obtain a third stereoscopic scene graph comprises:

and performing simulated illumination processing on the second three-dimensional scene graph according to the light source position and the second environment light source information to obtain a third three-dimensional scene graph.

8. The method according to claim 1, wherein the generating a target image or a target video based on the third stereoscopic scene graph comprises:

receiving an input of a user, the input being used to control a shooting direction of a virtual camera;

responding to the input, and carrying out simulated shooting on the third stereoscopic scene graph based on the shooting direction to obtain the target image; alternatively, the first and second electrodes may be,

performing surrounding shooting on the object in the third stereoscopic scene graph based on the plurality of shooting directions to obtain a plurality of video frames;

generating the target video based on the plurality of video frames.

9. An image processing apparatus, characterized in that the apparatus comprises:

the analysis module is used for analyzing the acquired first image and determining depth information of the first image, first ambient light source information and position information of an object in the first image;

the eliminating module is used for carrying out illumination elimination processing on the first three-dimensional scene graph according to the first environment light source information to obtain a second three-dimensional scene graph, and the first three-dimensional scene graph is constructed and obtained on the basis of the depth information;

10. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions when executed by the processor implementing the steps of the method of any of claims 1 to 8.

11. A readable storage medium, on which a program or instructions are stored, which when executed by a processor, carry out the steps of the method according to any one of claims 1 to 8.