CN116051695A - Image processing method and device thereof - Google Patents

Abstract

The application discloses an image processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: performing human body detection on image frame data of the video to obtain a human body detection frame comprising a target object; performing three-dimensional human body reconstruction on the target object according to the human body detection frame to obtain three-dimensional human body data of the target object under each view angle of N view angles; wherein the N view angles are different view angles taking a shooting view angle of the target object in the image frame data as a reference view angle, N is an integer greater than 0; and generating a target image corresponding to the image frame data according to the three-dimensional human body data and the image frame data of the target object under each view angle of the N view angles.

Description

Image processing method and device thereof

technical field

The present application belongs to the technical field of image processing, and in particular relates to an image processing method and device thereof.

Background technique

Human body multi-view action replication special effects are mostly seen in movie clips and short video platforms for mobile phones. At present, when realizing multi-view action copying, it first uses multiple cameras to shoot two-dimensional (2Dimensional, 2D) images of the same action from multiple perspectives, and then passes Video post-processing, superimposing other perspective 2D images on the reference 2D image to obtain a 2D special effect image, so as to realize a video with multi-view action replication special effects.

However, it requires the photographer to use multiple cameras, and each camera takes a two-dimensional image of a corresponding angle of view, and then performs post-processing of the video. The operation is cumbersome, and the requirements for camera equipment and post-processing are relatively high.

Contents of the invention

The purpose of the embodiments of the present application is to provide an image processing method and device thereof, so as to solve the problem that the electronic device cannot automatically realize the video with the special effect of multi-view motion replication.

In the first aspect, the embodiment of the present application provides an image processing method, the method comprising:

Human body detection is performed on the image frame data of the video to obtain a human body detection frame including the target object;

According to the human body detection frame, perform three-dimensional human body reconstruction on the target object, and obtain the three-dimensional human body data of the target object under each of the N viewing angles; wherein, the N viewing angles are based on the image frame data The shooting angle of view of the target object described in is a different angle of view of the reference angle of view, and N is an integer greater than 0;

Generate a target image corresponding to the image frame data according to the three-dimensional human body data at each of the N viewing angles and the image frame data.

In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

The detection module is used to detect the human body on the image frame data of the video, and obtain the human body detection frame including the target object;

A three-dimensional reconstruction module, configured to perform three-dimensional human body reconstruction on the target object according to the human body detection frame, and obtain three-dimensional human body data of the target object under each of the N viewing angles; wherein, the N viewing angles are Taking the shooting angle of view of the target object in the image frame data as a different angle of view of the reference angle of view, N is an integer greater than 0;

A generating module, configured to generate a target image corresponding to the image frame data according to the three-dimensional human body data and the image frame data in each of the N viewing angles.

In the third aspect, the embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, the memory stores programs or instructions that can run on the processor, and the programs or instructions are processed by the The steps of the method described in the first aspect are realized when the controller is executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .

In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.

In a sixth aspect, an embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the method described in the first aspect.

In the embodiment of this application, it will detect the human body on the image frame data of the video to obtain the human body detection frame including the target object, and perform three-dimensional human body reconstruction on the target object according to the human body detection frame to obtain each of the N viewing angles The 3D human body data of the target object, and then according to the 3D human body data and image frame data of each of the N viewing angles, the target image corresponding to the image frame data is generated, that is, the electronic device only needs to analyze the target in the image frame data of the video After 3D human body reconstruction, the target image corresponding to the image frame data with multi-angle motion replication effects can be obtained, and then a video with multi-angle motion replication effects can be realized. The operation is simple, and no redundant camera equipment and post-processing are required.

Description of drawings

Fig. 1 is a flow chart of the image processing method provided by the embodiment of the present application;

2 is a schematic diagram of the relationship between the image frame data, the 3D human body data of the target object under the reference viewing angle, the 3D human body data of the target object under different target viewing angles, and the target image corresponding to the image frame data in the embodiment of the present application;

FIG. 3 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

Fig. 5 is a schematic structural diagram of an electronic device provided by another embodiment of the present application.

Detailed ways

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or described herein and that references to "first", "second", etc. to distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

The image processing method provided by the embodiment of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

Please refer to FIG. 1 , which is a flowchart of an image processing method provided by an embodiment of the present application. The method can be applied to electronic equipment. The electronic device may be a mobile phone, a tablet computer, a notebook computer, and the like. As shown in FIG. 1, the method may include steps 1100 to 1300, which will be described in detail below.

Step 1100, perform human body detection on video image frame data to obtain a human body detection frame including a target object.

Wherein, the image frame data refers to the image frame data extracted by the electronic device during the video recording process or from the recorded video, and is used to make image frames with multi-view action replication special effects. Wherein, the target object is a person who needs to perform three-dimensional human body reconstruction in the image frame data.

In this embodiment, during the video recording process of the electronic device, the image frame at the current time point may be acquired in real time as the image frame data, or an image frame may be extracted every predetermined number of image frames (for example, every 5 image frames). image frame as image frame data. Further, human body detection is performed on the acquired image frame data to obtain a human body detection frame including the target object in the image frame data.

It can be understood that the human body detection frame may be a dotted line frame or a solid line frame. Of course, the human body detection frame may also be in other representation forms, which is not limited in this embodiment. Exemplarily, during video recording, the electronic device may acquire the image frame data 21 shown in FIG. 2 , and perform human body detection on the image frame data 21 to obtain the human body detection frame 22 including the target object 201 .

After performing step 1100 to carry out human body detection to the image frame data of the video, after obtaining the human body detection frame including the target object, enter:

Step 1200, perform 3D human body reconstruction on the target object according to the human body detection frame, and obtain 3D human body data of the target object in each of the N viewing angles.

Wherein, the N angles of view are different angles of view taking the shooting angle of view of the target object in the image frame data as a reference angle of view, and N is an integer greater than 0. Exemplarily, the shooting angle of view of the target object 201 in the image frame data 21 shown in FIG. 2 may be used as a reference angle of view, and correspondingly, the N angles of view may be three angles of view: left angle of view, top angle of view and right angle of view.

In some embodiments of the present application, step 1200 performs 3D human body reconstruction on the target object according to the human body detection frame, and obtains the 3D human body data of the target object under each of the N viewing angles, which may further include Steps 1210 to 1230 are as follows:

Step 1210, according to the human body detection frame, perform cropping processing on the image frame data to obtain a person image; wherein, the person image includes the target object.

In this step 1210, the electronic device will trim the image frame data according to the human body detection frame to obtain a person image including only the target object, and record the position information of the human body detection frame, for example, the human body detection frame 22 shown in FIG. 2 can be recorded. It can be understood that the position information of the human body detection frame recorded here is used for subsequent projection of the three-dimensional human body data of the target object under different viewing angles.

Step 1220, according to the person image, obtain first parameters for performing 3D human body reconstruction on the target object.

Wherein, the first parameter includes a body shape parameter β of the target object and a joint rotation parameter θ of the target object.

In this step 1220, the electronic device will input the cropped character image into the deep network of the human body reconstruction algorithm, and then obtain the body shape parameter β of the target object and the joint rotation parameter θ of the target object.

Step 1230, according to the first parameter and the preset 3D human body reconstruction model, obtain the 3D human body data of the target object in each of the N viewing angles.

Optionally, in this step 1230, according to the first parameter and the preset 3D human body reconstruction model, obtaining the 3D human body data of the target object in each of the N viewing angles may further include the following steps 1231 to 1233:

Step 1231, according to the body shape parameters of the target object, the joint rotation parameters of the target object and the preset 3D human body reconstruction model, obtain the first 3D vertex coordinate information of the target object under the reference viewing angle.

Wherein, the preset 3D human body reconstruction model may be a skinned multi-person linear model (Skinned Multi-PersonLinear Model, SMPL), or other models used for 3D human body reconstruction, which is not limited in this embodiment.

In this step 1231, the electronic device inputs the body shape parameter β of the target object and the joint rotation parameter θ of the target object into the SMPL model, so as to obtain the first three-dimensional vertex coordinate information of the target object under the reference viewing angle. Specifically, the first three-dimensional vertex coordinate information V of the target object under the reference viewing angle may include: position information of each of the 6890 three-dimensional vertices representing the target object under the reference viewing angle. It can be understood that by rendering the first 3D vertex information V of the target object under the reference viewing angle, the 3D human body data 202 of the target object under the reference viewing angle shown in FIG. 2 can be obtained.

Step 1232, according to the first 3D vertex coordinate information and the target viewing angle, obtain second 3D vertex coordinate information of the target object under the target viewing angle; wherein, the N viewing angles include the target viewing angle.

It can be understood that the second 3D vertex coordinate information of the target object under the target viewing angle may include: position information of each of the 6890 3D vertices representing the target object under the target viewing angle.

Optionally, in step 1232, according to the first 3D vertex coordinate information and the target viewing angle, obtaining the second 3D vertex coordinate information of the target object under the target viewing angle may further include: obtaining the target viewing angle relative to The second parameter of the reference viewing angle; wherein, the second parameter is the rotation parameter and translation parameter of the camera of the electronic device relative to the reference viewing angle at the target viewing angle; according to the second parameter and the first 3D vertex coordinate information, obtaining second 3D vertex coordinate information of the target object under the target viewing angle.

Specifically, the electronic device takes any of the N viewing angles as the target viewing angle i, and obtains the rotation parameter R_i and the translation parameter t_i of the camera of the electronic device relative to the reference viewing angle at the target viewing angle i, and according to the rotation parameter R_i and the translation parameter t_i, transforming the first 3D vertex coordinate information V of the target object under the reference viewing angle into the second 3D vertex coordinate information V_i of the target object under the target viewing angle. Wherein, the second three-dimensional vertex coordinate information V_i, the rotation parameter R_i and the translation parameter t_i of the target object under the target viewing angle i satisfy the following formula 1:

V_i＝R_i*V+t_i (1)

Step 1233: Render the second 3D vertex coordinate information to obtain 3D human body data of the target object under the target viewing angle.

In this step 1233, the electronic device can render the second 3D vertex coordinate information of the target object under the target viewing angle, and then obtain the 3D human body data M_3d of the target object under the target viewing angle, wherein the 3D human body data can be called Smoothing network for 3D human body surface. For example, referring to FIG. 2, the electronic device renders the second 3D vertex information of the target object in the left view, and then the 3D human body data 203 of the target object in the left view can be obtained, and the second 3D vertex information of the target object in the top view can be obtained. By rendering the 2D and 3D vertex information, the 3D human body data 204 of the target object in the top view can be obtained, and by rendering the second 3D vertex information of the target object in the right view, the 3D data of the target object in the right view can be obtained Human Body Data 205 .

After performing the above step 1200 to perform three-dimensional human body reconstruction on the target object according to the human body detection frame, and obtain the three-dimensional human body data of the target object under each of the N viewing angles, enter:

Step 1300: Generate a target image corresponding to the image frame data according to the 3D human body data of the target object in each of the N viewing angles and the image frame data.

Wherein, the target image corresponding to the image frame data can be understood as a two-dimensional special effect image obtained by performing multi-angle motion copy special effect processing on the image frame data.

In some embodiments of the present application, in step 1300, the target image corresponding to the image frame data is generated according to the three-dimensional human body data and the image frame data of the target object under each of the N viewing angles. It may further include: according to a preset projection model, project the 3D human body data of the target object under each of the N viewing angles to the first position corresponding to the viewing angle in the image frame data, and generate the image frame data the corresponding target image.

Specifically, based on the human body detection frame information of the target object, the rendered three-dimensional human body data M_3d corresponding to the N viewing angles can be projected to the corresponding positions in the image frame data to obtain the target image M_2d corresponding to the image frame data, which specifically meets the following requirements Formula 2:

M_2d＝π(M_3d) (2)

where π(·) represents the camera projection model.

Referring to FIG. 2 , the electronic device projects the three-dimensional human body data 203 of the target object in the left view to the left side of the target object 201 in the image frame data 21, and projects the three-dimensional human body data 204 of the target object in the top view to the image frame data The upper side of the target object 201 in 21, project the 3D human body data 205 of the target object in the right view to the right side of the target object 201 in the image frame data 21, and then obtain the reference view, the left view, the top view and the right view. Angle of view Action copied target image 22 from four angles of view.

According to the embodiment of the present application, it will detect the human body on the image frame data of the video, obtain the human body detection frame including the target object, and perform three-dimensional human body reconstruction on the target object according to the human body detection frame, so as to obtain each of the N viewing angles The 3D human body data of the target object, and then according to the 3D human body data and image frame data of each of the N viewing angles, generate the target image corresponding to the image frame data, that is, the electronic device only needs to analyze the target object in the image frame data of the video After 3D human body reconstruction, the target image corresponding to the image frame data with multi-angle motion replication effects can be obtained, and then a video with multi-angle motion replication effects can be realized. The operation is simple, and no redundant camera equipment and post-processing are required.

It should be noted that the image processing apparatus provided in the embodiment of the present application can implement various processes implemented in the method embodiment in FIG. 1 , and details are not repeated here to avoid repetition.

It should be noted that, the image processing method provided in the embodiment of the present application may be executed by an image processing device. In the embodiment of the present application, the image processing device executed by the image processing device is taken as an example to describe the image processing device provided in the embodiment of the present application.

Referring to FIG. 3 , the embodiment of the present application further provides an image processing device 300 , and the image processing device 300 includes a detection module 301 , a three-dimensional reconstruction module 302 , and a generation module 303 .

The detection module 301 is used to detect the human body on the image frame data of the video, and obtain the human body detection frame including the target object;

The three-dimensional reconstruction module 302 is configured to perform three-dimensional human body reconstruction on the target object according to the human body detection frame, and obtain three-dimensional human body data of the target object under each of the N viewing angles; wherein, the N viewing angles are different viewing angles with the shooting viewing angle of the target object in the image frame data as the reference viewing angle, and N is an integer greater than 0;

The generating module 303 is configured to generate a target image corresponding to the image frame data according to the 3D human body data of the target object under each of the N viewing angles and the image frame data.

In one embodiment, the 3D reconstruction module 302 is specifically used for:

According to the human body detection frame, performing cropping processing on the image frame data to obtain a person image; wherein the person image includes the target object;

Obtaining first parameters for performing three-dimensional human body reconstruction on the target object according to the person image; wherein, the first parameters include body shape parameters of the target object and joint rotation parameters of the target object;

According to the first parameter and the preset three-dimensional human body reconstruction model, the three-dimensional human body data of the target object in each of the N viewing angles is obtained.

In one embodiment, the 3D reconstruction module 302 is specifically used for:

Obtaining the first three-dimensional vertex coordinate information of the target object under the reference viewing angle according to the body shape parameters of the target object, the joint rotation parameters of the target object, and the preset three-dimensional human body reconstruction model;

Obtaining second 3D vertex coordinate information of the target object under the target viewing angle according to the first 3D vertex coordinate information and the target viewing angle; wherein, the N viewing angles include the target viewing angle;

According to the second 3D vertex coordinate information, the 3D human body data of the target object under the target viewing angle is obtained.

In one embodiment, the 3D reconstruction module 302 is specifically used for:

Obtaining a second parameter of the target viewing angle relative to the reference viewing angle; wherein, the second parameter is a rotation parameter and a translation parameter of the camera of the electronic device at the target viewing angle relative to the reference viewing angle;

According to the second parameter and the first three-dimensional vertex coordinate information, second three-dimensional vertex coordinate information of the target object under the target viewing angle is obtained.

In one embodiment, the generating module 303 is specifically configured to:

According to a preset projection model, project the three-dimensional human body data of the target object under each of the N viewing angles to the first position corresponding to the viewing angle in the image frame data, and generate a corresponding image frame data. target image.

In the embodiment of this application, it will detect the human body on the image frame data of the video to obtain the human body detection frame including the target object, and perform three-dimensional human body reconstruction on the target object according to the human body detection frame to obtain each of the N viewing angles The 3D human body data of the target object, and then according to the 3D human body data and image frame data of each of the N viewing angles, the target image corresponding to the image frame data is generated, that is, the electronic device only needs to analyze the target in the image frame data of the video After 3D human body reconstruction, the target image corresponding to the image frame data with multi-angle motion replication effects can be obtained, and then a video with multi-angle motion replication effects can be realized. The operation is simple, and no redundant camera equipment and post-processing are required.

The image processing apparatus in the embodiment of the present application may be an electronic device, or may be a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or other devices other than the terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) Devices, robots, wearable devices, ultra-mobile personal computers (ultra-mobile personalcomputer, UMPC), netbooks or personal digital assistants (personal digital assistant, PDA), etc., can also serve as servers, network attached storage (Network Attached Storage, NAS) , a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which are not specifically limited in this embodiment of the present application.

The image processing device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The image processing apparatus provided in the embodiment of the present application can realize various processes realized in the method embodiment in FIG. 1 , and details are not repeated here to avoid repetition.

Optionally, as shown in FIG. 4 , the embodiment of the present application also provides an electronic device 400, including a processor 401 and a memory 402, and the memory 402 stores programs or instructions that can run on the processor 401. The When the programs or instructions are executed by the processor 401, the steps of the above-mentioned image processing method embodiments can be realized, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 5 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 500 includes but not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510 and an image Process chips and other components.

Those skilled in the art can understand that the electronic device 500 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 510 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 5 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components, and details will not be repeated here. .

Wherein, the processor 510 is configured to perform human body detection on the image frame data of the video to obtain a human body detection frame including the target object; perform three-dimensional human body reconstruction on the target object according to the human body detection frame, and obtain each of the N viewing angles The three-dimensional human body data of the target object under a viewing angle; wherein, the N viewing angles are different viewing angles based on the shooting angle of view of the target object in the image frame data, and N is an integer greater than 0; according to The three-dimensional human body data and the image frame data of the target object in each of the N viewing angles are used to generate a target image corresponding to the image frame data.

In the embodiment of this application, it will detect the human body on the image frame data of the video to obtain the human body detection frame including the target object, and perform three-dimensional human body reconstruction on the target object according to the human body detection frame to obtain each of the N viewing angles The 3D human body data of the target object, and then according to the 3D human body data and image frame data of each of the N viewing angles, the target image corresponding to the image frame data is generated, that is, the electronic device only needs to analyze the target in the image frame data of the video After 3D human body reconstruction, the target image corresponding to the image frame data with multi-angle motion replication effects can be obtained, and then a video with multi-angle motion replication effects can be realized. The operation is simple, and no redundant camera equipment and post-processing are required.

In one embodiment, the processor 510 is specifically configured to perform cropping processing on the image frame data according to the human body detection frame to obtain a person image; wherein, the person image includes the target object; according to the person image , to obtain the first parameters for three-dimensional human body reconstruction of the target object; wherein, the first parameters include the body shape parameters of the target object and the joint rotation parameters of the target object; according to the first parameters and A 3D human body reconstruction model is preset to obtain 3D human body data of the target object in each of the N viewing angles.

In one embodiment, the processor 510 is specifically configured to obtain the target under the reference viewing angle according to the body shape parameters of the target object, the joint rotation parameters of the target object, and the preset three-dimensional human body reconstruction model. The first three-dimensional vertex coordinate information of the object; according to the first three-dimensional vertex coordinate information and the target viewing angle, obtain the second three-dimensional vertex coordinate information of the target object under the target viewing angle; wherein, the N viewing angles include the the target viewing angle; rendering the second three-dimensional vertex coordinate information to obtain the three-dimensional human body data of the target object under the target viewing angle.

In one embodiment, the processor 510 is specifically configured to obtain a second parameter of the target viewing angle relative to the reference viewing angle; wherein, the second parameter is the relative The rotation parameter and the translation parameter of the reference viewing angle; according to the second parameter and the first 3D vertex coordinate information, obtain the second 3D vertex coordinate information of the target object under the target viewing angle.

In one embodiment, the processor 510 is specifically configured to project, according to a preset projection model, the three-dimensional human body data of the target object under each of the N viewing angles into the image frame data corresponding to the viewing angle In the first position, generate the target image corresponding to the image frame data.

It should be understood that, in the embodiment of the present application, the input unit 504 may include a graphics processor (Graphics Processing Unit, GPU) 5041 and a microphone 5042, and the graphics processor 5041 is compatible with the image capture device (such as Camera) to process the image data of still pictures or videos. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072 . The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts, a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.

The memory 509 can be used to store software programs as well as various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required by at least one function (such as a sound playing function, image playback function, etc.), etc. Furthermore, memory 509 may include volatile memory or nonvolatile memory, or, memory 509 may include both volatile and nonvolatile memory. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synch link DRAM , SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM). The memory 509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

The processor 510 may include one or more processing units; optionally, the processor 510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the foregoing modem processor may not be integrated into the processor 510 .

The embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or an instruction, and when the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is realized, and can achieve the same To avoid repetition, the technical effects will not be repeated here.

Wherein, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and the like.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the above image processing method embodiment Each process can achieve the same technical effect, so in order to avoid repetition, it will not be repeated here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

The embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the various processes in the above image processing method embodiment, and can achieve the same technical effect , to avoid repetition, it will not be repeated here.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims (10)

1. An image processing method, the method comprising:

performing human body detection on image frame data of the video to obtain a human body detection frame comprising a target object;

performing three-dimensional human body reconstruction on the target object according to the human body detection frame to obtain three-dimensional human body data of the target object under each view angle of N view angles; wherein the N view angles are different view angles taking a shooting view angle of the target object in the image frame data as a reference view angle, N is an integer greater than 0;

and generating a target image corresponding to the image frame data according to the three-dimensional human body data and the image frame data of the target object under each view angle of the N view angles.

2. The method according to claim 1, wherein the reconstructing the three-dimensional human body of the target object according to the human body detection frame to obtain three-dimensional human body data of the target object at each of N view angles includes:

cutting the image frame data according to the human body detection frame to obtain a figure image; wherein the character image includes the target object;

obtaining a first parameter for reconstructing a three-dimensional human body of the target object according to the figure image; wherein the first parameter includes a body type parameter of the target object and a joint rotation parameter of the target object;

and obtaining three-dimensional human body data of the target object under each view angle of N view angles according to the first parameters and a preset three-dimensional human body reconstruction model.

3. The method according to claim 2, wherein the obtaining three-dimensional human body data of the target object at each of N view angles according to the first parameter and a preset three-dimensional human body reconstruction model includes:

obtaining first three-dimensional vertex coordinate information of the target object under the reference view angle according to the body type parameter of the target object, the joint rotation parameter of the target object and the preset three-dimensional human body reconstruction model;

obtaining second three-dimensional vertex coordinate information of the target object under the target view angle according to the first three-dimensional vertex coordinate information and the target view angle; wherein the N views include the target view;

and rendering the second three-dimensional vertex coordinate information to obtain three-dimensional human body data of the target object under the target view angle.

4. A method according to claim 3, wherein said obtaining second three-dimensional vertex coordinate information of said target object at said target perspective from said first three-dimensional vertex coordinate information and target perspective comprises:

acquiring a second parameter of the target viewing angle relative to the reference viewing angle; the second parameter is a rotation parameter and a translation parameter of a camera of the electronic device relative to the reference view angle when the target view angle is;

and obtaining second three-dimensional vertex coordinate information of the target object under the target view angle according to the second parameter and the first three-dimensional vertex coordinate information.

5. The method according to claim 1, wherein generating the target image corresponding to the image frame data from the three-dimensional human body data and the image frame data of the target object for each of the N views includes:

and according to a preset projection model, projecting the three-dimensional human body data of the target object under each view angle of N view angles to a first position corresponding to the view angle in the image frame data, and generating a target image corresponding to the image frame data.

6. An image processing apparatus, characterized in that the apparatus comprises:

the detection module is used for detecting the human body of the image frame data of the video to obtain a human body detection frame comprising a target object;

the three-dimensional reconstruction module is used for carrying out three-dimensional human body reconstruction on the target object according to the human body detection frame to obtain three-dimensional human body data of the target object under each view angle of N view angles; wherein the N view angles are different view angles taking a shooting view angle of the target object in the image frame data as a reference view angle, N is an integer greater than 0;

and the generation module is used for generating a target image corresponding to the image frame data according to the three-dimensional human body data of the target object and the image frame data under each view angle of the N view angles.

7. The apparatus of claim 6, wherein the three-dimensional reconstruction module is configured to:

cutting the image frame data according to the human body detection frame to obtain a figure image; wherein the character image includes the target object;

obtaining a first parameter for reconstructing a three-dimensional human body of the target object according to the figure image; wherein the first parameter includes a body type parameter of the target object and a joint rotation parameter of the target object;

and obtaining three-dimensional human body data of the target object under each view angle of N view angles according to the first parameters and a preset three-dimensional human body reconstruction model.

8. The apparatus of claim 7, wherein the three-dimensional reconstruction module is configured to:

obtaining first three-dimensional vertex coordinate information of the target object under the reference view angle according to the body type parameter of the target object, the joint rotation parameter of the target object and the preset three-dimensional human body reconstruction model;

obtaining second three-dimensional vertex coordinate information of the target object under the target view angle according to the first three-dimensional vertex coordinate information and the target view angle; wherein the N views include the target view;

and obtaining three-dimensional human body data of the target object under the target visual angle according to the second three-dimensional vertex coordinate information.

9. The apparatus of claim 8, wherein the three-dimensional reconstruction module is configured to:

acquiring a second parameter of the target viewing angle relative to the reference viewing angle; the second parameter is a rotation parameter and a translation parameter of a camera of the electronic device relative to the reference view angle when the target view angle is;

and obtaining second three-dimensional vertex coordinate information of the target object under the target view angle according to the second parameter and the first three-dimensional vertex coordinate information.

10. The apparatus of claim 6, wherein the generating module is specifically configured to:

and according to a preset projection model, projecting the three-dimensional human body data of the target object under each view angle of N view angles to a first position corresponding to the view angle in the image frame data, and generating a target image corresponding to the image frame data.

Images