WO2020037679A1 - 视频处理方法、装置及电子设备 - Google Patents

视频处理方法、装置及电子设备 Download PDF

Info

Publication number
WO2020037679A1
WO2020037679A1 PCT/CN2018/102332 CN2018102332W WO2020037679A1 WO 2020037679 A1 WO2020037679 A1 WO 2020037679A1 CN 2018102332 W CN2018102332 W CN 2018102332W WO 2020037679 A1 WO2020037679 A1 WO 2020037679A1
Authority
WO
WIPO (PCT)
Prior art keywords
face image
target
dimensional
video
expression
Prior art date
Application number
PCT/CN2018/102332
Other languages
English (en)
French (fr)
Inventor
李建亿
朱利明
Original Assignee
太平洋未来科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 太平洋未来科技（深圳）有限公司 filed Critical 太平洋未来科技（深圳）有限公司
Priority to PCT/CN2018/102332 priority Critical patent/WO2020037679A1/zh
Priority to CN201811029389.7A priority patent/CN109151340B/zh
Publication of WO2020037679A1 publication Critical patent/WO2020037679A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction

Definitions

  • the present invention relates to the field of image processing technology, and in particular, to a video processing method, device, and electronic device.
  • the generation technology of virtual objects is more and more applied in video production.
  • the reconstruction of three-dimensional human faces is particularly important for the generation of virtual objects.
  • the two-dimensional face image is mainly obtained through the camera of the mobile phone, and the effect of the reconstructed three-dimensional face image depends partly on the image quality obtained by the previous camera device, and the obtained image quality depends partly on the processing effect of the shake when the mobile phone is taken.
  • mobile phones mainly perform anti-shake processing through software, and there are not many measures for hardware improvement.
  • the video processing method, device and electronic equipment provided by the embodiments of the present invention are used to solve at least the foregoing problems in related technologies.
  • An embodiment of the present invention provides a video processing method, including:
  • a convolutional neural network model In response to a user instruction, obtain a first face image in a target picture and a second face image to be replaced in a video; use a convolutional neural network model to output a first three-dimensional face image corresponding to the first face image Obtaining an expression parameter of the second face image, adjusting the first three-dimensional face image according to the expression parameter to obtain a second three-dimensional face image, and mapping the second three-dimensional face image to two Obtain the target face image in the dimension space, and replace the second face image with the target face image.
  • obtaining the expression parameters of the second face image and adjusting the first three-dimensional face image according to the expression parameters includes obtaining the second face image at a preset feature point. A first expression parameter; obtaining a second expression parameter of the first three-dimensional face image at the preset feature point; replacing the second expression parameter with the first expression parameter.
  • replacing the second face image with the target face image includes: adjusting the size of the target face image according to attribute information of a person corresponding to the second face image, and The second face image is replaced with the adjusted target face image.
  • the method further comprises: acquiring illumination information in the video, and processing the target face image according to the illumination information.
  • the processing the target face image according to the illumination information includes: determining a direction of incident light according to a position of the target face image in the video frame; and according to the illumination information and all The direction of the incident light is used to generate the lighting effect of the target face image.
  • the target picture is obtained through an image acquisition device, which includes a lens, an autofocus voice coil motor, a mechanical image stabilizer, and an image sensor, and the lens is fixed on the autofocus voice coil motor,
  • the lens is used to acquire an image
  • the image sensor transmits the image acquired by the lens to the recognition module
  • the autofocus voice coil motor is mounted on the mechanical image stabilizer
  • the processing module is based on the inside of the lens
  • the feedback of the lens shake detected by the gyroscope drives the action of the mechanical image stabilizer to achieve lens shake compensation.
  • the mechanical image stabilizer includes a movable plate, a base plate, and a compensation mechanism.
  • Each of the movable plate and the base plate is provided with a through hole through which the lens passes, and the auto-focusing voice coil motor is installed at
  • the movable plate is mounted on the substrate, and the size of the substrate is larger than the movable plate.
  • the compensation mechanism drives the movable plate and the movable plate under the driving of the processing module.
  • the lens moves to achieve lens shake compensation;
  • the compensation mechanism includes a first compensation component, a second compensation component, a third compensation component, and a fourth compensation component installed around the substrate, wherein the first compensation component and The third compensation component is disposed opposite to each other, the second compensation component is disposed opposite to the fourth compensation component, and a line between the first compensation component and the third compensation component is connected to the first compensation component and the first compensation component.
  • the lines between the three compensation components are perpendicular to each other; the first compensation component, the second compensation component, the third compensation component, and the fourth compensation component all include a driving member, a rotating shaft, and a one-way bearing.
  • the driving member is controlled by the processing module, and the driving member is drivingly connected to the rotating shaft to drive the rotating shaft to rotate;
  • the rotating shaft is connected to the inner ring of the one-way bearing to Driving the inner ring of the one-way bearing to rotate;
  • the rotating ring gear is sleeved on the one-way bearing and connected to the outer ring of the one-way bearing, and an outer surface of the rotating ring gear is provided with a ring in its circumferential direction External teeth
  • the bottom surface of the movable plate is provided with a plurality of rows of strip grooves arranged at even intervals, the strip grooves are engaged with the external teeth, and the external teeth can slide along the length direction of the strip grooves ;
  • the rotatable direction of the one-way bearing of the first compensation component is opposite to the rotatable direction of the one-way bearing of the third compensation component, and the rotatable direction of the one-way bearing of the second compensation component is different from that The rotatable direction of the one-way
  • the driving member is a micro motor, the micro motor is electrically connected to the processing module, and a rotary output end of the micro motor is connected to the rotating shaft; or the driving member includes a memory alloy wire and a crank A connecting rod, one end of the memory alloy wire is fixed on the fixing plate and connected with the processing module through a circuit, and the other end of the memory alloy wire is connected with the rotating shaft through the crank connecting rod to drive The rotation shaft rotates.
  • the image acquisition device is provided on a mobile phone
  • the mobile phone includes a bracket, the bracket includes a mobile phone mount and a retractable support rod;
  • the mobile phone mount includes a retractable connection plate and two opposite ends of the connection plate.
  • a folding plate group one end of the supporting rod is connected with the middle of the connecting plate through a damping hinge;
  • the folding plate group includes a first plate body, a second plate body and a third plate body, wherein the first plate One of the opposite ends of the body is hinged to the connection plate, and the other of the opposite ends of the first plate body is hinged to one of the opposite ends of the second plate body; the first The other end of the opposite ends of the two plates is hinged to one of the opposite ends of the third plate;
  • the second plate is provided with an opening for the corner of the mobile phone to be inserted; and the mobile phone mount is used to install a mobile phone.
  • the first plate body, the second plate body, and the third plate body are folded in a right triangle state, the second plate body is a hypotenuse of a right triangle, and the first plate body and the third plate body are A right-angled side of a right triangle, wherein the first A side plate and a side surface of the connecting plate bonded side by side, one of the opposite ends of the third plate member and the other end opposite ends of the first plate against body.
  • one side of the third plate body is provided with a first connection portion, and a side surface of the connection plate that is in contact with the third plate body is provided with a first fit that is matched with the first connection portion.
  • a second connection portion is provided on one end of the opposite ends of the first plate body, and a second connection is provided on the other end of the opposite ends of the third plate body to cooperate with the second connection portion.
  • the other end of the support rod is detachably connected with a base.
  • An acquisition module configured to acquire a first face image in a target picture and a second face image to be replaced in a video in response to a user's instruction; an output module, configured to output the first person using a convolutional neural network model A first three-dimensional face image corresponding to the face image; an adjustment module configured to obtain expression parameters of the second face image, and adjust the first three-dimensional face image according to the expression parameters to obtain a second three-dimensional face image A face image; a replacement module, configured to map the second three-dimensional face image to a two-dimensional space to obtain a target face image, and replace the second face image with the target face image.
  • the adjustment module is specifically configured to obtain a first expression parameter of the second face image at a preset feature point; obtain a second expression of the first three-dimensional face image at the preset feature point Parameter; replacing the second expression parameter with the first expression parameter.
  • the replacement module is specifically configured to adjust the size of the target face image according to the attribute information of the person corresponding to the second face image, and replace the second face image with the adjusted position.
  • the target face image is described.
  • the apparatus further includes a processing module, configured to acquire lighting information in the video, and process the target face image according to the lighting information.
  • the processing module is specifically configured to determine a direction of incident light according to a position of the target face image in the video frame; and generate the target person according to the illumination information and the direction of the incident light. Lighting effect of face image.
  • Another aspect of the embodiments of the present invention provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein,
  • the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute any one of the video processing methods in the foregoing embodiments of the present invention. .
  • the video processing method, device and electronic equipment provided by the embodiments of the present invention do not need to collect multiple pictures from multiple angles, and only need a single picture to replace the face in the video, thereby obtaining the target face ; At the same time, considering the influence of facial expression factors, the relevance of the target face to the video is improved. On the other hand, by improving the image stabilization structure of the image acquisition device, the image acquisition quality is improved.
  • FIG. 1 is a flowchart of a video processing method according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a video processing method according to an embodiment of the present invention.
  • FIG. 3 is a structural diagram of a video processing apparatus according to an embodiment of the present invention.
  • FIG. 4 is a structural diagram of a video processing apparatus according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a hardware structure of an electronic device that executes a video processing method provided by an embodiment of the method of the present invention
  • FIG. 6 is a structural diagram of an image acquisition device according to an embodiment of the present invention.
  • FIG. 7 is a structural diagram of an optical image stabilizer provided by an embodiment of the present invention.
  • FIG. 8 is an enlarged view of a portion A of FIG. 7;
  • FIG. 9 is a schematic bottom view of a movable plate of a micro memory alloy optical image stabilizer provided by an embodiment of the present invention.
  • FIG. 10 is a structural diagram of a stent provided by an embodiment of the present invention.
  • FIG. 11 is a schematic diagram of a state of a stent provided by an embodiment of the present invention.
  • FIG. 12 is another schematic state diagram of a stent according to an embodiment of the present invention.
  • FIG. 13 is a structural state diagram of a mounting base and a mobile phone provided according to an embodiment of the present invention.
  • FIG. 1 is a flowchart of a video processing method according to an embodiment of the present invention.
  • a video processing method provided by an embodiment of the present invention includes:
  • the user wants to interact with the video and replace the second face image in the video with the first face image in the target picture.
  • the target picture may be a picture stored in the user's electronic device, or may be a picture currently taken by the user.
  • the instruction may be a user's operation on the interactive identification, or a preset user operation trajectory, which is not limited in the present invention.
  • the second face image to be replaced and the first face image of the target picture in the video corresponding to the instruction are obtained.
  • the CNN is a deep feedforward artificial neural network.
  • the basic structure of CNN includes two layers. One is a feature extraction layer. The input of each neuron is connected to the local acceptance domain of the previous layer and the local features are extracted. Once the local feature is extracted, it is connected to other features. The positional relationship between them is also determined; the second is the feature mapping layer.
  • Each computing layer of the network consists of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal.
  • Convolutional neural networks usually include one-dimensional convolutional neural networks, two-dimensional convolutional neural networks, and three-dimensional convolutional neural networks. A large number of mathematical models of these convolutional neural networks have been introduced in the prior art. The type of convolutional neural network is limited.
  • the convolutional neural network model needs to be trained in advance.
  • the two-dimensional face image in the input sample set is used as the input end of the model, and the three-dimensional face image corresponding to the two-dimensional face image is used as the output end of the model. .
  • Gaussian algorithm 68 Channels
  • the three-dimensional face image is adjusted to the face direction according to the face direction, and voxelized reconstruction is performed.
  • a regression algorithm is used to calculate the cross-entropy loss function (normalized mean square error), and the loss function is converged to a minimum value through training, and the convolutional neural network model training is completed.
  • the first face image is input into a trained convolutional neural network model, thereby obtaining a first three-dimensional face image corresponding to the first face image.
  • S103 Obtain an expression parameter of the second face image, and adjust the first three-dimensional face image according to the expression parameter to obtain a second three-dimensional face image.
  • the expression of the person in the second face image in the video is related to the content played in the video, so the expression in the first three-dimensional face image needs to be adjusted according to the expression parameters in the second face image.
  • a key point capable of identifying a facial expression can be preset as a feature point
  • a first facial expression parameter of a second facial image at a preset characteristic point can be acquired
  • the preset characteristic point at a first three-dimensional human face can be acquired.
  • the attribute information includes, but is not limited to, a face shape, a facial feature, a figure, and the like of the person corresponding to the second face image.
  • the person's figure is matched, and the second face image is replaced with the adjusted target face image.
  • the video processing method provided by the embodiment of the present invention does not need to collect multiple pictures from multiple angles, and only needs a single picture to replace the face in the video to obtain the target face; at the same time, considering the influence of facial expression factors, Improved the relevance of the target face to the video.
  • FIG. 2 is a flowchart of a video processing method according to an embodiment of the present invention. As shown in FIG. 2, this embodiment is a specific implementation solution of the embodiment shown in FIG. 1, so the detailed implementation method and beneficial effects of each step in the embodiment shown in FIG. 1 are not described again.
  • the video processing provided by the embodiment of the present invention Methods including:
  • S202 Use a convolutional neural network model to output a first three-dimensional face image corresponding to the first face image.
  • S203. Acquire expression parameters of the second face image, and adjust the first three-dimensional face image according to the expression parameters to obtain a second three-dimensional face image.
  • the target face image is taken from the target picture, there may be situations where its lighting effect is different from the lighting effect of the video scene and cannot be integrated with the video scene, so that the user feels the inconsistency between the two visually. Therefore, it is also necessary to perform a light and shadow effect processing on the target image. Specifically, it can be performed by the following steps.
  • S205 Acquire illumination information in the video, and process the target face image according to the illumination information.
  • the direction of the incident light is first determined according to the position of the target face image in the video screen, that is, the position direction of the target face image in the video screen can be used as the direction of the reflected light to obtain
  • the normal direction of any point on the target's face is calculated by the light's reflection law.
  • the direction of the light source's incidence can also be obtained separately.
  • the normal directions of several points on the target's face can also be obtained separately.
  • Several light sources will be obtained by the law of light reflection.
  • An average value of the incident directions is obtained to obtain an average incident direction of the light source, which is not limited in the present invention.
  • an illumination effect of the target face image is generated. Specifically, target illumination information of the target face is calculated according to the direction of the incident light and the illumination information; and a lighting effect of the target face image is generated according to the target illumination information.
  • the reflection of the human face can generally be regarded as a specular reflection. Because the specular reflection will cause the target face to reflect light, making the target face appear unclear in the user's eyes. Therefore, when the lighting information is known, it can be based on the target person.
  • the roughness of the face surface calculates the target illumination information of the incident light reflected on the target face surface within a certain range around the incident direction. Obtain a clear illumination effect of the target face by expanding the range of the incident direction, because the smoother the surface of the target face, the stronger the specular reflection effect. If the target illumination information is obtained based only on the incident direction determined in step S101, the result is The light direction is unique, and the lighting effect of the target face will be blurred due to the specular emission.
  • the target illumination information includes the illumination intensity and direction of the emitted light, and the illumination effect of the target face is generated according to the illumination intensity and direction of the emitted light.
  • the video processing method provided by the embodiment of the present invention does not need to collect multiple pictures from multiple angles, and only needs a single picture to replace the face in the video to obtain the target face; at the same time, considering the influence of facial expression factors, Improved the relevance of the target face to the video.
  • the fusion degree between the target face and the video scene is improved, and the user's interactive experience is improved.
  • FIG. 3 is a structural diagram of a video processing apparatus according to an embodiment of the present invention. As shown in FIG. 3, the device specifically includes: an acquisition module 100, an output module 200, an adjustment module 300, and a replacement module 400. among them,
  • An obtaining module 100 is configured to obtain a first face image in a target picture and a second face image to be replaced in a video in response to an instruction of a user; an output module 200 is configured to output the first face image using a convolutional neural network model.
  • a replacement module 400 configured to map the second three-dimensional face image to a two-dimensional space to obtain a target face image, and replace the second face image with the target face image.
  • the adjustment module 300 is specifically configured to obtain a first expression parameter of the second face image at a preset feature point; obtain a second expression of the first three-dimensional face image at the preset feature point Parameter; replacing the second expression parameter with the first expression parameter.
  • the replacement module 400 is specifically configured to adjust the size of the target face image according to the attribute information of the person corresponding to the second face image, and replace the second face image with the adjusted position.
  • the target face image is described.
  • the video processing apparatus provided by the embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 1, and its implementation principles, methods, and functional uses are similar to the embodiment shown in FIG. 1, and details are not described herein again.
  • FIG. 4 is a structural diagram of a video processing apparatus according to an embodiment of the present invention.
  • the device specifically includes: an acquisition module 100, an output module 200, an adjustment module 300, a replacement module 400, and a processing module 500. among them,
  • An obtaining module 100 is configured to obtain a first face image in a target picture and a second face image to be replaced in a video in response to an instruction of a user; an output module 200 is configured to output the first face image using a convolutional neural network model.
  • a replacement module 400 configured to map the second three-dimensional face image to a two-dimensional space to obtain a target face image, and replace the second face image with the target face image;
  • the processing module 500 is configured to obtain illumination information in the video, and process the target face image according to the illumination information.
  • the processing module 500 is specifically configured to determine a direction of incident light according to a position of the target face image in the video frame; and generate the target person according to the illumination information and the direction of the incident light. Lighting effect of face image.
  • the adjustment module 300 is specifically configured to obtain a first expression parameter of the second face image at a preset feature point; obtain a second expression of the first three-dimensional face image at the preset feature point Parameter; replacing the second expression parameter with the first expression parameter.
  • the replacement module 400 is specifically configured to adjust the size of the target face image according to the attribute information of the person corresponding to the second face image, and replace the second face image with the adjusted position.
  • the target face image is described.
  • the video processing apparatus provided by the embodiment of the present invention is specifically configured to execute the method provided by the embodiment shown in FIG. 2, and its implementation principles, methods, and functional uses are similar to the embodiment shown in FIG. 2, and details are not described herein again.
  • the above-mentioned video processing apparatus may be used as one of the software or hardware functional units, which are independently provided in the above-mentioned electronic device, or may be used as one of the functional modules integrated in the processor to perform the video processing of the embodiments of the present invention. method.
  • FIG. 5 is a schematic diagram of a hardware structure of an electronic device that executes a video processing method provided by an embodiment of the method of the present invention.
  • the electronic device includes:
  • One or more processors 510 and a memory 520 are taken as an example in FIG. 5.
  • the device for performing the video processing method may further include an input device 530 and an output device 530.
  • the processor 510, the memory 520, the input device 530, and the output device 540 may be connected through a bus or other methods.
  • the connection through the bus is taken as an example.
  • the memory 520 is a non-volatile computer-readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, as corresponding to the video processing method in the embodiment of the present invention.
  • the processor 510 executes various functional applications of the server and data processing by running non-volatile software programs, instructions, and modules stored in the memory 520, that is, implementing the video processing method.
  • the memory 520 may include a storage program area and a storage data area, where the storage program area may store an operating system and an application program required for at least one function; the storage data area may store a program created by using a video processing apparatus according to an embodiment of the present invention Data, etc.
  • the memory 520 may include a high-speed random access memory 520, and may further include a non-volatile memory 520, for example, at least one magnetic disk memory 520, a flash memory device, or other non-volatile solid-state memory 520.
  • the memory 520 may optionally include a memory 520 remotely disposed with respect to the processor 55, and these remote memories 520 may be connected to the video processing device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the input device 530 may receive inputted numeric or character information, and generate key signal inputs related to user settings and function control of the video processing device.
  • the input device 530 may include a device such as a pressing module.
  • the one or more modules are stored in the memory 520, and when executed by the one or more processors 510, execute the video processing method.
  • the electronic devices in the embodiments of the present invention exist in various forms, including but not limited to:
  • Mobile communication equipment This type of equipment is characterized by mobile communication functions, and its main goal is to provide voice and data communication.
  • Such terminals include: smart phones (such as iPhone), multimedia phones, feature phones, and low-end phones.
  • Ultra-mobile personal computer equipment This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has the characteristics of mobile Internet access.
  • Such terminals include: PDA, MID and UMPC devices, such as iPad.
  • Portable entertainment equipment This type of equipment can display and play multimedia content.
  • Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.
  • an image acquisition device for acquiring an image is provided on the electronic device, and a software or hardware image stabilizer is often provided on the image acquisition device to ensure the quality of the acquired image.
  • Most of the existing image stabilizers are powered by coils that generate Loren magnetic force in the magnetic field to drive the lens.
  • the lens needs to be driven in at least two directions, which means that multiple coils need to be arranged, which will give the whole.
  • the miniaturization of the structure brings certain challenges, and it is easy to be affected by external magnetic fields, which will affect the anti-shake effect. Therefore, the Chinese patent published as CN106131435A provides a miniature optical anti-shake camera module, which realizes memory alloy wires through temperature changes.
  • the control chip of the micro memory alloy optical anti-shake actuator can control the change of the driving signal to change the temperature of the memory alloy wire. Control the elongation and shortening of the memory alloy wire, and calculate the position and moving distance of the actuator based on the resistance of the memory alloy wire. When the micro memory alloy optical image stabilization actuator moves to the specified position, the resistance of the memory alloy wire at this time is fed back. By comparing the deviation of this resistance value and the target value, the movement on the micro memory alloy optical image stabilization actuator can be corrected. deviation.
  • the above technical solution can compensate the lens for the shake in the first direction, but when the subsequent shake in the second direction occurs, it is too late due to the memory alloy wire. Deformation in an instant, so it is easy to cause untimely compensation, and it is impossible to accurately realize lens shake compensation for multiple shakes and continuous shakes in different directions. Therefore, it is necessary to improve its structure in order to obtain better image quality and facilitate subsequent 3D Image generation.
  • this embodiment improves the learning image stabilization device and design it as a mechanical image stabilization device 3000.
  • the specific structure is as follows:
  • the mechanical image stabilizer 3000 of this embodiment includes a movable plate 3100, a base plate 3200, and a compensation mechanism 3300.
  • Each of the movable plate 3100 and the base plate 3200 is provided with a through hole through which the lens 1000 passes.
  • An autofocus voice coil motor 2000 is mounted on the movable plate 3100, and the movable plate 3100 is mounted on the base plate 3200.
  • the size of the base plate 3200 is larger than the movable plate 3100, and the movable plate 3100 passes above it.
  • the auto-focusing voice coil motor limits its up and down movement, and the compensation mechanism 3300 drives the movable plate 3100 and the lens 1000 on the movable plate 3100 to move under the driving of the processing module to achieve shake compensation of the lens 1000.
  • the compensation mechanism 3300 in this embodiment includes a first compensation component 3310, a second compensation component 3320, a third compensation component 3330, and a fourth compensation component 3340 installed around the substrate 3200.
  • a compensation component 3310 and the third compensation component 3330 are disposed opposite to each other, the second compensation component 3320 is disposed opposite to the fourth compensation component 3340, and a connection line between the first compensation component 3310 and the third compensation component 3330
  • the connection lines between the first compensation component 3310 and the third compensation component 3330 are perpendicular to each other, that is, a compensation component, a second compensation component 3320, and a third compensation component 3330 are respectively arranged in the front, rear, left, and right directions of the movable plate 3100.
  • the first compensation component 3310 can make the movable plate 3100 move forward
  • the third compensation component 3330 can make the movable plate 3100 move backward
  • the second compensation component 3320 can make the movable plate 3100 move left
  • the fourth compensation component 3340 can make The movable plate 3100 moves to the left
  • the first compensation component 3310 can cooperate with the second compensation component 3320 or the fourth compensation component 3340 to realize the operation of the movable plate 3100 in an inclined direction.
  • the third component 3330 may be compensated 1000 compensation and the second compensation component 3320 or the fourth compensation component 3340 cooperate to achieve movement of the movable plate 3100 to the tilt direction, the lens implemented in the respective direction of jitter.
  • the first compensation component 3310, the second compensation component 3320, the third compensation component 3330, and the fourth compensation component 3340 in this embodiment each include a driving member 3301, a rotating shaft 3302, a one-way bearing 3303, and a rotating ring gear 3304.
  • the driving member 3301 is controlled by the processing module, and the driving member 3301 is drivingly connected to the rotating shaft 3302 to drive the rotating shaft 3302 to rotate.
  • the rotating shaft 3302 is connected to the inner ring of the one-way bearing 3303 to drive the inner ring of the one-way bearing 3303 to rotate.
  • the rotating ring gear 3304 is sleeved on the one-way bearing 3303 and is connected to the one-way bearing 3303.
  • the outer ring of the one-way bearing 3303 is fixedly connected.
  • the outer surface of the rotating ring gear 3304 is provided with a ring of external teeth along its circumferential direction.
  • the shaped groove 3110 is meshed with the external teeth, and the external teeth can slide along the length direction of the strip groove 3110; wherein the rotatable direction of the one-way bearing 3303 of the first compensation component 3310 and the external teeth.
  • the rotation direction of the one-way bearing 3303 of the third compensation component 3330 is opposite, and the rotation direction of the one-way bearing 3303 of the second compensation component 3320 is opposite to the rotation direction of the one-way bearing 3303 of the fourth compensation component 3340.
  • One-way bearing 3303 is a bearing that can rotate freely in one direction and lock in the other direction.
  • the driving member 3301 of the first compensation component 3310 causes the rotating shaft 3302 to drive
  • the inner ring of the one-way bearing 3303 rotates.
  • the one-way bearing 3303 is locked. Therefore, the inner ring of the one-way bearing 3303 can drive the outer ring to rotate, which in turn drives the rotating ring gear 3304 to rotate.
  • the engagement of the groove 3110 drives the movable plate 3100 to move in a direction that can compensate for shake.
  • the third compensation component 3330 can be used to drive the movable plate 3100 to rotate.
  • the one-way bearing 3303 of the first compensation component 3310 is in a rotatable state, so the ring gear on the first compensation component 3310 follows the movable plate 3100, and will not affect the activity Reset of board 3100.
  • the one-way bearing 3303 and the rotating ring gear 3304 can reduce the overall thickness of the entire mechanical vibration stabilizer 3000 by concealing parts of the one-way bearing 3303 and the rotating ring gear 3304 in the mounting holes.
  • a part of the entire compensation component is directly placed in the mounting hole.
  • the driving member 3301 in this embodiment may be a micro motor, the micro motor is electrically connected to the processing module, a rotation output end of the micro motor is connected to the rotating shaft 3302, and the micro motor is controlled To the processing module.
  • the driving member 3301 is composed of a memory alloy wire and a crank connecting rod. One end of the memory alloy wire is fixed on the fixing plate and is connected to the processing module through a circuit. The other end of the memory alloy wire passes The crank link is connected to the rotating shaft 3302 to drive the rotating shaft 3302 to rotate.
  • the processing module calculates the elongation of the memory alloy wire according to the feedback from the gyroscope, and drives the corresponding circuit to the shape memory alloy.
  • the temperature of the wire is increased, and the shape memory alloy wire is stretched to drive the crank link mechanism.
  • the crank of the crank link mechanism drives the rotation shaft 3302 to rotate the inner ring of the one-way bearing 3303.
  • the inner The ring drives the outer ring to rotate, and the rotating ring gear 3304 drives the movable plate 3100 through the strip groove 3110.
  • the following describes the working process of the mechanical image stabilizer 3000 of this embodiment in detail with reference to the above structure.
  • the two shake directions are opposite, and the movable plate 3100 needs to be compensated for forward motion, and then Left motion compensation once.
  • the gyroscope feeds the detected lens 1000 shake direction and distance in advance to the processing module.
  • the processing module calculates the required movement distance of the movable plate 3100, and then drives the first compensation component 3310.
  • the driving member 3301 causes the rotating shaft 3302 to drive the inner ring of the one-way bearing 3303.
  • the one-way bearing 3303 is locked, so the inner ring can drive the outer ring to rotate, which in turn drives the rotating ring gear 3304 to rotate, and the rotating ring gear 3304 passes
  • the strip groove 3110 drives the movable plate 3100 to move forward, and then the third compensation component 3330 drives the movable plate 3100 to reset.
  • the gyroscope feeds back the detected lens 1000 shake direction and distance to the processing module in advance, and the processing module calculates the motion distance required for the motion board 3100 to drive the second compensation component 3320.
  • the driving member 3301 causes the rotating shaft 3302 to drive the inner ring of the one-way bearing 3303.
  • the one-way bearing 3303 is locked, so the inner ring can drive the outer ring to rotate, which in turn drives the rotating ring gear 3304 to rotate, and the rotating ring gear 3304 passes
  • the strip groove 3110 drives the movable plate 3100 to move forward, and because the external teeth of the ring gear 3304 can slide along the length direction of the strip groove 310, when the movable plate 3100 moves to the left, the movable plate 3100 and the first compensation
  • the sliding fitting between the component 3310 and the third compensation component 3330 does not affect the leftward movement of the movable plate 3100.
  • the fourth compensation component 3340 is used to drive the movable plate 3100 to reset.
  • the above is just two simple jitters.
  • the basic working process is the same as the principle described above.
  • the detection feedback of the shape memory alloy resistance and the detection feedback of the gyroscope are existing technologies, and are not described here too.
  • the mechanical compensator provided by this embodiment not only is not affected by external magnetic fields and has a good anti-shake effect, but also can accurately compensate the lens 1000 in the case of multiple shakes, and the compensation is timely and accurate. Greatly improved the quality of the acquired images, and simplified the difficulty of subsequent 3D image processing.
  • the electronic device in this embodiment is specifically a mobile phone with the image acquisition device, and the mobile phone includes a stand.
  • the purpose of the mobile phone including a stand is because of the uncertainty of the image acquisition environment, it is necessary to use the stand to support and fix the mobile phone in order to obtain more stable image quality.
  • the bracket 6000 in this embodiment includes a mobile phone mounting base 6100 and a retractable supporting rod 6200.
  • the supporting rod 6200 and the middle portion of the mobile phone mounting base 6100 pass through a damping hinge. It is connected such that when the supporting rod 6200 is rotated to the state of FIG. 11, the bracket 6000 may form a selfie stick structure, and when the supporting rod 6200 is rotated to the state of FIG. 12, the bracket 6000 may form a mobile phone bracket 6000 structure.
  • the applicant found that the combination of the mobile phone mounting base 6100 and the support pole 6200 takes up a lot of space. Even if the support pole 6200 is retractable, the mobile phone mounting base 6100 cannot undergo structural changes and the volume will not be further reduced. Putting it in a pocket or a small bag causes the inconvenience of carrying the bracket 6000. Therefore, in this embodiment, a second step improvement is performed on the bracket 6000, so that the overall accommodation of the bracket 6000 is further improved.
  • the mobile phone mounting base 6100 of this embodiment includes a retractable connecting plate 6110 and a folding plate group 6120 installed at opposite ends of the connecting plate 6110.
  • the support rod 6200 and the connecting plate 6110 The middle part is connected by a damping hinge;
  • the folding plate group 6120 includes a first plate body 6121, a second plate body 6122, and a third plate body 6123, wherein one of the two opposite ends of the first plate body 6121 is connected to the first plate body 6121.
  • the connecting plate 6110 is hinged, and the other end of the opposite ends of the first plate body 6121 is hinged to one of the opposite ends of the second plate body 6122; The other end is hinged to one of opposite ends of the third plate body 6123; the second plate body 6122 is provided with an opening 6130 for inserting a corner of the mobile phone.
  • the first plate body 6121, the second plate body 6122, and the third plate body 6123 are folded into a right triangle state, and the second plate body 6122 is a hypotenuse of a right-angled triangle, and the first plate body 6121 and the third plate 6123 are right-angled sides of a right triangle, wherein one side of the third plate body 6123 and one of the connection plate 6110 The sides are attached side by side, and the other end of the opposite ends of the third plate body 6123 and one of the opposite ends of the first plate body 6121 abut against each other.
  • This structure can make the three folding plates in a self-locking state, and When the two corners of the lower part of the mobile phone are inserted into the two openings 6130 on both sides, the lower sides of the mobile phone 5000 are located in two right-angled triangles.
  • the mobile phone 5000 can be completed through the joint work of the mobile phone, the connecting plate 6110, and the folding plate group 6120.
  • the triangle state cannot be opened under external force.
  • the triangle state of 6120 pieces of folding plate group can only be released after the mobile phone is pulled out from the opening 6130.
  • the connecting plate 6110 When the mobile phone mounting base 6100 is not in working state, the connecting plate 6110 is reduced to a minimum length, and the folding plate group 6120 and the connecting plate 6110 are folded to each other.
  • the user can fold the mobile phone mounting base 6100 to a minimum volume, and due to the support
  • the scalability of the lever 6200 allows the entire bracket 6000 to be accommodated in the smallest volume, which improves the collection of the bracket 6000. Users can even put the bracket 6000 directly into their pockets or small handbags, which is very convenient.
  • a first connection portion is also provided on one side of the third plate body 6123, and a side surface where the connection plate 6110 is in contact with the third plate body 6123 is provided with the first connection portion.
  • a first mating portion that mates with a connecting portion.
  • the first connecting portion of this embodiment is a convex strip or protrusion (not shown in the figure), and the first matching portion is a card slot (not shown in the figure) opened on the connecting plate 6110.
  • This structure not only improves the stability when the 6120 pieces of the folding plate group are in a triangle state, but also facilitates the connection between the 6120 pieces of the folding plate group and the connecting plate 6110 when the mobile phone mounting base 6100 needs to be folded to a minimum state.
  • a second connection portion is also provided at one end of the opposite ends of the first plate body 6121, and the other end of the opposite ends of the third plate body 6123 is provided with the second connection portion.
  • the second connecting portion is a second matching portion that is matched with the second fitting portion, and the second connecting portion and the second fitting portion are engaged and connected.
  • the second connecting portion may be a protrusion (not shown in the figure), and the second mating portion is an opening 6130 or a card slot (not shown in the figure) that cooperates with the protrusion.
  • a base (not shown in the figure) can be detachably connected to the other end of the support rod 6200.
  • the support rod 6200 can be stretched to A certain length and place the bracket 6000 on a plane through the base, and then place the mobile phone in the mobile phone mounting base 6100 to complete the fixing of the mobile phone; and the detachable connection of the support bar 6200 and the base can make the two can be carried separately, further The accommodating of the bracket 6000 and the convenience of carrying are improved.
  • the device embodiments described above are only schematic, and the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, which may be located in One place, or can be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the objective of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without creative labor.
  • An embodiment of the present invention provides a non-transitory computer-readable storage storage medium, where the computer storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device is caused
  • the video processing method in any of the foregoing method embodiments is executed on the above.
  • An embodiment of the present invention provides a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions When executed by an electronic device, the electronic device is caused to execute the video processing method in any of the foregoing method embodiments.
  • each embodiment can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware.
  • the above-mentioned technical solution in essence or a part that contributes to the existing technology may be embodied in the form of a software product, and the computer software product may be stored in a computer-readable storage medium, the computer-readable record A medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer).
  • machine-readable media include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagation signals (e.g., carrier waves , Infrared signals, digital signals, etc.), the computer software product includes a number of instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments Methods.

Abstract

本发明实施例提供一种视频处理方法、装置及电子设备，所述方法包括：响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。通过上述方法、装置及电子设备，无需采集多角度多张图片，仅需单张图片即可对视频中的人脸进行替换，从而得到目标人脸；同时考虑到脸部表情因素的影响，提高了目标人脸与视频的关联性。

Description

视频处理方法、装置及电子设备 技术领域
本发明涉及图像处理技术领域，尤其涉及一种视频处理方法、装置及电子设备。
背景技术
近年来，虚拟对象的生成技术被越来越多的应用在视频制作中，其中，三维人脸的重构对虚拟对象的生成尤为重要。发明人在实现本发明的过程中发现，现有技术一般采用人脸替换视频中的目标人脸来实现虚拟对象的生成，但并没有考虑到脸部的表情因素从而使得虚拟对象与视频的关联性较差，从而导致生成的虚拟对象与视频环境不匹配，用户感觉虚假。
此外，在视频制作中生成虚拟对象越来越多的用于手机视频编辑。二维人脸图像主要通过手机的摄像头获得，而重构的三维人脸图像的效果部分取决于前期摄像装置获取的图像质量，而获取图像质量又部分的取决于手机拍摄时对抖动的处理效果，目前的手机主要通过软件进行防抖处理，硬件改进的措施不多。
发明内容
本发明实施例提供的视频处理方法、装置及电子设备，用以至少解决相关技术中的上述问题。
本发明实施例一方面提供了一种视频处理方法，包括：
响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
进一步地，所述获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，包括：获取所述第二人脸图像在预设特征点的第一表情参数；获取所述第一三维人脸图像在所述预设特征点的第二表情参数；将所述第二表情参数替换成所述第一表情参数。
进一步地，所述将所述第二人脸图像替换成所述目标人脸图像，包括：根据所述第二人脸图像对应人物的属性信息对所述目标人脸图像的尺寸进行调整，将所述 第二人脸图像替换成调整后的所述目标人脸图像。
进一步地，所述方法还包括：获取所述视频中的光照信息，根据所述光照信息对所述目标人脸图像进行处理。
进一步地，所述根据所述光照信息对所述目标人脸图像进行处理，包括：根据所述目标人脸图像在所述视频画面中的位置确定入射光的方向；根据所述光照信息和所述入射光的方向，生成所述目标人脸图像的光照效果。
进一步地，通过图像获取设备获取所述目标图片，所述图像获取设备包括镜头、自动聚焦音圈马达、机械防抖器以及图像传感器，所述镜头固装在所述自动聚焦音圈马达上，所述镜头用于获取图像，所述图像传感器将所述镜头获取的图像传输至所述识别模块，所述自动聚焦音圈马达安装在所述机械防抖器上，所述处理模块根据镜头内的陀螺仪检测到的镜头抖动的反馈驱动所述机械防抖器的动作，实现镜头的抖动补偿。
进一步地，所述机械防抖器包括活动板、基板以及补偿机构，所述活动板和所述基板的中部均设有所述镜头穿过的通孔，所述自动聚焦音圈马达安装在所述活动板上，所述活动板安装在所述基板上，且所述基板的尺寸大于所述活动板，所述补偿机构在所述处理模块的驱动下带动所述活动板和活动板上的镜头动作，以实现镜头的抖动补偿；所述补偿机构包括安装在所述基板四周的第一补偿组件、第二补偿组件、第三补偿组件以及第四补偿组件，其中所述第一补偿组件和所述第三补偿组件相对设置，所述第二补偿组件与所述第四补偿组件相对设置，所述第一补偿组件与第三补偿组件之间的连线与所述第一补偿组件与第三补偿组件之间的连线相互垂直；所述第一补偿组件、第二补偿组件、第三补偿组件以及第四补偿组件均包括驱动件、转轴、单向轴承以及转动齿圈；所述驱动件受控于所述处理模块，所述驱动件与所述转轴传动连接，以带动所述转轴转动；所述转轴与所述单向轴承的内圈相连接，以带动所述单向轴承的内圈转动；所述转动齿圈套设在所述单向轴承上并与所述单向轴承的外圈相连接，所述转动齿圈的外表面沿其周向设有一圈外齿，所述活动板的底面设有多排均匀间隔布设的条形槽，所述条形槽与所述外齿相啮合，且所述外齿可沿所述条形槽的长度方向滑动；其中，所述第一补偿组件的单向轴承的可转动方向与所述第三补偿组件的单向轴承的可转动方向相反，所述第二补偿组件的单向轴承的可转动方向与所述第四补偿组件的单向轴承的可转动方向相反。
进一步地，所述固定板的四周开设有四个贯穿的安装孔，所述安装孔上安装有所述单向轴承和所述转动齿圈。
进一步地，所述驱动件为微型电机，所述微型电机与所述处理模块电连接，所述微型电机的转动输出端与所述转轴相连接；或，所述驱动件包括记忆合金丝和曲柄连杆，所述记忆合金丝一端固定于所述固定板上，并与所述处理模块通过电路相连接，所述记忆合金丝另一端通过所述曲柄连杆与所述转轴相连接，以带动所述转轴转 动。
进一步地，所述图像获取设备设置于手机上，手机包括支架，所述支架包括手机安装座和可伸缩的支撑杆；所述手机安装座包括可伸缩的连接板和安装于连接板相对两端的折叠板组，所述支撑杆的一端与所述连接板中部通过阻尼铰链相连接；所述折叠板组包括第一板体、第二板体及第三板体，其中，所述第一板体的相对两端中的一端与所述连接板相铰接，所述第一板体的相对两端中的另一端与所述第二板体的相对两端中的一端相铰接；所述第二板体相对两端的另一端与所述第三板体相对两端中的一端相铰接；所述第二板体设有供手机边角插入的开口；所述手机安装座用于安装手机时，所述第一板体、第二板体和第三板体折叠呈直角三角形状态，所述第二板体为直角三角形的斜边，所述第一板体和所述第三板体为直角三角形的直角边，其中，所述第三板体的一个侧面与所述连接板的一个侧面并排贴合，所述第三板体相对两端中的另一端与所述第一板体相对两端中的一端相抵。
进一步地，所述第三板体的一个侧面设有第一连接部，所述连接板与所述第三板体相贴合的侧面设有与所述第一连接部相配合的第一配合部，所述支架手机安装座用于安装手机时，所述第一连接部和所述第一配合部卡合连接。
进一步地，所述第一板体相对两端中的一端设有第二连接部，所述第三板体相对两端中的另一端设有与所述第二连接部相配合的第二配合部，所述支架手机安装座用于安装手机时，所述第二连接部和所述第二配合部卡合连接。
进一步地，所述支撑杆的另一端可拆卸连接有底座。
本发明实施例的另一方面提供了一种视频处理装置，包括：
获取模块，用于响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；输出模块，用于利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；调整模块，用于获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；替换模块，用于将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
进一步地，所述调整模块具体用于，获取所述第二人脸图像在预设特征点的第一表情参数；获取所述第一三维人脸图像在所述预设特征点的第二表情参数；将所述第二表情参数替换成所述第一表情参数。
进一步地，所述替换模块具体用于，根据所述第二人脸图像对应人物的属性信息对所述目标人脸图像的尺寸进行调整，将所述第二人脸图像替换成调整后的所述目标人脸图像。
进一步地，所述装置还包括处理模块，用于获取所述视频中的光照信息，根据所述光照信息对所述目标人脸图像进行处理。
进一步地，所述处理模块具体用于，根据所述目标人脸图像在所述视频画面中 的位置确定入射光的方向；根据所述光照信息和所述入射光的方向，生成所述目标人脸图像的光照效果。
本发明实施例的又一方面提供一种电子设备，包括：至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行本发明实施例上述任一项视频处理方法。
由以上技术方案可见，本发明实施例提供的视频处理方法、装置及电子设备，无需采集多角度多张图片，仅需单张图片即可对视频中的人脸进行替换，从而得到目标人脸；同时考虑到脸部表情因素的影响，提高了目标人脸与视频的关联性。另一方面，通过改进图像获取设备的防抖结构，改善了图像获取质量。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明实施例中记载的一些实施例，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。
图1为本发明一个实施例提供的视频处理方法流程图；
图2为本发明一个实施例提供的视频处理方法流程图；
图3为本发明一个实施例提供的视频处理装置结构图；
图4为本发明一个实施例提供的视频处理装置结构图；
图5为执行本发明方法实施例提供的执行视频处理方法的电子设备的硬件结构示意图；
图6为本发明一个实施例提供的图像获取设备的结构图；
图7为本发明一个实施例提供的光学防抖器的结构图；
图8为图7的A部放大图；
图9为本发明一个实施例提供的微型记忆合金光学防抖器的活动板的底面示意图；
图10为本发明一个实施例提供的支架的结构图；
图11为本发明一个实施例提供的支架的一个状态示意图；
图12为本发明一个实施例提供的支架的另一个状态示意图；
图13为本发明一个实施例提供的安装座与手机相连接时的结构状态图。
具体实施方式
为了使本领域的人员更好地理解本发明实施例中的技术方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本发明实施例一部分实施例，而不是全部的实施例。基于本发明实施例中的实施例，本领域普通技术人员所获得的所有其他实施例，都应当属于本发明实施例保护的范围。
本发明实施例的执行主体为电子设备，所述电子设备包括但不限于手机、平板电脑、笔记本电脑、带摄像头的台式电脑等。下面结合附图，对本发明的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互结合。图1为本发明实施例提供的视频处理方法流程图。如图1所示，本发明实施例提供的视频处理方法，包括：
S101，响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像。
在播放视频的过程中，用户希望与视频进行交互，将视频中的第二人脸图像替换成目标图片中的第一人脸图像。具体地，该目标图片可以是存储在用户电子设备中的图片，也可以是用户当前拍摄的图片。该指令可以是用户对交互标识的操作，也可以预先设定的用户操作轨迹，本发明在此不做限定。
在本步骤中，当接收到用户的操作指令后，获取该指令对应的视频中待替换的第二人脸图像以及目标图片的第一人脸图像。
S102，利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像。
具体地，卷积神经网络CNN是一种深度前馈人工神经网络。CNN的基本结构包括两层，其一为特征提取层，每个神经元的输入与前一层的局部接受域相连，并提取该局部的特征，一旦该局部特征被提取后，它与其它特征间的位置关系也随之确定下来；其二是特征映射层，网络的每个计算层由多个特征映射组成，每个特征映射是一个平面，平面上所有神经元的权值相等。卷积神经网络通常包括一维卷积神经网络、二维卷积神经网络和三维卷积神经网络，现有技术已有这些卷积神经网络数学模型的大量介绍，此处不再赘述，也不对卷积神经网络的类型进行限定。
在进行本步骤之前，需要预先对卷积神经网络模型进行训练。首先构建一个卷积神经网络，并获取一定数量的训练样本集，输入样本集中的二维人脸图像作为模型的输入端，与该二维人脸图像对应的三维人脸图像作为模型的输出端。其次利用人脸特征点识别算法得到人像68个特征点,包括眉毛,眼睛,鼻子,嘴巴,脸庞的关键点坐标位置；对每个特征点通过高斯算法形成一个代表周围6个像素的通道(68个通道) 作为卷积神经网络的输入。再次，将三维人脸图像根据人脸方向调整到正脸方向，进行体素化重构。最后，使用回归算法计算交叉熵损失函数(归一化均方误差)，通过训练使损失函数收敛到最小值，该卷积神经网络模型训练完成。
将第一人脸图像输入到训练好的卷积神经网络模型中，从而得到该第一人脸图像对应的第一三维人脸图像。
S103，获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像。
视频中第二人脸图像中人的表情是与视频中播放的内容相关的，因此需要根据该第二人脸图像中表情参数对得到的第一三维人脸图像中的表情进行调整。具体地，可以预先设定能够标识人脸表情的关键点作为特征点，获取第二人脸图像在预设特征点的第一表情参数，并获取所述预设特征点在第一三维人脸图像中的第二表情参数，将所述第二表情参数替换成所述第一表情参数，实现了表情的嫁接，从而使得替换后的人脸表情与视频播放的内容相关。
S104，将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
在本步骤中，属性信息包括但不限于第二人脸图像对应的人物脸型、脸部特征、身材等。根据人物脸型、脸部特征、身材对目标人脸图像的脸部进行调整，使两张脸尽可能的重合尺寸，同时使目标人脸图像中人脸的尺寸与视频中第二人脸图像对应的人物身材进相匹配，将该第二人脸图像替换成调整后的所述目标人脸图像。
本发明实施例提供的视频处理方法，无需采集多角度多张图片，仅需单张图片即可对视频中的人脸进行替换，从而得到目标人脸；同时考虑到脸部表情因素的影响，提高了目标人脸与视频的关联性。
图2为本发明实施例提供的视频处理方法流程图。如图2所示，本实施例为图1所示实施例的具体实现方案，因此不再赘述图1所示实施例中各步骤的具体实现方法和有益效果，本发明实施例提供的视频处理方法，包括：
S201，响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像。
S202，利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像。S203，获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像。
S204，将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
由于目标人脸图像取自于目标图片，可能会出现其光照效果和视频场景的光照效果不同、无法与视频场景相融合的情况，使得用户从视觉上感受到两者间的不协调。因此，还需要对目标图像进行光影效果的处理。具体地，可以通过如下步骤进行。
S205，获取所述视频中的光照信息，根据所述光照信息对所述目标人脸图像进行处理。
在本步骤中，首先根据所述目标人脸图像在所述视频画面中的位置确定入射光的方向，即可以将目标人脸图像在所述视频画面中的位置方向作为反射光的方向，获取目标人脸表面任意一点的法线方向，通过光的反射定律计算出视频中光源的入射方向；也可以分别获取目标人脸表面若干点的法线方向，将通过光的反射定律得到的若干光源入射方向求取平均值，得到光源的平均入射方向，本发明在此不做限定。
其次，根据所述光照信息和所述入射光的方向，生成所述目标人脸图像的光照效果。具体地，根据所述入射光方向和所述光照信息，计算出所述目标人脸的目标光照信息；依据所述目标光照信息生成所述目标人脸图像的光照效果。
人脸的反射一般可看作是镜面反射，由于镜面反射会造成目标人脸反光的现象，使得目标人脸呈现在用户眼中不清晰，因此，在已知光照信息的情况下，可以根据目标人脸表面的粗糙程度计算出入射方向周围一定范围内的入射光在目标人脸表面反射的目标光照信息。通过扩大入射方向的范围来获取目标人脸清晰的光照效果，因为目标人脸的表面越光滑，则镜面反射效果越强烈，如果此时只根据步骤S101确定的入射方向得到目标光照信息，其出射光方向是唯一的，会因为镜面发射使得目标人脸光照效果较为模糊，此时通过选择入射方向周围一定范围内的入射光，出射光的方向不是单一的，可增加目标人脸光照效果的清晰度。目标光照信息包括出射光的光照强度和方向，根据出射光的光照强度和方向生成目标人脸的光照效果。
本发明实施例提供的视频处理方法，无需采集多角度多张图片，仅需单张图片即可对视频中的人脸进行替换，从而得到目标人脸；同时考虑到脸部表情因素的影响，提高了目标人脸与视频的关联性。此外，通过对目标人脸进行光影效果的处理，提高了目标人脸与视频场景的融合度，提高了用户的交互体验。
图3为本发明实施例提供的视频处理装置结构图。如图3所示，该装置具体包括：获取模块100，输出模块200、调整模块300和替换模块400。其中，
获取模块100，用于响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；输出模块200，用于利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；调整模块300，用于获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；替换模块400，用于将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
可选地，调整模块300具体用于，获取所述第二人脸图像在预设特征点的第一表情参数；获取所述第一三维人脸图像在所述预设特征点的第二表情参数；将所述第二表情参数替换成所述第一表情参数。
可选地，替换模块400具体用于，根据所述第二人脸图像对应人物的属性信息对所述目标人脸图像的尺寸进行调整，将所述第二人脸图像替换成调整后的所述目标人脸图像。
本发明实施例提供的视频处理装置具体用于执行图1所示实施例提供的所述方法，其实现原理、方法和功能用途等与图1所示实施例类似，在此不再赘述。
图4为本发明实施例提供的视频处理装置结构图。如图5所示，该装置具体包括：获取模块100，输出模块200、调整模块300、替换模块400和处理模块500。其中，
获取模块100，用于响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；输出模块200，用于利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；调整模块300，用于获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；替换模块400，用于将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像；处理模块500，用于获取所述视频中的光照信息，根据所述光照信息对所述目标人脸图像进行处理。
可选地，处理模块500具体用于，根据所述目标人脸图像在所述视频画面中的位置确定入射光的方向；根据所述光照信息和所述入射光的方向，生成所述目标人脸图像的光照效果。
可选地，调整模块300具体用于，获取所述第二人脸图像在预设特征点的第一表情参数；获取所述第一三维人脸图像在所述预设特征点的第二表情参数；将所述第二表情参数替换成所述第一表情参数。
可选地，替换模块400具体用于，根据所述第二人脸图像对应人物的属性信息对所述目标人脸图像的尺寸进行调整，将所述第二人脸图像替换成调整后的所述目标人脸图像。
本发明实施例提供的视频处理装置具体用于执行图2所示实施例提供的所述方法，其实现原理、方法和功能用途和图2所示实施例类似，在此不再赘述。
上述这些本发明实施例的视频处理装置可以作为其中一个软件或者硬件功能单元，独立设置在上述电子设备中，也可以作为整合在处理器中的其中一个功能模块，执行本发明实施例的视频处理方法。
图5为执行本发明方法实施例提供的视频处理方法的电子设备的硬件结构示意图。根据图5所示，该电子设备包括：
一个或多个处理器510以及存储器520，图5中以一个处理器510为例。执行所述的视频处理方法的设备还可以包括：输入装置530和输出装置530。
处理器510、存储器520、输入装置530和输出装置540可以通过总线或者其他 方式连接，图5中以通过总线连接为例。
存储器520作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，如本发明实施例中的所述视频处理方法对应的程序指令/模块。处理器510通过运行存储在存储器520中的非易失性软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现所述视频处理方法。
存储器520可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据本发明实施例提供的视频处理装置的使用所创建的数据等。此外，存储器520可以包括高速随机存取存储器520，还可以包括非易失性存储器520，例如至少一个磁盘存储器520件、闪存器件、或其他非易失性固态存储器520件。在一些实施例中，存储器520可选包括相对于处理器55远程设置的存储器520，这些远程存储器520可以通过网络连接至所述视频处理装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置530可接收输入的数字或字符信息，以及产生与视频处理装置的用户设置以及功能控制有关的键信号输入。输入装置530可包括按压模组等设备。
所述一个或者多个模块存储在所述存储器520中，当被所述一个或者多个处理器510执行时，执行所述视频处理方法。
本发明实施例的电子设备以多种形式存在，包括但不限于:
(1)移动通信设备:这类设备的特点是具备移动通信功能，并且以提供话音、数据通信为主要目标。这类终端包括:智能手机(例如iPhone)、多媒体手机、功能性手机，以及低端手机等。
(2)超移动个人计算机设备:这类设备属于个人计算机的范畴，有计算和处理功能，一般也具备移动上网特性。这类终端包括:PDA、MID和UMPC设备等，例如iPad。
(3)便携式娱乐设备:这类设备可以显示和播放多媒体内容。该类设备包括:音频、视频播放器(例如iPod)，掌上游戏机，电子书，以及智能玩具和便携式车载导航设备。
(4)其他具有数据交互功能的电子装置。
优选的，所述电子设备上设置有用于获取图像的图像获取设备，图像获取设备上为保证获取图像的质量往往设置有软件或硬件防抖器。现有的防抖器大多由通电线圈在磁场中产生洛伦磁力驱动镜头移动，而要实现光学防抖，需要在至少两个方向上驱动镜头，这意味着需要布置多个线圈，会给整体结构的微型化带来一定挑战，而且容易受外界磁场干扰，进而影响防抖效果，因此公开号为CN106131435A的中国专利提供了一种微型光学防抖摄像头模组，其通过温度变化实现记忆合金丝的拉伸和缩短，以此拉动自动聚焦音圈马达移动，实现镜头的抖动补偿，微型记忆合金光学防抖致动器的控制芯片可以控制驱动信号的变化来改变记忆合金丝的温度，以此控 制记忆合金丝的伸长和缩短，并且根据记忆合金丝的电阻来计算致动器的位置和移动距离。当微型记忆合金光学防抖致动器上移动到指定位置后反馈记忆合金丝此时的电阻，通过比较这个电阻值与目标值的偏差，可以校正微型记忆合金光学防抖致动器上的移动偏差。
但是申请人发现，由于抖动的随机性和不确定性，仅仅依靠上述技术方案的结构是无法实现在多次抖动发生的情况下能够对镜头进行精确的补偿，这是由于形状记忆合金的升温和降温均需要一定的时间，当抖动向第一方向发生时，上述技术方案可以实现镜头对第一方向抖动的补偿，但是当随之而来的第二方向的抖动发生时，由于记忆合金丝来不及在瞬间变形，因此容易造成补偿不及时，无法精准实现对多次抖动和不同方向的连续抖动的镜头抖动补偿，因此需要对其结构上进行改进，以期获得更好的图像质量，从而便于后续三维图像的生成。
结合附图7-9所示，本实施例对学防抖器进行改进，将其设计为机械防抖器3000，其具体结构如下：
本实施例的所述机械防抖器3000包括活动板3100、基板3200以及补偿机构3300，所述活动板3100和所述基板3200的中部均设有所述镜头1000穿过的通孔，所述自动聚焦音圈马达2000安装在所述活动板3100上，所述活动板3100安装在所述基板3200上，且所述基板3200的尺寸大于所述活动板3100，所述活动板3100通过其上方的自动聚焦音圈马达限位其上下的移动，所述补偿机构3300在所述处理模块的驱动下带动所述活动板3100和活动板3100上的镜头1000动作，以实现镜头1000的抖动补偿。
具体的，本实施例的所述补偿机构3300包括安装在所述基板3200四周的第一补偿组件3310、第二补偿组件3320、第三补偿组件3330以及第四补偿组件3340，其中所述第一补偿组件3310和所述第三补偿组件3330相对设置，所述第二补偿组件3320与所述第四补偿组件3340相对设置，所述第一补偿组件3310与第三补偿组件3330之间的连线与所述第一补偿组件3310与第三补偿组件3330之间的连线相互垂直，即一补偿组件、第二补偿组件3320、第三补偿组件3330分别布设在活动板3100的前后左右四个方位，第一补偿组件3310可使得活动板3100向前运动，第三补偿组件3330可使得活动板3100向后运动，第二补偿组件3320可使得活动板3100向左运动，第四补偿组件3340可使得活动板3100向左运动，而且第一补偿组件3310可以与第二补偿组件3320或者第四补偿组件3340相配合实现活动板3100向倾斜方向的运动，第三补偿组件3330也可以与第二补偿组件3320或者第四补偿组件3340相配合实现活动板3100向倾斜方向的运动，实现可以对镜头1000在各个抖动方向上的补偿。
具体的，本实施例的所述第一补偿组件3310、第二补偿组件3320、第三补偿组件3330以及第四补偿组件3340均包括驱动件3301、转轴3302、单向轴承3303以及 转动齿圈3304。所述驱动件3301受控于所述处理模块，所述驱动件3301与所述转轴3302传动连接，以带动所述转轴3302转动。所述转轴3302与所述单向轴承3303的内圈相连接，以带动所述单向轴承3303的内圈转动；所述转动齿圈3304套设在所述单向轴承3303上并与所述单向轴承3303的外圈固定连接，所述转动齿圈3304的外表面沿其周向设有一圈外齿，所述活动板3100的底面设有多排均匀间隔布设的条形槽3110，所述条形槽3110与所述外齿相啮合，且所述外齿可沿所述条形槽3110的长度方向滑动；其中，所述第一补偿组件3310的单向轴承3303的可转动方向与所述第三补偿组件3330的单向轴承3303的可转动方向相反，所述第二补偿组件3320的单向轴承3303的可转动方向与所述第四补偿组件3340的单向轴承3303的可转动方向相反。
单向轴承3303是在一个方向上可以自由转动，而在另一个方向上锁死的一种轴承，当需要使得活动板3100向前移动时，第一补偿组件3310的驱动件3301使得转轴3302带动单向轴承3303的内圈转动，此时，单向轴承3303处于锁死状态，因此单向轴承3303的内圈可以带动外圈转动，进而带动转动齿圈3304转动，转动齿圈3304通过与条形槽3110的啮合带动活动板3100向可以补偿抖动的方向运动；当抖动补偿后需要活动板3100复位时，可以通过第三补偿组件3330带动活动板3100转动，第三补偿组件3330的运行过程过程与第一补偿组件3310同理，此时，第一补偿组件3310的单向轴承3303处于可转动状态，因此第一补偿组件3310上的齿圈为与活动板3100随动状态，不会影响活动板3100的复位。
优选的，为了降低整个机械防抖器3000的整体厚度，本实施例在所述固定板的四周开设有四个贯穿的安装孔(图中未示出)，所述安装孔上安装有所述单向轴承3303和所述转动齿圈3304，通过将单向轴承3303和转动齿圈3304的部分隐藏在安装孔内，以降低整个机械防抖器3000的整体厚度。或者直接将整个补偿组件的部分置于所述安装孔内。
具体，本实施例的所述驱动件3301可以是微型电机，所述微型电机与所述处理模块电连接，所述微型电机的转动输出端与所述转轴3302相连接，所述微型电机受控于所述处理模块。或者，所述驱动件3301由记忆合金丝和曲柄连杆组成，所述记忆合金丝一端固定于所述固定板上，并与所述处理模块通过电路相连接，所述记忆合金丝另一端通过所述曲柄连杆与所述转轴3302相连接，以带动所述转轴3302转动，具体为处理模块根据陀螺仪的反馈计算出记忆合金丝的伸长量，并驱动相应的电路对该形状记忆合金丝进行升温，该形状记忆合金丝伸长带动曲柄连杆机构运动，曲柄连杆机构的曲柄带动转轴3302转动，使得单向轴承3303的内圈转动，单向轴承3303处于锁死状态时，内圈带动外圈转动，转动齿圈3304通过条形槽3110带动活动板3100运动。
下面结合上述结构对本实施例的机械防抖器3000的工作过程进行详细的描述， 以镜头1000两次抖动为例，两次抖动方向相反，且需要使得活动板3100向前运动补偿一次，并随后向左运动补偿一次。需要活动板3100向前运动补偿时，陀螺仪事先将检测到的镜头1000抖动方向和距离反馈给所述处理模块，处理模块计算出需要活动板3100的运动距离，进而驱动第一补偿组件3310的驱动件3301使得转轴3302带动单向轴承3303的内圈转动，此时，单向轴承3303处于锁死状态，因此内圈可以带动外圈转动，进而带动转动齿圈3304转动，转动齿圈3304通过条形槽3110带动活动板3100向前运动，随后第三补偿组件3330带动活动板3100复位。需要活动板3100向左运动补偿时，陀螺仪事先将检测到的镜头1000抖动方向和距离反馈给所述处理模块，处理模块计算出需要活动板3100的运动距离，进而驱动第二补偿组件3320的驱动件3301使得转轴3302带动单向轴承3303的内圈转动，此时，单向轴承3303处于锁死状态，因此内圈可以带动外圈转动，进而带动转动齿圈3304转动，转动齿圈3304通过条形槽3110带动活动板3100向前运动，而且由于转动齿圈3304的外齿可沿所述条形槽3110的长度方向滑动，在活动板3100向左运动时，活动板3100与第一补偿组件3310和第三补偿组件3330之间为滑动配合，不会影响活动板3100向左运动，在补偿结束后，再通过第四补偿组件3340带动活动板3100复位。
当然上述仅仅为简单的两次抖动，当发生多次抖动时，或者抖动的方向并非往复运动时，可以通过驱动多个补偿组件以补偿抖动，其基础工作过程与上述描述原理相同，这里不过多赘述，另外关于形状记忆合金电阻的检测反馈、陀螺仪的检测反馈等均为现有技术，这里也不过多描述。
结合上述说明可知，本实施例提供的机械补偿器不仅不会受到外界磁场干扰，防抖效果好，而且可以实现在多次抖动发生的情况下能够对镜头1000进行精确的补偿，补偿及时准确，大大改善改了获取图像的质量，简化了后续三维图像的处理难度。
进一步地，本实施例的电子设备具体为带有所述图像获取设备的手机，该手机包括支架。手机包含支架的目的是由于图像获取环境的不确定性，因此需要使用支架对手机进行支撑和固定，以期获得更稳定的图像质量。
另外，申请人发现，现有的手机支架仅仅具有支撑手机的功能，而不具有自拍杆的功能，因此申请人对支架做出第一步改进，将手机支架6000和支撑杆6200相结合，结合附图10所示，本实施例的所述支架6000包括手机安装座6100和可伸缩的支撑杆6200，支撑杆6200与手机安装座6100的中部(具体为下述基板3200的中部)通过阻尼铰链相连接，使得支撑杆6200在转动至图11的状态时，支架6000可形成自拍杆结构，而支撑杆6200在转动至图12的状态时，支架6000可形成手机支架6000结构。
而结合上述支架结构申请人又发现，手机安装座6100与支撑杆6200结合后占用空间较大，即使支撑杆6200可伸缩，但是手机安装座6100无法进行结构的变化，体积不会进一步缩小，无法将其放入衣兜或者小型的包内，造成支架6000携带不便的 问题，因此本实施例对支架6000做出第二步改进，使得支架6000的整体收容性得到进一步的提高。
结合图11-13所示，本实施例的所述手机安装座6100包括可伸缩的连接板6110和安装于连接板6110相对两端的折叠板组6120，所述支撑杆6200与所述连接板6110中部通过阻尼铰链相连接；所述折叠板组6120包括第一板体6121、第二板体6122及第三板体6123，其中，所述第一板体6121的相对两端中的一端与所述连接板6110相铰接，所述第一板体6121的相对两端中的另一端与所述第二板体6122的相对两端中的一端相铰接；所述第二板体6122相对两端的另一端与所述第三板体6123相对两端中的一端相铰接；所述第二板体6122设有供手机边角插入的开口6130。
结合附图13所示，所述手机安装座6100用于安装手机时，所述第一板体6121、第二板体6122和第三板体6123折叠呈直角三角形状态，所述第二板体6122为直角三角形的斜边，所述第一板体6121和所述第三板体6123为直角三角形的直角边，其中，所述第三板体6123的一个侧面与所述连接板6110的一个侧面并排贴合，所述第三板体6123相对两端中的另一端与所述第一板体6121相对两端中的一端相抵，该结构可以使得三个折叠板处于自锁状态，并且将手机下部的两个边角插入到两侧的两个开口6130时，手机5000的下部两侧位于两个直角三角形内，通过手机、连接板6110和折叠板组6120件的共同作可以完成手机5000的固定，三角形状态在外力情况下无法打开，只有从开口6130抽出手机后才能解除折叠板组6120件的三角形状态。
而在手机安装座6100不处于工作状态时，将连接板6110缩小至最小长度，并且将折叠板组6120件与连接板6110相互折叠，用户可以将手机安装座6100折叠呈最小体积，而由于支撑杆6200的可伸缩性，因此可以将整个支架6000收容呈体积最小的状态，提高了支架6000的收荣幸，用户甚至可以直接将支架6000放入衣兜或小的手包内，十分方便。
优选的，本实施例还在所述第三板体6123的一个侧面设有第一连接部，所述连接板6110与所述第三板体6123相贴合的侧面设有与所述第一连接部相配合的第一配合部，所述支架6000手机安装座6100用于安装手机时，所述第一连接部和所述第一配合部卡合连接。具体的，本实施例的第一连接部为一个凸条或凸起(图中未示出)，第一配合部为开设在连接板6110上的卡槽(图中未示出)。该结构不仅提高了折叠板组6120件处于三角形状态时的稳定性，而且在需要将手机安装座6100折叠至最小状态时也便于折叠板组6120件与连接板6110的连接。
优选的，本实施例还在所述第一板体6121相对两端中的一端设有第二连接部，所述第三板体6123相对两端中的另一端设有与所述第二连接部相配合的第二配合部，所述支架6000手机安装座6100用于安装手机时，所述第二连接部和所述第二配合部卡合连接。第二连接部可以是凸起(图中未示出)，第二配合部为与凸起相配合的开口6130或卡槽(图中未示出)。该结构提高了叠板组件处于三角形状态时的稳 定性
另外，本实施例还可以在所述支撑杆6200的另一端可拆卸连接有底座(图中未示出)，在需要固定手机并且使手机5000具有一定高度时，可以将支撑杆6200拉伸呈一定长度，并通过底座将支架6000置于一个平面上，再将手机放置到手机安装座6100内，完成手机的固定；而支撑杆6200和底座的可拆卸连接可以使得两者可以单独携带，进一步提高了支架6000的收容性和携带的方便性。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
本发明实施例提供了一种非暂态计算机可读存存储介质，所述计算机存储介质存储有计算机可执行指令，其中，当所述计算机可执行指令被电子设备执行时，使所述电子设备上执行上述任意方法实施例中的视频处理方法。
本发明实施例提供了一种计算机程序产品，其中，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，其中，当所述程序指令被电子设备执行时，使所述电子设备执行上述任意方法实施例中的视频处理方法。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，所述计算机可读记录介质包括用于以计算机(例如计算机)可读的形式存储或传送信息的任何机制。例如，机器可读介质包括只读存储器(ROM)、随机存取存储器(RAM)、磁盘存储介质、光存储介质、闪速存储介质、电、光、声或其他形式的传播信号(例如，载波、红外信号、数字信号等)等，该计算机软件产品包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明实施例的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (10)

  1. 一种视频处理方法，其特征在于，包括：
    响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；
    利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；
    获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；
    将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
  2. 根据权利要求1所述的方法，其特征在于，所述获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，包括：
    获取所述第二人脸图像在预设特征点的第一表情参数；
    获取所述第一三维人脸图像在所述预设特征点的第二表情参数；
    将所述第二表情参数替换成所述第一表情参数。
  3. 根据权利要求1所述的方法，其特征在于，所述将所述第二人脸图像替换成所述目标人脸图像，包括：
    根据所述第二人脸图像对应人物的属性信息对所述目标人脸图像的尺寸进行调整，将所述第二人脸图像替换成调整后的所述目标人脸图像。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述方法还包括：
    获取所述视频中的光照信息，根据所述光照信息对所述目标人脸图像进行处理。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述光照信息对所述目标人脸图像进行处理，包括：
    根据所述目标人脸图像在所述视频画面中的位置确定入射光的方向；
    根据所述光照信息和所述入射光的方向，生成所述目标人脸图像的光照效果。
  6. 一种视频处理装置，其特征在于，包括：
    获取模块，用于响应于用户的指令，获取目标图片中的第一人脸图像和视频中待替换的第二人脸图像；
    输出模块，用于利用卷积神经网络模型输出所述第一人脸图像对应的第一三维人脸图像；
    调整模块，用于获取所述第二人脸图像的表情参数，根据所述表情参数对所述第一三维人脸图像进行调整，得到第二三维人脸图像；
    替换模块，用于将所述第二三维人脸图像映射到二维空间得到目标人脸图像，并将所述第二人脸图像替换成所述目标人脸图像。
  7. 根据权利要求6所述的装置，其特征在于，所述调整模块具体用于，获取所述第二人脸图像在预设特征点的第一表情参数；获取所述第一三维人脸图像在所述 预设特征点的第二表情参数；将所述第二表情参数替换成所述第一表情参数。
  8. 根据权利要求6所述的装置，其特征在于，所述替换模块具体用于，根据所述第二人脸图像对应人物的属性信息对所述目标人脸图像的尺寸进行调整，将所述第二人脸图像替换成调整后的所述目标人脸图像。
  9. 根据权利要求6-8任一项所述的装置，其特征在于，所述装置还包括：
    处理模块，用于获取所述视频中的光照信息，根据所述光照信息对所述目标人脸图像进行处理。
  10. 一种电子设备，其特征在于，包括：至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1至5中任一项所述的视频处理方法。
PCT/CN2018/102332 2018-08-24 2018-08-24 视频处理方法、装置及电子设备 WO2020037679A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/102332 WO2020037679A1 (zh) 2018-08-24 2018-08-24 视频处理方法、装置及电子设备
CN201811029389.7A CN109151340B (zh) 2018-08-24 2018-09-05 视频处理方法、装置及电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/102332 WO2020037679A1 (zh) 2018-08-24 2018-08-24 视频处理方法、装置及电子设备

Publications (1)

Publication Number Publication Date
WO2020037679A1 true WO2020037679A1 (zh) 2020-02-27

Family

ID=64826840

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/102332 WO2020037679A1 (zh) 2018-08-24 2018-08-24 视频处理方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN109151340B (zh)
WO (1) WO2020037679A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112017141A (zh) * 2020-09-14 2020-12-01 北京百度网讯科技有限公司 视频数据处理方法和装置
CN112102157A (zh) * 2020-09-09 2020-12-18 咪咕文化科技有限公司 视频换脸方法、电子设备和计算机可读存储介质
CN112989955A (zh) * 2021-02-20 2021-06-18 北方工业大学 基于空时双流异构嫁接卷积神经网络人体动作识别方法
US11222466B1 (en) 2020-09-30 2022-01-11 Disney Enterprises, Inc. Three-dimensional geometry-based models for changing facial identities in video frames and images

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109961507B (zh) * 2019-03-22 2020-12-18 腾讯科技（深圳）有限公司 一种人脸图像生成方法、装置、设备及存储介质
CN111860044A (zh) * 2019-04-26 2020-10-30 北京陌陌信息技术有限公司 人脸换脸方法、装置、设备及计算机存储介质
CN111860045A (zh) * 2019-04-26 2020-10-30 北京陌陌信息技术有限公司 人脸换脸方法、装置、设备及计算机存储介质
CN111861948B (zh) * 2019-04-26 2024-04-09 北京陌陌信息技术有限公司 图像处理方法、装置、设备及计算机存储介质
CN110503703B (zh) * 2019-08-27 2023-10-13 北京百度网讯科技有限公司 用于生成图像的方法和装置
CN110868554B (zh) * 2019-11-18 2022-03-08 广州方硅信息技术有限公司 直播中实时换脸的方法、装置、设备及存储介质
CN111461959B (zh) * 2020-02-17 2023-04-25 浙江大学 人脸情绪合成方法及装置
CN111491124B (zh) * 2020-04-17 2023-02-17 维沃移动通信有限公司 视频处理方法、装置及电子设备
CN113792705B (zh) * 2021-09-30 2024-04-23 北京跳悦智能科技有限公司 一种视频表情迁移方法及系统、计算机设备
CN114004922B (zh) * 2021-10-29 2023-11-24 腾讯科技（深圳）有限公司 骨骼动画显示方法、装置、设备、介质及计算机程序产品
CN115195757B (zh) * 2022-09-07 2023-08-04 郑州轻工业大学 电动公交起步驾驶行为建模及识别训练方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070127844A1 (en) * 2005-12-07 2007-06-07 Sony Corporation Image processing apparatus, image processing method, program, and data configuration
CN104156993A (zh) * 2014-07-18 2014-11-19 小米科技有限责任公司 一种切换图片中人脸图像的方法及装置
CN104484858A (zh) * 2014-12-31 2015-04-01 小米科技有限责任公司 人物图像处理方法及装置
CN105118082A (zh) * 2015-07-30 2015-12-02 科大讯飞股份有限公司 个性化视频生成方法及系统
CN107067429A (zh) * 2017-03-17 2017-08-18 徐迪 基于深度学习的人脸三维重建和人脸替换的视频编辑系统及方法
CN107341827A (zh) * 2017-07-27 2017-11-10 腾讯科技（深圳）有限公司 一种视频处理方法、装置和存储介质
CN108388889A (zh) * 2018-03-23 2018-08-10 百度在线网络技术（北京）有限公司 用于分析人脸图像的方法和装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3763215B2 (ja) * 1998-09-01 2006-04-05 株式会社明電舎 三次元位置決め方法及び装置並びに上記方法を実現するソフトウェアを記録した媒体
KR100874962B1 (ko) * 2007-04-16 2008-12-19 (주)에프엑스기어 사용자지정 안면이미지가 반영된 영상 컨텐츠 제작 시스템
CN102479388A (zh) * 2010-11-22 2012-05-30 北京盛开互动科技有限公司 基于人脸跟踪和分析的表情互动方法
CN106599817A (zh) * 2016-12-07 2017-04-26 腾讯科技（深圳）有限公司 一种人脸替换方法及装置
CN107316020B (zh) * 2017-06-26 2020-05-08 司马大大（北京）智能系统有限公司 人脸替换方法、装置及电子设备
CN107481318A (zh) * 2017-08-09 2017-12-15 广东欧珀移动通信有限公司 用户头像的替换方法、装置和终端设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070127844A1 (en) * 2005-12-07 2007-06-07 Sony Corporation Image processing apparatus, image processing method, program, and data configuration
CN104156993A (zh) * 2014-07-18 2014-11-19 小米科技有限责任公司 一种切换图片中人脸图像的方法及装置
CN104484858A (zh) * 2014-12-31 2015-04-01 小米科技有限责任公司 人物图像处理方法及装置
CN105118082A (zh) * 2015-07-30 2015-12-02 科大讯飞股份有限公司 个性化视频生成方法及系统
CN107067429A (zh) * 2017-03-17 2017-08-18 徐迪 基于深度学习的人脸三维重建和人脸替换的视频编辑系统及方法
CN107341827A (zh) * 2017-07-27 2017-11-10 腾讯科技（深圳）有限公司 一种视频处理方法、装置和存储介质
CN108388889A (zh) * 2018-03-23 2018-08-10 百度在线网络技术（北京）有限公司 用于分析人脸图像的方法和装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102157A (zh) * 2020-09-09 2020-12-18 咪咕文化科技有限公司 视频换脸方法、电子设备和计算机可读存储介质
CN112017141A (zh) * 2020-09-14 2020-12-01 北京百度网讯科技有限公司 视频数据处理方法和装置
US11222466B1 (en) 2020-09-30 2022-01-11 Disney Enterprises, Inc. Three-dimensional geometry-based models for changing facial identities in video frames and images
CN112989955A (zh) * 2021-02-20 2021-06-18 北方工业大学 基于空时双流异构嫁接卷积神经网络人体动作识别方法
CN112989955B (zh) * 2021-02-20 2023-09-29 北方工业大学 基于空时双流异构嫁接卷积神经网络人体动作识别方法

Also Published As

Publication number Publication date
CN109151340B (zh) 2021-08-27
CN109151340A (zh) 2019-01-04

Similar Documents

Publication Publication Date Title
WO2020037679A1 (zh) 视频处理方法、装置及电子设备
WO2020037676A1 (zh) 三维人脸图像生成方法、装置及电子设备
WO2020037680A1 (zh) 基于光线的三维人脸优化方法、装置及电子设备
US11509817B2 (en) Autonomous media capturing
US9479736B1 (en) Rendered audiovisual communication
US10684467B2 (en) Image processing for head mounted display devices
WO2020037678A1 (zh) 基于遮挡图像生成三维人脸图像方法、装置及电子设备
KR102005106B1 (ko) 증강 및 가상 현실을 위한 시스템 및 방법
WO2020037681A1 (zh) 视频生成方法、装置及电子设备
US20080165195A1 (en) Method, apparatus, and software for animated self-portraits
CN107533356A (zh) 头像控制系统
CN108537870B (zh) 图像处理方法、装置及电子设备
WO2019200719A1 (zh) 三维人脸模型生成方法、装置及电子设备
CN113168737A (zh) 用于三维模型共享的方法和系统
WO2019205284A1 (zh) Ar成像方法和装置
TWI255141B (en) Method and system for real-time interactive video
US11620780B2 (en) Multiple device sensor input based avatar
BR112015014629A2 (pt) método para operar um sistema que possui um monitor, uma câmera e um processador
CN108377398A (zh) 基于红外的ar成像方法、系统及电子设备
WO2020056689A1 (zh) 一种ar成像方法、装置及电子设备
WO2020056691A1 (zh) 一种交互对象的生成方法、装置及电子设备
CN115442658B (zh) 直播方法、装置、存储介质、电子设备及产品
WO2024031882A1 (zh) 视频处理方法、装置及计算机可读存储介质
CN116095353A (zh) 基于体积视频的直播方法、装置、电子设备及存储介质
WO2020056693A1 (zh) 一种图片合成方法、装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18930990

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18930990

Country of ref document: EP

Kind code of ref document: A1