CN110969569A - Method and device for generating test-mirror video - Google Patents
Method and device for generating test-mirror video Download PDFInfo
- Publication number
- CN110969569A CN110969569A CN201811139042.8A CN201811139042A CN110969569A CN 110969569 A CN110969569 A CN 110969569A CN 201811139042 A CN201811139042 A CN 201811139042A CN 110969569 A CN110969569 A CN 110969569A
- Authority
- CN
- China
- Prior art keywords
- image
- video frame
- person
- video
- face area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000014509 gene expression Effects 0.000 claims abstract description 60
- 238000012360 testing method Methods 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims description 31
- 230000000007 visual effect Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 20
- 238000004891 communication Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 101150052583 CALM1 gene Proteins 0.000 description 1
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 101150095793 PICALM gene Proteins 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 101150014174 calm gene Proteins 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Studio Devices (AREA)
Abstract
The disclosure relates to a generation method and device of a trial video. The method comprises the following steps: determining the expression and face angle of a target character in a first video frame of a first video; determining a first image matched with the expression and face angle of the target character in the first video frame from the image corresponding to the trial image character; and replacing the first video frame according to the first image to obtain a mirror test video corresponding to the mirror test person. The method and the device can rapidly generate the mirror test video according to the images of the plurality of face angles of the plurality of expressions of the mirror test person, thereby greatly improving the generation efficiency of the mirror test video and saving the mirror test time.
Description
Technical Field
The present disclosure relates to the field of video technologies, and in particular, to a method and an apparatus for generating a trial video.
Background
Trial (audio) generally refers to a method of taking a snapshot or video to determine whether a person's expressive power, skill, or other qualities can take on the task of performing. For example, a video is shot to determine whether a person is suitable for the actor or whether an actor is suitable for showing a character in a video.
Currently, the trial typically takes a significant amount of time. How to save the time of trying the lens is a problem to be solved urgently.
Disclosure of Invention
In view of this, the present disclosure provides a method and an apparatus for generating a trial video.
According to an aspect of the present disclosure, a method for generating a fitting video is provided, including:
determining the expression and face angle of a target character in a first video frame of a first video;
determining a first image matched with the expression and face angle of the target character in the first video frame from the image corresponding to the trial image character;
and replacing the first video frame according to the first image to obtain a mirror test video corresponding to the mirror test person.
In one possible implementation manner, determining a first image matching the expression and face angle of the target person in the first video frame from images corresponding to a trial image person includes:
determining an image which is in the same expression type as the target character in the first video frame in the image corresponding to the trial image character as a candidate image;
and determining the candidate image closest to the face angle of the target person in the first video frame as the first image.
In one possible implementation, performing replacement processing on the first video frame according to the first image includes:
replacing the first video frame with the first image.
In one possible implementation, performing replacement processing on the first video frame according to the first image includes:
intercepting a face area of the test-mirror person from the first image;
and replacing the face area of the target person in the first video frame according to the face area of the trial image person in the first image.
In a possible implementation manner, replacing the face area of the target person in the first video frame according to the face area of the trial person in the first image includes:
adjusting the face area of the test-mirror person in the first image;
and replacing the face area of the target person in the first video frame with the face area of the first image after the adjustment of the trial image person.
In a possible implementation manner, the adjusting the face area of the trial image person in the first image includes one or two of the following:
adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame;
and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
In one possible implementation, the visual parameters include one or more of brightness, hue, saturation, and resolution.
According to another aspect of the present disclosure, there is provided a generation apparatus of a fitting video, including:
the first determining module is used for determining the expression and the face angle of a target character in a first video frame of a first video;
the second determining module is used for determining a first image matched with the expression and face angle of the target character in the first video frame from the image corresponding to the trial image character;
and the replacing module is used for replacing the first video frame according to the first image to obtain a mirror test video corresponding to the mirror test person.
In one possible implementation manner, the second determining module includes:
the first determining sub-module is used for determining an image, which is in the same expression type as the target character in the first video frame, in the image corresponding to the trial image character as a candidate image;
and the second determining sub-module is used for determining the candidate image closest to the face angle of the target person in the first video frame as the first image.
In one possible implementation, the replacement module is configured to:
replacing the first video frame with the first image.
In one possible implementation, the replacement module includes:
the intercepting submodule is used for intercepting a face area of the test mirror figure from the first image;
and the replacing sub-module is used for replacing the face area of the target person in the first video frame according to the face area of the trial image person in the first image.
In one possible implementation, the replacing sub-module includes:
the adjusting unit is used for adjusting the face area of the test-mirror person in the first image;
and the replacing unit is used for replacing the face area of the target person in the first video frame with the face area of the first image after the adjustment of the trial image person.
In a possible implementation, the adjusting unit is configured to perform one or both of the following:
adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame;
and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
In one possible implementation, the visual parameters include one or more of brightness, hue, saturation, and resolution.
According to another aspect of the present disclosure, there is provided a generation apparatus of a fitting video, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above method.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.
In the embodiment of the disclosure, the expression and the face angle of the target person in the first video frame of the first video are determined, the first image matched with the expression and the face angle of the target person in the first video frame is determined from the image corresponding to the test lens person, and the first video frame is subjected to replacement processing according to the first image to obtain the test lens video corresponding to the test lens person, so that the test lens video can be quickly generated according to the images of the test lens person with the multiple expressions and the multiple face angles, the generation efficiency of the test lens video can be greatly improved, and the time for testing the lens can be saved.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
Fig. 1 shows a flowchart of a generation method of a trial video according to an embodiment of the present disclosure.
Fig. 2 is a schematic diagram illustrating a method for generating a fitting video according to an embodiment of the present disclosure, in which images of multiple expressions of a fitting person are acquired from multiple angles.
Fig. 3 shows an exemplary flowchart of step S12 of a generation method of a trial video according to an embodiment of the present disclosure.
Fig. 4 shows an exemplary flowchart of step S13 of a generation method of a trial video according to an embodiment of the present disclosure.
Fig. 5 shows an exemplary flowchart of step S132 of a generation method of a trial video according to an embodiment of the present disclosure.
Fig. 6 shows a block diagram of a generation apparatus of a trial video according to an embodiment of the present disclosure.
Fig. 7 shows an exemplary block diagram of a generation apparatus of a trial video according to an embodiment of the present disclosure.
Fig. 8 is a block diagram illustrating an apparatus 800 for generation of a try-on video in accordance with an example embodiment.
Fig. 9 is a block diagram illustrating an apparatus 1900 for generation of a try-on video, according to an example embodiment.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
Fig. 1 shows a flowchart of a generation method of a trial video according to an embodiment of the present disclosure. The execution subject of the method for generating the trial video may be a device for generating the trial video. For example, the generation method of the trial video may be executed by a terminal device or a server or other processing device. The terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, or a wearable device. In some possible implementations, the generation method of the trial video may be implemented by a processor calling computer readable instructions stored in a memory. As shown in fig. 1, the method may include steps S11 through S13.
In step S11, the expression and face angle of the target character in the first video frame of the first video are determined.
In the disclosed embodiment, the first video may represent a video used to generate a fitting video. The "first" of the first video is merely for convenience of expression and reference herein, and does not imply that there must be a corresponding first video in a particular implementation of the present disclosure.
In the disclosed embodiments, the first video frame represents a certain video frame of the first video. The "first" of the first video frames is merely for convenience of expression and reference herein, and does not imply that there must be a first video frame corresponding thereto in a particular implementation of the present disclosure.
In one possible implementation, determining an expression of a target person in a first video frame of a first video includes: and inputting the first video frame into a first neural network, and outputting the expression category of the target character in the first video frame by the first neural network. In this implementation, a first neural network is used to identify an expression category of a person.
In one possible implementation, determining a face angle of a target person in a first video frame of a first video includes: and inputting the first video frame into a second neural network, and outputting the face angle of the target person in the first video frame by the second neural network. In this implementation, a second neural network is used to identify face angles.
In step S12, a first image that matches the expression and face angle of the target person in the first video frame is determined from the images corresponding to the trial characters.
In one possible implementation, before step S12, the method may further include: images of a plurality of expressions of a try-on person are acquired from a plurality of angles. For example, the plurality of expressions includes a happy expression, a sad expression, an angry expression, an aversive expression, a frightened expression, a surprised expression, and a calm expression. In this implementation, an image of a certain expression of the test person can be acquired through a plurality of image pickup devices at different angles at the same time. The shooting device can be a camera or a mobile phone.
In the disclosed embodiments, the span of the plurality of angles is not limited. For example, there may be a plurality of angles within 180 degrees; as another example, there may be multiple angles within 360 degrees; as another example, there may be multiple angles within 90 degrees. The span of the plurality of angles may also be less than 90 degrees, and the embodiments of the present disclosure are not described in detail.
In the disclosed embodiments, the included angle between adjacent angles is not limited. For example, the included angle between adjacent angles is 10 degrees; as another example, the included angle between adjacent angles is 5 degrees; as another example, the included angle between adjacent angles is 20 degrees.
Fig. 2 is a schematic diagram illustrating a method for generating a fitting video according to an embodiment of the present disclosure, in which images of multiple expressions of a fitting person are acquired from multiple angles. In the example shown in fig. 2, images corresponding to different expressions may be acquired every 10 degrees from the left 90 degrees to the right 90 degrees. For example, images corresponding to different expressions may be acquired from left 90 degrees, left 80 degrees, left 70 degrees, left 60 degrees, left 50 degrees, left 40 degrees, left 30 degrees, left 20 degrees, left 10 degrees, front 0 degrees, right 10 degrees, right 20 degrees, right 30 degrees, right 40 degrees, right 50 degrees, right 60 degrees, right 70 degrees, right 80 degrees, and right 90 degrees, respectively.
In one possible implementation, images of different angles and different expressions of the test-mirror figure can be labeled and stored. For example, an image of a happy expression of 0 degrees on the front is labeled "happy _ front _0. jpg", an angry expression of the left side face shifted by 10 degrees from the front is labeled "angry _ left _10. jpg", and a surprised expression of the right side face shifted by 30 degrees from the front is labeled "surprised _ right _30. jpg".
In the embodiment of the disclosure, the image corresponding to the try-on person comprises images with a plurality of angles and a plurality of expressions. For example, if the plurality of angles include 19 different angles from 90 degrees to 90 degrees from the left and the plurality of expressions include 7 expressions, the image corresponding to the test person includes 19 × 7 — 133 images.
In step S13, the first video frame is replaced with the first image, and a fitting video corresponding to the fitting person is obtained.
In a possible implementation manner, for each video frame of the first video, step S11 and step S12 may be respectively executed, so as to respectively determine a first image matching with each video frame of the first video, and thus respectively perform replacement processing on each video frame of the first video by using the first image, so as to obtain a fitting video corresponding to a fitting person.
In one possible implementation, performing replacement processing on a first video frame according to a first image includes: the first video frame is replaced with the first image. In this implementation, by replacing the first video frame with the first image, it is possible to replace each of the video frames including the target person in the first video with an image corresponding to the trial person, and thus obtain a trial video corresponding to the trial person.
In the embodiment of the disclosure, the expression and the face angle of the target person in the first video frame of the first video are determined, the first image matched with the expression and the face angle of the target person in the first video frame is determined from the image corresponding to the test lens person, and the first video frame is subjected to replacement processing according to the first image to obtain the test lens video corresponding to the test lens person, so that the test lens video can be quickly generated according to the images of the test lens person with the multiple expressions and the multiple face angles, the generation efficiency of the test lens video can be greatly improved, and the time for testing the lens can be saved.
Fig. 3 shows an exemplary flowchart of step S12 of a generation method of a trial video according to an embodiment of the present disclosure. As shown in fig. 3, step S12 may include step S121 and step S122.
In step S121, an image of the image corresponding to the trial image character, which is the same as the expression category of the target character in the first video frame, is determined as a candidate image.
For example, if the expression category of the target person in the first video frame is happy, an image whose expression category is happy in the image corresponding to the trial image person may be determined as a candidate image.
In step S122, the candidate image closest to the face angle of the target person in the first video frame is determined as the first image.
For example, if the face angle of the target person in the first video frame is 38 degrees, the face angle of the candidate image a is 30 degrees, and the face angle of the candidate image B is 40 degrees, the candidate image B may be determined as the first image.
Fig. 4 shows an exemplary flowchart of step S13 of a generation method of a trial video according to an embodiment of the present disclosure. As shown in fig. 4, step S13 may include step S131 and step S132.
In step S131, a face region of the test person is cut out from the first image.
In one possible implementation manner, the region defined by the face bounding box in the first image may be used as the face region of the person trying on the mirror in the first image.
In step S132, the face area of the target person in the first video frame is replaced according to the face area of the trial person in the first image.
In one possible implementation, only the face region of the target person in the first video frame may be replaced, while the other regions (e.g., the background region) in the first video frame are retained.
Fig. 5 shows an exemplary flowchart of step S132 of a generation method of a trial video according to an embodiment of the present disclosure. As shown in fig. 5, step S132 may include step S1321 and step S1322.
In step S1321, the face area of the test-mirror person in the first image is adjusted.
In one possible implementation manner, the adjusting the face area of the trial image person in the first image includes: and adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame.
As an example of this implementation, adjusting the size of the face region of the test-mirror person in the first image according to the size of the face region of the target person in the first video frame includes: the size of the face region of the test-mirror person in the first image is adjusted to be the same as the size of the face region of the target person in the first video frame.
In another possible implementation manner, the adjusting the face area of the trial image person in the first image includes: and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
As one example of this implementation, the visual parameters include one or more of brightness, hue, saturation, and resolution.
As an example of this implementation, the visual parameters of the face region of the trial character in the first image may be adjusted to coincide with the visual parameters of the face region of the target character in the first video frame.
In another possible implementation manner, the adjusting the face area of the trial image person in the first image includes: adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame; and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
In step S1322, the face area of the target person in the first video frame is replaced with the face area of the first image after the adjustment of the test-mirror person.
The human face area of the trial lens person in the first image is adjusted, so that the adjusted human face area of the trial lens person in the first image can adapt to the first video frame, and the obtained trial lens video is more natural.
Fig. 6 shows a block diagram of a generation apparatus of a trial video according to an embodiment of the present disclosure. As shown in fig. 6, the apparatus includes: a first determining module 61, configured to determine an expression and a face angle of a target person in a first video frame of a first video; a second determining module 62, configured to determine, from the image corresponding to the trial image character, a first image that matches the expression and face angle of the target character in the first video frame; and the replacing module 63 is configured to perform replacement processing on the first video frame according to the first image to obtain a mirror-fitting video corresponding to the mirror-fitting person.
Fig. 7 shows an exemplary block diagram of a generation apparatus of a trial video according to an embodiment of the present disclosure. As shown in fig. 7:
in one possible implementation, the second determining module 62 includes: the first determining sub-module 621 is configured to determine, as a candidate image, an image that is in the same expression category as the target person in the first video frame in the image corresponding to the trial image person; and a second determining sub-module 622 for determining the candidate image closest to the face angle of the target person in the first video frame as the first image.
In one possible implementation, the replacement module 63 is configured to: the first video frame is replaced with the first image.
In one possible implementation, the replacement module 63 includes: an intercepting submodule 631, configured to intercept a face region of the trial person from the first image; the replacing sub-module 632 is configured to replace the face area of the target person in the first video frame according to the face area of the test-mirror person in the first image.
In one possible implementation, the replacement submodule 632 includes: the adjusting unit is used for adjusting the face area of the test-mirror person in the first image; and the replacing unit is used for replacing the face area of the target person in the first video frame with the face area of the first image after the adjustment of the test-mirror person.
In one possible implementation, the adjustment unit is used for one or both of the following: adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame; and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
In one possible implementation, the visual parameters include one or more of brightness, hue, saturation, and resolution.
In the embodiment of the disclosure, the expression and the face angle of the target person in the first video frame of the first video are determined, the first image matched with the expression and the face angle of the target person in the first video frame is determined from the image corresponding to the test lens person, and the first video frame is subjected to replacement processing according to the first image to obtain the test lens video corresponding to the test lens person, so that the test lens video can be quickly generated according to the images of the test lens person with the multiple expressions and the multiple face angles, the generation efficiency of the test lens video can be greatly improved, and the time for testing the lens can be saved.
Fig. 8 is a block diagram illustrating an apparatus 800 for generation of a try-on video in accordance with an example embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 8, the apparatus 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the apparatus 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed status of the device 800, the relative positioning of components, such as a display and keypad of the device 800, the sensor assembly 814 may also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration/deceleration of the device 800, and a change in the temperature of the device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the device 800 to perform the above-described methods.
Fig. 9 is a block diagram illustrating an apparatus 1900 for generation of a try-on video, according to an example embodiment. For example, the apparatus 1900 may be provided as a server. Referring to fig. 9, the device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by the processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.
The device 1900 may also include a power component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input/output (I/O) interface 1958. The device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, MacOS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the apparatus 1900 to perform the above-described methods.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (16)
1. A method for generating a trial video, comprising:
determining the expression and face angle of a target character in a first video frame of a first video;
determining a first image matched with the expression and face angle of the target character in the first video frame from the image corresponding to the trial image character;
and replacing the first video frame according to the first image to obtain a mirror test video corresponding to the mirror test person.
2. The method of claim 1, wherein determining a first image from the images corresponding to the trial image character that matches the expression and face angle of the target character in the first video frame comprises:
determining an image which is in the same expression type as the target character in the first video frame in the image corresponding to the trial image character as a candidate image;
and determining the candidate image closest to the face angle of the target person in the first video frame as the first image.
3. The method of claim 1, wherein performing replacement processing on the first video frame according to the first image comprises:
replacing the first video frame with the first image.
4. The method of claim 1, wherein performing replacement processing on the first video frame according to the first image comprises:
intercepting a face area of the test-mirror person from the first image;
and replacing the face area of the target person in the first video frame according to the face area of the trial image person in the first image.
5. The method of claim 4, wherein replacing the face region of the target person in the first video frame based on the face region of the trial person in the first image comprises:
adjusting the face area of the test-mirror person in the first image;
and replacing the face area of the target person in the first video frame with the face area of the first image after the adjustment of the trial image person.
6. The method of claim 5, wherein adjusting the face region of the trial person in the first image comprises one or both of:
adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame;
and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
7. The method of claim 6, wherein the visual parameters include one or more of brightness, hue, saturation, and resolution.
8. An apparatus for generating a test-view video, comprising:
the first determining module is used for determining the expression and the face angle of a target character in a first video frame of a first video;
the second determining module is used for determining a first image matched with the expression and face angle of the target character in the first video frame from the image corresponding to the trial image character;
and the replacing module is used for replacing the first video frame according to the first image to obtain a mirror test video corresponding to the mirror test person.
9. The apparatus of claim 8, wherein the second determining module comprises:
the first determining sub-module is used for determining an image, which is in the same expression type as the target character in the first video frame, in the image corresponding to the trial image character as a candidate image;
and the second determining sub-module is used for determining the candidate image closest to the face angle of the target person in the first video frame as the first image.
10. The apparatus of claim 8, wherein the replacement module is configured to:
replacing the first video frame with the first image.
11. The apparatus of claim 8, wherein the replacement module comprises:
the intercepting submodule is used for intercepting a face area of the test mirror figure from the first image;
and the replacing sub-module is used for replacing the face area of the target person in the first video frame according to the face area of the trial image person in the first image.
12. The apparatus of claim 11, wherein the replacement sub-module comprises:
the adjusting unit is used for adjusting the face area of the test-mirror person in the first image;
and the replacing unit is used for replacing the face area of the target person in the first video frame with the face area of the first image after the adjustment of the trial image person.
13. The apparatus of claim 12, wherein the adjustment unit is configured to one or both of:
adjusting the size of the face area of the test-mirror person in the first image according to the size of the face area of the target person in the first video frame;
and adjusting the visual parameters of the face area of the test-mirror person in the first image according to the visual parameters of the face area of the target person in the first video frame.
14. The apparatus of claim 13, wherein the visual parameters comprise one or more of brightness, hue, saturation and resolution.
15. An apparatus for generating a test-view video, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the method of any one of claims 1 to 7.
16. A non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811139042.8A CN110969569A (en) | 2018-09-28 | 2018-09-28 | Method and device for generating test-mirror video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811139042.8A CN110969569A (en) | 2018-09-28 | 2018-09-28 | Method and device for generating test-mirror video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110969569A true CN110969569A (en) | 2020-04-07 |
Family
ID=70026759
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811139042.8A Pending CN110969569A (en) | 2018-09-28 | 2018-09-28 | Method and device for generating test-mirror video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110969569A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111601033A (en) * | 2020-04-27 | 2020-08-28 | 北京小米松果电子有限公司 | Video processing method, device and storage medium |
| CN112017141A (en) * | 2020-09-14 | 2020-12-01 | 北京百度网讯科技有限公司 | Video data processing method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140198177A1 (en) * | 2013-01-15 | 2014-07-17 | International Business Machines Corporation | Realtime photo retouching of live video |
| CN104376589A (en) * | 2014-12-04 | 2015-02-25 | 青岛华通国有资本运营（集团）有限责任公司 | Method for replacing movie and TV play figures |
| CN105118082A (en) * | 2015-07-30 | 2015-12-02 | 科大讯飞股份有限公司 | Personalized video generation method and system |
| CN108259788A (en) * | 2018-01-29 | 2018-07-06 | 努比亚技术有限公司 | Video editing method, terminal and computer readable storage medium |
-
2018
- 2018-09-28 CN CN201811139042.8A patent/CN110969569A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140198177A1 (en) * | 2013-01-15 | 2014-07-17 | International Business Machines Corporation | Realtime photo retouching of live video |
| CN104376589A (en) * | 2014-12-04 | 2015-02-25 | 青岛华通国有资本运营（集团）有限责任公司 | Method for replacing movie and TV play figures |
| CN105118082A (en) * | 2015-07-30 | 2015-12-02 | 科大讯飞股份有限公司 | Personalized video generation method and system |
| CN108259788A (en) * | 2018-01-29 | 2018-07-06 | 努比亚技术有限公司 | Video editing method, terminal and computer readable storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111601033A (en) * | 2020-04-27 | 2020-08-28 | 北京小米松果电子有限公司 | Video processing method, device and storage medium |
| US11368632B2 (en) | 2020-04-27 | 2022-06-21 | Beijing Xiaomi Pinecone Electronics Co., Ltd. | Method and apparatus for processing video, and storage medium |
| CN112017141A (en) * | 2020-09-14 | 2020-12-01 | 北京百度网讯科技有限公司 | Video data processing method and device |
| CN112017141B (en) * | 2020-09-14 | 2024-06-04 | 北京百度网讯科技有限公司 | Video data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210326587A1 (en) | Human face and hand association detecting method and a device, and storage medium | |
| CN107692997B (en) | Heart rate detection method and device | |
| CN111553864B (en) | Image restoration method and device, electronic equipment and storage medium | |
| WO2022134382A1 (en) | Image segmentation method and apparatus, and electronic device, storage medium and computer program | |
| CN110569777B (en) | Image processing method and device, electronic device and storage medium | |
| CN110933488A (en) | Video editing method and device | |
| CN110858924B (en) | Video background music generation method and device and storage medium | |
| CN111243011A (en) | Key point detection method and device, electronic equipment and storage medium | |
| CN113065591B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN111523346B (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN109934240B (en) | Feature updating method and device, electronic equipment and storage medium | |
| CN108174269B (en) | Visual audio playing method and device | |
| CN110633715B (en) | Image processing method, network training method and device and electronic equipment | |
| CN109992754B (en) | Document processing method and device | |
| CN109756783B (en) | Poster generation method and device | |
| CN110969569A (en) | Method and device for generating test-mirror video | |
| CN112102300A (en) | Counting method and device, electronic equipment and storage medium | |
| CN110929545A (en) | Human face image sorting method and device | |
| CN112541875A (en) | Depth image processing method and device, electronic equipment and storage medium | |
| CN111694571B (en) | Compiling method and device | |
| CN110121115B (en) | Method and device for determining wonderful video clip | |
| CN110333903B (en) | Method and device for determining page loading duration | |
| CN110858921A (en) | Program video processing method and device | |
| CN110955800A (en) | Video retrieval method and device | |
| CN114445298A (en) | Image processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200509 Address after: 310052 room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: 200241, room 2, floor 02, building 555, Dongchuan Road, Minhang District, Shanghai Applicant before: CHUANXIAN NETWORK TECHNOLOGY (SHANGHAI) Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200407 |