CN107424218B - 3D try-on-based sequence diagram correction method and device - Google Patents
3D try-on-based sequence diagram correction method and device Download PDFInfo
- Publication number
- CN107424218B CN107424218B CN201710611858.5A CN201710611858A CN107424218B CN 107424218 B CN107424218 B CN 107424218B CN 201710611858 A CN201710611858 A CN 201710611858A CN 107424218 B CN107424218 B CN 107424218B
- Authority
- CN
- China
- Prior art keywords
- frame
- image
- sequence
- sequence diagram
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
Abstract
The invention discloses a sequence diagram correction method based on 3D try-on, which comprises the steps of detecting each frame of image of a video generated in the 3D try-on, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprise a yaw angle; selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas; redetection is carried out on each frame of image in the effective sequence diagram set so as to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprise redetection yaw angles; correcting the re-detected yaw angle according to the sequence of each frame of image in the effective sequence diagram set; the method disclosed by the invention improves the accuracy of the identification of the fitting part through multiple detections, reduces the error when the fitting part is identified, and thus improves the accuracy of the fitting; the invention also discloses a sequence diagram correction device based on the 3D try-on, and the sequence diagram correction device also has the beneficial effects.
Description
Technical Field
The invention relates to the field of computer vision, in particular to a sequence diagram correction method and device based on 3 try-on.
Background
People usually try on the purchased articles before buying the articles when shopping at ordinary times, but usually need to be repeatedly worn and replaced when trying on each article, which can greatly influence the experience of purchasers. For example, when women buy the earrings, the wearing of a plurality of earrings is usually tried, the patience of the purchasers is affected when the earrings are repeatedly replaced, and the earrings are complex to wear.
With the development of science and technology, based on the above problems, there is a 3D fitting method, which includes the steps of turning left and right body parts of a purchaser and recording the process. For example, when the purchaser wants to purchase the earring, the purchaser may first turn the head left or right, and the process is recorded by the camera. After the video is processed, the purchaser can add each article to be purchased to the video which is recorded just before, and the effect after wearing is watched through the animation generated by the computer, so that the process of repeated try-on is omitted.
However, in the prior art, the try-on part needs to be identified first, and when the identification has an error, the problem of low try-on precision occurs, that is, the article to be worn cannot be accurately worn at the position to be worn, which may affect the judgment of the try-on result by the purchaser and affect the experience of the user.
Disclosure of Invention
In view of the above, the main objective of the present invention is to provide a sequence diagram correction method based on 3D fitting, which can effectively reduce the error generated when identifying the fitting part; another object of the present invention is to provide a sequence diagram correction apparatus based on 3D fitting, which can effectively solve the problem of low fitting accuracy caused by recognition error.
In order to solve the technical problem, the invention provides a sequence diagram correction method based on 3D try-on, which comprises the following steps:
detecting each frame of image of a video generated in a 3D fitting process, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprise a yaw angle;
selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas;
redetection is carried out on each frame of image in the effective sequence diagram set so as to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprise redetection yaw angles;
and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
Optionally, the method further comprises:
selecting images with preset frame numbers at intervals in sequence in the video to form an original frame sequence atlas;
the detecting each frame of image in the video to generate detection data corresponding to each frame of image in the video, wherein the detecting data including a yaw angle comprises:
detecting each frame image in the original frame sequence diagram set to generate detection data corresponding to each frame image in the original frame sequence diagram set, wherein the detection data comprises a yaw angle.
Optionally, the selecting the frames of images with the yaw angle within a first threshold range to form an effective sequence atlas includes:
and selecting images with the yaw angles within a first threshold value range and with preset frame numbers at intervals in sequence to form an effective sequence image set.
Optionally, the detecting each frame image of the video generated in the 3D fitting includes:
positioning the fitting part;
carrying out normalization processing on the positioning area;
and calculating the initial position of the preset position in the positioning area.
Optionally, the correcting the re-detected yaw angle according to the sequence of each frame of image in the effective sequence diagram set includes:
sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value;
and when the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence chart set, re-assigning the re-detected yaw angles.
Optionally, the apparatus includes:
a detection module: detecting each frame of image of a video generated in a 3D fitting process, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprise a yaw angle;
the effective sequence atlas generation module: selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas;
a re-detection module: redetection is carried out on each frame of image in the effective sequence diagram set so as to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprise redetection yaw angles;
a correction module: and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
Optionally, the apparatus further comprises:
an original frame sequence atlas generation module: selecting images with preset frame numbers at intervals in sequence in the video to form an original frame sequence atlas;
the detection module specifically comprises:
and detecting each frame of image in the original frame sequence diagram set to generate detection data corresponding to each frame of image in the original frame sequence diagram set, wherein the detection data comprises a module of yaw angle.
Optionally, the effective sequence atlas generation module specifically includes:
and selecting images with the yaw angles within a first threshold value range and with preset frame numbers at intervals in sequence to form an effective sequence image set.
Optionally, the detection module includes:
a positioning unit: positioning the fitting part;
a normalization unit: carrying out normalization processing on the positioning area;
a calculation unit: and calculating the initial position of the preset position in the positioning area.
Optionally, the correction module includes:
the re-detection yaw angle sequencing unit: sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value;
an assignment unit: and when the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence chart set, re-assigning the re-detected yaw angles.
According to the 3D try-on-based sequence diagram correction method provided by the invention, after each frame in a video is detected, each frame in the video is re-detected, and the detection data is corrected. The accuracy of the recognition of the fitting part is improved through multiple detections, and the error in the process of recognizing the fitting part is reduced, so that the accuracy of the fitting is improved, and the experience of a purchaser is improved. The invention also provides a sequence diagram correction device based on the sequence diagram correction method, which has the beneficial effects and is not repeated herein.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a flowchart of a first sequence diagram correction method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a second sequence diagram calibration method according to an embodiment of the present invention;
fig. 3 is a flowchart of a specific implementation method in S101;
fig. 4 is a flowchart of a specific implementation method in S104;
FIG. 5 is a flowchart of a sequence diagram correction method in a specific scenario according to an embodiment of the present invention;
fig. 6 is a block diagram of a sequence diagram correction apparatus according to an embodiment of the present invention.
Detailed Description
The core of the invention is to provide a sequence diagram correction method based on 3D fitting, in the prior art, after the rotating video of the fitting part is recorded, the fitting part needs to be identified first, but after the identification is finished, the data generated by the identification, such as yaw data, is not corrected. When the identification has errors, errors are generated in data generated by the identification, so that the positioned area is not accurate, errors occur in positioning of key parts, such as earlobes, eyes, nose bridges and the like, and when the video is tried on, the articles tried on in the video cannot accurately appear at the key parts, namely, the articles to be worn cannot accurately appear at the positions to be worn. The above situation may affect the judgment of the fitting result by the purchaser, and affect the experience of the user.
The sequence diagram correction method provided by the invention detects each frame in the video, then detects each frame in the video again, and corrects the detection data. The accuracy of the recognition of the fitting part is improved through multiple detections, and the error in the process of recognizing the fitting part is reduced, so that the accuracy of the fitting is improved, and the experience of a purchaser is improved.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a first sequence diagram correction method according to an embodiment of the present invention, including:
s101: detecting each frame of image of a video generated in the 3D fitting, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprises a yaw angle.
In this step, the video generated in the 3D fitting process is that the purchaser twists the body part left and right first, and records the process. For example, when the purchaser wants to purchase the earring, the purchaser may first turn the head left or right, and the process is recorded by the camera. After the video is recorded, various data need to be detected and verified, and the aim of improving the fitting precision is achieved.
In this step, the detection data includes a yaw angle, but in a normal case, when data of each frame image in a video is detected, a coordinate system used is a three-dimensional coordinate system in the image, and therefore the detection data further includes a pitch angle and a roll angle. The yaw angle is the angle of left-right torsion of the fitting part, the pitch angle is the angle of up-down inclination of the fitting part, and the roll angle is the angle of left-right inclination of the fitting part. In actual operation, the yaw angle is usually the data of the main transformation, so in the present invention, the detection data used is the yaw angle. Of course, other detection data, such as the pitch angle and the roll angle, may be used to assist in image screening, and are not described herein.
The method for detecting each item of data will be described in detail in the following embodiments, and will not be described herein.
S102: and selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas.
In this step, the principle of selecting the image is that the selected image includes as much as possible all images of the fitting part exposed during the left-right twisting process of the fitting part, for example, when the article to be fitted by the purchaser is an earring, the face of the purchaser faces the screen as the origin of coordinates, the first threshold range is substantially-90 degrees to 90 degrees, and in the interval of-90 degrees to 90 degrees, the effect of fitting can be clearly seen by the purchaser through the images recorded in the video; when the article to be try-on by the purchaser is clothes, the first threshold range is approximately-180 degrees to 180 degrees by taking the body of the purchaser facing the screen as the origin of coordinates, and in the interval of-180 degrees to 180 degrees, the effect of try-on can be clearly seen by the purchaser through the image in the recorded video. In the embodiment of the present invention, the value range of the first threshold range corresponds to different value ranges according to different selections of the origin of coordinates, and the selection angle is not strictly limited to-90 degrees to 90 degrees, or-180 degrees to 180 degrees, as long as the selected image is ensured to enable a purchaser to clearly see the effect of trying on the image recorded in the video, which is not specifically limited herein.
S103: and performing redetection on each frame of image in the effective sequence diagram set to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprises redetection yaw angles.
In this step, the re-detection method is substantially similar to the detection method for detecting the image of each frame of the video in S101, and the detailed process will be described in detail in the following embodiments, which are not repeated herein.
The re-detection data generated by re-detection may include a re-detection pitch angle, a re-detection roll angle, and the like, in addition to the re-detection yaw angle, and the re-detection data is similar to the detection data in S101, and specific conditions are described in detail in S101 and will not be described herein.
The re-detection data generated by re-detection should at least include the detection data generated during the above-mentioned detection, for example, when only the yaw angle exists in the detection data, the re-detection data should also include the re-detected yaw angle, but the re-detection data may further include data not included in the detection data, such as re-detected roll angle, re-detected pitch angle, etc., for screening the images in the valid sequence diagram set. Of course, the re-detection data may also correspond to the detection data one to one, that is, data not included in the detection data is not included, and the specific setting manner is determined according to specific situations, and the embodiment of the present invention is not particularly limited herein.
S104: and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
In this step, the re-detection data generated in S103 is substituted for the detection data generated in S101, so as to make the detection generated data more accurate.
The step of correcting the re-detected yaw angle may be further performed, and will be described in detail in the following embodiments, which will not be further expanded herein.
According to the sequence diagram correction method based on the 3D try-on, provided by the embodiment of the invention, after each frame in a video is detected, each frame in the video is re-detected, and the detection data is corrected. The accuracy of the recognition of the fitting part is improved through multiple detections, and the error in the process of recognizing the fitting part is reduced, so that the accuracy of the fitting is improved, and the experience of a purchaser is improved.
On the basis of the embodiment of the invention, partial repeated images can be further removed, so that the correction method provided by the invention can be rapidly and effectively carried out. The detailed process will be described in detail in the following examples.
Referring to fig. 2 and fig. 3, fig. 2 is a flowchart of a second sequence diagram correction method according to an embodiment of the present invention, and fig. 3 is a flowchart of an implementation method in S101.
Referring to fig. 2, the steps include:
s201: and selecting images with preset frame numbers at intervals in sequence in the video to form an original frame sequence atlas.
In this step, the video is a video generated in the 3D fitting, which has been described in detail in S101 in the foregoing embodiment of the present invention and is not described herein again.
In this step, the purpose of selecting images sequentially spaced by a predetermined number of frames is to eliminate some repeated images in advance to increase the generation speed of the try-on video. The preset number of frames may be 3 frames, 4 frames, 5 frames, etc., as the case may be, and is not particularly limited herein.
In addition to the above-mentioned selecting of the images sequentially spaced by the preset number of frames, the original frame sequence atlas may be generated by extracting one frame of image from the images with the preset number of frames, for example, the original frame sequence atlas may be generated by extracting one frame of image from every 3 frames of images, of course, the extraction method may be random extraction, or may be extraction according to a fixed rule, and is not limited specifically here.
Further, the video may be first cut into a predetermined number of frames, for example, the video is 30 frames per second, the video is first cut into 15 frames per second, and the above steps are continuously performed, but the present screening step may not be performed, as the case may be.
S202: detecting each frame image in the original frame sequence diagram set to generate detection data corresponding to each frame image in the original frame sequence diagram set, wherein the detection data comprises a yaw angle.
This step is substantially similar to S101, and is to detect an image and generate detection data, except that in this step, an original sequence atlas is detected, and the rest is described in detail in the above embodiment of the present invention, and is not described again here.
S203: and selecting images with the yaw angles within a first threshold value range and with preset frame numbers at intervals in sequence to form an effective sequence image set.
In this step, the images selected at the preset frame number in sequence are the same as the images selected at the preset frame number in sequence in S201, and the specific situation and the expansion are described in detail in S201, and are not described again here.
In this step, repeated images can be further removed, and an effective sequence atlas is screened out.
Besides the above methods, other methods can be selected to select the effective sequence atlas, such as: the method comprises the steps of firstly sequencing all frame images in an original frame sequence diagram set according to the magnitude of a yaw angle value, and then selecting images with the yaw angle within a first threshold value range at intervals in sequence and with preset yaw angle degrees to form an effective sequence diagram set. The predetermined yaw angle may be 0.5 degrees, 1 degree, 2 degrees, or the like. Of course, the effective sequence atlas may also be generated in a manner of extracting one frame of image from an image with a preset yaw angle degree, where the extraction method may be to randomly extract the image, or to extract the image according to a certain rule, and is not limited herein.
Compared with the method for selecting the images with a certain number of frames, the method for selecting the images with a certain number of intervals of the yaw angle can effectively eliminate the repeated images, and does not need to worry about the condition that the finally generated try-on video is not consistent due to the fact that too many non-repeated images are eliminated.
S204: and performing redetection on each frame of image in the effective sequence diagram set to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprises redetection yaw angles.
In this step, after the redetection is performed, a method similar to the above-mentioned method for selecting images with a certain degree of yaw angle interval may also be adopted, and the images with a certain degree of yaw angle interval for redetection are selected to form a new effective sequence atlas, and the subsequent steps such as S205 and the like are executed. The method for screening images in this step is similar to the method for screening images in S203, and is already described in detail in S203, and is not repeated herein.
S205: and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
S204 and S205 are the same as S103 and S104 in the above embodiments of the invention, and the specific situation has been described in detail in the above embodiments, please refer to the above embodiments, and will not be described herein.
In the embodiment of the present invention, the detection process and the re-detection process may specifically include:
s301: and positioning the fitting part.
In this step, the fitting site is positioned by a preset algorithm to acquire an image of the fitting site. In the embodiment of the present invention, a specific algorithm is not limited as long as the fitting portion can be located.
S302: and carrying out normalization processing on the positioning area.
The specific normalization process includes scaling the size of the localization area to the same size, and the like. The normalization of the positioning region in this step is to accurately position the key part in the positioning region, and the specific normalization step is not specifically limited herein.
S303: and calculating the initial position of the preset position in the positioning area.
In this step, the preset positions are some key parts in the region located in S301, for example, when the located region is a human face, the key parts include eyes, a mouth, a nose bridge, an earlobe and the like; when the positioning area is the upper half of the human body, the key parts comprise shoulders, arms, abdomen and the like. When the posture of a person in the video deflects, the positioning area also deflects, and correspondingly, the coordinates of the key part also change correspondingly. After the initial position of the key portion is calculated, that is, after the initial position of the preset position is calculated, the detection data of each frame image in the video may be calculated according to the initial position and the subsequent change.
In the detection process, in order to improve the detection speed, a method for tracking a key part is used for optimizing the detection speed, wherein data detected by the method for tracking the key part is most accurate in the process that the key part rotates towards two sides from the direction opposite to the screen, so in the re-detection process, the image in the interval that the key part rotates towards one side from the direction opposite to the screen is usually selected to be re-detected, for example, when the first threshold interval selected in the above steps is-90 degrees to 90 degrees, the image is selected to be re-detected from 0 degrees to-90 degrees, and then the image is re-detected from 0 degrees to 90 degrees. In the embodiment of the present invention, the re-detection step is the same as the detection step, and the method used in S301 to S303 is described in detail. Of course, besides the above-mentioned selection manner of the re-detection angle, there may be other selection manners of re-detection supervision, which is not specifically limited herein.
The sequence diagram correction method based on the 3D try-on can eliminate partial repeated images, thereby greatly improving the detection speed, increasing the generation speed of the try-on video and effectively ensuring the quick implementation of the correction method provided by the invention.
When a large error occurs in the calculation of the yaw angle in the detection process, the yaw angle needs to be reassigned, and the specific process will be described in detail in the following embodiments of the invention.
Referring to fig. 4, fig. 4 is a flowchart of an implementation method in S104, where the method specifically includes:
s401: and sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value.
In this step, the re-detected yaw angles are sorted in order of magnitude. In actual operation, since the fitting part is continuously rotated, the change sequence of the yaw angle of each frame image should be continuously changed correspondingly, i.e. the change sequence of the yaw angle should be consistent with the sequence of each frame image in the video. When the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence diagram set, the calculation of the yaw angles has a large error, and at the moment, the value of the yaw angles of the images with the large error needs to be re-assigned.
S402: and when the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence chart set, re-assigning the re-detected yaw angles.
In this step, a specific assignment process may be to select 2 images before and after the image with the error, calculate the yaw angle of the image with the error according to the linear relationship between the maximum yaw angle and the minimum yaw angle in the 4 images, and assign a new value. In the embodiment of the present invention, different numbers of images may be selected, for example, 5 images before and after the image with the error is selected, and the selection manner is many, and the data appearing here is only used for illustration and is not a specific limitation to the above method.
In addition to the above method, there may be another method to re-assign the re-detected yaw angle, for example, re-calculate the re-detected yaw angle of the image with the error, and the like, and is not limited herein.
According to the sequence diagram correction method based on 3D fitting provided by the embodiment of the invention, when a larger error occurs in calculation of the yaw angle, the yaw angle of the image with the yaw angle error can be reassigned according to a plurality of images before and after the image with the yaw angle error, so that each yaw angle in the effective sequence diagram set is corrected, and errors caused by calculation errors of the yaw angle are reduced.
In the implementation process of the invention, for example, when the article to be tried on by the purchaser is glasses, the purchaser records a video of 60 frames per second of images about the rotation of the face of the purchaser for 36 seconds through the camera.
Referring to fig. 5, fig. 5 is a flowchart of a sequence diagram correction method in a specific scenario according to an embodiment of the present invention, including the steps of:
s501: and selecting images which are sequentially spaced by 2 frames in the video to form an original frame sequence atlas.
In this step, 20 images per second, for a total of 720 images, are selected.
S502: detecting each frame image in the original frame sequence diagram set to generate detection data corresponding to each frame image in the original frame sequence diagram set, wherein the detection data comprises a yaw angle.
S503: and sequencing the frame images in the original frame sequence diagram set according to the magnitude of the yaw angle value.
S504: and selecting images with the yaw angles within-90 degrees to 90 degrees and sequentially spaced by 0.5 degree to form an effective sequence atlas.
In the step, the coordinate origin is selected in the direction of the face facing the screen, and images with yaw angles sequentially spaced by 0.5 degrees are selected, wherein the images generally have 360 frames.
S505: and performing redetection on each frame of image in the effective sequence diagram set to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprises redetection yaw angles.
S506: and sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value.
S507: and when the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence chart set, re-assigning the re-detected yaw angles.
The sequence diagram correction method based on the 3D try-on is a step which may occur when an article which is required to be tried on by a purchaser is glasses. In the embodiment of the present invention, the specific data is presented only to explain the idea of the present invention, and when other specific scenarios occur, the specific data therein may be changed or other necessary steps may be added to achieve the purpose of the present invention.
In the following, the sequence diagram correction apparatus based on 3D fitting provided by the embodiment of the present invention is introduced, and the sequence diagram correction apparatus described below and the sequence diagram correction method described above may be referred to correspondingly.
Fig. 6 is a block diagram of a sequence diagram correction apparatus according to an embodiment of the present invention, and referring to fig. 6, the sequence diagram correction apparatus according to the embodiment of the present invention may include:
the detection module 100: detecting each frame of image of a video generated in the 3D fitting, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprises a yaw angle.
Valid sequence atlas generation module 200: and selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas.
The re-detection module 300: and performing redetection on each frame of image in the effective sequence diagram set to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprises redetection yaw angles.
The correction module 400: and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
The embodiment of the present invention may further include an original frame sequence atlas generation module 500: and selecting images with preset frame numbers at intervals in sequence in the video to form an original frame sequence atlas.
The detection module 100 may specifically be: and detecting each frame of image in the original frame sequence diagram set to generate detection data corresponding to each frame of image in the original frame sequence diagram set, wherein the detection data comprises a module of yaw angle.
The effective sequence atlas generation module 200 may specifically be: and selecting images with the yaw angles within a first threshold value range and with preset frame numbers at intervals in sequence to form an effective sequence image set.
The detection module 100 may comprise a positioning unit 101: positioning the fitting part; the normalization unit 102: carrying out normalization processing on the positioning area; the calculation unit 103: and calculating the initial position of the preset position in the positioning area.
The correction module 400 may include a redetection yaw ranking unit 401: sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value; assignment unit 402: and when the sequence of the re-detection yaw angles is inconsistent with the sequence of the images in the effective sequence diagram set, re-assigning the re-detection yaw angles.
The sequence diagram correction apparatus of this embodiment is configured to implement the foregoing sequence diagram correction method, and therefore specific implementations of the sequence diagram correction apparatus can be found in the foregoing embodiment parts of the sequence diagram correction method, for example, the detection module 100, the effective sequence diagram set generation module 200, the re-detection module 300, and the correction module 400, which are respectively configured to implement steps S101, S102, S103, and S104 in the foregoing sequence diagram correction method, so that the specific implementations thereof may refer to descriptions of corresponding respective part embodiments, and are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The sequence diagram correction method and device based on 3D fitting provided by the present invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A sequence diagram correction method based on 3D fitting, the method comprising:
detecting each frame of image of a video generated in a 3D fitting process, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprise a yaw angle;
selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas;
redetection is carried out on each frame of image in the effective sequence diagram set so as to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprise redetection yaw angles;
and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
2. The method of claim 1, further comprising:
selecting images with preset frame numbers at intervals in sequence in the video to form an original frame sequence atlas;
detecting each frame of image in the video to generate detection data corresponding to each frame of image in the video, wherein the detecting data comprises a yaw angle, and the detecting comprises the following steps:
detecting each frame image in the original frame sequence diagram set to generate detection data corresponding to each frame image in the original frame sequence diagram set, wherein the detection data comprises a yaw angle.
3. The method of claim 2, wherein said selecting said frames of images having said yaw angle within a first threshold range to form a valid sequence atlas comprises:
and selecting images with the yaw angles within a first threshold value range and with preset frame numbers at intervals in sequence to form an effective sequence image set.
4. The method of claim 3, wherein the detecting each frame of image of the video generated in the 3D fitting comprises:
positioning the fitting part;
carrying out normalization processing on the positioning area;
and calculating the initial position of the preset position in the positioning area.
5. The method according to any one of claims 1 to 4, wherein said correcting said re-detected yaw angle according to an order of frames of images in said set of valid sequence diagrams comprises:
sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value;
and when the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence chart set, re-assigning the re-detected yaw angles.
6. A sequence diagram correction apparatus based on 3D fitting, the apparatus comprising:
a detection module: detecting each frame of image of a video generated in a 3D fitting process, and generating detection data corresponding to each frame of image in the video, wherein the detection data comprise a yaw angle;
the effective sequence atlas generation module: selecting each frame of the image with the yaw angle within a first threshold value range to form an effective sequence atlas;
a re-detection module: redetection is carried out on each frame of image in the effective sequence diagram set so as to generate redetection data corresponding to each frame of image in the effective sequence diagram set, wherein the redetection data comprise redetection yaw angles;
a correction module: and correcting the re-detection yaw angle according to the sequence of each frame of image in the effective sequence diagram set.
7. The apparatus of claim 6, further comprising:
an original frame sequence atlas generation module: selecting images with preset frame numbers at intervals in sequence in the video to form an original frame sequence atlas;
the detection module specifically comprises:
and detecting each frame of image in the original frame sequence diagram set to generate detection data corresponding to each frame of image in the original frame sequence diagram set, wherein the detection data comprises a module of yaw angle.
8. The apparatus according to claim 7, wherein the effective sequence atlas generation module is specifically configured to:
and selecting images with the yaw angles within a first threshold value range and with preset frame numbers at intervals in sequence to form an effective sequence image set.
9. The apparatus of claim 8, wherein the detection module comprises:
a positioning unit: positioning the fitting part;
a normalization unit: carrying out normalization processing on the positioning area;
a calculation unit: and calculating the initial position of the preset position in the positioning area.
10. The apparatus of any one of claims 6 to 9, wherein the correction module comprises:
the re-detection yaw angle sequencing unit: sequencing the re-detection yaw angles of each frame of image in the effective sequence diagram set according to the magnitude of the numerical value;
an assignment unit: and when the sequence of the re-detected yaw angles is inconsistent with the sequence of the images in the effective sequence chart set, re-assigning the re-detected yaw angles.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710611858.5A CN107424218B (en) | 2017-07-25 | 2017-07-25 | 3D try-on-based sequence diagram correction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710611858.5A CN107424218B (en) | 2017-07-25 | 2017-07-25 | 3D try-on-based sequence diagram correction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107424218A CN107424218A (en) | 2017-12-01 |
| CN107424218B true CN107424218B (en) | 2020-11-06 |
Family
ID=60430453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710611858.5A Active CN107424218B (en) | 2017-07-25 | 2017-07-25 | 3D try-on-based sequence diagram correction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107424218B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001223963A (en) * | 2000-02-07 | 2001-08-17 | Sony Corp | System and method for generating video information for displaying on plural screens and recording medium |
| CN1596377A (en) * | 2002-10-01 | 2005-03-16 | 索尼株式会社 | Optical scan device, image position calibration method, and image display device |
| CN101561710A (en) * | 2009-05-19 | 2009-10-21 | 重庆大学 | Man-machine interaction method based on estimation of human face posture |
| CN103456008A (en) * | 2013-08-26 | 2013-12-18 | 刘晓英 | Method for matching face and glasses |
| EP2680225A2 (en) * | 2012-06-28 | 2014-01-01 | Samsung Medison Co., Ltd. | Diagnosis imaging apparatus and operation method thereof |
| CN104392485A (en) * | 2014-11-12 | 2015-03-04 | 天脉聚源（北京）教育科技有限公司 | Method, device and system for displaying three-dimensional model |
| CN105704398A (en) * | 2016-03-11 | 2016-06-22 | 咸阳师范学院 | Video processing method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5168580B2 (en) * | 2008-12-01 | 2013-03-21 | 富士通株式会社 | Driving simulation device, wide-angle camera image simulation device, and image deformation synthesis device |
-
2017
- 2017-07-25 CN CN201710611858.5A patent/CN107424218B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001223963A (en) * | 2000-02-07 | 2001-08-17 | Sony Corp | System and method for generating video information for displaying on plural screens and recording medium |
| CN1596377A (en) * | 2002-10-01 | 2005-03-16 | 索尼株式会社 | Optical scan device, image position calibration method, and image display device |
| CN101561710A (en) * | 2009-05-19 | 2009-10-21 | 重庆大学 | Man-machine interaction method based on estimation of human face posture |
| EP2680225A2 (en) * | 2012-06-28 | 2014-01-01 | Samsung Medison Co., Ltd. | Diagnosis imaging apparatus and operation method thereof |
| CN103456008A (en) * | 2013-08-26 | 2013-12-18 | 刘晓英 | Method for matching face and glasses |
| CN104392485A (en) * | 2014-11-12 | 2015-03-04 | 天脉聚源（北京）教育科技有限公司 | Method, device and system for displaying three-dimensional model |
| CN105704398A (en) * | 2016-03-11 | 2016-06-22 | 咸阳师范学院 | Video processing method |
Non-Patent Citations (3)
| Title |
|---|
| "A measurement method of attitude angles based on artificial labeling feature points";Feng Jin et al;《Proceeding of the 11th World Congress on Intelligent Control and Automation》;20150305;5164-5168 * |
| "Integrated face detection, tracking, and pose estimation";Masayuki Miyama et al;《2012 IEEE 11th International Conference on Signal Processing》;20130404;1056-1059 * |
| "一种多视角人脸检测方法";马颖哲 等;《计算机工程与科学》;20110131;第33卷(第1期);132-137 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107424218A (en) | 2017-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11749025B2 (en) | Eye pose identification using eye features | |
| AU2015224394B2 (en) | Estimating device and estimation method | |
| WO2017092334A1 (en) | Method and device for image rendering processing | |
| JP5772821B2 (en) | Facial feature point position correction apparatus, face feature point position correction method, and face feature point position correction program | |
| JP4655235B2 (en) | Information processing apparatus and method, and program | |
| JP6216024B1 (en) | Trained model generation method and signal data discrimination device | |
| US20150161435A1 (en) | Frontal face detection apparatus and method using facial pose | |
| CN107424218B (en) | 3D try-on-based sequence diagram correction method and device | |
| KR20210103275A (en) | Apparatus and method for synthesizing face model | |
| US9563816B2 (en) | Information processing apparatus, method for controlling information processing apparatus, and storage medium | |
| JP6510451B2 (en) | Device, method and program for specifying the pupil area of a person in an image | |
| US9721151B2 (en) | Method and apparatus for detecting interfacing region in depth image | |
| CN116416671B (en) | Face image correcting method and device, electronic equipment and storage medium | |
| WO2021245928A1 (en) | Image processing device, image processing method, and image processing program | |
| CN113344904A (en) | Surface defect detection method and related product | |
| JPWO2021193352A5 (en) | Image tracking device, image tracking method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |