KR102318394B1 - playing back section moving apparatus using learning - Google Patents

Abstract

The present invention relates to a device for moving a playback section in makeup demonstration videos through learning and, more specifically, to a device for moving a playback section in makeup demonstration videos through learning, which marks time stamps for a playback start time and a playback end time for each makeup object recognized in a makeup demonstration video. To this end, the device comprises: a video image extraction unit individually extracting makeup demonstration images included in a video related to makeup demonstration by category according to preset conditions; a video image collection unit collecting the extracted makeup demonstration images according to the preset conditions and storing the same by category; a learning unit performing learning on the basis of the makeup demonstration images collected and stored by the moving image collection unit to learn individual makeup products and makeup poses; a video image analysis unit receiving an arbitrary makeup demonstration video related to a makeup demonstration to recognize and estimate a makeup object image included in the makeup demonstration video; and a time stamp unit additionally marking time stamps on the makeup demonstration video on the basis of recognition and estimation of the makeup object image to provide movement to each time stamp section of the makeup demonstration video.

Description

Apparatus for moving image section of makeup method through learning {playing back section moving apparatus using learning}

The present invention relates to an apparatus for moving a video section of a makeup method through learning, and more particularly, moving a section of a makeup method video through learning by marking time stamps for a playback start time and a playback end time for each makeup object recognized in a makeup demonstration video It's about the device.

Many viewers follow the makeup methods of famous YouTubers by referring to videos of famous YouTubers demonstrating various makeup methods for each makeup product provided through the Internet. Multiple makeup poses for multiple makeup products may be inserted into this makeup demonstration video, or makeup poses for one makeup product may be inserted.

Viewers who watch such a video demonstration of how to make up will go through a try and error when trying to move the video playback section with the makeup product they want to watch or when trying to move the video playback section with the makeup pose they want to watch. do. In this way, it takes a lot of time for viewers to find the makeup method playback time they want to watch through try and error, and it is not difficult to find the exact start and end points. Furthermore, there is a problem in that viewers' discomfort will inevitably increase if they go through such a try and error every time they watch various videos demonstrating various makeup methods uploaded by numerous YouTubers every day.

Republic of Korea Patent Publication No. 10-0405328 Republic of Korea Patent Publication No. 10-2019-0100515 Korean Patent Publication No. 10-2020-0043112 Republic of Korea Patent Publication No. 10-1948040

Accordingly, the present invention was created to solve the above-described problems, and by inserting and marking a time stamp in a video showing a YouTuber's makeup method, a YouTuber viewer can conveniently and accurately move to a point where they want to watch. Its purpose is to provide

However, the objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The above-described object of the present invention is a video image extractor for individually extracting a makeup demonstration image included in a video related to a makeup demonstration by category according to a preset condition, and collects the extracted makeup demonstration image according to a preset condition and by category The video image collection unit that stores, the video image collection unit collects by category by category, and the learning unit learns individual makeup products and makeup poses by performing learning based on the stored makeup demonstration images, receiving arbitrary makeup demonstration videos related to makeup demonstration A video image analysis unit that recognizes and estimates the makeup method object image included in the makeup method demonstration video, and additionally marks a time stamp on the makeup method demonstration video based on the recognition and estimation of the makeup method object image to move sections of the makeup method demonstration video by time stamp It can be achieved by providing an apparatus for moving the video section of the makeup method through learning, characterized in that it includes a time stamp unit to provide.

In addition, the video image extraction unit extracts a product logo extractor that extracts a product logo image included in the makeup demonstration image, a product model extraction unit that extracts a product model image included in the makeup demonstration image, and a makeup pose image included in the makeup demonstration image. Includes a pose extractor to extract,

The video image collection unit includes a product logo collection unit that collects and stores a plurality of product logo images extracted from the product logo extraction unit by category, a product model collection unit that collects and stores a plurality of product model images extracted from the product model extraction unit by category, and a pose collecting unit that collects and stores the plurality of makeup pose images extracted by the pose extracting unit by category.

In addition, the learning unit learns makeup products by performing learning based on product logo images and product model images for each category provided by the product logo collection unit and the product model collection unit, and makeup pose images by category provided by the product learning unit and pose collection unit and a pose learning unit that learns a makeup pose by performing learning based on .

In addition, the video image analysis unit receives the makeup method demonstration video and extracts the product logo image and product model image, which are the makeup method object images, and is included in the makeup method demonstration video by learning from the product learning unit based on the extracted product logo image and product model image The product recognition unit that recognizes the makeup product that has been used as an input, receives the makeup demonstration video, extracts the makeup pose image that is the makeup method object image, and the makeup pose included in the makeup method demonstration video by learning the pose learning unit based on the extracted makeup pose image and a pose estimator for estimating.

In addition, the time stamp unit includes a product time stamp unit for marking time stamps for the start and end of playback of the cosmetic product in the makeup method demonstration video based on the recognition of the cosmetic product by the product recognition unit;

A pose time stamp unit for marking time stamps for starting and ending reproduction of a makeup pose in a video for demonstrating a makeup method based on the makeup pose estimation of the pose estimator is included.

The apparatus further includes a pose validity determining unit that determines the validity of a makeup pose based on the makeup pose estimation recognized by the pose estimation unit.

In addition, the pose validity determining unit includes a pose start measurement unit that measures a start time of a makeup pose based on the makeup pose estimation recognized by the pose estimation unit, a pose end measurement unit that measures an end time of a makeup pose, a start time of a makeup pose, and and a pose validity time determination unit that determines the validity time of the makeup pose by comparing it with a preset condition based on the end time.

In addition, the pose time stamp unit marks time stamps for the start and end of the makeup pose on the makeup method demonstration video when the validity of the makeup pose is recognized according to the determination result of the pose validity time determination unit.

According to the present invention as described above, there is an effect that a YouTuber viewer can conveniently and accurately move a section to a point that he or she wants to watch by inserting and marking a time stamp in a video showing a YouTuber's makeup method.

The following drawings attached to this specification illustrate a preferred embodiment of the present invention, and serve to further understand the technical spirit of the present invention together with the detailed description of the present invention, so the present invention is limited to the matters described in those drawings It should not be construed as being limited.
1 is a diagram schematically showing a moving image extracting unit and a moving image collecting unit of a makeup method image section moving device through learning according to an embodiment of the present invention;
2 is a view for explaining a demonstration sequence of makeup products and makeup poses included in a YouTuber's makeup method demonstration video according to an embodiment of the present invention;
3 is a diagram schematically showing the learning unit 300 of the apparatus for moving the video section of the makeup method through learning according to an embodiment of the present invention;
4 is a diagram schematically showing a moving image analysis unit of the apparatus for moving the video section of the makeup method through learning according to an embodiment of the present invention;
5 is a diagram schematically showing a pause validity determination unit, a calculation unit, and a time stamp unit according to an embodiment of the present invention;
6 is a view showing the insertion and marking of the reproduction start time and the reproduction end time of a makeup product and a makeup pose, respectively, in a video of a YouTuber demonstrating a makeup method according to an embodiment of the present invention.

Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In addition, one embodiment described below does not unreasonably limit the content of the present invention described in the claims, and it cannot be said that the entire configuration described in the present embodiment is essential as a solution for the present invention. In addition, descriptions of the prior art and matters obvious to those skilled in the art may be omitted, and the description of the omitted components (methods) and functions may be sufficiently referenced within the scope not departing from the technical spirit of the present invention.

The apparatus for moving a video section of a makeup method through learning according to an embodiment of the present invention learns a video of a demonstration of a makeup method demonstrated by a famous YouTuber, etc. By recognizing and estimating the makeup products and makeup poses included in the makeup demonstration video provided arbitrarily based on numerous learnings and marking the time stamp, viewers who watch the YouTuber video can quickly and It is a device that is provided to make it convenient. Hereinafter, with reference to the accompanying drawings, an apparatus for moving a video section of a makeup method through learning according to an embodiment of the present invention will be described in detail.

As shown in FIG. 1 , a video 10a of a YouTuber demonstrating a makeup method is accessible through the Internet, and a viewer can view the video 10a of a YouTuber demonstrating a makeup method through a connection means such as a computer. can watch At this time, as shown in FIG. 1 , in the YouTuber's makeup method demonstration video 10a, several YouTuber videos 11, 12, 13 can be provided through the Internet, and one YouTuber can use several makeup methods. Demonstration videos can also be provided. In the YouTuber's makeup demonstration video, a makeup pose for one makeup product may be demonstrated for each individual video, or makeup poses corresponding to several makeup products may be demonstrated for each individual video.

On the other hand, as shown in FIG. 2 , in a video demonstrating a makeup method, in general, a YouTuber first introduces a cosmetic product to be demonstrated before demonstrating the makeup method, followed by a video demonstrating the makeup method for the cosmetic product. Therefore, the makeup demonstration video largely includes an image of a makeup product (product logo, product model) and an image of the makeup pose of the makeup product. Of course, the video may include various additional images other than these images, but in an embodiment of the present invention, the description will focus on makeup products and makeup poses.

As shown in FIG. 1 , the moving image extraction unit 100 searches for individual makeup method demonstration videos provided by various YouTubers through the Internet, and extracts images included in the makeup method demonstration video. Such a video for demonstrating the makeup method may be updated daily by a corresponding YouTuber, and thus, the operation from extracting a video image to learning, which will be described later, may be continuously performed automatically or manually according to a preset time.

In order to extract the image included in the makeup demonstration video, the video image extraction unit 100 includes a product logo extraction unit 110 , a product model extraction unit 120 , and a pose extraction unit 130 .

The product logo extraction unit 110 sequentially receives a video demonstration of a makeup method, sets the area of the product logo image included in the videos 11, 12, and 13, and extracts the product logo image based on the set area. Product logo images are inexhaustible for each cosmetic manufacturer and brand. Therefore, in order to learn such a product logo, a product logo image is extracted.

The product model extraction unit 120 sequentially receives the video of demonstration of makeup method, sets the area of the product model image included in the videos 11, 12, and 13, and extracts the product model image based on the set area. Product model images are extracted and learned to recognize products by manufacturers/brands presented by numerous YouTubers through learning, which will be described later.

The pose extraction unit 130 sequentially receives the makeup demonstration video, sets the area of the makeup pose image included in the videos 11, 12, and 13, and extracts the makeup pose image based on the set area. Therefore, by performing learning using the extracted makeup pose image, the eyebrow makeup pose included in the video can be recognized. In this case, the makeup pose image may vary for each makeup part. That is, each makeup pose is different for each makeup part, such as a lipstick pose image, an eyebrow makeup pose image, and a skin pose image. can recognize On the other hand, the makeup poses of each YouTuber may be slightly different from each other even for the same makeup area. This difference in makeup poses can be overcome by learning to be described later. Therefore, in order to increase the learning accuracy, it is desirable to extract various makeup pose images for each YouTuber.

The moving image collection unit 200 collects and stores the product logo image, product model image, and makeup pose image extracted by the above-described moving image extraction unit 100 by category. As an example, YouTuber A's makeup demonstration video includes product logo images and product model images for a1 product, a2 product, and a3 product, and includes b1 makeup pose, b2 makeup pose, and b3 makeup pose image for each product. . In addition, the video showing YouTuber B's makeup method includes a product logo image, a product model image, and a makeup pose image for each product. Each of these images can be collected and stored by category by product logo, by product model, and by makeup pose. The collected and stored various images may be stored in a database (not shown). The database stores various kinds of moving images and images related to moving images and various data referred to in the present invention.

The learning unit 300 performs learning by using the product logo image, the product model image, and the makeup pose image collected and stored in the moving image collection unit 200 . To this end, the learning unit 300 includes a product learning unit 310 and a pose learning unit 320 .

The product learning unit 310 receives the product logo image and the product model image collected and stored by the product logo collection unit 210 and the product model collection unit 220 and performs learning by a preset learning algorithm. The pose learning unit 320 receives the makeup pose image collected and stored by the pose collection unit 230 and performs learning by a preset learning algorithm. The above-described learning algorithm may be performed based on a previously known learning algorithm, and the present invention is not limited to a specific learning algorithm. By performing learning by the product learning unit 310 and the pose learning unit 320 , products for each manufacturer/brand can be recognized, and a makeup pose can be estimated for each makeup part. Since the recognition accuracy increases as the amount of basic data increases, it may be desirable for the learning unit 300 to search and extract videos for demonstrating makeup methods of YouTubers, which are newly updated every day, and to perform learning from time to time.

Meanwhile, the pose learning unit 320 learns various makeup poses for each makeup part or makeup product. In this case, with respect to the makeup pose, a pose estimation error may sometimes occur due to a learning error. That is, as an example, when a YouTuber takes a pose of rubbing his eyes, he may make a pose estimation error by learning to apply eyebrow makeup (or eye makeup). In order to solve such a pose estimation error, a pose validity determining unit 500 to be described later is included in the present invention and will be described later.

1 and 3, by searching through the Internet for videos containing makeup products and makeup methods of various YouTubers, extracting and learning images for product logos, product models, and makeup poses included therein. When a video demonstrating makeup method is input, the product logo, product model, and makeup pose included in the video can be recognized. This will be described later with reference to the accompanying FIGS. 4 to 6 .

As shown in FIG. 4 , the moving image analysis unit 400 extracts each image related to the product logo, product model, and makeup pose included in any one arbitrarily input YouTuber makeup demonstration video 11, and extracts the extracted Recognize and estimate makeup products and makeup poses, respectively, through images. To this end, it includes a product logo extraction unit 410 , a product model extraction unit 420 , a pose extraction unit 430 , a product recognition unit 440 , and a pose estimation unit 450 .

The product logo extraction unit 410 extracts the product logo image included in the inputted makeup demonstration video 11 according to a preset extraction area. The product model extraction unit 420 and the pose extraction unit 430 also extract a product logo and a makeup pose image according to a preset extraction area, respectively.

The product recognition unit 440 recognizes a cosmetic product based on each product logo image and product model image provided by the product logo extraction unit 410 and the product model extraction unit 420 . The pose estimation unit 450 estimates a makeup pose based on the makeup pose image provided by the pose extraction unit 430 . Such recognition and estimation is possible because sufficient learning has been performed by the learning unit 300 described above, and learning can be continuously repeated by finding and generating new learning data when a new makeup method demonstration video is updated. The above-described moving image analysis unit 400 repeatedly analyzes the makeup method demonstration videos 12 and 13 of each YouTuber sequentially input as described above.

When the analysis is completed by the moving image analysis unit 400, it is possible to recognize and estimate the makeup products and makeup poses respectively included in the cosmetic demonstration videos 11, 12, and 13 for each YouTuber.

On the other hand, as described above, when the learning is completed by the learning unit 300 and the makeup pose is estimated, the makeup pose of the YouTuber and the arbitrary motion pose of the YouTuber are similar to each other, so that an error in estimation of the makeup pose may occur. . In order to solve this problem, the pause validity determination unit 500 is included in the present invention.

The pose validity determining unit 500 examines and determines the validity of the makeup pose determined by the pose estimating unit 450 . To this end, the pose validity determination unit 500 includes a pose start measurement unit 510 , a pause end measurement unit 520 , and a pose validity time determination unit 530 .

The pose start measurement unit 510 measures a video playback start time at which the makeup pose starts playing in the makeup method demonstration video based on the makeup pose estimated by the pose estimation unit 450 .

The pose end measurement unit 520 measures the video playback end time at which the makeup pose in the makeup method demonstration video is finished using the makeup pose estimated by the pose estimation unit 540 as a reference.

The pose valid time determining unit 530 receives the video playback start time and end time of the makeup pose respectively, calculates a makeup pose time, and compares the calculated makeup pose time with a preset makeup time for the makeup pose. review and judge the effectiveness of That is, in the case of the eyebrow makeup pose, assuming that the preset makeup time condition is set in the range of 2 to 3 minutes, any YouTuber's arbitrary motion is recognized as the eyebrow makeup pose by the pose estimator 450 and Accordingly, if the effective time of the pose is calculated to be 30 seconds, the arbitrary motion behavior of the YouTuber is a noise action, not a eyebrow makeup pose, so this arbitrary motion action can be excluded. Therefore, in order for the pose validity determining unit 500 to examine and determine the validity of the corresponding makeup pose, it is necessary to pre-store the effective makeup times for various makeup poses in the database.

On the other hand, when the validity of the makeup pose is verified by the pose validity determining unit 500, the time stamp for each product and each makeup pose is marked for convenient and free movement of the YouTuber's makeup demonstration video section. carry out To this end, it includes a time stamp unit 600 having a product time stamp unit 610 and a pause time stamp unit 620 . In general, in a video of a YouTuber demonstrating a makeup method, as shown in FIG. 2 , cosmetics (cosmetic products) to be demonstrated by the YouTuber are first shown to the viewer, and then a makeup method using the cosmetic is demonstrated. Of course, the video for demonstrating the makeup method may include other additional images (as an example, chatting or additional explanations about products, etc.), but these additional images may be excluded from the present invention as described above.

Therefore, the product time stamp unit 610 marks the reproduction start time and reproduction end time of the cosmetic product on the cosmetic demonstration video based on the reproduction time data transmitted from the product reproduction time calculation unit 600 .

In addition, when the pose validity time determination unit 530 determines the validity of the makeup pose, the pose time stamp unit 620 reproduces the makeup pose based on the reproduction time data transmitted from the makeup pose time calculation unit 720 . The start time and playback end time are marked on the cosmetic demonstration video.

On the other hand, the product playback time calculation unit 710 calculates the start and end of the playback time of the cosmetics in the cosmetic demonstration video, respectively, based on the product recognition of the product recognition unit 440 . The makeup pose time calculation unit 720 calculates the start and end times of the makeup pose in the cosmetic demonstration video, respectively, based on the makeup pose estimation of the pose estimation unit 450 . The makeup pose time calculation unit 720 may transmit the calculated start time and end time to the pose effective time determination unit 530 so that the pose validity time determination unit 530 verifies the validity of the makeup pose.

As described above, learning about product logos, product models, and makeup poses is performed according to FIGS. 1 and 3, and the makeup method demonstration video arbitrarily input according to FIGS. 4 and 5 is analyzed through learning through learning. You can mark a time stamp on each YouTuber's makeup demonstration video for each pose.

As an example, as shown in FIG. 6 , assuming that the makeup method demonstration video for YouTuber A's eye makeup method and lipstick makeup method is played from time t11 to time t24, analysis of YouTuber A's makeup method demonstration video Accordingly, it can be seen that the video playback time for “eye makeup products” is played from t11 to t14, and the playback start time and end time can be displayed on the video 10. In addition, it can be seen that the moving picture for “lipstick cosmetic product” is played from t21 to t24, and the playback start time and end time can be displayed on the moving picture 10 .

Similarly, by analyzing the playback start time and end time for the eye makeup pose, it can be inserted and marked in the video 10, and the playback start time and end time for the lipstick makeup pose are analyzed and inserted and marked in the video 10 can do. In this way, as shown in FIG. 6, a time stamp can be additionally marked on a YouTuber's makeup demonstration video.

By adding such a time stamp, a viewer who reproduces it can conveniently move to a section with a cosmetic product or makeup pose that he or she wants. That is, videos of various YouTubers' makeup method demonstrations including the above-described time marking are continuously updated in the database, and viewers can play videos of famous YouTubers' makeup method demonstrations stored in the database through the provided smartphone application. , You can freely move between sections according to the operation of the app.

In the description of the present invention, the description may be omitted for matters obvious to those skilled in the art and those skilled in the art, and the description of these omitted components (methods) and functions will be sufficiently referenced within the scope not departing from the technical spirit of the present invention. will be able In addition, the above-described components of the present invention have been described for convenience of description of the present invention, and components not described herein may be added within a range that does not depart from the technical spirit of the present invention.

The description of the configuration and function of each part has been described separately for convenience of description, and if necessary, any configuration and function may be implemented by being integrated into other components, or may be implemented in more subdivision.

As mentioned above, although it has been described with reference to one embodiment of the present invention, the present invention is not limited thereto, and various modifications and applications are possible. That is, those skilled in the art will readily understand that many modifications are possible without departing from the gist of the present invention. In addition, if it is determined that the detailed description of the known functions related to the present invention and its configuration or the coupling relationship for each configuration of the present invention may unnecessarily obscure the gist of the present invention, it should be noted that the detailed description is omitted. something to do.

10, 10a, 10b : YouTuber makeup demonstration video
11: 1st YouTuber video
12 : 2nd YouTuber video
13 : 3rd YouTuber video
21 : YouTuber
22: cosmetic product or product model
23 : product logo
100: video image extraction unit
110: product logo extraction unit
120: product model extraction unit
130: pose extraction unit
200: video image collection unit
210: product log collection unit
220: product model collection unit
230: pose collection unit
300: study
310: Product Learning Department
320: pose learning unit
400: video image analysis unit
410: product logo extraction unit
420: product model extraction unit
430: makeup pose extraction unit
440: product recognition unit
450: pose estimation unit
500: pose validity judgment unit
510: pose start measurement unit
520: Pose end measurement unit
530: pose valid time determination unit
600: timestamp part
610: product timestamp part
620: pose time stamp part
710: product playback time calculation unit
720: makeup pose time calculation unit

Claims (8)

A video image extraction unit that individually extracts makeup demonstration images included in a video related to makeup demonstration by category according to preset conditions;
A video image collection unit that collects the extracted makeup demonstration images according to preset conditions and stores them by category;
A learning unit for learning individual makeup products and makeup poses by performing learning based on the makeup method demonstration images collected and stored by category in the moving image collection unit;
A video image analysis unit for receiving an arbitrary makeup method demonstration video related to the makeup method demonstration, and recognizing and estimating the makeup method object image included in the makeup method demonstration video;
Based on the recognition and estimation of the makeup method object image, by additionally marking a time stamp on the makeup method demonstration video, a time stamp unit for providing a section movement for each time stamp of the makeup method demonstration video. Video section moving device.
The method of claim 1,
The video image extraction unit,
A product logo extraction unit for extracting a product logo image included in the makeup method demonstration image,
a product model extraction unit for extracting a product model image included in the makeup method demonstration image;
and a pose extraction unit for extracting a makeup pose image included in the makeup method demonstration image,
The video image collection unit,
a product logo collection unit for collecting and storing a plurality of product logo images extracted by the product logo extraction unit by category;
a product model collection unit that collects and stores a plurality of product model images extracted by the product model extraction unit by category;
and a pose collection unit for collecting and storing a plurality of makeup pose images extracted by the pose extraction unit by category.
3. The method of claim 2,
The learning unit,
A product learning unit for learning makeup products by performing learning based on the product logo images and product model images for each category provided by the product logo collection unit and the product model collection unit,
and a pose learning unit for learning makeup poses by performing learning based on the makeup pose images for each category provided by the pose collection unit.
4. The method of claim 3,
The video image analysis unit,
The makeup method demonstration video is received, the product logo image and the product model image that are the makeup method object images are extracted, and the cosmetic product included in the makeup method demonstration video by the learning of the product learning unit based on the extracted product logo image and the product model image A product recognition unit that recognizes
A pose estimating unit that receives the makeup method demonstration video, extracts a makeup pose image that is a makeup method object image, and estimates a makeup pose included in the makeup method demonstration video by learning the pose learning unit based on the extracted makeup pose image A device for moving the video section of the makeup method through learning, characterized in that.
5. The method of claim 4,
The time stamp unit,
A product time stamp unit for marking time stamps for the start and end of reproduction of a cosmetic product on the video demonstration of the makeup method based on the recognition of the cosmetic product by the product recognition unit;
and a pose time stamp unit for marking time stamps for starting and ending reproduction of a makeup pose on the makeup demonstration video based on the makeup pose estimation of the pose estimator.
6. The method of claim 5,
The apparatus for moving the video section of the makeup method through learning, characterized in that it further comprises a pose validity determining unit that determines the validity of the makeup pose based on the makeup pose estimation recognized by the pose estimation unit.
7. The method of claim 6,
The pose validity determination unit,
a pose start measurement unit for measuring the start time of a makeup pose based on the makeup pose estimation recognized by the pose estimation unit;
a pose end measurement unit that measures the end time of the makeup pose;
and a pose valid time determining unit that determines the effective time of the makeup pose by comparing it with a preset condition based on the start time and the end time of the makeup pose.
8. The method of claim 7,
The pause time stamp unit,
When the validity of the makeup pose is recognized according to the determination result of the pose valid time determining unit, time stamps for the start and end of the makeup pose are marked on the makeup demonstration video.

Images