KR102299960B1 - Apparatus and method for recommending keyword related to video - Google Patents

Abstract

The apparatus for recommending a keyword related to an image includes a receiver that receives an image captured from a producer terminal, an object analyzer that detects an object from the received image, and analyzes the detected object, and recognizes the behavior of the analyzed object. A behavior recognition unit, an image recognition unit for recognizing the image based on the behavior of the recognized object, a similarity analysis unit for analyzing the similarity by comparing the image with the similar content loaded in the enterprise in relation to the recognized image, and the analysis and a recommendation unit for recommending a keyword for the image based on the obtained similarity.

Description

Apparatus and method for recommending keywords related to video {APPARATUS AND METHOD FOR RECOMMENDING KEYWORD RELATED TO VIDEO}

The present invention relates to an apparatus and method for recommending a keyword related to an image.

One-person media means that an individual plans and produces content alone and distributes the produced content. With the advent of single-person media, anyone can produce and distribute broadcast content, enabling individuals to participate in media as producers rather than consumers.

A creator refers to a creator who creates and uploads such one-person media. Creators have daily conversations with viewers through one-person media, and can create content with friendly materials and show them to viewers.

In relation to the technology for producing such one-person media, Korean Patent Application Laid-Open No. 2012-0067884, which is a prior art, discloses a personal broadcast receiving terminal, a personal broadcast server, and an apparatus and method for providing a personal broadcast service.

In general, viewers who watch single-person media only watch content in their area of interest. Taking this into consideration, creators should choose a title that will attract viewers' attention in order to ensure that the one-person media they produce can be viewed by many viewers. In addition, creators must select relevant keywords that allow their one-person media to be exposed at the top of the platform.

When a viewer watches content in a field of interest, the viewer decides whether to watch the content through the title, so a device that recommends a keyword related to the video that recommends the title of the video to induce the viewer's interest and methods are provided.

SUMMARY The present invention provides an apparatus and method for recommending keywords related to a video that recommends keywords so that, when a viewer performs a content search to view content that a viewer is interested in, a video created by a creator can be displayed at the top of the search results.

By comparing videos made by creators with videos made by famous creators and recommending keywords used in videos made by famous creators, the video created by creators can be exposed as videos related to videos made by famous creators. An object of the present invention is to provide an apparatus and method for recommending keywords related to an image.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention is a receiver for receiving an image photographed from a producer terminal, an object analysis unit for detecting an object from the received image, and analyzing the detected object , a behavior recognition unit for recognizing the behavior of the analyzed object, an image recognition unit for recognizing the image based on the recognized behavior of the object, by comparing the image with similar contents loaded into the enterprise in relation to the recognized image It is possible to provide a keyword recommendation apparatus including a similarity analyzer for analyzing the similarity and a recommender for recommending a keyword for the image based on the analyzed similarity.

According to an embodiment, the object analyzer divides the received image into a plurality of pixel sets, detects at least one of a background and an object through a boundary line included in the image based on the divided plurality of pixel sets, and , a category may be classified with respect to the detected at least one background and object.

According to an embodiment, the behavior recognition unit detects a plurality of body joints through pose prediction for the detected object, measures an angle between the detected plurality of body joints, and combines the measured angles between each joint. based on the behavior of the object may be recognized.

According to an embodiment, the image recognition unit lists keywords related to the image based on at least one of the recognized background information, object information, and behavior information, and recognizes the image based on the keywords related to the listed image. can

According to an embodiment, the similarity analyzer calculates a similarity by comparing at least one of background information, object information, and behavior information for each frame with respect to the image and the similar content, and calculates a similarity level of the similar content based on the calculated similarity. and extracts a keyword related to , and the recommendation unit may recommend a keyword related to the extracted similar content as a keyword for the image.

The above-described problem solving means are merely exemplary, and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description.

According to any one of the above-described problem solving means of the present invention, when a viewer watches content in a field he or she is interested in, the viewer decides whether to watch the content through the title, so that the viewer's interest can be induced. An apparatus and method for recommending a keyword related to an image for which a title of an image is recommended may be provided.

Provided are an apparatus and method for recommending keywords related to a video that recommends keywords so that, when a viewer performs a content search to view content that a viewer is interested in, the video created by the creator can be displayed at the top of the search results. .

By comparing videos made by creators with videos made by famous creators and recommending keywords used in videos made by famous creators, the video created by creators can be exposed as videos related to videos made by famous creators. An apparatus and method for recommending a keyword related to an image may be provided.

1 is a block diagram of a keyword recommendation system according to an embodiment of the present invention.
2 is a block diagram of a keyword recommendation apparatus according to an embodiment of the present invention.
3A and 3B are exemplary views for explaining a process of recognizing an action of an object according to an embodiment of the present invention.
4 is a flowchart of a method of recommending a keyword related to an image in the keyword recommendation apparatus according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware.

Some of the operations or functions described as being performed by the terminal or device in the present specification may be instead performed by a server connected to the terminal or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal or device connected to the server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a keyword recommendation system according to an embodiment of the present invention. Referring to FIG. 1 , the keyword recommendation system 1 may include a producer terminal 110 and a keyword recommendation device 120 . The producer terminal 110 and the keyword recommendation device 120 exemplarily show components that can be controlled by the keyword recommendation system 1 .

Each component of the keyword recommendation system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1 , the keyword recommendation apparatus 120 may be connected to the producer terminal 110 at the same time or at a time interval.

A network refers to a connection structure that enables information exchange between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World). Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, etc. are included, but are not limited thereto.

The producer terminal 110 may photograph an image on a specific subject by a producer, and transmit the captured image to the keyword recommendation device 120 . The image on a specific subject is, for example, related to food, game, travel, and the like, and may be an image filmed by a producer directly appearing on the image as a broadcast host, or an image filmed without appearing through a narration.

The keyword recommendation apparatus 120 may receive the captured image from the producer terminal 110 .

The keyword recommendation apparatus 120 may detect an object from the received image and analyze the detected object. For example, the keyword recommendation apparatus 120 divides the received image into a plurality of pixel sets, detects at least one of a background and an object through a boundary line included in the image based on the divided plurality of pixel sets, and detects A category may be classified for the at least one background and object.

The keyword recommendation apparatus 120 may recognize the behavior of the analyzed object. For example, the keyword recommendation apparatus 120 detects a plurality of body joints through pose prediction for the detected object, measures angles between the detected plurality of body joints, and based on a combination of the measured angles between each joint. This allows us to recognize the behavior of an object.

The keyword recommendation apparatus 120 may recognize an image based on the recognized behavior of the object. For example, the keyword recommendation apparatus 120 may list keywords related to an image based on at least one of the recognized background information, object information, and behavior information, and recognize the image based on the keywords related to the listed image.

The keyword recommendation apparatus 120 may analyze the similarity by comparing the similar content and the image loaded into the enterprise in relation to the recognized image. For example, the keyword recommendation apparatus 120 compares at least one of background information, object information, and behavior information for all frames with respect to images and similar content to calculate a similarity, and based on the calculated similarity, keywords related to similar content can be extracted.

The keyword recommendation apparatus 120 may recommend a keyword for the image based on the analyzed similarity. For example, the keyword recommendation apparatus 120 may recommend a keyword related to the extracted similar content as a keyword for the image.

2 is a block diagram of a keyword recommendation apparatus according to an embodiment of the present invention. Referring to FIG. 2 , the keyword recommendation device 120 includes a receiver 210 , an object analyzer 220 , a behavior recognizer 230 , an image recognizer 240 , a similarity analyzer 250 , and a recommender 260 . ) may be included.

The receiving unit 210 may receive the captured image from the producer terminal 110 .

The object analyzer 220 may detect an object from the received image and analyze the detected object. To this end, the object analyzer 220 divides the received image into a plurality of pixel sets using a Fully Convolution Networks (FCN) algorithm, and uses the background and At least one of objects (eg, people, things, animals, etc.) may be detected. In this case, when the same object is detected, the object analyzer 220 may determine the number of detected identical objects.

The object analyzer 220 may classify categories with respect to at least one background and object detected using a support vector machine (SVM) algorithm. In this case, the object analyzer 220 may recognize a plurality of object information (classes) through the classified categories and perform behavior recognition for each object.

The behavior recognition unit 230 may recognize the behavior of the analyzed object. For example, when the object analysis unit 220 detects a broadcast host (eg, BJ, YouTuber, etc.), a human-shaped game character, etc. as an object from the image, the behavior recognition unit 230 performs an LSTM (Long A plurality of body joints may be detected through pose prediction for the detected object using the Short Term Memory) algorithm.

The behavior recognition unit 230 may measure an angle between a plurality of detected body joints. For example, the behavior recognition unit 230 may measure an angle between joints such as a neck, chest, pelvis, shoulder, arm, and knee.

The behavior recognition unit 230 may recognize the behavior of the object based on a combination of measured angles between each joint. For example, the behavior recognition unit 230 may measure the joint angle of the right or left arm of the broadcast host, and recognize that the broadcast host is currently eating (eating) based on a combination of the measured angles between the joints. have. For another example, the behavior recognition unit 230 measures the angle between the joints of the game character, such as shoulders, arms, and knees, and based on the combination of the measured angles between the joints, the game character is prone, running, dying, shooting, etc. Recognize that you are taking action.

In this way, the behavior recognition unit 230 may recognize what kind of action is being taken for each object (person, game character, etc.) for each frame.

The image recognition unit 240 may recognize an image based on the recognized behavior of the object. For example, the image recognition unit 240 may list keywords related to an image based on at least one of the recognized background information, object information, and behavior information. In this case, since the image recognition unit 240 can know what action the object is taking with respect to the entire frame, keywords of the corresponding image can be listed using the information for each frame.

The image recognition unit 240 may recognize an image based on keywords related to the listed images. The process of recognizing an image by listing keywords related to the image will be described in detail with reference to FIGS. 3A and 3B .

3A and 3B are exemplary views for explaining a process of recognizing an action of an object according to an embodiment of the present invention.

Referring to FIG. 3A , the object analyzer 220 may recognize the broadcast host 300 and two chickens 310 as objects. The behavior recognition unit 230 may measure the angle between the body joints 320 of the broadcast host 300 to recognize that the broadcast host 300 is currently eating.

The image recognition unit 240 lists keywords related to the image, such as '[2 chickens] [1 person] [meal], based on the recognized background information, object information, behavior information, etc., and based on the listed keywords The video can be recognized as a 'mukbang video'.

Referring to FIG. 3B , the object analyzer 220 may recognize the game character 330 as an object. The action recognition unit 230 may measure the angle between the body joints 340 of the game character 330 to recognize that the action of the game character 330 is currently shooting.

The image recognition unit 240 lists keywords related to the image as '[game name] [character] [character behavior (shooting)]' based on the recognized background information, object information, behavior information, etc., and based on the listed keywords Thus, the video can be recognized as a 'game broadcast'.

Through this, the image recognition unit 240 may recognize the image as eating broadcast, outdoor street broadcast, travel broadcast, game broadcast, and the like.

Returning to FIG. 2 again, the similarity analysis unit 250 may analyze the similarity by comparing the similar content and the image loaded into the enterprise in relation to the recognized image. Here, the similar content may be extracted based on a famous creator, the number of subscribers who subscribe to the video produced by the famous creator, the number of views, the viewing time, the number of likes, and the like.

The similarity analyzer 250 may analyze the similarity by comparing the extracted similar content and the image. At this time, the similarity analysis unit 250 compares the extracted similar content and the image based on the priority to analyze the similarity, and if the similarity between the similar content and the image is low, compares the next similar content and the image according to the priority. Thus, the similarity can be analyzed.

The similarity analyzer 250 may calculate a similarity by comparing at least one of background information for each frame, object information, and behavior information for an image and similar content. For example, the similarity analyzer 250 may calculate the similarity by comparing background information, object information, and behavior information for each frame of an image with background information, object information, and behavior information for all frames of similar content.

The similarity analyzer 250 may extract keywords related to similar content based on the calculated similarity. For example, the similarity analyzer 250 may extract the title and tag of the similar content as keywords when the similarity between the image and the similar content is equal to or greater than 90%.

The recommender 260 may recommend a keyword for the image based on the analyzed similarity. For example, the recommendation unit 260 may recommend a keyword related to the extracted similar content as a keyword for the image by using a Recurrent Neural Network (RNN) algorithm.

4 is a flowchart of a method of recommending a keyword related to an image in the keyword recommendation apparatus according to an embodiment of the present invention. The method of recommending a keyword related to an image in the keyword recommendation apparatus 120 illustrated in FIG. 4 includes steps that are time-series processed in the keyword recommendation system 1 according to the embodiment illustrated in FIGS. 1 to 3B . Accordingly, even if the content is omitted below, it is also applied to the method of recommending a keyword related to an image in the keyword recommendation apparatus 120 according to the embodiment shown in FIGS. 1 to 3B .

In step S410 , the keyword recommendation apparatus 120 may receive the captured image from the producer terminal 110 .

In operation S420, the keyword recommendation apparatus 120 may detect an object from the received image and analyze the detected object.

In step S430 , the keyword recommendation apparatus 120 may recognize the behavior of the analyzed object.

In step S440 , the keyword recommendation apparatus 120 may recognize an image based on the recognized behavior of the object.

In step S450 , the keyword recommendation device 120 may analyze the similarity by comparing the similar content and the image loaded into the enterprise with respect to the recognized image.

In operation S460, the keyword recommendation apparatus 120 may recommend a keyword for the image based on the analyzed similarity.

In the above description, steps S410 to S460 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be switched.

The method of recommending a keyword related to an image in the keyword recommendation apparatus described with reference to FIGS. 1 to 4 may also be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by the computer. can Also, the method of recommending a keyword related to an image in the keyword recommendation apparatus described with reference to FIGS. 1 to 4 may be implemented in the form of a computer program stored in a medium executed by a computer.

Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

110: producer terminal
120: keyword recommendation device
210: receiver
220: object analysis unit
230: behavior recognition unit
240: image recognition unit
250: similarity analysis unit
260: recommendation

Claims (5)

In the device for recommending keywords related to images,
a receiver for receiving an image photographed from a producer terminal;
an object analysis unit detecting an object from the received image and analyzing the detected object;
a behavior recognition unit for recognizing the behavior of the analyzed object;
an image recognition unit for recognizing a subject of the image based on the behavior of the recognized object;
a similarity analysis unit for analyzing the similarity by comparing the image with the similar content loaded into the enterprise in relation to the subject of the recognized image; and
A recommendation unit for recommending a keyword for the image based on the analyzed similarity,
The similarity analyzer calculates a similarity by comparing at least one of background information, object information, and behavior information for each frame with respect to the image and the similar content;
extracting a keyword related to the similar content based on the calculated similarity;
The recommendation unit recommends a keyword related to the extracted similar content as a keyword for the image,
The image recognition unit recognizes the subject of the image based on a keyword related to the image obtained based on the background information about the image, the object information, and the behavior information,
The similar content is extracted according to priority based on at least one of a famous creator related to the subject of the video, the number of subscribers to the video produced by the famous creator, the number of views and viewing time of the produced video,
The similarity analyzer extracts, as the keyword, a title and tag of a specific similar content whose analyzed similarity between the similar content extracted according to the priority and the image is equal to or greater than a threshold value;
wherein the recommendation unit recommends a keyword related to the extracted specific similar content as a keyword for the image.
The method of claim 1,
The object analyzer divides the received image into a plurality of pixel sets,
detecting at least one of a background and an object through a boundary line included in the image based on the divided set of the plurality of pixels,
The keyword recommendation apparatus for classifying a category with respect to the detected at least one background and object.
3. The method of claim 2,
The behavior recognition unit detects a plurality of body joints through pose prediction for the detected object,
Measuring the angle between the detected plurality of body joints,
The keyword recommendation device for recognizing the behavior of the object based on the measured angle combination between each joint.
delete delete

Images