KR20200009888A - Method for Providing and Recommending Related Tag Using Image Analysis - Google Patents

Abstract

The present invention relates to a method for generating and recommending a relational tag based on image analysis. More specifically, provided is a method for automatically recommending a relational tag implying a description suitable for a photo through analysis of a person included in image data, generating the tag and uploading the tag to an SNS server.

Description

Relational tag generation based on multiple metadata analysis and its method {Method for Providing and Recommending Related Tag Using Image Analysis}

The present invention relates to a method for generating and recommending a relational tag based on multiple metadata analysis. More specifically, an individual relational tag which automatically includes a description suitable for a photo through analysis of information included in an image, sound, and text data is provided. To create and recommend.

Through social networking services, many users share their pictures. In the past, a method of sharing photos among users registered as acquaintances is increasingly used to share photos regardless of whether people with the same interests are registered by using hashtags indexed by interests or topics. have.

Hashtags consist of short texts that describe the photos, and since the photos with the same hashtags are easily searched, they help to collaborate and share hobby among users by expanding the exposure frequency and range of their photos. Representative social network services such as Instragram and Facebook allow users to register posted images and related image tags. On the other hand, the more active users are, the more tendency is to use dozens of hashtags to increase their exposure range and exposure frequency. However, in the conventional case, the user inputs or selects a corresponding hash tag directly at every photo posting, and thus it takes a lot of time to register the photo, and it is difficult to classify a uniform published image by using different words on the same subject. It is happening. For example, in the United States, various expressions such as "# U.S.A", "#USA", "America", and "United States" are possible, so that it is very difficult to input all possible expressions in the subject as hashtags.

On the other hand, recent developments in image processing technology have developed technologies such as detecting a person's face from a photograph and taking a specificity analysis to estimate a person's name, detect a smile, detect a sex, and detect an age. Also, background extraction has been developed to estimate the area in which the background is photographed or to identify the letters on the background. However, these technologies are only used to identify criminals and organize digital albums irrespective of hashtags.

The present invention is to solve the above problems, it is recommended to extract a text-based relational tag from the image data and upload it to the social network service server so that the photos uploaded by various users are systematically classified by a predetermined algorithm It aims at enabling efficient management of data and improving user convenience.

In an automated relational tag generation method based on analysis, a tag generation application executed in a terminal is loaded with image data including at least one person as an object from a storage unit of the terminal or inputted by being captured by a camera unit of the terminal. Receiving first step; A second step of generating, by the tag generation application, relationship analysis result information by analyzing at least one of the number, gender, age, or emotion of a person included in the image data as the object by applying a preset image analysis algorithm; ; A third step of generating, by the tag generation application, a textualized relational tag about a person or persons included as the object based on the relationship analysis result information; Transmitting the relational tag generated in the tag generation application to a tag management server through a network; A fifth step of collecting and storing the relational tag transmitted to the tag management server and constructing it into a tag database using a predetermined tag analysis algorithm; A sixth step of comparing a newly generated relational tag stored in the tag management server with the tag database and recommending a relevant tag to a user; And a seventh step of uploading the tag selected by the user and the image data among the recommended tags to the social network service server.

According to the image analysis-based relational tag generation and recommendation method according to the present invention, the text-based relational tag is recommended from the image data and uploaded to the social network service server along with the image data by various users. By uploading photos systematically classified by a certain algorithm, it is possible to efficiently manage big data and improve user convenience.

1 is a block diagram of each component for explaining a method of generating and recommending a relational tag based on image analysis according to an embodiment of the present invention;
2 is a detailed configuration diagram of a terminal in an image analysis-based relational tag generation and recommendation method according to an embodiment of the present invention, and FIG. 3 is a detailed configuration diagram showing a tag generation application according to an embodiment of the present invention;
4 is a schematic flowchart of an image analysis-based relational tag generation and recommendation method according to an embodiment of the present invention.

Hereinafter, a relational tag generation and recommendation method based on image analysis according to the present invention will be described with reference to the drawings. The tag generation system according to the present invention includes a terminal 10 and a tag generation application 100. The terminal 10 is not particularly limited as a device that can be connected to a communication network, and may be, for example, a smart phone, a cellular phone, a PC, a tablet, or the like. The tag generation application 100 may be installed and driven in the user terminal 10 as an app or a program.

As illustrated in FIG. 2, the terminal 10 includes a tag generation application 100, a communication unit, a storage unit, a display unit, and a user input unit. The communication unit performs data communication by connecting to a communication network such as the Internet, and may be, for example, a WiFi modem, a CDMA modem, an LTE modem, or a LAN card. The storage unit stores a variety of data including application and image data (data_img), and may be, for example, a storage device such as a hard drive, a flash memory, an SSD, a DRAM, a flash memory, and the like. The display unit provides a user with visualized information such as an image and text through a screen. The user input unit converts a user's input into an electrical signal to provide a form of data that can be processed in the terminal 10 and may be, for example, a keyboard, a mouse, or a touch screen. According to an exemplary embodiment, the camera apparatus may further include a camera unit configured to convert the subject into image data data_img through the image pickup device. The tag generation application 100 includes an image receiving module 110, an image analysis module 121, a relational tag generating module 131, a recommendation module 140, and an upload module 150 as shown in FIG. 3. It is composed. These modules are functionally divided for the purpose of illustrating the invention. The image receiving module 110 performs a function of receiving image data data_img including at least one person as an object from the terminal 10. In this case, when the terminal 10 further includes a camera unit, the image receiving module 110 may load image data data_img stored in the storage unit of the terminal 10 or may be captured by the camera unit of the terminal 10. It can be implemented to receive image data (data_img). Such images may be, for example, secondary photographic images such as JPEG, GIF, or BMP file formats, illustration images, or two-dimensional video images such as MP4, MOV, or AVI file formats, or three-dimensional stereoscopic image formats such as Intel Real Sense. It may be a still image or a moving image.

The image analysis module 121 analyzes at least one of the number, gender, age, or emotion of a person included as an object in the image data data_img by applying a predetermined image analysis algorithm to analyze the relationship analysis result information info_relation. Perform the function to create. The image analysis method based on such an analysis algorithm is a stage in which facial recognition, gender recognition, age recognition, and emotion recognition are available as of the filing date. The image analysis module 121 may be implemented using commercially available programs. For example, programs such as Face Recognition, Animetrics Face Recognition, Skybiometry Face Detection and Recognition, Face ++, FaceMark, EmoVu by Eyeris, FaceRect, EyeFace, Kairos, etc. Provide all or part of facial recognition, gender recognition, age recognition, and emotion recognition. The image analysis module 121 is not necessarily installed inside the terminal 10 but may be an external image analysis server connected through a communication network. According to this embodiment, it is possible to generate a more advanced automation tag by applying a complex image analysis algorithm that is impossible to perform or takes too long with limited computing resources of the mobile terminal 10. The relational tag generation module 131 performs a function of generating a textualized relational tag (tag_relation) about a person or people included as an object based on the relationship analysis result information (info_relation). For example, when the relationship analysis result information (info_relation) of the image analysis module 121 has two persons photographed in the image data data_img, and the two genders are different from each other, "#lover", "#Lover", " You can create "lover" related relational tags (tag_relation) such as #lover "," #love ", and" #date ". More preferably, as a method of calculating the relative distance between the recognized faces, when it is close to the threshold value or less, it may be estimated as a "lover" relationship.

In addition, when the relationship analysis result information (info_relation) of the image analysis module 121 has two or more persons photographed in the image data (data_img) and the genders are the same, "#friend", "#friend", " You can create "friends" and related relational tags (tag_relation) such as #friendship, "#friends", and "#friends". More preferably, as a method of calculating the relative distance between the recognized faces, it may be estimated as a "friend" relationship when it is close to the threshold value or less.

As another example, the relationship analysis result information (info_relation) of the image analysis module 121 is the number of people photographed in the image data (data_img) three or more, at least one of them is an adult male, at least one of them If it is an adult female and at least one of them is a male or female child, one may be allowed to generate a “family” related tag (relationship tag_relation) such as “#happy”, “#loved family” and the like.

10: terminal 20: tag management server
30: social network service (SNS) server
100: tag generation application 110: image receiving module

Claims (1)

In the automated relational tag generation and recommendation method based on image analysis,
The first step of the tag generation application to be executed in the terminal, the image data containing at least one person as an object to retrieve the image data from the storage of the terminal or to be imaged by the camera of the terminal;
A second step of generating, by the tag generation application, relationship analysis result information by analyzing at least one of the number, gender, age, or emotion of a person included as the object in the image data by applying a preset image analysis algorithm;
A third step of generating, by the tag generation application, a textualized relational tag about a person or persons included as the object based on the relationship analysis result information;
A fourth step of transmitting the relational tag generated in the tag generation application to a tag management server through a network;
A fifth step of collecting and storing the relational tag transmitted to the tag management server and constructing it into a tag database using a predetermined tag analysis algorithm;
A sixth step of comparing a newly generated relational tag stored in the tag management server with the tag database and recommending a relevant tag to a user;
And a seventh step of uploading the tag selected by the user and the image data among the recommended tags to the social network service server.
The third step,
When the result of the relationship analysis of the second step is that the number of people is two and the two genders are different, the tag generation application generates a "lover" related relational tag,
When the relationship analysis result information of the second step is two or more people and the same gender, the tag generation application generates a "friend" relational tag,
The tag generation result when the relationship analysis result information of the second stage is three or more persons, at least one of which is an adult male, at least one of which is an adult female, and at least one of which is a male or female child. Application generated relational tags and recommendation method based on image analysis, characterized by generating "family" related relational tags

Images