KR102023687B1 - Method for sharing images, and system and application implementing the same method - Google Patents

Abstract

An image clustering method of a server operated by at least one processor, the method comprising: extracting at least one image included in each web document from a plurality of web documents and contents describing the image, and extracting images from the plurality of web documents If there are duplicate images among them, determining one representative image representing the duplicate images, and connecting image description contents corresponding to each of the duplicate images to the representative image, a plurality of linked images Determining image information including an image description tag and a timestamp of the representative image from image description contents, mapping the image information and the representative image to an image storage, and requesting an image clustering request according to a clustering keyword. If present, the image store Extracting images associated with the clustering keyword, and generating at least one clustering event by clustering at least some of the extracted images based on the similarity of the extracted images.

Description

How to share images on a server, and the systems and applications that implement them {METHOD FOR SHARING IMAGES, AND SYSTEM AND APPLICATION IMPLEMENTING THE SAME METHOD}

The present invention relates to image clustering.

Various images are posted on web pages such as news and blogs, and photographs taken from user terminals are uploaded on various sharing platforms such as social media services. The web with such various images becomes a huge image repository, and users can search for the images they want.

However, the same images are duplicated on different web pages or a large number of similar images shot at the same event. Therefore, in order to obtain a desired result among a large number of images that are not properly classified, the user must refine and search a search word, and for this, the user must have prior knowledge of a person or an event. In addition, since it is difficult to reflect the latestness of the image, the search result may not vary significantly depending on the search time. Recently, there have been attempts to categorize and provide images by event, but there is still no difference from the prior art of exposing images in time order because the images are arranged and exposed in the event. Accordingly, what is needed is a method of classifying and sharing images in a manner that is meaningful to the user.
(Patent Document 1) KR10-1029160 B
(Patent Document 2) US2015-0078673 A

The problem to be solved by the present invention is to classify various images from various sources by attributes such as time, event, person, etc., and to provide a method of clustering images through a tag describing the image, and a system and application implementing the same. In addition, an object of the present invention is to provide a sharing method for generating content that can be shared based on clustered images, and a system and application implementing the same.

An image clustering method of a server operated by at least one processor according to an embodiment, the method comprising: extracting at least one image included in each web document and contents describing the image from a plurality of web documents; If duplicate images exist among the images extracted from the web document, determining one representative image representing the duplicate images, and connecting image description contents corresponding to each of the duplicate images to the representative image, wherein the representative image Determining image information including an image description tag and a timestamp of the representative image from a plurality of image description contents connected to the image, mapping the image information and the representative image, and storing the image information in an image storage; Image clustering request In the image store it comprises a step of extracting an image associated with the keyword grouping and generates at least one event cluster by cluster at least a portion of the extracted image on the basis of the similarity of the extracted image.

The storing in the image storage may include determining the image description tag based on keywords appearing in the plurality of image description contents, and determining a timestamp of the representative image based on a creation date of web documents including the duplicate images. Can be.

The generating of the clustering event may extract images associated with the clustering keyword from the image store based on image description tags of the images stored in the image store.

The generating of the clustering event may extract the images created in the search period as the search images based on time stamps of the images stored in the image storage.

The generating of the clustering event may extract the name of the clustering event and the tags associated with the clustering event from image description tags or image description contents of the images bundled into the clustering event.

The image clustering method may include extracting at least one candidate person from the specific image description content when the specific image description content extracted from an arbitrary web document includes person information having the same name or a double expression. Evaluating whether each candidate is a person in a specific image extracted from the random web document based on contents and person information of each candidate person, and determining a person in the specific image based on an evaluation score of each candidate person. The method may further include tagging the determined person identification information to the specific image.

The image clustering method may further include transmitting clustered images of the at least one clustering event to a user terminal, wherein the clustered images are divided and displayed for each clustering event on an interface screen of the user terminal. When there are a plurality of characters in the images belonging to each other, they may be classified and displayed by the characters on the interface screen.

The transmitting of the clustered images in the clustering event may generate content displayed on the interface screen while the clustered images are randomly crossed or sequentially crossed in a display order, and the content may be transmitted to the user terminal. have.

An image sharing method of a server operated by at least one processor according to another embodiment, the method comprising: receiving a request for sharing a plurality of images grouped into a single event, identifying a person of each of the plurality of images, and identifying the same person Estimating a person's movement in the images classified as and sorting the images of the person, and transmitting the arranged plurality of images to the user terminal.

The sorting may include calculating an optical flow of coordinates for all image pairs in the images classified as the same person, and sorting the images in the order of successively making the person movement based on the calculation result. have.

The sorting may include classifying the plurality of images for each person and sorting the images classified for each person when a plurality of persons appear in the plurality of images.

The transmitting of the plurality of images may include generating content displayed on an interface screen of the user terminal while the plurality of images are randomly crossed or sequentially in a display order, and transmitting the content to the user terminal. It may include a step.

The transmitting of the plurality of images may further transmit at least one tag associated with the single event or the plurality of images to the user terminal.

An image sharing method of a server operated by at least one processor according to another embodiment, the method comprising: retrieving at least one stored image identical or similar to an input image in an image store; image description information mapped to the stored image Generating a list of tags associated with the input image by using; and transmitting the shared information of the input image and the tag list to a user terminal, wherein the image storage is extracted from a plurality of web documents. An image and image description information describing the image may be mapped and managed.

The input image may be an image scraped from link information requested to be shared from the user terminal.

The information of the input image and the tag list may be displayed on an interface screen of the user terminal, and the sharing information of the input image may be thumbnail-type content.

An image sharing method of a user terminal operated by at least one processor according to another embodiment, the method comprising: transmitting a requested search keyword to a server, at least one cluster event related to the search keyword from the server, and each cluster; Receiving at least one tag associated with the event, and displaying on the first interface screen, the image and the at least one tag grouped in each grouping event classified by the grouping event. Images clustered in each cluster event and at least one tag are information obtained from images extracted from a plurality of web documents and image description information describing the image.

The image sharing method may include: transmitting a sharing request for a random cluster event received on the first interface screen to the server, receiving a plurality of images clustered in the random cluster event from the server, and a second interface screen The method may further include displaying the received plurality of images.

The plurality of images may be arranged in an order of continuously making a movement of a person, and when a plurality of people appear in the plurality of images, the plurality of images may be arranged for each person.

In the displaying of the plurality of images, the generated content may be displayed on the second interface screen in a form in which the plurality of images are randomly crossed or sequentially crossed in a display order.

According to an embodiment of the present invention, various images of various sources may be clustered by attributes such as time, event, and person. According to an embodiment of the present invention, a large amount of images pouring out on the web may be processed to generate content that can be shared. According to an embodiment of the present invention, the user may obtain meaningful content from a keyword of interest, and visually confirm the image search result through the aligned images.

1 is a schematic structural diagram of a system according to an embodiment of the present invention.
2 is a flow chart illustrating a method of storing images in an image store in accordance with one embodiment of the present invention.
3 is an exemplary view illustrating an image storing method according to an embodiment of the present invention.
4 is a flowchart of an image clustering method according to an embodiment of the present invention.
5 is an exemplary view illustrating an image clustering method according to an embodiment of the present invention.
6 is an exemplary view of a clustered image provided to a user terminal according to an embodiment of the present invention.
7 is a flowchart illustrating a person identification method of an image according to an exemplary embodiment of the present invention.
8 is an example of a service using a person identification method of an image according to an embodiment of the present invention.
9 and 10 each illustrate a cluster image sharing method according to an embodiment of the present invention.
11 is a diagram illustrating a system for sharing a web document in a messenger service according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise. In addition, the terms “… unit”, “… unit”, “module”, etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. have.

1 is a schematic structural diagram of a system according to an embodiment of the present invention.

Referring to FIG. 1, the system 10 includes a server 100 in which the functions described in the present invention are implemented, and a user terminal 200 in communication with the server 100. The system 10 includes an image store 300 in which images processed by the server 100 are stored. The user terminal 200 mounts an application programmed to provide the functions described in the present invention, displays information provided by the server 100 through the application, and transmits information input by the user to the server 100. do.

The server 100 executes the operations described in the present invention by at least one processor. The server 100 classifies various images from various sources by attributes such as time, event, and person, and clusters the images through a tag describing the image attributes. The server 100 generates content that can be shared based on the clustered images, and provides an image search result for the search request of the user terminal 200. The operation method of the server 100 will be described in detail below.

The user terminal 200 includes an application stored in a computer readable storage medium and hardware such as a processor, a memory, a display, a communication module, and the like. The application includes instructions for executing the operation described by way of example in the present invention. The processor runs the application to execute the programmed operation. The display may display a user interface screen provided by an application and receive a user input, for example, a touch input. The communication module communicates with the server 100 through a communication network.

The user terminal 200 may be implemented in various forms, for example, in the form of a mobile terminal such as a smartphone, a pad-type terminal such as a smart pad, a computer of various forms such as a laptop computer, a wearable device, a TV terminal, or the like. Can be implemented.

2 is a flowchart illustrating a method of storing images in an image store according to an embodiment of the present invention, and FIG. 3 is an exemplary view illustrating an image storing method according to an embodiment of the present invention.

Referring to FIG. 2, the server 100 extracts at least one image included in each web document and contents describing the corresponding image from the plurality of web documents (S110). Web documents (or simply referred to as 'documents') are digital documents published on the web such as news, blogs, social media services, etc., and include all documents accessible by the server 100 via a network. The web document is composed of at least one image and text, and the same / similar image may be extracted from the plurality of web documents. Since the image and the text included in the web document are correlated with each other, the server 100 may extract content related to the image or content describing the image from the text. Contents describing the image may include a title appearing in the document from which the image is extracted, a word included in the document, a word, and the like may be extracted from metadata of the document.

The server 100 tags content describing the image for each image (S120).

The server 100 determines whether the extracted images overlap, and extracts duplicate images (S130). The duplicate image extraction method may be selected from various image processing techniques. For example, the server 100 may determine duplicate images by comparing feature values (eg, 128 bit hash values) representing each image.

The server 100 determines a representative image among the duplicate images and connects the image description contents tagged to each of the duplicate images to the representative image (S140). That is, image information in which a plurality of web document information is connected to a single image is generated. The server 100 may determine the representative image as the representative image or the image having the fastest creation date among duplicate images, but may select the representative image in various ways.

The server 100 determines an image description tag of the representative image and a timestamp of the representative image based on the plurality of image description contents, maps the image description tag and the timestamp to the representative image, and stores the image description tag and the timestamp in the image storage 300 ( S150). The image description tag may be obtained from words that frequently appear in a plurality of documents describing the same image. The image description tag that appears frequently in a plurality of documents may be regarded as the most representative description of the image. The server 100 may determine the earliest creation date among web documents including duplicate images as a time stamp of the representative image. Alternatively, when a specific event time (eg, a record release date, a performance date, an event schedule, etc.) is described in the web document, the server 100 may determine a time stamp of the representative image based on the document content. Most of the news about the events that occurred on the day is written on the day, but since the web document may include information about the events that have already occurred, it may contain information about the events that will occur in the future. Can also be used as a timestamp.

Referring to FIG. 3, when N documents include image # 1, and each document includes contents describing image # 1, the server 100 generates a plurality of web documents based on the same image # 1. Connect. The server 100 extracts a word (eg, red velvet, album) that frequently appears among image description contents included in web documents as an image description tag, and displays the earliest creation date among web documents. Can be extracted with a timestamp of # 1. If the document 1 further includes the image # 2, the server 100 connects a plurality of web documents around the image # 2.

The representative image # 1, the image description tag, and the timestamp stored in the image storage 300 may be stored as shown in Table 1. The representative image may be stored as feature values and / or image files.

representation
image image
Description Tag /
Timestamp Image descriptions included in web documents Image # 1 Red velvet
album
2016.09.21 Document 1: Red Velvet New Album Coming Soon (2016.09.22)
Document 2: red velvet style vs transformation (2016.09.21)
Document 3: red velvet is back with a new album (2016.09.21)
Document 4: New Red Velvet Style Album (2016.09.25)
…
Document N: Fresh Red Velvet New Song (September 21, 2016) Image # 2 - -

Next, a method of grouping the same / similar images according to clustering criteria such as time / person / event type will be described. FIG. 4 is a flowchart of an image clustering method according to an embodiment of the present invention, and FIG. 6 is an exemplary diagram illustrating an image clustering method according to an exemplary embodiment of the present invention, and FIG. 6 is an exemplary diagram of a clustered image provided to a user terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the server 100 receives an image clustering request including a clustering keyword (S210). When the server 100 receives a request for an image search for the search keyword from the user terminal 200, the server 100 may start the image clustering operation by applying the search keyword as a clustering keyword. Alternatively, the server 100 may periodically perform an image clustering operation for a specific keyword, or perform an image clustering operation for the corresponding event in advance whenever a specific event occurs.

The server 100 extracts the images related to the clustering keyword from the image storage 300 in which the image description tag and the time stamp are mapped to the image extracted from the web document (S220). At this time, the server 100 is an image of a time stamp is created in a specific search period (for example, today, three days before today, a week ago from today, a month ago from today, a year ago from today, a certain period in the past, etc.) Extract them. The search period can be set by the user or the server. Meanwhile, the server 100 maps image description tags and timestamps to images extracted from a large amount of web documents before image clustering, and stores the image description tags and timestamps in the image storage 300, and clusters in the image storage 300 when clustering the images. Assume that you extract an image associated with a keyword. However, the server 100 may map the image description tag and the time stamp to the image extracted from the web document while grouping the images, and may store them in the image storage 300.

The server 100 sorts the images associated with the clustering keyword in the timestamp order (S230). Referring to FIG. 5A, since the images associated with the clustering keyword are extracted from the image storage 300 in which duplicate images existing in the web document are organized into a single image, the images extracted from the image storage 300 may be extracted. It consists of images that are not identical. For example, if the search period for the clustering keyword is from November 1 to November 3, the M image having the time stamp of November 1 (1101_image_1 to 1101_image_m) and the time stamp of November 2 N images (1102_image_1 to 1102_image_n) and K images (1103_image_1 to 1103_image_k) having a time stamp of November 3 are arranged in chronological order. Each image is extracted from web documents such as news, blogs, cafes, etc., and image description tags are tagged with metadata. In addition, each image may additionally be tagged with an image title or an image source.

The server 100 generates at least one clustering event by binding the images based on the similarity of the images arranged in the timestamp order (S240). The similarity of the images may include the similarity between images or the similarity of an image description tag tagged to the image. Referring to (b) of FIG. 5, the server 100 generates clustering event # 1 and clustering event # 2 by grouping images having similarity or more among M images having a timestamp of November 1st, and having a timestamp. Cluster event # 3 is generated by grouping images with similarity above the reference value among N images on November 2, and cluster event # 4 by grouping images with similarity above the reference value among K images with time stamp on November 3 Can be. In this case, the server 100 may exclude an image that does not belong to any clustering event among the aligned images, or may expose a single image as a search result image.

Meanwhile, in steps S230 and S240, it has been described that the server 100 arranges the images in time stamp order to bundle the images in divided time units (for example, one day unit). However, without having to sort images in advance by sorting them in timestamp order (for example, today, yesterday, and yesterday), the server 100 extracts images created during a specific search period (for example, three days before today). The entire extracted images may be grouped based on similarity.

The server 100 maps an event name and a cluster tag to each cluster event (S250). The server 100 analyzes the image description tag of the images belonging to the cluster to create an event name representing the cluster (for example, ooo verse, ooo rehearsal, ooo game, movie ooo premiere, etc.), and the cluster. Analyze image description tags of images belonging to to create a cluster tag that represents a cluster. The cluster tag may include all the image description tags of the images belonging to the cluster, or may include the high frequency tag among the image description tags of the images belonging to the cluster. In addition to the image description tag, the server 100 may determine keywords associated with the image description tag as a cluster tag, or determine keywords extracted from information associated with the search keyword as a cluster tag. For example, when the search keyword is a person, the server 100 extracts a synonym / similar word about the person's appearance movie / casting broadcast and the like, and selects a keyword determined to be highly related to the search keyword or a clustering event from the extracted information. Can be determined by cluster tag. At least some of the cluster tags may be displayed on the user terminal 200 together with the clustered images. Clustered images, event names, and cluster tags by cluster events are stored in the image storage 300.

The server 100 transmits clustered images to a specific clustering event or a cluster tag to the user terminal 200 (S260). The clustered images are displayed on the user interface screen of the user terminal 200.

Referring to FIG. 6A, the user terminal 200 displays images on the user interface screen 400 for each cluster event. When a plurality of cluster events exist, the user terminal 200 may display the images in chronological order. For example, the UI screen 400a displays an event name 410, a clustered image 420, and at least one tag 430 for each clustering event. The tag may be a tag corresponding to a cluster event or a tag associated with a cluster event. The user interface screen 400 may further display a share button 440, a date 450, and the like for each clustering event. The user interface screen 400 may further display a search keyword name 460 used for group event search. When the user selects the tag 430, the user terminal 200 may display a search result related to the tag 430 on a sharing screen (eg, a chat room screen of a messenger application).

Referring to FIG. 6B, when the user selects the sharing button 440, the user terminal 200 shares a clustering event corresponding to the sharing button 440 with a sharing screen (eg, a chat room screen of a messenger application). 400b). In this case, the shared cluster event may display the entire clustered image on the shared screen, but the thumbnail type or the content (played) is displayed while the clustered images are automatically or manually moved sequentially or randomly. In the future, it may be referred to as 'moving image content' or 'summary content'.

The sharing screen 400b may display the tags related to the shared content together with the shared content.

7 is a flowchart of a method for identifying a person in an image according to an embodiment of the present invention, and FIG. 8 is an example of a service using the method for identifying a person in an image according to an embodiment of the present invention.

Referring to FIG. 7, there are numerous web documents such as news, blogs, social media services, etc. When the images extracted from the web documents overlap, the server 100 determines the representative image among the duplicate images and duplicates the representative image. Link tagged image descriptions to each of the images. However, since the content of the image description is text, the image search result may be inaccurate when there is an identical expression or a double expression. The image classifier can classify people, but because the number of people is constantly increasing, the learning-based image classifier is not highly scalable. Therefore, the server 100 checks the person in the image as follows based on the image description content extracted with the image.

The server 100 manages person information for each person (S310). The person information includes feature values of the person image, unique person identification information (ID) assigned to the person, and additional information (eg, a job) for identifying the person. The server 100 may use an external person database without directly managing person information.

When the server 100 receives the image and the image description content extracted from the web document, the server 100 extracts a candidate person from the image description content (S320). If it is not necessary to identify a person in the image from the image description content or if it is certain that the person is a specific person, the following person identification procedure may be omitted, and a procedure of tagging the image description tag on the extracted image may be performed immediately.

The server 100 evaluates whether a candidate person is a person in the image based on keywords included in the image description content and person information of the candidate person (S330). The server 100 may mash up various APIs such as a person database, a movie database, and a TV program database to evaluate whether the candidate person is a person in the image.

The server 100 checks the person in the image based on the evaluation score of the candidate person (S340). When there are a plurality of candidate people, the server 100 may determine a person in the image based on the evaluation score of each candidate person. The server 100 may finally determine the person by comparing the image feature value of the person identified with the feature value of the image extracted from the web document to improve accuracy.

The server 100 adds the confirmed person ID to the image description tag of the image (S350).

For example, when the server 100 receives an image description content of 'being drawn from Kim's sketchbook of Kim Bum Soo, Yu Hee-yeol' together with a person image extracted from a web document, the person name 'Kim Bum Soo' included in the image description content is input. As a basis, we extract candidates (singer Kim Bum-soo, announcer Kim Bum-soo, manager Kim Bum-soo).

The server 100 evaluates the degree of conformity with the image description content among the candidates based on keywords included in the image description content (for example, a music broadcast program name 'Yoo Hee-yeol's sketchbook', etc.) and the person information of the candidate person. After that, the highest score 'singer Kim Bum Soo' can be identified as the person in the image. In this case, evaluation criteria and evaluation methods may be set in various ways.

Through such a method of identifying a person in an image, the server 100 may tag and store accurate person information when storing images extracted from a web document in the image storage described with reference to FIG. 2. When the server 100 provides a clustered image to the user terminal 200, the server 100 may cluster the images with accurate person information. Therefore, when a user requests a search by inputting a person name tag (for example, #Kim Beom-Soo), the server 100 may provide an image grouped by the singer Kim Bum-Soo and an image grouped by the announcer Kim Bum-Soo.

Referring to FIG. 8, the person identification method in the image of the server 100 described above may be applied to the person identification service included in the image.

For example, the server 100 may be requested to confirm the person in the image. The user may request a person identification may vary. According to an embodiment of the present disclosure, when a user reads a web document containing an image and wonders who the image is included in the web document, the user transmits the link information (URL) of the web document or the web document to the server 100 to transmit the web. You can request to identify a person from an image in a document. Then, the server 100 may add the person identification information as shown in FIG. 8 to the image of the web document and display it on the user terminal 200. According to another embodiment, the user may request to confirm the person of the image by transmitting an image to be identified to the server 100.

When a plurality of people are included in the image, the server 100 may evaluate who each person is and check the person name based on the evaluation result.

When the image description content is input together with the image, the server 100 determines a candidate person from the image description content, and evaluates each of the plurality of people by comparing the image feature values of the narrowed candidate person with the feature value of the input image. . For example, when an image description content of 'TWICE OO stage' is input together with the image, the server 100 determines a candidate person as a 'TWICE member' based on the image description content, and the person in the image is TWICE. The likelihood of each member being evaluated can be evaluated.

Even if only an image is input, the server 100 finds the same stored image as the input image in the image storage 300. The server 100 may determine the candidate person based on the image description tag or the image description content corresponding to the stored image, and compare each feature of the narrowed candidate person with the feature value of the input image to evaluate each person. have.

If there is no stored image identical to the input image in the image storage 300, the server 100 compares the feature values of the person images stored in the image storage 300 with the feature values for each person in the input image to evaluate each of the plurality of persons. Can be.

As illustrated in FIG. 8, the user terminal 200 may further show an evaluation score together with the person's name. Each person name is generated as a tag, and when the user selects the person name or person name tag on the interface screen, the user terminal 200 may display a search result related to the selected person name tag.

9 and 10 each illustrate a cluster image sharing method according to an embodiment of the present invention.

9 and 10, the server 100 generates the clustered images as content in a form that users can share and provides them to the user terminal 200. The server 100 may provide the user terminal 200 with 'moving image content' or 'summary content' in which the clustered images are continuously reproduced. Services for sharing content by a user may vary, but the following description will be given by sharing clustered images with chatters on a chat room screen of a messenger application.

First, a method of generating summary content by using the clustered images by the server 100 will be described.

Referring to FIG. 9, the server 100 brings a clustered image included in a clustering event (S410).

The server 100 checks the characters in each of the clustered images and sorts them by the characters (S420).

The server 100 checks the movement between the same person images among the same person images arranged for each person (S430). The server 100 may track an optical flow of image coordinates to find a motion.

The server 100 calculates a continuous motion score for all image pairs from the optical flow of coordinates for all image pairs in the image in which the same person appears (S440). When many coordinates of two images move continuously in the same direction, the movement score is calculated to be high. The server 100 may store the relationship of all image pairs obtainable from the N images in an N × N matrix.

The server 100 estimates the direction of movement in the image based on the motion scores of all the image pairs, and sequentially arranges the images in the order in which the movements are continuous (S450). For example, when the images of person # 1 included in the cluster event are IMG1 to IMG5, the server 100 may include IMG1 and IMG2, IMG1 and IMG3, IMG1 and IMG4, IMG1 and IMG5, IMG2 and IMG3, IMG1 and IMG4, Based on the movement scores of IMG2 and IMG5, IMG3 and IMG4, IMG3 and IMG5, IMG4 and IMG5, we find a sequence of movements (the sequence of making human movements), namely IMG2, IMG3, IMG1, IMG5, and IMG4. Place the images in order.

The server 100 generates summary content (moving image content) that is displayed while the images sequentially arranged are successively (randomly or sequentially) (S460). Since the images are placed in an order that makes the character's movements natural based on the optical flow, the characters appear to move naturally when the placed images are passed continuously (randomly or sequentially). The summary content may be shared and transmitted to the chat room of the messenger application as shown in FIG.

The summary content may be generated by selecting from various templates. For example, the summary content may be generated as a template in which images of a single person follow in sequence. The summary content may be generated by concatenating the summary content of the plurality of characters when there are a plurality of characters in the cluster event.

The server 100 may generate a summary content by selecting a plurality of images among the clustered images, or the server 100 may be selected from the plurality of images selected from the user terminal 200 or the user terminal 200. The summary content may be generated based on the person.

Next, a method of displaying content related to a clustering event received from the server 100 by the user terminal 200 on the screen will be described.

Referring to FIG. 10A, the user terminal 200 receives a plurality of images clustered as a clustering event from the server 100 and displays the plurality of images on the search result screen 500a. The user terminal 200 may request a cluster event for the search keyword from the server 100. In this case, the user may directly input a search keyword to request a cluster event or select a search keyword displayed on an arbitrary interface screen to request a cluster event. The user interface screen 500a may further display another collection in a card view form when a search result related to the search keyword exists in another collection in addition to the cluster event.

The search result screen 500a may display representative images 510 of the plurality of images. In this case, the search result screen 500a may display a tag 520 (eg, #TWICE, #Tzuyu, #Stage, #M Countdown, etc.) related to the clustering event. The user interface screen 500a may display a share button 530. The user interface screen 500a may display an image expansion button 540.

Referring to FIG. 10B, when the image expand button 540 of the search result screen 500a is selected, the images of the clustering event hidden on the search result screen 500b are expanded and displayed. At this time, the images are sorted by person, and even in the same person image, the images are arranged and displayed according to the movement direction (copper line). Meanwhile, in FIG. 10A, when a representative image is selected, only images connected to the representative image may be unfolded.

When the share button 530 is selected, the server 100 may generate a plurality of images as summary content and transmit the plurality of images to the user terminal 200. Alternatively, when the user selects a button (not shown) for requesting generation of summary content for images included in the clustering event on the search result screen 500a or 500b, the server 100 may summarize the plurality of images. The generated content may be generated and transmitted to the user terminal 200. The search result screen 500a or 500b may further include an image selection button (not shown) so that a user may select images among the images included in the group event to generate summary content.

Referring to FIG. 10C, the user terminal 200 may receive the summary type content 610 of the clustering event from the server 100 and display it on the sharing screen (eg, a chat room screen) 600. have. When the user makes a selection such as tapping the summary content 610, the images included in the summary content 610 are displayed in succession (randomly or sequentially) in order. The sharing screen 600 may display a tag 620 related to the summary content (eg, #TWICE, #Tzuyu, #stage, etc.) together with the summary content 610.

When the recommendation tag or the representative tag is selected, the user terminal 200 may display the corresponding search result on the sharing screen 600 or a separate interface screen.

11 is a diagram illustrating a system for sharing a web document in a messenger service according to an embodiment of the present invention.

Referring to FIG. 11, it is assumed that the server 100 shares content through a messenger application of the user terminal 200. The server 100 may be divided into a messaging server 10, an image analysis server 12, a scrap server 14, and the like according to a function provided by a messenger service. The server 100 may include a messaging server 10 and an image analysis server 12. In addition, the interworking method between the scrap server 14 and the method of communicating with the user terminal 200 may be variously changed according to the interworking rule or service scenario. Meanwhile, the server 100 includes at least one of the messaging server 10, the image analysis server 12, and the scrap server 14, and a server function not included in the server 100 is an external server system (eg, For example, the system may be provided from an external server system in conjunction with a system operated by a third party.

When specific link (URL) information 710 is input to the chat room screen 700, the user terminal 200 transmits specific link (URL) information to the scrap server 14, and the scrap server 14 transmits a specific link ( Access the content server corresponding to the URL) information and scrape the content. The scraped content includes an image and image description information (eg, a web document title) included in the web document.

The scrap server 14 sends the scraped content to the image analysis server 12 to request shared content.

The image analysis server 12 generates thumbnails by analyzing information such as whether a person appears in the scraped image, a location of a face, a location of an object, and the like. The image analysis server 12 extracts the same / similar images from the scraped image in the image storage 300. Here, as shown in Table 1, the image storage 300 stores image description contents included in image description tags or web documents for each image. Accordingly, the image analysis server 12 generates, in the image storage 300, a tag list associated with the scraped image based on the image description tag mapped to the extracted identical / similar image or the image description contents included in the web documents. can do.

The thumbnail and tag list of the scraped content may be delivered to the user terminal 200 through the scrap server 14.

The chat room screen 700 displays a thumbnail 720 of content scraped from the link (URL) information 710 and at least one tag 730 included in the tag list. When the tag 730 is selected, the user terminal 200 may display the search result corresponding to the tag 730 on the chat window 700 or a separate interface screen.

As described above, according to an exemplary embodiment of the present invention, various images of various sources may be clustered by attributes such as time, event, and person. According to an embodiment of the present invention, a large amount of images pouring out on the web may be processed to generate content that can be shared. According to an embodiment of the present invention, the user may obtain meaningful content from a keyword of interest, and visually confirm the image search result through the aligned images.

The embodiments of the present invention described above are not only implemented through the apparatus and the method, but may be implemented through a program for realizing a function corresponding to the configuration of the embodiments of the present invention or a recording medium on which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (7)

An image sharing method of a server operated by at least one processor,
Receiving a request for sharing a plurality of images clustered as a single event;
Identify a person included in each image based on person identification information included in each of the plurality of images, extract a plurality of image pairs by combining images including the same person among the plurality of images, and extract the plurality of image pairs. Compute a continuous motion score of each of the plurality of image pairs by tracking an optical flow of corresponding coordinates in the image pairs of the image pairs, and successively move the character's motion based on the motion score of each of the plurality of image pairs. Arranging the images including the characters in the order of making the image;
Sending the plurality of aligned images to a user terminal,
The motion score is given a higher score as the more coordinates moving in the same direction in the two images of each image pair. In claim 1,
The sorting step
And when the plurality of persons appear in the plurality of images, classifying the plurality of images for each person and sorting the images classified for each person. In claim 1,
The transmitting of the plurality of images
Generating content displayed on an interface screen of the user terminal while the plurality of images are randomly passed or sequentially passed in a display order; and
Transmitting the content to the user terminal
Image sharing method comprising a. In claim 1,
The transmitting of the plurality of images
And transmitting at least one tag associated with the single event or the plurality of images to the user terminal. delete delete delete

Images