KR100922693B1 - System and method for searching person - Google Patents

Abstract

A person searching system and method are disclosed. The person search system may include a similarity calculator configured to calculate a similarity between an input user query and an image included in a document, a face index calculation that detects a face region from an image included in the document, and calculates a face index using the face region. The image retrieval unit may be configured to include a recency calculator configured to calculate recency of the document using a time of creation of the document, and an image matching the user query using a goodness of fit according to at least one of the face index, the similarity, and the recency. It may include an image search unit for searching.

Portrait, Image, Similarity, Face Index, Freshness, Goodness of Fit

Description

People Search System and Methods {SYSTEM AND METHOD FOR SEARCHING PERSON}

The present invention relates to a person searching system and method, and more particularly, to determine a goodness of fit according to similarity, face index, and recency based on an image included in a document, and to match an image matching a user query according to the goodness of fit as a person search result. A person searching system and method for providing the same.

Recently, as image retrieval is possible, many modeling methods for retrieving an image corresponding to a user query have been proposed. For example, the conventional image search has a method of exposing images containing a word sorted according to the importance of the word.

Recently, however, when a user searches for an image by inputting a query including a person's name, there are many cases where a user wants to search for an image including a face area. For example, when a user searches for an image by inputting a name of a celebrity, there are many cases where an image including a face of the celebrity is searched for when searching for an image.

The conventional image search has a problem in that it is difficult to search for an image including a face region among images corresponding to a user query. In particular, in the case of modeling the image search in the prior art, the image processing technique is not applied, and sometimes a search result different from the user's intention is often exposed.

Accordingly, there is a demand for a method of performing a more accurate person search by searching for an image in which a person is specified when searching for an image.

The present invention provides a person searching system and method for performing an image search reflecting a user's search intention by searching for an image matching a user query related to a person's name.

The present invention provides a person searching system and method capable of searching for an image more suitable for a user query by searching for an image using a similarity between the user query and the image.

The present invention provides a person retrieval system and method that can reflect a user search intention for face-oriented person searching by calculating a face index through a face region detected in an image.

The present invention provides a person searching system and method that can provide an image search result that can reflect the user's recent interest when searching for a person by calculating the freshness of the document according to the document creation time.

In a person searching system according to an embodiment of the present invention, a person searching system may include a similarity calculator configured to calculate a similarity between an input user query and an image included in a document, and detect a face region from the image included in the document. A face index calculator for calculating a face index using an area, a freshness calculator for calculating the freshness of the document using a writing time of the document, and at least one of the face index, the similarity, and the freshness It may include an image search unit for searching for an image matching the user query using the goodness of fit.

In this case, the face index calculator may include: a face region ratio calculator configured to calculate a face region ratio occupied by the face region in the entire image from the attributes of the face region, a face number calculator configured to calculate the number of faces from the attributes of the face region; A face size calculator may be configured to calculate a face size from an attribute of the face region, and a skin color ratio calculator may be configured to calculate a skin color ratio from an attribute of the face region.

According to an embodiment of the present invention, a person searching method may include calculating a similarity between an input user query and an image included in a document, detecting a face region from the image included in the document, and using the face region to extract a face index. Calculating a recency of the document by using the time of creation of the document and an image matching the user query using a goodness of fit according to at least one of the face index, the similarity, and the recency. Searching may be included.

In this case, the calculating of the face index may include calculating a proportion of the face area occupied by the face area in the entire image from the property of the face area, calculating the number of faces from the property of the face area, Calculating a face size from an attribute and calculating a skin color ratio from the attribute of the face region.

According to the present invention, there is provided a person searching system and method for performing an image search reflecting a user's search intention by searching for an image matching a user query related to a person name.

According to the present invention, there is provided a person searching system and method capable of searching for an image more suitable for a user query by searching for an image using a similarity between the user query and the image.

According to the present invention, there is provided a person searching system and method that can reflect a user search intention of searching for a face-oriented person by calculating a face index through a face region detected in an image.

According to the present invention, there is provided a person searching system and method that can provide a more recent image by calculating the freshness of the document according to the time of document creation.

Hereinafter, with reference to the contents described in the accompanying drawings will be described in detail an embodiment according to the present invention. However, the present invention is not limited or limited by the embodiments. Like reference numerals in the drawings denote like elements. The person searching method according to an embodiment of the present invention may be performed by a person searching system.

1 is a block diagram showing the overall configuration of a person searching system according to an embodiment of the present invention.

Referring to FIG. 1, a person searching system 100 according to an exemplary embodiment of the present invention may include a similarity calculator 101, a face index calculator 102, a freshness calculator 103, and an image searcher 104. It may include.

The similarity calculator 101 may calculate a similarity between the input user query 105 and an image included in the document. In this case, the similarity calculator 101 may use a document stored in the database 106. In one example, database 106 can store and maintain at least one document that includes an image. In this case, the document may include a digital document, and may include news, blog / cafe article, UCC material, and the like. However, the document may include a face image of the person.

For example, the similarity calculator 101 may calculate the similarity using text related to an image included in a document among texts included in the document. That is, it may be difficult for the person searching system 100 to immediately determine whether the image included in the document is related to the user query 105.

Accordingly, the person searching system 100 may determine whether or not the image included in the document is related to the user query 105 by calculating the similarity using the text related to the image included in the document. If the user query 105 of "Jeonhyun" is input, the similarity calculator 101 may calculate the similarity between the user query 105 and the image using text related to the image of "Jeonhyun" in the document. .

In this case, the text associated with the image may be text associated with the user query 105. That is, the text associated with the image may be a word or sentence that includes the user query 105. Referring to the above example, the text associated with the image may be "Jeonhyun" itself or a phrase or sentence including "Jeonhyun".

 For example, the similarity calculator 101 detects the frequency of exposure of the user query, the total number of documents to which the user query is exposed, and the number of entire texts included in the document through the text related to the image. Similarity can be calculated. The similarity calculator 101 will be described in detail with reference to FIG. 2.

The face index calculator 102 may detect a face region from an image included in the document, and calculate a face index using the face region. The person searching system 100 may extract an image having a large weight of the face from the image and provide the same to the user by using the face index. Since the user wants an image in which the face of the person is highlighted when searching for a person, the person search system 100 may provide a person image with a face area highlighted.

For example, when an overlapping region exists between a plurality of face regions, the face index calculator 102 may detect a face region from which the overlapping region is excluded. For example, the face index calculator 102 may calculate a face area ratio, a face number, a face size, and a skin color ratio.

In this case, the face index calculator 102 may calculate the face index by applying the skin color ratio and the face region ratio to the weighted sum of the face size and the number of faces. The face index calculator 102 is described in detail with reference to FIGS. 3 to 7.

The freshness calculation unit 103 may calculate the freshness of the document by using the time of creation of the document. In the case of person search, a user often has an intention to search for the latest figure of the person, and therefore, the person search system 100 according to an embodiment of the present invention may be included in a more recent document through the recency of the document. It can provide a person image.

For example, the recency calculator 103 may calculate the recency of the document according to a ratio between a time point at which the document is created and a current time point at which the user query is input and a preset threshold time point. Details of the freshness calculator 103 are described with reference to FIG. 8.

The image retrieval unit 104 may search for an image matching the user query using a goodness of fit according to at least one of the face index, the similarity, and the freshness. That is, according to an embodiment of the present invention, the person search system 100 may perform a person search by searching for a face image having a high degree of fitness for a user query input by a user through a fitness calculation.

For example, the image search unit 104 may calculate the fitness by applying weights to the face index, the similarity, and the freshness, respectively. The image search unit 104 may determine a search ranking of the image according to the calculated fitness, and search for the image according to the search ranking.

2 is a diagram illustrating an example for describing an operation of a similarity search unit of a person searching system according to an exemplary embodiment of the present invention.

Referring to FIG. 2, a user query 105 may be input. At this time, the user query 105 may be "A". At this time, "A" may be a query associated with the person name. For example, it may be "A" itself or a query that includes "A".

The similarity calculator 101 may then calculate the similarity between the input user query 105 and the image included in the document. In this case, the similarity calculator 101 may use a document stored in the database 106. In one example, database 106 can store and maintain at least one document that includes an image. In this case, the document may include a digital document, and may include news, blog / cafe article, UCC material, and the like. However, the document may include a face image of the person.

For example, suppose document 202 and document 203 are stored in database 105. In this case, the document 202 and the document 203 may include an image. The similarity calculator 101 may calculate the similarity between the user query 105 and the image using the text included in the document 202 and the document 203.

In one example, the similarity calculator 101 may calculate the similarity between texts using text related to an image included in a document among the texts included in the document. Referring to FIG. 2, the text related to the image may be “photographing A picture”, “starring: A, B”. In this case, the text associated with the image may be text associated with the user query 105. That is, the text associated with the image may include the user query 105 itself or a similar word, a phrase or a sentence including the user query 105, and the like.

In one example, the similarity calculator 101 may generate a similarity 201 for each document. That is, the higher the similarity, the higher the similarity between the image included in the document and the user query 105. In addition, if the similarity is high, it may be determined that the person image searched by the person searching system 100 is close to the person search result requested by the user.

In one example, the similarity calculation unit 101 is a terminal frequency of the user query 105, the total number of inverse document frequency of the user query 105 is exposed on the document through the text associated with the image and the The similarity may be calculated by detecting the document length of the entire text included in the document. In this case, the similarity calculator 101 applies a weight to each of the frequency of exposure of the user query 105 on the document, the total number of documents to which the user query 105 is exposed, and the number of the entire texts included in the document. Can be calculated. In one example, the similarity calculator 101 may calculate the similarity through the modified 2-Poisson model.

As a result, the person searching system 100 may search for an image highly related to the user query 105 by calculating a similarity between the user query 105 and the image.

3 is a block diagram showing the overall configuration of the face index calculation unit of the person searching system according to an embodiment of the present invention.

Referring to FIG. 3, the face index calculator 102 may include a face region ratio calculator 301, a face number calculator 302, a face size calculator 303, and a skin color ratio calculator 304. have. In this case, the face index calculator 102 may detect a face region from an image included in the document, and calculate a face index using the face region. As shown in FIG. 1, the face index calculator 102 may detect a face region from an image included in at least one document stored in the database 106.

The face index calculator 102 may detect a face region in which the duplicate region is excluded when there is a duplicate region between the plurality of face regions. When the face area is detected by excluding the overlapped area, an error due to the overlapped area can be reduced, thereby improving accuracy in calculating the face index later.

The face region ratio calculator 301 may calculate a face region ratio occupied by the face region in the entire image from the attributes of the face region. In this case, the face region ratio calculator 301 may calculate the face region ratio using the result of summing each of the face regions in the image and the total size of the image. According to an embodiment of the present invention, the face index may have a larger value as the face area ratio increases.

The face number calculator 302 may calculate the number of faces from the attributes of the face area. According to an embodiment of the present invention, the smaller the number of faces, the greater the degree of emphasizing the face area in the image, so that the smaller the number of faces, the higher the face index. That is, even if the size of the entire face area is the same, an image having a smaller number of faces may have a larger face index.

The face size calculator 303 may calculate the size of the face area from the attributes of the face area. According to an embodiment of the present invention, even if there is an image having the same face area ratio, an image having a large size of the face area may exhibit a larger face index.

The skin color ratio calculator 304 may calculate the skin color ratio of the face region from the attributes of the face region. Skin color may be distinguished through brightness, saturation, and color, which represent color. For example, the skin color ratio calculation unit 304 may set the skin color step by step, and calculate the ratio of the skin color corresponding to each step. In this case, the skin color ratio calculator 304 may calculate the skin color ratio by using the result of summing each of the face areas in the image and the result of summing each of the skin colors of the face area.

In this case, the face index calculator 102 may include a face region ratio calculator 301, a face number calculator 302, a face size calculator 303, and a skin color ratio calculator 304. In addition, the face index calculator 102 may include all of the face area ratio calculator 301, the face number calculator 302, the face size calculator 303, and the skin color ratio calculator 304. Each of the face region ratio calculator 301, the face number calculator 302, the face size calculator 303, and the skin color ratio calculator 304 will be described in detail with reference to FIGS. 4 to 7.

For example, the face index calculator 103 may calculate the face index by applying the skin color ratio and the face region ratio to the weighted sum of the face size and the number of faces. According to an embodiment of the present invention, the person searching system 100 may first expose an image having a higher face index to a user by setting a high search ranking.

4 is a diagram illustrating an example for describing an operation of a face region ratio calculator according to an exemplary embodiment of the present invention.

The face region ratio calculator 301 may calculate the face region ratio occupied by the face region in the entire image from the attributes of the face region. In this case, the face region ratio calculator 301 may calculate the face region ratio using the result of summing each of the face regions in the image and the total size of the image. For example, the face region ratio calculator 103 may calculate the face region ratio according to Equation 1 below.

는 얼굴 영역 비율을 의미하고,

는 전체 이미지에서 각각의 얼굴 영역의 사이즈를 의미한다. 예를 들어, 얼굴 영역이 2개인 경우,

는 2개의 얼굴 영역의 사이즈를 합산한 결과를 의미한다. 다만, 얼굴 영역 간에 중복되는 영역이 존재하는 경우, 얼굴 영역의 사이즈는 중복되는 영역을 제외한 부분에 대한 사이즈를 의미할 수 있다.

는 이미지의 전체 사이즈로, 가로 길이와 세로 길이의 곱으로 결정될 수 있다.here,

Means the face area ratio,

Denotes the size of each face region in the entire image. For example, if you have two face zones,

Denotes the result of summing the sizes of two facial regions. However, when there is an overlapping area between the face areas, the size of the face area may mean a size of a portion excluding the overlapping area.

Is the overall size of the image, which may be determined as the product of the width and length.

Referring to FIG. 4, the image 401 and the image 402 have the same overall size of the image. However, since the image 401 has a smaller size of the face area than the image 402, the face area ratio of the image 401 is smaller than that of the image 402 according to Equation 1. The person search system 100 may expose the image 402 to the user as an image search result in preference to the image 401.

5 is a diagram illustrating an example for describing an operation of a face count calculator according to an exemplary embodiment of the present invention.

The face number calculator 302 may calculate the number of faces from the attributes of the face area. For example, the number of faces calculated by the face number calculator 302 may be an input value of a function according to Equation 2 below, and the output value of the function may be used when calculating a face index.

는 얼굴 개수 계산부(302)가 계산한 얼굴 개수를 의미하고, A와 B는 상수를 의미한다. 수학식 2를 참고하면, 이미지(501)는 이미지(502)보다 얼굴 개수가 작기 때문에,

는 이미지(501)가 이미지(502)보다 더 큰 값을 가질 수 있다.here,

Denotes the number of faces calculated by the face number calculator 302, and A and B denote constants. Referring to Equation 2, since the image 501 has a smaller number of faces than the image 502,

May have a larger value than image 502.

Therefore, as the number of faces increases, it is difficult to select a face area to be searched in the entire image. Therefore, a face index may be higher in an image having a relatively small number of faces than an image having a large number of faces. That is, the image having a small number of faces is relatively higher than the image having a large number of faces, which means that the face index is higher.

6 is a diagram illustrating an example for describing an operation of a face size calculator according to an exemplary embodiment of the present invention.

The face size calculator 303 may calculate a face size from attributes of the face area. At this time, the face size may be said to be dependent on the ratio of the face area and the skin color in the face area.

For example, the face size calculator 303 may calculate a face size, and the calculated face size may be an input value of a function according to Equation 3, and the output value of the function may be used when calculating a face index.

는 얼굴 영역에 대한 얼굴 사이즈를 의미하고,

는 이미지의 전체 사이즈를 의미한다. 이 때, 이미지 상에 얼굴이 복수 개이고, 얼굴 영역 간에 중복되는 영역이 존재하는 경우, 얼굴 사이즈 계산부(303)는 중복되는 영역을 제외하고 얼굴 사이즈를 계산할 수 있다. 일례로, 얼굴 사이즈가 클수록 얼굴 지수는 높게 계산될 수 있다.here,

Means the face size for the face area,

Means the total size of the image. In this case, when there are a plurality of faces on the image and there are overlapping areas between the face areas, the face size calculator 303 may calculate the face size except for the overlapping areas. For example, the larger the face size, the higher the face index may be calculated.

Referring to FIG. 6, the image 601 and the image 602 have the same face area ratio. However, it can be seen that the face size is larger in the image 602 than in the image 601. Thus, even if the face area is the same, the image 602 having a larger face size may have a larger face index than the image 601.

7 is a view showing an example for explaining the operation of the skin color ratio calculator according to an embodiment of the present invention.

The skin color ratio calculator 304 may calculate the skin color ratio from the attributes of the face area. For example, the skin color ratio calculator 304 may calculate the skin color ratio of the face region according to Equation 4 below.

는 얼굴 영역의 피부색 비율을 의미하고,

는 전체 이미지에서 각각의 얼굴 영역의 크기를 의미한다. 또한,

는 전체 이미지에서 각각의 피부색이 나타내는 영역의 크기를 의미한다. 예를 들어, 얼굴 영역이 2개인 경우,

는 2개의 얼굴 영역의 사이즈를 합산한 결과를 의미한다. 이 때, 얼굴 영역이 중복되는 경우,

는 중복되는 영역을 제외한 얼굴 영역의 사이즈를 의미한다. 또한,

는 얼굴 영역에서 피부색 각각에 대응되는 영역의 사이즈를 합산한 결과를 의미한다.here,

Means the skin color ratio of the face area,

Means the size of each face area in the whole image. Also,

Means the size of the area indicated by each skin color in the entire image. For example, if you have two face zones,

Denotes the result of summing the sizes of two facial regions. At this time, if the face area overlaps,

Denotes the size of the face region excluding the overlapping region. Also,

Denotes a result of summing the sizes of the regions corresponding to the skin colors in the face region.

Referring to FIG. 7, when a face region is detected in an image, the skin color ratio calculator 304 may determine the skin color distribution of the face region by calculating the skin color ratio of the face region. The skin color may have a different distribution depending on contrast, saturation, color, etc. in the facial area. The skin color ratio calculator 304 may determine the skin color distribution of the face region in the image of FIG. 7 through image processing.

4 to 7, the face index calculator 102 may calculate a face index by calculating a face region ratio, a face number, a face size, and a skin color ratio from an image included in a document.

For example, the face index calculator 102 may calculate the face index according to Equation 5 using Equations 1 to 4 below. The face index calculator 102 may calculate a face index by applying the skin color ratio and the face region ratio to the weighted sum of the face size and the number of faces.

는 얼굴 영역 비율,

는 피부색 비율,

는 얼굴 사이즈에 대한 함수값,

는 얼굴 개수에 대한 함수값을 의미한다. 또한,

과

는 각각 얼굴 사이즈와 얼굴 개수에 대한 함수값에 적용되는 가중치를 의미한다.Here, FV (I) means a face value (Face Value),

The face area ratio,

Is skin color ratio,

Is a function value for face size,

Denotes a function value for the number of faces. Also,

and

Denote weights applied to the function values for the face size and the number of faces, respectively.

8 is a diagram illustrating an example for explaining the operation of the freshness calculator of the person searching system according to an embodiment of the present invention.

The freshness calculation unit 103 may calculate the freshness of the document by using the time of creation of the document. In this case, the freshness calculator 103 may calculate the freshness of the document according to a difference between a time point of creating the document and a current time point at which the user query 105 is input and a preset threshold time point. In one example, the recency calculation unit 103 may calculate the recency of the document according to the following equation (6).

은 문서의 최신성을 의미하고,

는 문서의 작성 시점 및 사용자 질의(105)가 입력된 현재 시점 간의 차이를 의미한다. 그리고,

는 미리 설정한 임계 시점을 의미하고, K는 상수를 의미한다. 수학식 6은 일례에 불과하고, 시스템의 구성에 따라 변경될 수 있다.here,

Means the document's freshness,

Denotes a difference between a document creation time point and a current time point when the user query 105 is input. And,

Denotes a preset threshold time point, and K denotes a constant. Equation 6 is only an example and may be changed according to the configuration of the system.

8 shows an example in which Equation 6 is graphically represented. Referring to FIG. 8, the horizontal axis represents the viewpoint and the vertical axis represents the freshness. If the time point is 0, it means the current time point at which the user query 105 is input, and P means the time point at which the document is created. The time point can be set up to MAXAGE. The graph shown in Figure 8 shows the monotonic susceptibility form. The shape of the graph shown in FIG. 8 may vary depending on the configuration of the system.

If P is 0, that is, when the document creation time point is the current time point, the freshness value is 1. When P is MAXAGE, that is, when the document creation time point is a preset threshold time point, the freshness value is 0. When P has a value between 0 and MAXAGE, the freshness decreases as the time point increases according to Equation 6.

The image retrieval unit 104 may search for an image matching the user query from the document by calculating a goodness of fit through at least one of a face index, a similarity, and a freshness.

For example, the image search unit 104 may calculate a fitness by applying weights to the face index, the similarity, and the freshness, respectively. For example, the image search unit 104 may calculate a goodness of fit through Equation 7 below.

는 적합도를 의미하고,

는 유사도,

는 얼굴 지수,

는 최신성을 의미한다. 그리고, S, Q, R은 각각 유사도, 얼굴 지수, 최신성에 적용되는 가중치를 의미한다. 상기 수학식 7에서는 얼굴 지수, 유사도 및 최신성을 전부 이용하는 경우를 나타낸다. 다만, 위에서 언급했듯이, 이미지 검색부(104)는 수학식 7에 한정되지 않고, 얼굴 지수, 유사도 및 최신성을 각각 이용하거나 또는 조합하여 적합도를 계산할 수 있다.here,

Means goodness of fit,

Is similarity,

Face index,

Means freshness. S, Q, and R mean weights applied to similarity, face index, and freshness, respectively. In Equation 7, the face index, the similarity, and the freshness are all used. However, as mentioned above, the image retrieval unit 104 is not limited to the equation (7), and may calculate the fitness by using or combining the face index, the similarity, and the freshness, respectively.

  The image search unit 104 may determine a search ranking according to the fitness and search for an image according to the search ranking. That is, the images may be sorted according to the search ranking, and the sorted images may be provided to the user according to the search exposure criteria.

9 is a flowchart illustrating the entire process of the person searching method according to an embodiment of the present invention.

In step S901, the similarity between the input user query and the image included in the document may be calculated. In this case, in step S901, the similarity may be calculated using text related to an image included in the document among the text included in the document. In this case, the document may include a digital document, and may include news, blog / cafe article, UCC material, and the like. However, the document may include a face image of the person. Here, the text associated with the image may be text associated with the user query.

For example, in step S901, the degree of similarity is detected by detecting the frequency of exposure of the user query on the document, the total number of documents to which the user query is exposed, and the number of whole texts included in the document through the text associated with the image. Can be calculated. At this time, if the similarity is high, it may be determined that the searched person image is close to the person search result requested by the user.

In operation S902, a face region may be detected from an image included in a document, and a face index may be calculated using the detected face region. Since the user wants an image in which the face of the person is highlighted when searching for a person, a face image is emphasized and a person image having a high face index may be provided to the user as a search result. For example, in operation S902, when a duplicate area exists between a plurality of face areas, a face area in which the duplicate area is excluded may be detected from the image. By eliminating the redundant region, the accuracy of the calculated face index can be improved.

In this case, step S902 may include calculating a ratio of the face area occupied by the face area in the entire image from the attributes of the face area. In this case, the step of calculating the face area ratio may calculate the face area ratio using the result of summing each of the face areas in the image and the total size of the image. For example, the larger the facial region ratio, the higher the facial index.

In addition, step S902 may include calculating a number of faces from attributes of the face area. In the calculating of the face index, the smaller the number of faces, the higher the face index may be. If the number of faces is small, the face index can be calculated higher because the face area can be further highlighted in the whole image.

In operation S902, a face size may be calculated from an attribute of the face area. At this time, even if there are images having the same face area ratio, the face index may be calculated higher because the larger the face size, the more the face area may be highlighted.

In operation S902, the skin color ratio may be calculated from the attributes of the face area. For example, by setting the skin color step by step, the ratio of the skin color corresponding to each step may be calculated. In the calculating of the skin color ratio, the skin color ratio may be calculated using the result of summing each of the face areas in the image and the result of summing each of the skin colors of the face area.

For example, in step S902, the face index may be calculated by applying the skin color ratio and the face region ratio to the weighted sum of each of the face size and the number of faces. Refer to Equation 5 for the process of calculating the face index.

In step S902, the step of calculating the area ratio of the face, the step of calculating the number of faces, the step of calculating the size of the face and the step of calculating the skin color ratio may be performed in parallel without being ordered.

In step 903, the freshness of the document can be calculated using the time of creation of the document. In this case, in step S903, the freshness of the document may be calculated according to a ratio between a difference between a time point at which the document is created and a current time point at which the user query is input and a preset threshold time point. That is, since the user wants to search for the latest image of the person during the person search, the image included in the recent document may be provided as a search result through step S903.

In step 904, an image matching the user query may be retrieved using a goodness of fit according to at least one of the face index, the similarity, and the freshness. For example, in step S904, the goodness of fit may be calculated by applying weights to each of the face index, the similarity, and the freshness. A process of calculating the goodness of fit may refer to Equation 7 above.

In operation S904, a search ranking may be determined according to the goodness of fit, and an image may be searched according to the search ranking. As a result, an image with high suitability may have a high search ranking, and an image having a high search ranking may be first exposed to a user as a person search result.

For example, the person searching method may be performed in parallel regardless of the order of steps S901, S902, and S903.

For details not described with reference to FIG. 9, reference may be made to the contents described with reference to FIGS. 1 to 8.

In addition, the person searching method according to an embodiment of the present invention includes a computer readable medium including program instructions for performing operations implemented by various computers. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium or program instructions may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

1 is a block diagram showing the overall configuration of a person searching system according to an embodiment of the present invention.

2 is a diagram illustrating an example for describing an operation of a similarity search unit of a person searching system according to an exemplary embodiment of the present invention.

3 is a block diagram showing the overall configuration of the face index calculation unit of the person searching system according to an embodiment of the present invention.

4 is a diagram illustrating an example for describing an operation of a face region ratio calculator according to an exemplary embodiment of the present invention.

5 is a diagram illustrating an example for describing an operation of a face count calculator according to an exemplary embodiment of the present invention.

6 is a diagram illustrating an example for describing an operation of a face size calculator according to an exemplary embodiment of the present invention.

7 is a view showing an example for explaining the operation of the skin color ratio calculator according to an embodiment of the present invention.

8 is a diagram illustrating an example for explaining the operation of the freshness calculator of the person searching system according to an embodiment of the present invention.

9 is a flowchart illustrating the entire process of the person searching method according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

100: People Search System

101: similarity calculation unit

102: face index calculator

103: freshness calculation unit

104: image search unit

105: user queries

106: database

Claims (33)

A similarity calculator which calculates a similarity between the input user query and the image included in the document; A face index calculator for detecting a face region from the image included in the document and calculating a face index using the face region; A recency calculator for calculating recency of the document using a time point at which the document is created; And An image search unit that searches for an image matching the user query using a goodness of fit according to at least one of the face index, the similarity, and the freshness Including, The image search unit, And a search ranking is determined according to the fitness, and an image is searched according to the search ranking. The method of claim 1, The similarity calculation unit, Calculating similarity using text related to an image included in the document among the text included in the document, The text associated with the image is And a text associated with the user query. The method of claim 2, The similarity calculation unit, And calculating the similarity by detecting a frequency of exposure of the user query, the total number of documents exposed to the user query, and the total number of texts included in the document through the text related to the image. Search system. The method of claim 1, The face index calculator, And if a duplicate area exists between the plurality of face areas, detecting a face area from which the duplicate area is excluded from the image. The method of claim 1, The face index calculator, A face region ratio calculation unit that calculates a face region ratio occupied by the face region in the entire image from the attributes of the face region Person search system comprising a. The method of claim 5, The face area ratio calculation unit, And calculating a face area ratio using the result of adding each face area in the image and the total size of the image. The method of claim 1, The face index calculator, A face count calculator for counting face counts from attributes of the face area Person search system comprising a. The method of claim 7, wherein The face index calculator, And the face index is calculated to be higher as the number of faces is smaller. The method of claim 1, The face index calculator, A face size calculator for calculating a face size from attributes of the face area Person search system comprising a. The method of claim 1, The face index calculator, Skin color ratio calculation unit for calculating the skin color ratio from the attributes of the face area Person search system comprising a. The method of claim 10, The skin color ratio calculation unit, The skin color ratio is calculated using the result of summing each of the face areas in the image and the result of summing each of the skin colors of the face area. The method of claim 1, The face index calculator, A face region ratio calculation unit that calculates a face region ratio occupied by the face region in the entire image from the attributes of the face region; A face count calculator configured to calculate a face count from an attribute of the face area; A face size calculator for calculating a face size from attributes of the face area; And Skin color ratio calculation unit for calculating the skin color ratio from the attributes of the face area Person search system comprising a. The method of claim 12, The face index calculator, And calculating the face index by applying the skin color ratio and the face region ratio to the weighted sum of each of the face size and the number of faces. The method of claim 1, The freshness calculation unit, And calculating the freshness of the document according to a difference between a time point at which the document is created and a current time point at which the user query is input, and a predetermined threshold time point. The method of claim 1, The image search unit, And calculating the fitness by applying weights to the face index, the similarity, and the freshness, respectively. delete Calculating a similarity between the input user query and the image included in the document; Detecting a face region from an image included in the document, and calculating a face index using the face region; Calculating the freshness of the document using the time of creation of the document; And Retrieving an image matching the user query using a goodness of fit according to at least one of the face index, the similarity, and the freshness Including, The step of retrieving the image, And a search ranking is determined according to the fitness, and an image is searched according to the search ranking. The method of claim 17, The step of calculating the similarity, Calculating similarity using text related to an image included in the document among the text included in the document, The text associated with the image is And a text associated with the user query. The method of claim 18, The step of calculating the similarity, The similarity is calculated by detecting a frequency of exposure of the user query, the total number of documents in which the user query is exposed, and the total number of texts included in the document through the text related to the image. How to search for people. The method of claim 17, The step of calculating the face index, And if a duplicate area exists between the plurality of face areas, detecting a face area from which the duplicate area is excluded from the image. The method of claim 17, The step of calculating the face index, Calculating a proportion of a face region occupied by the face region in the entire image from the attributes of the face region Person searching method comprising a. The method of claim 21, The step of calculating the face area ratio, And calculating a face area ratio using the result of adding each face area in the image and the total size of the image. The method of claim 17, The step of calculating the face index, Calculating the number of faces from the attributes of the face region Person searching method comprising a. The method of claim 23, wherein The step of calculating the face index, The smaller the number of faces, the higher the face index is calculated. The method of claim 17, The step of calculating the face index, Calculating a face size from attributes of the face area Person searching method comprising a. The method of claim 17, The step of calculating the face index, Calculating the skin color ratio from the attributes of the face area Person searching method comprising a. The method of claim 26, The step of calculating the skin color ratio, The skin color ratio is calculated using the result of summing each of the face areas in the image and the result of summing each of the skin colors of the face area. The method of claim 17, The step of calculating the face index, Calculating a proportion of the face area occupied by the face area in the entire image from the attributes of the face area; Calculating a number of faces from attributes of the face area; Calculating a face size from an attribute of the face area; And Calculating the skin color ratio from the attributes of the face area Person searching method comprising a. The method of claim 28, The step of calculating the face index, And calculating the face index by applying the skin color ratio and the face region ratio to the weighted sum of each of the face size and the number of faces. The method of claim 17, The step of calculating the freshness of the document, And calculating the freshness of the document according to a difference between a time point at which the document is created and a current time point at which the user query is input, and a predetermined threshold time point. The method of claim 17, The step of retrieving the image, And calculating the fitness by applying weights to the face index, the similarity, and the freshness, respectively. delete A computer-readable recording medium in which a program for executing the method of any one of claims 17 to 31 is recorded.

Images