WO2022044923A1

WO2022044923A1 - Information processing device

Info

Publication number: WO2022044923A1
Application number: PCT/JP2021/030217
Authority: WO
Inventors: 久央勝見; 渉山田; 桂一落合
Original assignee: 株式会社Ｎｔｔドコモ
Priority date: 2020-08-26
Filing date: 2021-08-18
Publication date: 2022-03-03
Also published as: JP7412575B2; JPWO2022044923A1

Abstract

A server 10 of one embodiment of the present invention comprises: a first acquisition unit 11 that acquires one or more candidate spot images for each of a plurality of candidate spots; a second acquisition unit 12 that acquires one or more famous spot images for each of a plurality of famous spots; a feature amount calculation unit 13 that uses each of a plurality of feature amount extraction methods to calculate a feature amount by each feature amount extraction method, for each candidate spot image and each famous spot image; a similarity degree calculation unit 14 that calculates, on the basis of the feature amounts of each image according to each feature amount extraction method, a degree of similarity for each feature amount extraction method, such calculation performed for each set consisting of a candidate spot image and a famous spot; and an extraction unit 15 that, on the basis of the similarity degrees for each combination of a feature amount extraction method and a set consisting of a candidate spot image and a famous spot, extracts a similar pair, which is a set consisting of a candidate spot image and a famous spot that are similar to each other.

Description

Information processing equipment

One aspect of the present invention relates to an information processing device.

Conventionally, the similarity between the two images is calculated by comparing the feature quantities of a plurality of types of the input search image with the feature quantities of a plurality of types of the registered images registered in advance, and the search image is based on the similarity degree. A mechanism for searching for registered images similar to the above is known (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2001-319232

In the above mechanism, it is generally determined that the input image and the registered image are similar to each other when the similarity between the input image and the registered image is equal to or higher than a predetermined threshold value. However, in such a process, it may be difficult to appropriately set a threshold value as a similar determination criterion in advance. Therefore, if an inappropriate threshold value is set, it may not be possible to properly extract a set of images that are similar to each other (that is, a set of subjects that are similar to each other).

One aspect of the present invention is to provide an information processing apparatus capable of easily and appropriately extracting a set of subjects similar to each other.

The information processing apparatus according to one aspect of the present invention has a first acquisition unit that acquires one or more first images for each of a plurality of first subjects, and one or more second images for each of the plurality of second subjects. A feature amount calculation unit for calculating the feature amount for each feature amount extraction method and a feature for each first image and each second image by using the second acquisition unit to be acquired and each of the plurality of feature amount extraction methods. A similarity calculation unit that calculates the similarity for each feature amount extraction method for each set of the first image and the second image based on the feature amount for each feature amount extraction method of each image calculated by the quantity calculation unit. , A set of first and second subjects that are similar to each other based on the degree of similarity for each combination of the first image and the second image set and the feature amount extraction method calculated by the similarity calculation unit. It is provided with an extraction unit for extracting pairs.

In the information processing apparatus according to one aspect of the present invention, for all combinations of one or more first images for each of the plurality of first subjects and one or more second images for each of the plurality of second subjects. The degree of similarity for each feature extraction method is calculated. Then, based on the similarity calculated for each combination of the first image and the second image pair and the feature amount extraction method in this way, a similar pair that is a pair of the first subject and the second subject that are similar to each other. Is extracted. According to the above configuration, it is not necessary to set a threshold value for the similarity in advance by performing a relative evaluation using the similarity between the plurality of first subjects and the plurality of second subjects. As a result, it is possible to prevent setting an inappropriate threshold value. Further, since it is not necessary to set a threshold value, it is possible to more easily perform the process of extracting a set of subjects similar to each other. Therefore, according to the information processing apparatus, it is possible to easily and appropriately extract a set of subjects similar to each other.

According to one aspect of the present invention, it is possible to provide an information processing apparatus capable of easily and appropriately extracting a set of subjects similar to each other.

It is a figure which shows the structure of the server which is the information processing apparatus which concerns on embodiment. It is a flowchart which shows an example of the operation of a server. It is a flowchart which shows the processing procedure of the 1st example of step S5 in FIG. It is a flowchart which shows the processing procedure of the 2nd example of step S5 in FIG. It is a flowchart which shows the processing procedure of the 3rd example of step S5 in FIG. It is a figure which shows an example of the hardware configuration of a server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 is a diagram showing a configuration of a server 10 which is an information processing device according to an embodiment. The server 10 is a device configured to be able to execute a process of extracting a similar pair, which is a set of candidate spots (first subject) and famous spots (second subject) that are similar to each other.

Famous spots are tourist spots (sightseeing spots) that have a certain degree of name recognition. Examples of famous spots include Niagara Falls, Machu Picchu and other well-known tourist destinations. Candidate spots are less well-known than famous spots and have fewer tourists than famous spots. Famous spots and candidate spots are listed in advance by, for example, the operator of the server 10.

As shown in FIG. 1, the server 10 includes a first acquisition unit 11, a second acquisition unit 12, a feature amount calculation unit 13, a similarity calculation unit 14, an extraction unit 15, and a presentation unit 16. To prepare for.

The first acquisition unit 11 acquires one or more candidate spot images _X _ik (first image) for each of a plurality of (NX in this embodiment) candidate spots _X _i (i = 1, ..., NX). do. In the present embodiment, the first acquisition unit 11 acquires a plurality of candidate spot images X _ik for each candidate spot X _i . Here, the candidate spot image X _ik indicates the kth image including the candidate spot X _i in the subject. However, the plurality of candidate spots _Xi may include candidate spots having only one candidate spot image.

The first acquisition unit 11 acquires a plurality of candidate spot images X _ik from, for example, the candidate spot image DB 20. The candidate spot image DB 20 is a database that stores one or more candidate spot images X _ik for each of the plurality of candidate spots X _i . In the candidate spot image DB 20, candidate spot images X _ik collected in advance by an operator or the like are accumulated. The candidate spot image X _ik is stored in the candidate spot image DB 20 in advance by being extracted from an image provided in, for example, a Google map (registered trademark).

The second acquisition unit 12 acquires one or more famous spot images Y _jm (second image) for each of a plurality of ( _NY pieces in this embodiment) famous spots Y _j (j = 1, ..., _NY ). do. In the present embodiment, the second acquisition unit 12 acquires a plurality of famous spot images Y _jm for each famous spot Y _j . Here, the famous spot image Y _jm indicates the m-th image including the famous spot Y _j as a subject. However, the plurality of famous spots _Yj may include a famous spot having only one famous spot image.

The second acquisition unit 12 acquires a plurality of famous spot images Y _jm from, for example, the famous spot image DB 30. The famous spot image DB 30 is a database that stores one or more famous spot images Y _jm for each of the plurality of famous spots Y _j . The famous spot image Y _jm collected in advance by an operator or the like is stored in the famous spot image DB 30. The famous spot image Y _jm is stored in the candidate spot image DB 20 in advance by being extracted from an image provided in, for example, a website accessible via a communication network such as the Internet and a Google map (registered trademark). Has been done.

The feature amount calculation unit 13 calculates the feature amount for each feature amount extraction method for each candidate spot image X _ik and each famous spot image Y _jm by using each of the plurality of feature amount extraction methods. For example, the feature amount calculation unit 13 previously performs a calculation using the feature amount extraction functions F _n (n = 1, ..., Nf) corresponding to each of a plurality of (Nf pieces in this embodiment) feature amount extraction methods. It is programmed to be executable. More specifically, the feature amount calculation unit 13 converts each image X _ik and Y _jm into an image feature amount vector by using the feature amount extraction function F _n . In the following description, the image feature quantity vector obtained by converting the image p using the feature quantity extraction function F _n is referred to as F _n (p).

Examples of the plurality of feature amount extraction methods include convolutional neural networks, Visual Concept, GIST, and the like. In this case, the feature amount calculation unit 13 calculates the feature amount (CNN feature amount) obtained by the convolutional neural network, the feature amount based on the Visual Concept, and the GIST feature amount for each image X _ik and Y _jm .

By the feature amount calculation unit 13, F _n (X _ik ) and F _n which are the feature amount (feature amount vector) of each image X _ik and Y _jm for each feature amount extraction method (that is, the feature amount extraction function F _n ). (Y _jm ) is obtained.

The similarity calculation unit 14 is based on the feature amount (F _n (X _ik ) and F _n (Y _jm )) for each feature amount extraction method of each image X _ik , Y _jm calculated by the feature amount calculation unit 13. , The similarity Sim _n (X _ik , Y _jm ) for each feature extraction method is calculated for each set (X _ik , Y _jm ) of the candidate spot image X _ik and the famous spot image Y _jm . Here, Sim _n (X _ik , Y _jm ) uses the feature amount F _n (X _ik ) of the candidate spot image X _ik obtained by using the feature amount extraction function F _n and the feature amount extraction function F _n . The degree of similarity between the feature amount F _n (Y _jm ) of the famous spot image Y _jm obtained in the above and the feature amount F n (Y jm) is shown.

The similarity calculation unit 14 may calculate the similarity for each feature amount extraction method by a calculation method predetermined according to the feature amount extraction method. For example, the similarity calculation unit 14 may calculate the cosine similarity between the feature amounts as the above-mentioned similarity for the feature amount obtained by the convolutional neural network and the feature amount based on the Visual Concept. Further, the similarity calculation unit 14 may calculate the L2 distance between the feature amounts as the similarity for the GIST feature amount.

The extraction unit 15 has a similarity degree for each combination of the set of the candidate spot image X _ik and the famous spot image Y _jm calculated by the similarity calculation unit 14 and the feature amount extraction method (that is, the feature amount extraction function F _n ). Based on Sim _n (F _n (X _ik ), F _n (Y _jm )), a similar pair that is a pair (X _i , Y _j ) of candidate spots X _i and famous spots Y _j that are similar to each other is extracted. A specific example of the processing of the extraction unit 15 will be described later using a flowchart.

The presentation unit 16 presents information on famous spots to the user. The presentation unit 16 presents the information of the candidate spot similar to the famous spot to the user in association with the information of the famous spot based on the information of the similar pair extracted by the extraction unit 15. The presentation unit 16 extracts, for example, a famous spot that matches the condition in response to a search request from the user (for example, an operation that requests information on a tourist spot that matches the condition input by the user). Then, the presentation unit 16 presents to the user information about the candidate spots having a similar pair relationship with the famous spots, together with the information about the extracted famous spots.

Next, an example of the operation of the server 10 will be described with reference to the flowchart of FIG.

In step S1, the first acquisition unit 11 has a plurality of candidate spots _X _i (i = 1, ..., NX) for each of the plurality of (NX in the present embodiment) candidate spots _X i from the candidate spot image DB 20. Get the image X _ik .

In step S2, the second acquisition unit 12 has a plurality of famous spots Y j (j = 1, ..., _NY ) for each of a plurality of (NY pieces in this embodiment) famous spots _Y _j (j = 1, ..., NY) from the famous spot image DB 30. Acquire the image Y _jm . Note that step S2 may be executed before step S1 or may be executed in parallel with step S1.

In step S3, the feature amount calculation unit 13 has, for each candidate spot image X _ik and each famous spot image Y _jm , the feature amount F _n (X _ik ), F for each feature amount extraction function F _n (feature amount extraction method). Calculate _n (Y _jm ).

In step S4, the similarity calculation unit 14 determines the similarity Sim _n (X _ik ,) for each feature amount extraction function F _n for each set (X _ik , Y _jm ) of the candidate spot image X _ik and the famous spot image Y _jm . Y _jm ) is calculated.

In step S5, the extraction unit 15 is similar based on the similarity Sim _n (X _ik , Y _jm ) for each feature amount extraction function F _n for each set (X _ik , Y _jm ) calculated in step S4. Extract pairs. Hereinafter, the first to third examples of the processing of the extraction unit 15 will be described with reference to FIGS. 3 to 5.

(First example)
A first example of the process of the extraction unit 15 (step S5 in FIG. 2) will be described with reference to FIG.

In step S11, the extraction unit 15 determines the total similarity Sim, which is the sum of the similarity for each feature amount extraction function F _n for each set (X _ik , Y _jm ) of the candidate spot image X _ik and the famous spot image Y _jm . Calculate _total (X _ik , Y _jm ). Here, the total similarity Sim _total (X _ik , Y _jm ) is expressed by the following equation (1).
Equation (1): Sim _total (X _ik , Y _jm ) = Sim ₁ (X _ik , Y _jm ) + ... + Sim _Nf (X _ik , Y _jm )

In step S12, the extraction unit 15 sets the total similarity Sim _total up to the top N (the top predetermined number) in the set (X _ik , Y _jm ) of all the candidate spot images X _ik and the famous spot image Y _jm . The set (X _ik , Y _jm ) to have is extracted. N is a number arbitrarily determined in advance by, for example, an operator or the like.

In step S13, the extraction unit 15 sets the candidate spot X _i and the famous spot Y _j corresponding to the pair (X _ik , Y _jm ) of the candidate spot image X _ik and the famous spot image Y _jm extracted in step S12 (the set of the candidate spot X i and the famous spot Y j). X _i , Y _j ) is extracted as a similar pair. For example, in step S12, the pair (X ₁₁ , Y ₂₃ ) of the candidate spot image X ₁₁ (the first image of the candidate spot X ₁ ) and the famous spot image Y ₂₃ (the third image of the famous spot Y ₂ ) is When extracted as a pair having a total similarity up to the top N ranks, the extraction unit 15 extracts a pair (X ₁ , Y ₂ ) of the candidate spot X ₁ and the famous spot Y ₂ as a similar pair.

According to the first example above, by considering the similarity Sim _n corresponding to each of the plurality of feature extraction functions F _n , the similarity corresponding to a single feature extraction method is used as compared with the case of using only the similarity. , It is possible to extract similar pairs with high accuracy.

(Second example)
A second example of the process of the extraction unit 15 (step S5 in FIG. 2) will be described with reference to FIG.

In step S21, the extraction unit 15 ranks up to the top N in the set (X _ik , Y _jm ) of all the candidate spot images X _ik and the famous spot image Y _jm for each feature amount extraction function F _n (upper predetermined). A set (X _ik , Y _jm ) having a similarity Sim _n of (number) is extracted. N is a number arbitrarily determined in advance by, for example, an operator or the like.

In step S22, the extraction unit 15 sets the candidate spot X _i and the famous spot Y _j corresponding to the set (X _ik , Y _jm ) of the candidate spot image X _ik and the famous spot image Y _jm extracted in step S21 (the set of the candidate spot X i and the famous spot Y j). X _i , Y _j ) are extracted as similar candidates. That is, for each of the feature quantity extraction functions F ₁ to F _Nf , a set TopN_F ₁ to TopN_F _Nf containing N similar candidates is extracted.

In step S23, the extraction unit 15 extracts a pair (X _i , Y _j ) of candidate spots X _i and famous spots Y _j extracted as similar candidates for a predetermined number or more of feature quantity extraction functions F _n as similar pairs. do. As an example, the extraction unit 15 extracts a set (X _i , Y _j ) extracted as a similar candidate for all the feature quantity extraction functions F ₁ to F _Nf as a similar pair. For example, when three feature quantity extraction functions F ₁ to F ₃ are used, a set TopN_F ₁ containing N similar candidates for the feature quantity extraction function F ₁ and N similar candidates for the feature quantity extraction function F ₂ . A set (X _i , Y _j ) included in any of the set TopN_F ₂ containing the above and the set TopN_F ₃ containing N similar candidates for the feature extraction function F ₃ is extracted as a similar pair. In other words, the pairs (X _i , Y _j ) included in the intersection (TopN_F ₁ ∩ TopN_F ₂ ∩ TopN_F ₃ ) of the sets TopN_F ₁ , TopN_F ₂ , and TopN_F ₃ are extracted as similar pairs.

According to the second example above, by considering the similarity Sim _n corresponding to each of the plurality of feature extraction functions F _n , as compared with the case where only the similarity corresponding to a single feature extraction method is used. , It is possible to extract similar pairs with high accuracy. More specifically, a pair (X _i , Y _j ) of spots (subjects) judged to be similar to each other from a plurality of viewpoints (feature amount extraction method) can be extracted as a similar pair.

(Third example)
A third example of the process of the extraction unit 15 (step S5 in FIG. 2) will be described with reference to FIG.

In step S31, the extraction unit 15 sets all the candidate spot images X _ik and the famous spot image Y _jm (X) for each combination (X _i , F _n ) of the candidate spot X _i and the feature amount extraction function F _n . A set (X _ik , Y _jm ) having a degree of similarity up to the upper N rank (upper predetermined number) among _ik , Y _jm ) is extracted.

In step S32, the extraction unit 15 is a candidate corresponding to the set (X _ik , Y _jm ) of the candidate spot image X _ik and the famous spot image Y _jm extracted in step S31 for each combination (X _i , F _n ). A set of spots X _i and famous spots Y _j (X _i , Y _j ) is extracted as a first similarity candidate. That is, for each combination (X _i , F _n ), a set TopN_F _n (X _i ) containing N first similar candidates is extracted.

In step S33, the extraction unit 15 extracts the famous spot Y _j extracted as the first similar candidate for the feature quantity extraction function F _n of a predetermined number or more for each candidate spot X _i as the second similar candidate. As an example, the extraction unit 15 uses the famous spot Y _j included in the set (X _i , Y _j ) extracted as the first similar candidate for all the feature quantity extraction functions F ₁ to F _Nf as the second similar candidate. Extract. For example, focusing on a certain candidate spot X _i , when three feature quantity extraction functions F ₁ to F ₃ are used, a set TopN_F ₁ (X _i ) containing N first similar candidates for the feature quantity extraction function F ₁ . ), The set TopN_F ₂ (X _i ) containing N first similar candidates for the feature extraction function F ₂ , and the set TopN_F ₃ (X i) containing N first similar candidates for the feature extraction function F ₃ . When a set (X _i , Y _j ) is included in any of _i ), the famous spot Y _j included in the set is extracted as a second similar candidate corresponding to the candidate spot X _i . In other words, the intersection of the sets TopN_F ₁ (X _i ), TopN_F ₂ (X _i ), and TopN_F ₃ (X _i ) (TopN_F ₁ (X _i ) ∩ TopN_F ₂ (X _i ) ∩ TopN_F ₃ (X _i ) The famous spot Y _j included in the set (X _i , Y _j ) included in is extracted as the second intersection candidate corresponding to the candidate spot X _i . In the following description, the set of the second similar candidates corresponding to a certain candidate spot X _i is referred to as g (X _i ). For example, when the famous spots Y ₂ and Y ₃ are extracted as the second similar candidate corresponding to the candidate spot X ₁ , g (X ₁ ) is a set including the famous spots Y ₂ and Y ₃ . That is, "g (X ₁ ) = {Y ₂ , Y ₃ }". By step S33, a set g (X ₁ ), ..., G (X _Nx ) of the second similar candidates corresponding to each of the candidate spots X ₁ , ..., X _Nx is obtained.

In step S34, the extraction unit 15 calculates tf (X _i , Y _j ), which is a score (first evaluation value) for each set (X _i , Y _j ) of the candidate spot X _i and the famous spot Y _j . tf (X _i , Y _j ) is a score that applies the idea of tf (Term Frequency) in tf-idf, which is one of the methods for evaluating the importance of words contained in a document.

The tf (X _i , Y _j ) for the set (X _i , Y _j ) of the specific candidate spot X _i and the specific famous spot Y _j is the second similar candidate corresponding to the specific candidate spot X _i . It is a value obtained by dividing the number of the specific famous spots Y _j included in the set g (X _i ) by the total number of famous spots included in the set g (X _i ). That is, tf (X _i , Y _j ) is expressed by the following equations (2-1) to (2-3).
Equation (2-1): tf (X _i , Y _j ) = N1 / N2
Equation (2-2): Formula of famous spot Y _j included in N1 = g (X _i ) (2-3): Total number of famous spots included in N2 = g (X _i )

For example, when g (X ₁ ) = {Y ₁ , Y ₂ , Y ₄ }, the number of famous spots Y ₁ included in g (X ₁ ) is "1" and is included in g (X ₁ ). Since the total number of famous spots is "3", "tf (X ₁ , Y ₁ ) = 1/3".

In step S35, the extraction unit 15 calculates idf (Y _j ), which is a score (second evaluation value) for each famous spot Y _j . idf (Y _j ) is a score that applies the concept of idf (Inverse Document Frequency) in tf-idf.

The idf ( _Y _j ) for the specific famous spot Y _j is the total number of candidate spots (NX in this embodiment) and the number of candidate spots including the specific famous spot Y _j as the second similar candidate (that is,). , The number of candidate spots X _i having the set g (X _i ) including the famous spot Y _j ). That is, idf (Y _j ) is expressed by the following equation (3).
Equation (3): idf (Y _j ) = total number of candidate spots / number of candidate spots X _i having a set g (X _i ) including the famous spot Y _j .

In step S36, the extraction unit 15 sets a similar pair based on tf (X _i , Y _j ) and idf (Y _j ) for each pair (X _i , Y _j ) of the candidate spot X _i and the famous spot Y _j . Extract. For example, the extraction unit 15 calculates tf-idf (X _i , Y _j ) as the final score of the set of spots (X _i , Y _j ). tf-idf (X _i , Y _j ) is expressed by the following equation (4).
Equation (4): tf-idf (X _i , Y _j ) = tf (X _i , Y _j ) × idf (Y _j )

Then, the extraction unit 15 is, for example, a set of spots (X _i , Y _j ) having a high tf-idf (X _i , Y _j ) (for example, tf-idf up to the upper M rank among all the sets of spots). A set of spots having (X _i , Y _j ) (X _i , Y _j )) is extracted as a similar pair.

According to the third example above, similar pairs can be extracted more appropriately by using a score to which the tf-idf method is applied. For example, it is possible to prevent a famous spot corresponding to a famous spot image having an average feature amount from being ranked high. An example of a famous spot image having an average feature amount is an image in which a certain degree of similarity with any candidate spot image is calculated. It is not desirable for such famous spots to be ranked high because they are not particularly similar to specific candidate spots. By using the score tf-idf (X _i , Y _j ) based on the above-mentioned tf-idf method, it is possible to prevent pairs including such famous spots from being extracted as similar pairs. Specifically, since the idf (Y _j ) of such a famous spot Y _j tends to be low, the score tf-idf (X _i , Y _j ) of the group including the famous spot Y _j should be lowered. Can be done.

Returning to FIG. 2, in step S6, in response to a search request from the user (for example, an operation of requesting information on a tourist spot that matches the condition input by the user), a famous spot that matches the condition is extracted. Then, the presentation unit 16 presents to the user information about the candidate spots having a similar pair relationship with the famous spots, together with the information about the extracted famous spots.

In the server 10 described above, all combinations of one or more candidate spot images X _ik for each of the plurality of candidate spots X _i and one or more famous spot images Y _jm for each of the plurality of famous spots Y _j . For each feature amount extraction method (feature amount extraction function F _n ), the similarity Sim _n (X _ik , Y _jm ) is calculated. Then, based on the similarity Sim _n (X _ik , Y _jm ) calculated for each combination of the candidate spot image X _ik and the famous spot image Y _jm and the feature amount extraction function F _n in this way, each other. Similar pairs that are a pair of similar candidate spots X _i and famous spots Y _j are extracted. According to the above configuration, it is not necessary to preset a threshold value for the similarity by performing a relative evaluation using the similarity between the plurality of candidate spots X _i and the plurality of famous spots Y _j . .. As a result, it is possible to prevent setting an inappropriate threshold value. Further, since it is not necessary to set a threshold value, it is possible to more easily perform the process of extracting a set of subjects similar to each other (in the present embodiment, a set of candidate spots and a set of famous spots). Therefore, according to the server 10, it is possible to easily and appropriately extract a set of subjects similar to each other.

Further, the server 10 includes a presentation unit 16. According to the above configuration, for example, it is possible to present information on a candidate spot similar to the famous spot to a user who is interested in the famous spot. This makes it possible to present the user with a new spot that the user did not know before. As a result, by guiding tourists (users) to new tourist spots (candidate spots similar to famous spots), it is possible to prevent tourists from concentrating on specific famous spots, which in turn eliminates tourism pollution and infectious diseases. It can be expected to take measures against infectious diseases and regional revitalization.

However, the server 10 is not limited to the above embodiment. For example, the server 10 does not have to include the presentation unit 16. For example, the presentation unit 16 described above may be mounted on an external server different from the server 10. In this case, if the server 10 transmits the information of the similar pair extracted by the extraction unit 15 to the external server, and the external server executes the same process as the presentation unit 16 described above based on the information of the similar pair. good. Further, in the above embodiment, a set of candidate spots and a set of famous spots is shown as an example of a set of subjects to be extracted as a similar pair, but a set of subjects to be extracted as a similar pair is described in the above example. Not limited.

The block diagram used in the explanation of the above embodiment shows a block of functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Further, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one physically or logically coupled device, or two or more physically or logically separated devices can be directly or indirectly (eg, for example). , Wired, wireless, etc.) and may be realized using these plurality of devices. The functional block may be realized by combining the software with the one device or the plurality of devices.

Functions include judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, solution, selection, selection, establishment, comparison, assumption, expectation, and assumption. There are broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc., but only these. I can't.

For example, the server 10 in one embodiment of the present disclosure may function as a computer that performs the communication control method of the present disclosure. FIG. 6 is a diagram showing an example of the hardware configuration of the server 10 according to the embodiment of the present disclosure. The server 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following explanation, the word "device" can be read as a circuit, device, unit, etc. The hardware configuration of the server 10 may be configured to include one or more of the devices shown in FIG. 1, or may be configured not to include some of the devices.

For each function in the server 10, by loading predetermined software (program) on hardware such as the processor 1001 and the memory 1002, the processor 1001 performs an operation and controls communication by the communication device 1004, or the memory 1002 and the memory 1002. It is realized by controlling at least one of reading and writing of data in the storage 1003.

The processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU: Central Processing Unit) including an interface with a peripheral device, a control device, an arithmetic unit, a register, and the like.

Further, the processor 1001 reads a program (program code), a software module, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used. For example, the extraction unit 15 may be realized by a control program stored in the memory 1002 and operating in the processor 1001, and may be realized in the same manner for other functional blocks. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be mounted by one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, or the like that can be executed to implement the communication control method according to the embodiment of the present disclosure.

The storage 1003 is a computer-readable recording medium, and is, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, an optical magnetic disk (for example, a compact disk, a digital versatile disk, a Blu-ray). It may consist of at least one (registered trademark) disk), smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing at least one of the memory 1002 and the storage 1003.

The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by the bus 1007 for communicating information. The bus 1007 may be configured by using a single bus, or may be configured by using a different bus for each device.

Further, the server 10 is configured to include hardware such as a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The hardware may implement some or all of each functional block. For example, processor 1001 may be implemented using at least one of these hardware.

Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiment described in the present specification. This embodiment can be implemented as an amendment or modification without departing from the spirit and scope of the present invention as determined by the description of the scope of claims. Therefore, the description herein is for purposes of illustration only and has no limiting implications for this embodiment.

The order of the processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in the present disclosure may be changed as long as there is no contradiction. For example, the methods described in the present disclosure present elements of various steps using exemplary order, and are not limited to the particular order presented.

The input / output information and the like may be stored in a specific location (for example, a memory) or may be managed using a management table. Information to be input / output may be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a true / false value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present disclosure may be used alone, in combination, or may be switched and used according to the execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Software, whether referred to as software, firmware, middleware, microcode, hardware description language, or other names, is an instruction, instruction set, code, code segment, program code, program, subprogram, software module. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. should be broadly interpreted.

Further, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, a website where the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL: Digital Subscriber Line), etc.) and wireless technology (infrared, microwave, etc.). When transmitted from a server or other remote source, at least one of these wired and wireless technologies is included within the definition of transmission medium.

The information, signals, etc. described in this disclosure may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

Further, the information, parameters, etc. described in the present disclosure may be expressed using an absolute value, a relative value from a predetermined value, or another corresponding information. It may be represented.

The names used for the above parameters are not limited in any respect. Further, mathematical formulas and the like using these parameters may differ from those expressly disclosed in this disclosure. The various names assigned to these various information elements are not limiting in any way, as the various information elements can be identified by any suitable name.

The statement "based on" used in this disclosure does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

Any reference to elements using designations such as "first" and "second" as used in this disclosure does not generally limit the quantity or order of those elements. These designations can be used in the present disclosure as a convenient way to distinguish between two or more elements. Therefore, references to the first and second elements do not mean that only two elements can be adopted, or that the first element must somehow precede the second element.

When "include", "including" and variations thereof are used in the present disclosure, these terms are as inclusive as the term "comprising". Is intended. Moreover, the term "or" used in the present disclosure is intended not to be an exclusive OR.

In the present disclosure, if articles are added by translation, for example, a, an and the in English, the disclosure may include the plural nouns following these articles.

In the present disclosure, the term "A and B are different" may mean "A and B are different from each other". The term may mean that "A and B are different from C". Terms such as "separate" and "combined" may be interpreted in the same way as "different".

10 ... server (information processing device), 11 ... first acquisition unit, 12 ... second acquisition unit, 13 ... feature amount calculation unit, 14 ... similarity calculation unit, 15 ... extraction unit, 16 ... presentation unit.

Claims

A first acquisition unit that acquires one or more first images for each of a plurality of first subjects,
A second acquisition unit that acquires one or more second images for each of a plurality of second subjects,
By using each of the plurality of feature amount extraction methods, a feature amount calculation unit for calculating the feature amount for each feature amount extraction method for each of the first image and each of the second images, and a feature amount calculation unit.
Based on the feature amount of each image calculated by the feature amount calculation unit for each feature amount extraction method, the similarity of each feature amount extraction method is calculated for each set of the first image and the second image. The similarity calculation unit to be calculated and
The first subject and the second subject that are similar to each other based on the similarity for each combination of the first image and the second image set and the feature amount extraction method calculated by the similarity calculation unit. An extraction unit that extracts similar pairs that are a set of subjects,
Information processing device equipped with.
The extraction unit
For each set of the first image and the second image, the total similarity, which is the sum of the similarity for each feature extraction method, is calculated.
Among all the sets of the first image and the second image, the set of the first image and the second image having the total similarity of the upper predetermined number is extracted.
The information processing apparatus according to claim 1, wherein the pair of the first subject and the second subject corresponding to the extracted pair of the first image and the second image is extracted as the similar pair.
The extraction unit
For each feature amount extraction method, a set of the first image and the second image having a predetermined number of similarities is extracted from all the sets of the first image and the second image.
For each feature amount extraction method, the pair of the first subject and the second subject corresponding to the extracted set of the first image and the second image is extracted as similar candidates.
The information processing apparatus according to claim 1, wherein the pair of the first subject and the second subject extracted as the similarity candidates for the feature amount extraction method of a predetermined number or more is extracted as the similarity pair.
The extraction unit
For each combination of the first subject and the feature amount extraction method, the first image and the second image having a predetermined number of similarities among all the sets of the first image and the second image. Extract the pair of
For each combination of the first subject and the feature amount extraction method, the set of the first subject and the second subject corresponding to the extracted set of the first image and the second image is the first similar candidate. Extracted as
For each of the first subjects, the second subject extracted as the first similarity candidate for the feature amount extraction method having a predetermined number or more is extracted as the second similarity candidate.
The first evaluation value for each set of the first subject and the second subject and the second evaluation value for each second subject are calculated.
The similar pair was extracted based on the first evaluation value and the second evaluation value for each pair of the first subject and the second subject.
The first evaluation value for a set of a specific first subject and a specific second subject is the number of the specific second subjects included in the set of the second similar candidates corresponding to the specific first subject. Is divided by the number of the second subjects included in the set.
The second evaluation value for a specific second subject is a value obtained by dividing the total number of the first subjects by the number of the first subjects including the specific second subject as the second similar candidate. Item 1. The information processing apparatus according to item 1.
A presentation unit that presents information on the second subject to the user is provided.
Based on the information of the similar pair extracted by the extraction unit, the presentation unit presents the information of the first subject similar to the second subject to the user in association with the information of the second subject. The information processing apparatus according to any one of claims 1 to 4.