JP7454893B1 - Similar content management system, similar content management method, and similar content management program - Google Patents

Abstract

[Problem] To develop a technology that can efficiently extract content such as avatars and icons that have the characteristics of real people, and easily and accurately manage information such as the presence or absence of a relationship between the content and the real person. provide. [Solution] A content input unit 1 for inputting target content whose relevance to the person is to be determined, a person image input unit 2 for inputting the person's image that is the person's face image, and an average a reference image input unit 5 that inputs a reference image that is a face image; a feature point extraction unit 6 that extracts feature points of the person's image and target content; and a feature point extraction unit 6 that extracts feature points of the person's image and target content, and It includes a feature point comparison unit 7 that compares contents, and a relationship determination unit 8 that determines the relationship between the subject content and the person whose appearance is similar to the person's image. [Selection diagram] Figure 1

Description

The present invention is a technology that can efficiently extract content such as avatars and icons that have the characteristics of a real person, and easily and accurately manage information such as the presence or absence of a relationship between the content and the real person. It is related to.

2. Description of the Related Art In recent years, with the improvement in processing power of electronic computers and the like, two-dimensional and three-dimensional computer graphics have been widely used in IT services such as games and SNS (Social Networking Services). These services not only use icons, avatars, etc. that imitate fictional characters, but also avatars, etc. that realistically represent real people such as users, and abstract images based on the characteristics of the person. Increasingly, avatars and the like are used. This has the effect of improving the user's sense of immersion in the game world, for example, when the main character, who is an avatar that represents the user's characteristics, fights against enemy characters in a game, and the icon representing the user on SNS. By using avatars that abstractly represent the users themselves, it is expected that interactions between users will be promoted in a virtual space that feels similar to the real world. Ru.

Patent Documents 1 and 2 both disclose technology for using avatars that imitate the actual appearance of the player himself or a co-player, respectively, in a computer game that uses a head-mounted display to represent a virtual space. .

JP 2019-012509 Publication JP 2019-139673 Publication

However, since there are various types of services such as games and SNS, and each service has a different world view and character design, users need to prepare avatars with different designs for each service. There is. Users need to manage these many avatars, such as the correspondence with each service and what the latest version is in case of repeated modifications, but regarding the technology to manage multiple avatars, etc. Both Patent Documents 1 and 2 do not disclose this at all.

Furthermore, it is important for a third party to be able to distinguish whether an avatar or the like having the characteristics of a specific user is an avatar or the like that is actually used by the user. For example, an avatar, etc. with the characteristics of a celebrity is a person impersonating the celebrity, and there is a concern about fraud, etc. by the person, so an avatar, etc. with the characteristics of a certain person is associated with the person in question. It is desirable to build a system that allows even a third party other than the person concerned to recognize whether or not the person in question is. However, such technology is not disclosed in Patent Documents 1 and 2.

The present invention has been made in view of the above-mentioned problems, and it efficiently extracts content such as avatars and icons that have the characteristics of a real person, and determines whether or not there is a relationship between the content and the real person. The purpose is to provide technology that allows information management to be performed easily and accurately.

In order to achieve the above object, the similar content management system according to claim 1 manages information regarding the relationship between target content that has external similarity to the person's image, which is a face image of the person. A similar content management system comprising: a feature point extracting means for extracting feature points corresponding to each element constituting the face from a facial image of the target content; and the target content extracted by the feature point extracting means. feature point comparison means for determining the presence or absence of external similarity between the target content and the person's image based on a comparison result between the feature point of the person's image and the feature point of the person's image that has a corresponding relationship with the feature point; and a relationship determining means for determining a relationship between the subject content and the subject content determined by the feature point comparing means to have the same appearance as the subject's image, and non-substitutability having a corresponding relationship with the subject content. information generating means for generating identification information of the target content and information regarding the relationship with the principal, which is recorded in a distributed ledger on a distributed network corresponding to the principal token, which is a token, and information generated by the information generating means; and an output instruction means for instructing the decentralized network to output the information obtained.

Further, in order to achieve the above object, in the similar content management system according to claim 2, in the above invention, the feature point comparing means compares the facial part of the target content at one or more feature points to be compared. determining that the target content has external similarity to the person's image when a distance between a position of a feature point in the image and a position of the feature point in the person's image is smaller than a first threshold; It is characterized by

Further, in order to achieve the above object, in the similar content management system according to claim 3, in the above invention, the feature point comparison means uses a face image as a reference in one or more feature points to be compared. a first vector whose starting point is the position of the feature point in the reference image and whose end point is the position of the feature point in the personal image; and a first vector whose starting point is the position of the feature point in the reference image and whose end point is the position of the feature point in the target content. If the value of the inner product with the second vector is greater than or equal to a second threshold value, it is determined that the target content has an appearance similarity with the person's image.

In addition, in order to achieve the above object, the similar content management method according to claim 4 provides information regarding the relationship between target content that has an appearance similarity to the person's image, which is a face image of the person. A similar content management method for managing similar content, comprising: a feature point extraction step of extracting feature points, which are points corresponding to each element constituting the face, from a facial image of a target content; A feature point that determines whether there is an external similarity between the target content and the person's image based on a comparison result between the feature point of the target content and a feature point of the person's image that has a corresponding relationship with the feature point. a comparison step, a relationship determination step of determining a relationship between the subject content and the subject content that was determined to have the same appearance as the person's image in the feature point comparison step; an information generation step of generating identification information of the target content and information regarding the relationship with the principal, which is recorded in a distributed ledger on a distributed network corresponding to the principal token, which is a fungibility token; and the information generating step. and an output instruction step of instructing the distributed network to output the information generated in the step.

In addition, in order to achieve the above object, the similar content management method according to claim 5 provides a method for managing feature points in the reference image, which is a reference face image, from among the one or more feature points; further comprising a feature point selection step of selecting a feature point for which a distance between the feature point and the corresponding feature point position in the person's image is equal to or greater than a third threshold, and in the feature point comparison step, the feature point selection step The method is characterized in that the feature points selected in step are compared.

Further, in order to achieve the above object, the similar content management program according to claim 6 provides information regarding the relationship between the subject content and the subject content that has an appearance similarity to the subject's image, which is the face image of the subject. A similar content management program that causes a computer to manage the content, the program having a feature point extraction function that causes the computer to extract feature points, which are points corresponding to each element constituting the face, from a face image of the target content; Based on the comparison result between the feature points of the target content extracted by the feature point extraction function and the feature points of the person's image that correspond to the feature points, the appearance difference between the target content and the person's image is determined. a feature point comparison function that determines the presence or absence of similarity; and a relationship determination function that determines the relationship between the person and the target content that is determined by the feature point comparison function to have the same appearance as the person's image. , generating identification information of the target content and information regarding the relationship with the principal, which is recorded in a distributed ledger on a distributed network corresponding to the principal token, which is a non-fungible token that has a correspondence with the principal; The present invention is characterized in that an information generation function and an output instruction function for instructing the distributed network to output the information generated by the information generation function are executed.

According to the present invention, content such as avatars and icons that have the characteristics of a real person can be efficiently extracted, and information such as the presence or absence of a relationship between the content and the real person can be easily and accurately managed. play.

1 is a schematic diagram showing the configuration of a similar content management system according to Embodiment 1. FIG. 7 is a flowchart illustrating a process for determining whether or not there is external similarity between target content and a person's image in the similar content management system according to the first embodiment. FIG. 2 is a schematic diagram showing the configuration of a similar content management system according to a second embodiment.

Hereinafter, embodiments of the present invention will be described in detail based on the drawings. The following embodiments describe examples that are considered to be the most appropriate embodiments of the present invention, and it goes without saying that the content of the present invention is not limited to the specific examples shown in this embodiment mode. It should not be interpreted in a limited manner. It goes without saying that even configurations other than the specific configurations shown in the embodiments are included in the technical scope of the present invention as long as they provide similar functions and effects.

(Embodiment 1)
First, a similar content management system according to the first embodiment will be described. As shown in FIG. 1, the similar content management system according to Embodiment 1 includes a content input unit 1 for inputting target content to be determined as to its relevance to the person, and a face image of the person. A personal image input unit 2 for inputting an image, a personal information input unit 3 for inputting personal information such as a name for identifying the person, and a one-to-one correspondence with the input personal image. A principal token generation unit 4 that generates a principal token that is a non-fungible token, a reference image input unit 5 that inputs a reference image that is an average face image, and a feature point that extracts feature points of the principal image and target content. The relationship between the extraction unit 6, the feature point comparison unit 7 that compares the person's image and the target content based on information such as the position of the extracted feature points, and the person whose target content has similarity in appearance with the person's image. a relationship determination unit 8 that determines gender, a transaction generation unit 9 that generates a transaction that includes information regarding the relationship between the target content and the person, and data that proves that the content of the transaction is in accordance with the person's intention. The present invention includes an electronic signature generation unit 10 that generates an electronic signature, and an output instruction unit 11 that instructs the distributed network to output the transaction and the electronic signature in correspondence with the person's token.

The content input unit 1 is for inputting target content whose relevance to the user is to be determined in the similar content management system according to the first embodiment. Specifically, the content input unit 1 inputs all kinds of video content (with no audio), such as photos and videos, as long as it includes images of people's faces, in addition to icons and avatars used in services such as games and SNS. It has a function to input content (including accompanying content) as target content.

The person's image input unit 2 is for inputting a person's image, which is an image including a face image of the person whose relevance with the target content is determined. Specifically, the personal image input unit 2 accepts any image, regardless of whether it is a moving image or a still image, or whether it is a two-dimensional or three-dimensional image, as long as it is clear enough to extract facial feature points (described later). It has an inputable configuration. As a more preferable aspect, it is preferable that the expression mode matches the expression mode of the target content, and it is also possible to input a plurality of personal images with different expression modes in accordance with various expression modes.

The personal information input section 3 is for inputting information for identifying the person, such as the person's name and date of birth. The contents of the personal information input through the personal information input section 3 are stored in a form linked to the personal token.

The principal token generation unit 4 is for generating principal tokens that are non-fungible tokens that have a one-to-one correspondence with the principal. "Non-fungible token" refers to a so-called NFT (Non-Fangible Token), a token that has unique data and cannot be replaced with other tokens, such as the Ethereum (registered trademark) standard. It is issued based on ERC721. The principal token in Embodiment 1 is issued based on ERC721 or other predetermined standards, and has a configuration in which the transaction history and other information of the token is stored on the blockchain.

"Blockchain" is a technology that uses cryptographic technology to synchronize data between multiple computers that make up a distributed network. Specifically, each block is composed of a collection of information related to the principal's token, such as agreed transaction records, and information for connecting with other blocks (information of the previous block), and each block is made up of multiple linked blocks. A blockchain is constructed by Even if data is tampered with on some of the computers, the correct data is selected by majority vote among the other computers, making it extremely difficult to destroy or tamper with the data.

The correspondence between the principal token and the principal information is the principal token's identifier and the principal's identification information (information that identifies the principal such as name and date of birth may be used, or commonly used identification information such as My Number). The format is one-to-one linking. More directly, the identifier of the principal token and the identification information of the principal may have the same content, or there may be no problem in a mode in which a correspondence relationship is set between the identifiers of the principal token and the identification information of the principal. Alternatively, the information may be linked to URL information on the Internet where the personal information is stored.

In addition, in the first embodiment, the blocks constituting the distributed ledger that correspond to the principal tokens record transaction records related to the transfer of the ownership name of the principal token, as well as information regarding the relationship between the target content and the principal. shall be carried out. The relationship between the target content and the person is determined by a relationship determining section 8, which will be described later, and specific information is generated by a transaction generating section 9.

The generation of the principal token may be performed by the principal token generation section 4 itself, or may be generated by an external system by issuing a predetermined command to an external system directly or indirectly connected to the principal token generation section 4. It may also be a form that is generated by the system. In addition, the specific format of the principal token is not limited to one that is compliant with ERC721, the Ethereum (registered trademark) standard, and is non-fungible and whose transaction history is stored on the blockchain. It may be in any format.

The reference image input unit 5 is for inputting a reference image that is an average face image. Specifically, the reference image input unit 5 may have a function of simply inputting an image to be a given reference image, or may have a function of inputting an unspecified number of face images and then generating the reference image. It is also possible to have the function of In addition, as a function of the reference image input unit 5, reference images corresponding to different attributes (such as reference images for men in a certain age group) are input and generated in advance, and the user's attributes are selected from among the plurality of reference images. It is also possible to select a reference image that matches or is similar to the reference image.

The feature point extraction unit 6 is for extracting feature points in a face image. "Feature points" refer to points corresponding to each element that makes up the face, for example, each organ such as the eyes, nose, mouth, ears, etc., more specifically, parts of each organ such as the inner and outer corners of the eyes, pupils, Points corresponding to the tip of the nose, the corners of the mouth, etc. In addition, feature points are not necessarily defined only for parts that have characteristics in appearance, but for example, parts that move characteristically when facial expressions change due to facial muscles or skeletal structure, etc. It is also possible to include portions that have no external features.

As a specific configuration of the feature point extracting section 6, for example, an image recognition technique realized by deep learning, machine learning, etc. may be used, or a plurality of image recognition techniques may be combined. Further, for example, when the target content is composed of three-dimensional computer graphics, the feature point extraction unit 6 may extract those corresponding to the feature portions from among the vertices on the surface subjected to the modeling process. Furthermore, the feature point extracting unit 6 extracts the position and semantic content (for example, that the feature point corresponds to the inner corner of the eye) for each feature point, but instead of the semantic content, the feature point A predetermined identification code (for example, a for the feature point corresponding to the center point of the pupil of the left eye and b for the feature point corresponding to the apex of the nasal tip) may be attached to each point. The semantic content or identification code is used to clarify the correspondence of each feature point between the target content, reference image, and person's image, and other information may be added if the correspondence can be clarified. It may also be a thing.

The feature point comparison unit 7 is for comparing the feature points of the target content and the person's image extracted by the feature point extraction unit 6 with each other. Specifically, the feature point comparison unit 7 has a corresponding relationship with the position derivation unit 13 that derives the positions of the feature points of the target content and the person's image, and each feature point and each feature point in the target content and the person's image. a distance derivation unit 14 that derives the distance between the feature points of the reference image; a direction derivation unit 15 that derives the displacement direction of each feature point from the corresponding feature point in the reference image; a position of each derived feature point; It includes a similarity determination unit 16 that compares the distance and direction and determines the similarity in appearance between the target content and the person shown in the person's image.

The position deriving unit 13 is for deriving the positions of the feature points of the target content and the person's image. The position information of each feature point derived by the position derivation unit 13 is output to the similarity determination unit 16, and the similarity determination unit 16 determines whether the positional deviation between the corresponding feature points is a predetermined threshold value (claims (corresponding to the first threshold value), it is determined that the positions of the corresponding feature points are the same.

In Embodiment 1, the positions of the feature points are normalized for each face image of the target content and the person's image (for example, the volume of the face part for a three-dimensional image, and the volume of the face part for a two-dimensional image). (The area is enlarged or reduced so that each area becomes equal to a certain reference value.) Then, the area is specified based on a predetermined origin and coordinate system. Furthermore, in the first embodiment, the same normalization process, origin, and coordinate system are used in all feature point extractions. The positions of feature points specified based on such criteria are defined as "positions of feature points in the image of the face part of the target content", "positions of the feature points in the person's image", and "features in the reference image" in the claims. This corresponds to an example of "point position".

The distance derivation unit 14 is for deriving the distance between each feature point in the person's image and the target content and the corresponding feature point in the reference image. Specifically, the distance derivation unit 14 derives the distance between the corresponding feature points, for example, the feature point corresponding to the inner corner of the right eye, in each of the person's image and the reference image, and also calculates the distance between the feature points in the person's image and the reference image. It has a function of deriving the distance between the feature points in each of the and reference images. For example, if the position coordinates of a feature point in the person's image are (x ₁ , y ₁ , z ₁ ) and the position coordinates of the corresponding feature point in the reference image are (x ₀ , y ₀ , z ₀ ), both feature points The distance between them is {(x ₁ −x ₀ ) ² +(y ₁ −y ₀ ) ² +(z ₁ −z ₀ ) ² } ^1/2 . In the first embodiment, it is assumed that the reference image is also normalized in the same way as the target content/person image, and the positions of feature points are expressed using a common origin and coordinate system. The information regarding the distance between feature points derived by the distance deriving unit 14 is output to the similarity determining unit 16, and is used in combination with information regarding the orientation of the feature points, which will be described later, for determining similarity.

The orientation deriving unit 15 is for deriving, for each feature point in the person's image and the target content, the direction of a line segment connecting each feature point to a feature point in a reference image that has a corresponding relationship. Specifically, the orientation deriving unit 15 derives, for each feature point in the person's image and the target content, the orientation of a line segment formed between each feature point and the corresponding feature point of the reference image. Has a function. For example, if the positional coordinates of a feature point in the person's image are (x ₁ , y ₁ , z ₁ ) and the positional coordinates of the corresponding feature point in the reference image are (x ₀ , y ₀ , z ₀ ), the ^The orientation of the line segment formed by is (x ₁ - x ₀ , y ₁ - y ₀ , z ₁ - _{z 0} ) _/ {( _x ¹ _{- 1} - z ₀ ) ² } ^1/2 . The information regarding the inter-feature point orientation derived by the orientation deriving section 15 is output to the similarity determining section 16, and is used in combination with the information regarding the inter-feature point distance derived by the distance deriving section 14 for similarity determination. More specifically, the similarity determination unit 16 performs the determination process using a vector that is a combination of distance and direction. That is, the first vector (x ₁ - x ₀ , y ₁ - y ₀ , z ₁ -z ₀ ) whose starting point is the feature point of the reference image and the end point is the feature point of the person's image, and Determination processing is performed based on a second vector (x ₂ −x ₀ , y ₂ −y ₀ , z ₂ −z ₀ ) whose starting point is the feature point (x ₂ , y ₂ , z ₂ ) of the target content. .

The similarity determining unit 16 determines whether each feature point is indicated by the target content or the person's image, based on the position of each feature point in the target content and the person's image, and the distance and orientation between each feature point and the feature point of the reference image. This is for determining the presence or absence of similarity between persons. Specifically, the similarity determination unit 16 determines that the positional deviation of each feature point of the target content from the corresponding feature point of the person's image is a predetermined threshold ("first threshold" in the claims). ), it is determined that the positions of the corresponding feature points are the same. Further, based on the derivation results of the distance deriving unit 14 and the direction deriving unit 15, the similarity determining unit 16 generates a first vector having the feature point on the reference image as the starting point and the feature point on the person's image as the end point, and the reference A second vector is generated with the feature point on the image as the starting point and the feature point on the target content as the end point, and the inner product of the first vector and the second vector is a constant threshold (“second threshold ), it is determined that the target content and the person's image have the same specificity with respect to the reference image. When it is determined that the feature points of the target content to be subjected to similarity determination are the same in position as the feature points in the person's image or the specificity of the feature points with respect to the reference image is the same, the similarity determination unit 16 It is determined that the person represented by the content is similar in appearance to the person shown in the person's image.

Note that the similarity determination unit 16 determines the similarity of the specificity of corresponding feature points with respect to the reference image based on the distance between feature points between the target content and the reference image and the features between the person's image and the reference image. The difference value of the distance between points is less than a predetermined threshold, and the line segment connecting the feature point of the target content and the feature point of the reference image and the line segment connecting the feature point of the person's image and the feature point of the reference image If the angles formed are less than a predetermined threshold, it may be determined that the feature points have the same specificity.

The relationship determination unit 8 determines whether the target content determined by the similarity determination unit 16 to have a similar appearance to the person's image has a relationship with the person shown in the person's image, that is, whether the content is related to the person concerned. For example, when the target content is a person's icon, avatar, etc., this is to determine whether or not it is the person's icon, avatar, etc.

Specifically, the relationship determination unit 8 stores information regarding the target content that has been determined to have similar appearance to the personal information input through the personal information input unit 3 (including the generation process and usage of the target content). content, characteristics of the service in which the target content is used, etc.), and if the target content matches the personal information (for example, the name, gender, age, birthplace, etc. of the user or creator of the target content) If the attribute information matches or resembles the personal information more than a certain number, etc.), it is presumed that there is a relationship, and if it does not match the personal information (if there is a contradiction in important information, or if the attribute information differs more than a certain number) , etc.) are determined to have no relationship. The relationship determining unit 8 outputs information to an information device such as a portable terminal managed by the person using the person's information regarding the target content estimated to be related, and based on the content of the response from the information device. Make the final judgment as to whether or not there is a relationship with the person. The response from the information device may be configured to automatically output an answer regarding the presence or absence of a relationship based on sensitive information of the person recorded in the information device, or a response from the person who has confirmed the information regarding the target content. A configuration may also be adopted in which answers are generated and output through operations.

The transaction generation unit 9 has a function of generating transactions to be stored in blocks in the distributed ledger, and corresponds to an example of information generation means in the claims. Specifically, the transaction generation unit 9 includes, as the content of the transaction, information related to the change in the name of the owner of the person's token, as well as information regarding the target content that has been determined by the similarity determination unit 16 to be similar in appearance to the person himself/herself. It is assumed that information regarding the determination result by the relationship determination unit 8 is generated. The transaction generation unit 9 generates information for identifying the target content that is determined to have similarity in appearance to the principal (if a non-fungible token that corresponds one-to-one with the target content is generated, the token is It has a function to generate a transaction that includes identifying information (in addition, it may also be the URL information of the website where the target content is recorded, the identification code of the target content, etc.) and information regarding the presence or absence of a relationship with the person.

The electronic signature generation unit 10 has a function of generating an electronic signature, which is data that proves that the creation and output of information included in a transaction is in accordance with the intention of the principal token holder. Specifically, the electronic signature generation unit 10 generates a hash value of the transaction generated by the transaction generation unit 9, and encrypts the hash value with a private key held by the holder of the principal token. It has a function to generate signatures. Note that the "hash value" is a fixed-length value obtained by performing a certain calculation procedure on the original data. Since the calculation procedure is irreversible, it is considered impossible to restore the original data from the hash value. Further, the "private key" is a sequence of numbers used for encryption processing, and is configured to be able to be decrypted using the corresponding "public key." Note that in the first embodiment, the transaction generation unit 9 and the electronic signature generation unit 10 are configured to independently create transactions and electronic signatures, which are one aspect of the retained name change information, but the present invention is not limited to this. , the transaction generation unit 9 and the electronic signature generation unit 10 may have the function of only instructing a predetermined external device to create a transaction and an electronic signature.

The output instruction unit 11 is for instructing the distributed network associated with the principal token to output the transaction generated by the transaction generation unit 9 and the electronic signature generated by the electronic signature generation unit 10. It is. Note that in the first embodiment, the output instruction section 11 is also provided with an output function itself, and specifically, the output instruction section 11 is connected directly or indirectly to the distributed network, and is connected to the distributed network. On the other hand, it is configured in such a manner that it can output predetermined data. The decentralized network generates a hash value of the output transaction, and compares the digital signature with the data (= hash value of the transaction) decrypted using the public key, and if the two values are different, the target tangible object is It is determined that the transaction is not by the owner of the transaction, and registration is rejected. If the two match, the transaction is determined to be by the owner, and the information constituted by the transaction is stored in the blocks that constitute the distributed ledger. The transaction output by the output instruction unit 11 includes information regarding the change of ownership name, and by outputting the transaction to the distributed network, this information is transferred to the distributed ledger provided on the distributed network. is stored in the blocks that make up the .

Next, the operation of the similarity determination unit 16 in the similar content management system according to the first embodiment will be explained. As shown in FIG. 2, the similarity determining unit 16 determines the distance between the two feature points based on the positions of the feature points of the target content and the positions of the feature points of the person's image, which are in a corresponding relationship with each other, derived by the position deriving unit 13. A distance is derived (step S101), and it is determined whether the distance is greater than or equal to a predetermined threshold _L0 (step S102). If it is less than the threshold L ₀ (step S102, No), the process moves to step S106, and if it is greater than or equal to the threshold L ₀ (step S102, Yes), the process moves to step S103.

Thereafter, the similarity determination unit 16 uses the feature points on the reference image as a starting point, based on the derivation results of the distance derivation unit 14 and direction derivation unit 15 regarding the feature points that correspond to each other in the person image and the reference image, and determines whether the person A first vector whose end point is a feature point on the image is generated (step S103). Further, the similarity determining unit 16 uses the feature points on the reference image as a starting point, based on the derivation results of the distance deriving unit 14 and the orientation deriving unit 15 regarding the feature points that correspond to each other in the target content and the reference image, and A second vector whose end point is a feature point on the content is generated (step S104).

After that, the similarity determination unit 16 calculates the inner product of the first vector and the second vector, and determines whether the calculation result is equal to or greater than a predetermined threshold value _M0 (step S105). If it is less than the threshold M ₀ (step S105, No), the similarity determination unit 16 determines that the target content has no similarity in appearance to the person's image (step S106), and ends the process. If the threshold value M is greater than or equal to ₀ (step S105, Yes), it is determined whether the processing has been completed for all feature points (step S107), and if it has not been completed (step S107, No), the process returns to step S101. The above process is repeated for other feature points. If it has been completed (Step S107, Yes), the similarity determining unit 16 determines that the target content has similarity in appearance to the person's image (Step S108), and ends all processing.

Next, advantages of the similar content management system according to the first embodiment will be explained. First, in the similar content management system according to the first embodiment, the relationship determining unit 8 determines the relationship between the target content and the person who has an appearance similarity to the person's image, and determines the relationship between the target content and the target content. It has a configuration that generates information including determination results regarding the relationship between the person and the person as a transaction. By adopting such a configuration, the user and other service users will be able to determine whether the target content, which has a similar appearance to the user, has a relationship with the user, or whether it is a coincidence or a third party has intentionally caused confusion. It has the advantage of being able to accurately recognize whether it is an unrelated object that was merely aimed at.

In addition, in determining the external similarity between the target content and the person's image, the similar content management system according to the first embodiment determines that the positions of feature points that correspond to each other in both are within the range of similarity. It has a configuration for determining whether or not. Since feature points are arranged corresponding to the shape of each part on the face, it is determined that the positional deviation of the corresponding feature points is within a predetermined threshold and within the range of similarity. In this case, similarity is also recognized in the shape of each part on the face corresponding to the feature point. By adopting such a configuration, the similar content management system according to the first embodiment has the advantage of being able to easily and quickly determine the similarity in appearance between the person's image and the target content.

Further, in the similar content management system according to the first embodiment, a first vector whose starting point is the feature point of the reference image and an end point of the feature point of the person's image, and a first vector whose starting point is the feature point of the reference image and an end point of the feature point of the person's image, are created between the feature points that have a correspondence relationship with each other. A second vector is set with the feature point as the starting point and the feature point of the target content as the end point, and when the inner product of the first vector and the second vector is a value greater than or equal to a predetermined threshold, It has a configuration in which it is determined that the shapes of various parts on the face have similarity.

If the inner product is a value greater than a predetermined threshold, the length of both vectors must be greater than a certain value, and the angle formed by both vectors is a small value. In some cases, the shape of the part on the face corresponding to the feature point will be significantly different from the reference image. In addition, if the angle formed by both vectors is a small value, the feature points of the person's image and the feature points of the target content are moving in similar directions compared to the feature points of the reference image. , in comparison with the facial area in the reference image, the facial area in the person's image and the facial area in the target content are different in shape to the same degree, that is, they have similar characteristic shapes. In this first embodiment, even in such a case, by adopting a configuration in which it is determined that the shape of each part on the face corresponding to the feature point has similarity in appearance, it is possible to This method has the advantage of being able to accurately determine the similarity of characteristic parts. This advantage lies not only in calculating the inner product of vectors, but also in recognizing similarity when the angle formed by both vectors is less than a certain threshold and the difference in length of both vectors is less than a certain threshold. The same problem occurs when

Furthermore, the similar content management system according to the first embodiment has a configuration in which information regarding the relationship between the target content and the principal is generated as a transaction, and is recorded in a block constituting a distributed ledger that corresponds to the principal token. have By adopting such a configuration, it is possible to prevent records from being altered or forged by a third party, and third parties such as the user and other service users can check the relationship between the target content and the user. It becomes possible to obtain accurate information about

(Embodiment 2)
Next, a similar content management system according to a second embodiment will be described. In the second embodiment, components having the same names and the same reference numerals as those in the first embodiment are assumed to exhibit the same functions as the components in the first embodiment, unless otherwise specified.

FIG. 3 is a schematic diagram showing the configuration of a similar content management system according to the second embodiment. As shown in FIG. 3, the similar content management system according to the second embodiment compares the person's image and the reference image, and identifies feature points at unique positions in the person's image based on the distances between corresponding feature points. a feature point selection unit 18 to select, and a position derivation unit 20 that derives positions in the target content, personal image, and reference image for feature points that have a correspondence relationship with the feature points selected by the feature point selection unit 18; A distance derivation unit 21 that derives a distance, an orientation derivation unit 22 that derives an orientation, and a similarity determination unit that determines whether there is external similarity between the target content and the person's image based on the derived position, distance, and orientation. 23.

The feature point selection unit 18 is for extracting feature points located at unique positions in the person's image from among a plurality of feature points set for the face image. Specifically, the feature point selection unit 18 selects a position that is unique compared to the position of a corresponding feature point in the reference image, specifically, a predetermined threshold value (“third threshold value” in the claims). It has the function of extracting feature points located at a distance of more than

Since facial parts such as eyes, nose, and mouth in the reference image have an average shape, the positions of the feature points arranged corresponding to these parts exist at average positions. Therefore, if the position of a feature point in the person's image is significantly different from that in the reference image, the facial region related to the feature point will have a shape that is significantly different from that in the reference image, and this different shape will affect the person's physical appearance. Becomes a feature. In the second embodiment, the feature point selection unit 18 extracts feature points corresponding to such physical features, and the feature point comparison unit 19 compares the feature points with the target content.

Next, advantages of the similar content management system according to the second embodiment will be explained. The similar content management system according to the second embodiment allows the feature point comparison unit 19 to compare only the feature points selected by the feature point selection unit 18 out of a large number of feature points, thereby quickly and accurately responding to the actual situation. This makes it possible to determine the visual similarity. The feature point selected by the feature point selection unit 18 is a feature point on the person's image that is located at a position deviated from the feature point in the reference image, and the feature point is different from the reference image in the person's image. This corresponds to the characteristic part. Therefore, by performing similarity determination on such feature points, it becomes possible to perform similarity determination on the characteristic portions of the person's face image. In content that is modeled after a person, such as a portrait, creative content is created while maintaining the physical characteristics of the person (for example, maintaining the features of the eyes and nose, but matching the drawing style of a specific creator). (e.g., creating a facial image with a different shape, or creating a full-body image with the appearance of another animal while maintaining its features, etc.) In such a case, if all feature points are compared, similarity may be denied based on the difference between the creative part of the target content and the person's image, but in the second embodiment, the features in the person's image By comparing feature points only with respect to feature points corresponding to parts, there is an advantage that it is possible to accurately determine the presence or absence of external similarity.

Further, in the similar content management system according to the second embodiment, the amount of information processing is reduced by adopting a configuration in which comparison processing is performed only on feature points selected by the feature point selection unit 18 instead of all feature points. This has the advantage of being significantly reduced. As a result, the similar content management system according to the second embodiment has the advantage of being able to easily and quickly determine whether there is any similarity in appearance.

Although the content of the present invention has been described above in Embodiments 1 and 2, the technical scope of the present invention should not be construed as being limited to the specific configuration described in the embodiments. It goes without saying that various modifications and applications of the above embodiments fall within the technical scope of the present invention as long as they can realize the functions of the present invention.

For example, the similar content management system according to the first and second embodiments uses both the distance between the corresponding target content and the feature points of the person's image and the inner product of the first vector and the second vector to determine the appearance. However, the visual similarity is determined only by the distance between feature points or the inner product of the first vector and the second vector (or the angle formed by both vectors and the difference in the length of the vectors). It is also possible to determine.

Furthermore, the similar content management systems according to the first and second embodiments adopt a configuration in which the personal image and the reference image are respectively input, and the feature points of both images are extracted by the feature point extraction unit 6, but the image input is not required. A configuration may be adopted in which only feature points extracted in advance are input and saved. Even if such a configuration is adopted, the function of determining the similarity in appearance does not deteriorate, but rather has the advantage that the amount of data to be stored can be reduced.

Furthermore, in the similar content management systems according to the first and second embodiments, regarding the feature point comparison processing by the feature point comparison units 7 and 19, a threshold value regarding the distance between feature points and an inner product of the above-mentioned first vector and second vector are used. Regarding the threshold values for each feature point, instead of setting them to a uniform value, a different threshold value may be determined for each feature point. By adopting such a configuration, it is possible to perform more accurate similarity judgments by setting the threshold to a small value for parts such as eyes where even a slight difference can completely change the impression of the shape. .

Further, in the similar content management system according to the first and second embodiments, a distributed ledger that is in correspondence with the principal token is used as an object for recording information regarding the relationship between the principal and the target content that is similar in appearance to the principal's image. Instead, it is also possible to generate a non-fungible token that is associated with personal information regardless of the personal image and record it in a distributed ledger that has a corresponding relationship with the non-fungible token. Furthermore, a non-fungible token may be generated for each target content, and information regarding the relationship with the principal may be recorded in a distributed ledger corresponding to each non-fungible token.

The present invention is a technology that can efficiently extract content such as avatars and icons that have the characteristics of a real person, and easily and accurately manage information such as the presence or absence of a relationship between the content and the real person. It is available as.

1 Content input section 2 Personal image input section 3 Principal information input section 4 Principal token generation section 5 Reference image input section 6 Feature point extraction section 7, 19 Feature point comparison section 8 Relationship determination section 9 Transaction generation section 10 Electronic signature generation section 11 Output instruction section 13, 20 Position derivation section 14, 21 Distance derivation section 15, 22 Orientation derivation section 16, 23 Similarity determination section 18 Feature point selection section

Claims (6)

A similar content management system that manages information regarding the relationship between target content that has external similarity to a person's image, which is a face image of the person, and the person,
Feature point extraction means for extracting feature points, which are points corresponding to each element constituting the face, from the facial image of the target content;
Based on the comparison result between the feature points of the target content extracted by the feature point extracting means and the feature points of the person's image that correspond to the feature points, the difference in appearance between the target content and the person's image is determined. feature point comparison means for determining the presence or absence of similarity;
relationship determination means for determining the relationship between the subject content and the person, which is determined by the feature point comparison means to have the same appearance as the image of the person;
Information that generates identification information of the target content and information regarding the relationship with the principal, which is recorded in a distributed ledger on a distributed network corresponding to the principal token, which is a non-fungible token that corresponds to the principal. a generating means;
output instruction means for instructing the distributed network to output the information generated by the information generation means;
A similar content management system comprising: The feature point comparing means is configured such that, among one or more feature points to be compared, a distance between a position of the feature point in the image of the face part of the target content and a position of the feature point in the person's image is a first threshold value. 2. The similar content management system according to claim 1, wherein if the content is smaller than , it is determined that the target content has an appearance similarity with the person's image. The feature point comparison means is configured to calculate, among one or more feature points to be compared, a first vector whose starting point is the position of the feature point in a reference image, which is a reference face image, and whose end point is the position of the feature point in the person's image. and a second vector whose starting point is the position of the feature point in the reference image and whose end point is the position of the feature point in the target content is greater than or equal to a second threshold, if the target content is 3. The similar content management system according to claim 1 or 2, wherein the similar content management system determines that the content has external similarity to the person's image. A similar content management method for managing information regarding a relationship between target content that has external similarity to a person's image, which is a face image of the person, and the person, the method comprising:
a feature point extraction step of extracting feature points, which are points corresponding to each element constituting the face, from the facial image of the target content;
Based on the comparison result between the feature points of the target content extracted in the feature point extraction step and the feature points of the person's image that have a corresponding relationship with the feature points, the appearance difference between the target content and the person's image is determined. a feature point comparison step for determining the presence or absence of similarity;
a relationship determination step of determining a relationship between the target content that is determined to have the same appearance as the person's image in the feature point comparison step, and the person;
Information that generates identification information of the target content and information regarding the relationship with the principal, which is recorded in a distributed ledger on a distributed network corresponding to the principal token, which is a non-fungible token that corresponds to the principal. a generation step;
an output instruction step of instructing the distributed network to output the information generated in the information generation step;
A similar content management method characterized by comprising: Among the one or more feature points, the distance between the position of the feature point in a reference image that is a reference face image and the position of the feature point in the person's image that corresponds to the feature point is equal to or greater than a third threshold. further comprising a feature point selection step of selecting a certain feature point;
5. The similar content management method according to claim 4, wherein in the feature point comparison step, feature points are compared for the feature points selected in the feature point selection step. A similar content management program that causes a computer to manage information regarding a relationship between target content that has external similarity to a person's image, which is a face image of the person, and the person,
to the computer,
a feature point extraction function that extracts feature points, which are points corresponding to each element that makes up the face, from the facial image of the target content;
Based on the comparison result between the feature points of the target content extracted by the feature point extraction function and the feature points of the person's image that have a corresponding relationship with the feature points, the appearance difference between the target content and the person's image is determined. feature point comparison function that determines the presence or absence of similarity;
a relationship determination function that determines a relationship between the subject content and the person, which is determined to have the same appearance as the person's image by the feature point comparison function;
Information that generates identification information of the target content and information regarding the relationship with the principal, which is recorded in a distributed ledger on a distributed network corresponding to the principal token, which is a non-fungible token that corresponds to the principal. generation function,
an output instruction function that instructs the distributed network to output the information generated by the information generation function;
A similar content management program characterized by executing the following.