JP2009251832A - User correlation diagram generation device, method, program, and system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a user to generate a user correlation diagram by his/her simple operation. <P>SOLUTION: The user registers his/her own face image in a correlation information generation server device 3 via a terminal device such as a mobile phone 1 in advance, and thereafter uploads a camera image at desired timing. The correlation information generation server device 3, when the user makes a generation request for a user correlation diagram, performs face recognition processing to the camera image uploaded from the user on the basis of the previously registered face image of the user, determines the correlation relationship between the user himself/herself and a co-shot user photographed together with the user himself/herself, and generates a user correlation diagram indicating the correlation relationship between the respective users. The user registers his/her own face image in advance, and thereafter at desired timing he/she can acquire a user correlation diagram centering the user at his/her desired time by performing simple input operation of only uploading a camera image to the correlation information generation server device 3 at desired timing. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention is a user correlation diagram creation device, a user correlation diagram creation method, a user correlation diagram creation program, and a program suitable for being applied to a server device or the like on a network that provides correlation information indicating a correlation between users. The present invention relates to a user correlation diagram creating system suitable for application to a system that provides correlation information indicating a correlation between users.
In particular, for example, uploading from each user via a terminal device such as a mobile phone, a PHS phone (PHS: Personal Handyphone System), a PDA device (PDA: Personal Digital Assistant), a portable game machine, a notebook or desktop personal computer device, etc. A user correlation diagram creation device, a user correlation diagram creation method, a user correlation diagram creation program, which detects correlation information indicating a correlation between the users based on the captured camera image and provides the correlation information to each user; The present invention relates to a user correlation diagram creation system.

  Today, systems are known that provide correlation information indicating the correlation between users. For example, in Japanese Patent Application Laid-Open No. 2003-132158 (Patent Document 1), each user registers his / her personal network information and obtains a personal network relationship based on the registered personal network information. Is disclosed. This personal network information management system has a function of providing, for example, a list of people who have a strong personal relationship with the user from the personal relationship information.

JP 2003-132158 A (pages 6-7, FIG. 1)

  However, in the case of the human network information management system disclosed in the above-mentioned patent document 1, since each user manually performs basic information for obtaining a human network relationship, the burden of this input is large. There's a problem.

  The present invention has been made in view of the above-described problems, and provides a user correlation diagram creation apparatus and a user correlation diagram capable of providing correlation information indicating a correlation between users with a simple input operation. It is an object to provide a method, a user correlation diagram creation program, and a user correlation diagram creation system.

In order to solve the above-described problem, the user correlation diagram creation apparatus according to the present invention provides:
A storage unit for storing at least a face image of each user and a camera image registered for each user;
When creating a user correlation diagram, the camera image registered in the storage unit is referred to by the user who has made the user correlation diagram creation request based on the face image of the user who has made the user correlation diagram creation request. Thus, the user who made the user correlation diagram in the camera image and the co-photographing user who is the user in the camera image together with the user are identified, and the correlation between the users A correlation determination unit for determining the relationship;
A co-photographing user detection unit that detects a face image of the same user as the co-copying user determined by the correlation determination unit from the storage unit;
While displaying the face image of the user who made the user correlation diagram creation request stored in the storage unit, the face image of the co-copy user detected from the storage unit by the co-copy user detection unit, A user correlation diagram creating unit that creates a user correlation diagram displaying correlation information indicating the correlation between the user himself / herself and the co-photographed user determined by the correlation determination unit.

Moreover, in order to solve the above-mentioned problem, the user correlation diagram creation method according to the present invention
A storage control unit storing at least a face image of each user and a camera image registered for each user in the storage unit;
At the time of creating the user correlation diagram, the correlation determining unit is registered in the storage unit by the user who has made the user correlation diagram creation request based on the face image of the user who has made the user correlation diagram creation request. The user who made the user correlation diagram in the camera image and the user who is the user in the camera image together with the user. Determining the correlation between each user;
A step of detecting, from the storage unit, a face image of the same user as the co-copy user determined in the step;
The user correlation diagram creation unit displays the face image of the user who made the user correlation diagram creation request stored in the storage unit, and the face image of the co-photographed user detected from the storage unit in the step And creating a user correlation diagram displaying correlation information indicating the correlation between the user himself / herself and the co-photographed user determined in the above step.

Moreover, in order to solve the above-mentioned problem, the user correlation diagram creation program according to the present invention provides:
Causing the computer to function as a storage control unit that stores at least a face image of each user and a camera image registered for each user in the storage unit;
When creating a user correlation diagram, the camera image registered in the storage unit is referred to by the user who has made the user correlation diagram creation request based on the face image of the user who has made the user correlation diagram creation request. Thus, the user who made the user correlation diagram in the camera image and the co-photographing user who is the user in the camera image together with the user are identified, and the correlation between the users Let the computer function as a correlation determination unit that determines the relationship,
Causing the computer to function as a co-photographing user detection unit that detects a face image of the same user as the co-copying user determined by causing the computer to function as the correlation determination unit;
The face image of the user who made the user correlation diagram creation request stored in the storage unit, and the face of the co-copy user detected from the storage unit by causing a computer to function as the co-copy user detection unit A user correlation diagram creating unit that creates a user correlation diagram that displays correlation information indicating the correlation between the user himself and the co-photographed user determined by causing the computer to function as the correlation determining unit while displaying an image As a computer to function.

Moreover, in order to solve the above-mentioned problem, the user correlation diagram creation system according to the present invention provides:
A terminal device that transmits at least a user's face image and a desired camera image;
A storage unit for storing the user's face image and the desired camera image transmitted via the terminal device, and a user's face image that has made a user correlation diagram creation request at the time of user correlation diagram creation Based on the user who made the request to create the user correlation diagram, the user who made the request to create the user correlation diagram in the camera image by referring to the camera image registered in the storage unit. And a co-photographing user who is a user in the camera image together with the user himself / herself, and a correlation determining unit for determining a correlation between the users, and the co-photographing user determined by the correlation determining unit A co-photographing user detection unit that detects a face image of the same user from the storage unit, and a face image of the user who made the user correlation diagram creation request stored in the storage unit The correlation indicating the correlation between the user himself and the co-printing user determined by the correlation determination unit and displaying the face image of the co-copying user detected from the storage unit by the co-copying user detection unit And a server device including a user correlation diagram creating unit that creates a user correlation diagram displaying information and transmits the created user correlation diagram to the terminal device.

  According to the present invention, at least a face image of each user and a camera image registered for each user are stored in the storage unit, and when the user correlation diagram is created, a request to create the user correlation diagram is made. Based on the user's face image, the user who made the user correlation diagram creation request refers to the camera image registered in the storage unit to create the user correlation diagram shown in the camera image. The user who made the request and the co-photographing user who is the user in the camera image together with the user himself / herself are determined, and the correlation between the users is determined.

  Then, the face image of the user who is the same as the determined co-photographing user is detected from the storage unit, and the face image of the user who made the user correlation diagram creation request stored in the storage unit, the storage unit In addition to displaying the face image of the co-photographed user detected from the above, a user correlation diagram is generated that displays correlation information indicating the correlation between the user himself and the co-photographed user.

  Thereby, the user can acquire a user correlation diagram indicating a correlation between the users with a simple input operation only by registering the user's face image and a desired camera image.

  According to the present invention, it is possible to create a user correlation diagram indicating a correlation between users by a simple input operation for registering a user's face image and a desired camera image.

  The present invention provides a user correlation diagram creation system that automatically creates and provides a correlation diagram indicating a correlation between users based on a camera captured image uploaded from a terminal device such as a mobile phone or a personal computer device. Can be applied to.

[Configuration of the embodiment]
〔System configuration〕
First, FIG. 1 shows a system configuration diagram of a user correlation diagram creating system according to the present embodiment. As can be seen from FIG. 1, in the case of this user correlation diagram creation system, each user's terminal device such as mobile phone 1 or personal computer device 2 (notebook type or desktop type) and each user's terminal device are uploaded. A correlation information creation server device 3 that creates and provides a user correlation diagram showing a correlation between users based on the captured image of the camera is connected to each other via a network 4 such as the Internet or a communication network. It consists of

[Configuration of mobile phone]
FIG. 2 shows a block diagram of the mobile phone 1. As shown in FIG. 2, a mobile phone 1 is an antenna that performs wireless communication such as voice call, videophone call, e-mail, and Web data (Web: World Wide Web) with the base station 6 shown in FIG. 11, a communication circuit 12, a speaker unit 13 for outputting received voices, a microphone unit 14 for collecting transmitted voices, an operation menu, e-mail, images (still images and moving images), etc. A display unit 15 for displaying a message, an operation unit 16 having a plurality of operation keys such as a cross key, a determination key, and a numeric key, and a light emitting unit 17 (LED) for notifying a user of incoming / outgoing calls with light. And have.

  The mobile phone 1 includes a camera unit 18 for capturing a still image or a moving image of a desired subject, and a vibration unit 19 for notifying a user of incoming / outgoing calls by vibrating the housing of the mobile phone. In addition to a timer 20 for counting time, a communication program for performing wireless communication processing via the base station 6 and various application programs, various data (contents) handled by these application programs are stored. It has a memory 21 and a control unit 22 that controls the operation of the entire mobile phone.

  In the memory 21, in addition to the communication program, a camera control program for controlling imaging of the camera unit 18, an e-mail management program for controlling creation and transmission / reception of e-mail, and a user schedule are registered. A schedule book management program for managing a schedule book, a web browsing program for browsing a web page published on a server device provided on a predetermined network such as a communication network or the Internet, and a telephone book A telephone directory management program for managing the music, a music player program for playing music content, uploading of camera image information captured by the camera unit 18 and the like, and user correlation with the correlation information creation server device 3 Need to create diagram The user correlates the application program for performing the uptake of the user correlation diagram is the created is stored.

In addition, the memory 21 includes a schedule book in which a user's desired schedule is registered, user names such as user acquaintances and friends, still images (face photos, etc.), address, telephone number, e-mail address, date of birth. Etc. are registered in the correlation information generation server device 3 based on the viewer function of the camera control program or the user correlation application program. The uploaded camera image (still image content) and moving image content, transmitted / received e-mail content, telephone / email outgoing / incoming history, and the like are stored.

[Configuration of server device]
FIG. 3 shows a block diagram of the correlation information creation server device 3. As shown in FIG. 3, the correlation information creation server device 3 performs communication control in the communication unit 31 that communicates with the mobile phone 1 and the personal computer device 2 shown in FIG. Communication control program, user correlation diagram creation program for performing user correlation diagram creation control, which will be described later, and control for detecting the same user among users reflected in camera images uploaded from each user And a memory 32 in which the same user discrimination program and the like are stored.

Further, the correlation information creation server device 3 controls communication of the communication unit 31 based on the communication program stored in the memory 32, and the same user based on the same user determination program stored in the memory 32. A control unit 35 that functions as the discrimination unit 33 and functions as the user correlation diagram creation unit 34 based on a user correlation diagram creation program stored in the memory 32, and stores a camera image and the like uploaded from each user A user database 5 (user DB).

[System operation]
〔user registration〕
First, each user who uses this user correlation diagram creation system performs user registration with the correlation information creation server device 3. For example, when performing user registration using the mobile phone 1, the user establishes a communication line with the correlation information creation server device 3 via the base station 6 and the network 4 and makes a user registration request. When this user registration request is made, the correlation information creation server device 3 gives the user's mobile phone 1 input fields for predetermined input items such as a user name, an e-mail address, a telephone number, and a desired password, A user registration screen including an attachment field for the user's face photograph (camera image showing the user's face) is transmitted.

On the user registration screen, the user inputs the user name, desired password, and the like, attaches a camera image of his / her face photo, and returns it to the correlation information creation server device 3. The correlation information creation server device 3 checks the input contents of each input field on the user registration screen returned from the user, and if there is no deficiency, forms a user area for the user in the user DB 5, and this user area The user's input contents and the user's face photo are registered. The correlation information creation server device 3 also issues a unique user identification number (user ID) to the user and registers this input content, and transmits this to the user's mobile phone 1. Thereby, user registration for the system is completed.

[Upload camera images]
Next, when such user registration is properly completed, the user can upload a desired still image (mainly a camera image) via a terminal device such as the mobile phone 1. The flowchart of FIG. 4 shows the flow of the still image upload operation in the mobile phone 1. The control unit 22 of the mobile phone 1 shown in FIG. 2 is based on the user correlation application program stored in the memory 21 when the user of the mobile phone 1 performs an activation operation of the user correlation application program via the operation unit 16. Then, the process shown in the flowchart of FIG. 4 is started.

  In this example, when the user of the mobile phone 1 performs an activation operation of the user correlation application program via the operation unit 16, the control unit 22 activates the user correlation application program stored in the memory 21. However, every time imaging is performed by the camera unit 18, the control unit 22 may automatically start the user correlation application program, or at the timing when imaging by the camera unit 18 ends. The control unit 22 may automatically start this user correlation application program. Further, as a setting of the mobile phone 1, the user may be able to select any one of the above-described activation methods.

  In step S1, the control unit 22 controls communication of the communication circuit 12 so as to connect the communication line to the correlation information creation server device 3 shown in FIG. The process proceeds to step S2 at the timing when the communication line is established.

  When this communication line is established, the control unit 22 displays a user ID and password input screen on the display unit 15. On this input screen, the user inputs the user ID previously issued from the correlation information creation server device 3 and the desired password registered at the time of user registration, and performs these transmission operations. When this transmission operation is performed, the control unit 22 transmits the user ID and password input by the user to the correlation information creation server device 3.

  The correlation information creation server device 3 compares the user ID and password transmitted from the user's mobile phone 1 with the user ID and password of the user registered in advance in the user DB 5, and if both match, Then, information indicating that the user authentication is successful is transmitted to the mobile phone 1 of the user. In step S2 of the flowchart of FIG. 4, the control unit 22 determines whether or not the information indicating that the user authentication is successful is received from the correlation information creation server device 3, and indicates that the user authentication is successful. If it is determined that the information has been received, the process proceeds to step S3. If it is determined that the information indicating that the user authentication has failed is received, the process shown in the flowchart of FIG.

  Next, when it is determined that the information indicating that the user authentication has been successful has been received and the process proceeds to step S3, the control unit 22 acquires the camera image stored in the memory 21 and performs the process. Proceed to S4. Specifically, when there is one camera image stored in the memory 21, the control unit 22 acquires the camera image as it is, and when there are a plurality of camera images stored in the memory 21, For example, a message such as “Please select an image to upload” is displayed on the display unit 15, and the camera image selected by the user is acquired from the memory 21.

  Further, when the user correlation application program is activated every time imaging is performed by the camera unit 18 as described above, the control unit 22 acquires the camera image captured immediately before starting the user correlation application program from the memory 21. To do.

  When the user correlation application program is activated at the timing when the imaging by the camera unit 18 is completed, the control unit 22 stores one or a plurality of camera images captured immediately before the user correlation application program is activated. 21. When a plurality of camera images are acquired, the control unit 22 displays a message such as “Please select an image to upload” on the display unit 15, for example, and stores the camera image selected by the user in the memory 21. May be obtained from

  Next, when the camera image is acquired from the memory 21 as described above, the control unit 22 determines the image size (data amount) of the acquired camera image in the user correlation diagram creation system in step S4 of FIG. It is determined whether or not the image size is within the range. If it is determined that the image size of the acquired camera image is within the range defined by the system, the process proceeds to step S5, and the image size of the acquired camera image is determined by the system. If it is determined that the image size is larger than the specified range, in step S6, the image size of the acquired camera image is reduced to an image size within the range specified by the system. Advances to step S5.

  In step S5, the control unit 22 controls the communication circuit 12 so as to transmit the acquired camera image to the correlation information creation server device 3, and ends all the processes in the flowchart of FIG.

  The correlation information creation server device 3 stores the camera image transmitted from the user's mobile phone 1 in the user area of the user in the user DB 5. As a result, the camera image transmitted from the user's cellular phone 1 is uploaded to the correlation information creation server device 3.

FIG. 5 shows the registration contents of the user DB 5. As can be seen from FIG. 5, in the user DB 5, one or a plurality of face photographs of each user registered at the time of the above-mentioned user registration, camera images uploaded from each user, etc., together with the above-mentioned input contents be registered. Thus, when a camera image is uploaded from each user, a user correlation diagram between each user registered in the system can be created.

[User correlation diagram transmission operation]
The flow of the user correlation diagram transmission operation in the correlation information creation server device 3 is shown in the flowchart of FIG. A user who wishes to create a user correlation diagram activates a user correlation application program stored in the memory 21 of the mobile phone 1 to establish a communication line with the correlation information creation server device 3, and Request drawing creation. The flowchart of FIG. 6 shows the control unit 35 of the correlation information creation server device 3 shown in FIG. 3 when such a user correlation diagram creation request is made via a user terminal device such as the cellular phone 1. Starts processing based on the user correlation diagram creation program stored in the memory 32.

  In step S11, the control unit 35 acquires the user ID and password from the terminal device of the user who has made the user correlation diagram creation request, and advances the process to step S12. In step S12, the control unit 35 performs a user authentication process by comparing the user ID and password acquired from the user terminal device with the user ID and password of the user registered in advance in the user DB 5, In step S13, it is determined whether or not the user who is currently making a user correlation diagram creation request is an authorized user of the system. If it is determined that the user is not a legitimate user of the system, the process proceeds to step S17, and an error message such as “password is incorrect” is returned to the terminal device of the user. The process shown in the flowchart ends. In this case, as a matter of course, a user correlation diagram is not created.

On the other hand, if it is determined in step S13 that the user who is currently making a user correlation diagram request is a legitimate user of the system, the control unit 35 advances the process to step S14. 7, a user correlation diagram is created as described later using the flowchart of FIG. 7, and the communication unit 31 shown in FIG. 3 is controlled to transmit the created user correlation diagram to the terminal device of the user in step S <b> 15. . Then, after the transmission of the user correlation diagram, in step S16, the communication unit 31 is controlled so as to disconnect the communication line established with the user's terminal device, and all of the flowcharts shown in FIG. The process ends.

[User correlation diagram creation operation]
Next, the user correlation diagram creation operation shown in step S14 of the flowchart of FIG. 6 will be described. The user correlation diagram is created by the control unit 35 of the correlation information creation server device 3 shown in FIG. 3 based on the same user discrimination program and the user correlation diagram creation program stored in the memory 32. In this example, in order to facilitate understanding, in FIG. 3, the same user determination program and the user correlation diagram creation program are illustrated separately, but the same user determination program is included in the user correlation diagram generation program. It should be understood that it is included.

  That is, in step S13 of the flowchart of FIG. 6, if it is determined that the user who is currently requested to create the user correlation diagram is an authorized user of the system, the control unit 35 performs the process of the flowchart of FIG. The process proceeds to step S21, the type and creation range of the user correlation diagram designated by the user is detected, and the process proceeds to step S22.

  Here, FIGS. 8A and 8B show camera images uploaded by the user 1. FIG. 8A is a camera image in which the user 1 and the user 3 are shown with the user 1 in the middle. In the case of this camera image, the correlation between the user 1 and the user 2 who are the principals is “directly”. Correlation ”. Similarly, in the case of the camera image in FIG. 8A, the correlation between the user 1 and the user 3 as the person is also a “direct correlation” correlation. On the other hand, in the case of the camera image of FIG. 8A, the correlation between the user 2 and the user 3 captured together with the user 1 who is the user is a “quasi-direct correlation” correlation.

  FIG. 8B is a camera image in which the user 1 and the user 5 are shown with the user 1 in the middle. In the case of this camera image, the correlation between the user 1 and the user 4 who are the principals is as follows. The correlation is “direct correlation”. Similarly, in the case of the camera image in FIG. 8B, the correlation between the user 1 and the user 5 as the person is also a “direct correlation” correlation. On the other hand, in the case of the camera image of FIG. 8B, the correlation between the user 2 and the user 3 captured together with the user 1 who is the user is a “quasi-direct correlation” correlation.

  On the other hand, as can be seen by comparing the camera image of FIG. 8A and the camera image of FIG. 8B, the other users 2, 2, 3, 4, and 5 are the other users of the two camera images. Are shown together with the user 1 who is the person. For this reason, the other users (= user 2 to user 5) shown in the two images each have a correlation of “indirect correlation”.

In the case of the user correlation diagram creation system of the embodiment, a user who makes a user correlation diagram creation request is:
"User correlation diagram for users with direct correlation only",
“User Correlation Diagram for Users Corresponding to Direct Correlation and Quasi-Indirect Correlation”,
"User correlation diagram for users with direct, quasi-direct, and indirect correlations"
A user correlation diagram of a desired type is designated from among the above. In step S21, the control unit 35 detects the type of the user correlation diagram designated by the user from such types of user correlation diagrams.

  FIG. 9 shows an example of a user correlation diagram created by the control unit 35, but the control unit 35 sends a user correlation diagram creation request (in this example, as will be described later). Draws a plurality of concentric circles spreading centering on user information such as the user 1) face photo, and places user information such as a corresponding user face photo on each circle. .

  Each circle is a circle indicating the range of correlation, and the first circle from the user 1 is a circle indicating the range of direct correlation and semi-direct correlation. In addition, the second circle from user 1 and the circle after the second circle indicate the range of indirect correlation, respectively, and the second circle from user 1 indicates the range of one indirect correlation. The third circle from user 1 indicates the range of two-indirect correlation, and the fourth circle from user 1 indicates the range of three-indirect correlation.

  When making a user correlation diagram creation request, the user designates the range of the user correlation diagram to be created, such as “direct correlation range”, “two-indirect correlation range”, and the like. In step S21, the control unit 35 detects the range of the user correlation diagram specified by the user in this way.

  Next, in step S21 of the flowchart of FIG. 7, the control unit 35 acquires from the user DB 5 a representative image of the user who has made this user correlation diagram creation request (a facial photograph of the user registered at the time of user registration). Then, the process proceeds to step S23. In step S23, the control unit 35 functions as the same user discriminating unit 33 based on the same user discriminating program stored in the memory 32, and is uploaded by the user who made the user correlation diagram creation request. Face recognition processing is performed on all camera images. Then, a camera image captured by a plurality of persons including the person is detected, and the process proceeds to step S24.

  In step S24, the control unit 35 functions as the same user determination unit 33, so that based on the camera image in which a plurality of persons including the person are captured, other users (co-photographing users) other than the person can be used. By recognizing the face image and referring to the user DB 5 based on the face image of the co-photographing user, the user information of the co-photographing user is acquired.

  Here, the fact that the user information of the co-copy user cannot be acquired from the user DB 5 means that the co-copy user is a user who is not registered in the system. For this reason, when the user information of the above-mentioned co-photographing user cannot be acquired from the user DB 5, the control unit 35 excludes the co-photographing user from the creation of the user correlation diagram and does not reflect it in the user correlation diagram shown in FIG. It has become. Thereby, a user correlation diagram only between users registered in the system can be created. Further, even when a user who is not acquainted with the user is reflected in the camera image, the user who is not acquainted can be excluded from the user correlation diagram to be created (however, a user who is not acquainted with the user is registered in the system). If the user is a user, the user correlation diagram may be reflected.)

  Next, the control unit 35 functions as the user correlation diagram creation unit 34 based on the user correlation diagram creation program stored in the memory 32, so that the user himself / herself (in this example) is centered as shown in FIG. Arranges user information such as a user's 1) face photo and user name, draws a circle that is concentric with the center, and draws the user information of the above-mentioned co-photographing user on the circle that is larger than the center. In the case of this example, the user information of the user 2 and the user 3) is arranged, and the process proceeds to step S26.

  In step S <b> 26, the control unit 35 functions as the same user determination unit 33, so that the camera uploaded by each co-photographing user based on the photo of each co-op user registered in the user DB 5. Face recognition processing is performed on the image, camera images taken by a plurality of persons including the co-photographing user himself / herself are detected, and the process proceeds to step S27.

  In step S27, the control unit 35 functions as the same user discriminating unit 33, thereby recognizing the face image of the co-photographing user reflected in the camera image together with the co-photographing user of the user 2 and the user 3, By referring to the user DB 5 based on the copy user's face image, the user information of the co-copy user is acquired, and the process proceeds to step S28.

  For example, in the example shown in FIG. 9, when the co-photographing user 4, the co-photographing user 5, and the co-photographing user 15 are detected from the camera image uploaded by the co-photographing user 2, the control unit 35 performs the co-copying user 4. Users of the copying user 4, the copying user 5, and the copying user 15 by referring to the user DB based on each facial photograph registered by the copying user 4, the copying user 5, and the copying user 15 Get information.

  Similarly, when the co-photographing user 14 and the co-photographing user 15 are detected from the camera image uploaded by the co-photographing user 3, the control unit 35 determines each face registered by the co-photographing user 14 and the co-photographing user 15. By referring to the user DB based on the photograph, user information of the co-photographing user 14 and the co-photographing user 15 is acquired. As described above, when the co-photographing user is not a user registered in the system, it is excluded from the user correlation diagram.

  Next, in step S28, the control unit 35 functions as the user correlation diagram creating unit 34, thereby drawing a circle that is slightly larger than the circle in which the user information of the co-photographing user is arranged, and this one larger The user information of the co-photographing user shown in the camera image together with the co-photographing user is arranged on the circle, and the process proceeds to step S29.

  Specifically, when the co-photographing user 4, the co-photographing user 5, and the co-photographing user 15 are detected from the camera image uploaded by the co-photographing user 2 shown in FIG. The user information of the co-photographing user 4, the co-photographing user 5, and the co-photographing user 15 acquired from the DB 5 is arranged on a circle that is slightly larger than the circle in which the user information of the co-photographing user 2 is arranged. Similarly, when the co-photographing user 14 and the co-photographing user 15 are detected from the camera image uploaded by the co-photographing user 3, the control unit 35 acquires the co-photographing user 14 and the co-photographing user 15 acquired from the user DB 5 in step S27. Are arranged on a circle that is slightly larger than the circle in which the user information of the co-photographing user 3 is arranged.

  Next, in step S29, the control unit 35 determines that the user correlation diagram in which the user information of each user is arranged in this manner is a range designated by the user who made the user correlation diagram creation request (explanation of step S21 above). The process proceeds to step S30 when it is determined that the specified range has been reached, and the process proceeds to step S26 when the specified range is not reached. Returning to step S26, the processes in steps S26 to S28 are repeated until the range specified by the user, such as “one indirect correlation range” or “two indirect correlation range”, is reached.

  Next, in step S29, the control unit 35 functions as the user correlation diagram creation unit 34 to detect the correlation between users arranged on each circle, and the user information of each user is A user correlation diagram is completed by performing a drawing process that connects lines of line types corresponding to the detected correlation.

  Specifically, as an example, in the case of the user correlation diagram creation system of the embodiment, when the correlation between the users is a direct correlation, the user information of each user is represented by a “straight line”. It comes to tie. Further, when the correlation between the users is a quasi-direct correlation, the user information of each user is connected by a “dotted line”.

  In addition, when a plurality of camera images showing direct correlation or quasi-direct correlation are detected, for example, three or five, the thickness of the straight or dotted line is set according to the number of the camera images. It gradually becomes thicker and draws.

  For example, when one camera image showing direct correlation is detected, the control unit 35 performs a drawing process for connecting user information of each user with a straight line having a thickness of 0.75 mm, and three camera images showing direct correlation are obtained. When detected, the control unit 35 performs a drawing process for connecting the user information of each user with a straight line having a thickness of 1 mm, and when five or more camera images showing a direct correlation are detected, the control unit 35 A drawing process for connecting user information of each user with a 3 mm straight line is performed.

  Similarly, for example, when one camera image showing quasi-direct correlation is detected, the control unit 35 performs a drawing process for connecting user information of each user with a dotted line having a thickness of 0.75 mm, and shows a quasi-direct correlation camera. When three images are detected, the control unit 35 performs a drawing process for connecting the user information of each user with a dotted line having a thickness of 1 mm, and when five or more camera images showing a quasi-direct correlation are detected, the control unit 35 35 performs a drawing process for connecting user information of each user with a dotted line having a thickness of 3 mm.

  Further, when the user has a direct correlation or quasi-direct correlation with a large number of users, for example, three or five, the control unit 35 shows a bold line surrounding the user information of the user. The drawing process is performed.

  In the case of the user correlation diagram shown in FIG. 9, it is shown that user 1 and user 2 are in direct correlation, and user 1 and user 3 are in quasi-direct correlation. Further, it is indicated that the user 2 and the user 5 have a direct correlation, and the user 5 and the user 7 have a quasi-direct correlation.

  Further, the user 8 and the user 12 have drawn a large number of camera images showing a direct correlation, so that a straight line connecting the user information of the user 8 and the user information of the user 12 is drawn with a bold line. Similarly, the user 4 and the user 6 have drawn a large number of straight lines connecting the user information of the user 4 and the user information of the user 6 because a large number of camera images showing a direct correlation have been detected. .

  In addition, since the user 15 has a direct correlation with each of the user 2, the user 3, the user 6, the user 16, and the user 17, the user information of the user 15 is drawn with a bold circle, for example. Processing has been done. Similarly, since the user 6 also has a direct correlation with each of the user 4, the user 9, the user 10, and the user 15, a drawing process in which the user information of the user 15 is surrounded by, for example, a bold circle. Has been made.

The user correlation diagram created in this way is transmitted to the terminal device of the user who made the user correlation diagram creation request by the control unit 35 in step S15 of the flowchart of FIG.

[Access to a desired user in the user correlation diagram]
Next, in the case of the user correlation diagram creation system of this embodiment, it becomes possible to access a desired user by e-mail, telephone, etc. among each user displayed in the user correlation diagram. ing.

  For example, when a user correlation chart is created using each user's face photograph as user information of each user, this user correlation chart is a line type indicating that each user's face photograph indicates the correlation of each user. Although a user correlation diagram is formed in a form connected by a line, a user who acquires such a user correlation diagram selects and operates a desired user on the terminal device such as the mobile phone 1 of the user. In this case, when the correlation information creation server device 3 transmits the created user correlation diagram, the user ID of each user displayed on the user correlation diagram is also displayed together with the user correlation diagram, such as the terminal device such as the mobile phone 1 of the user. Is sending to.

  When a desired user selection operation is performed on the user correlation diagram, the control unit 22 of the mobile phone 1 selects the user ID among the user IDs of each user transmitted from the correlation information creation server device 3 together with the user correlation diagram. The user ID of the user who has performed the operation is detected, and the detected user ID is transmitted to the correlation information creation server apparatus 3 to make an access request to the user who has performed the selection operation.

  When this access request is made, the correlation information creation server device 3 includes, among the contact information of each user registered in the user DB 5 (= the e-mail address and telephone number registered at the time of user registration), a mobile phone The contact information of the user specified by the user ID transmitted from the terminal device 1 or the like is acquired from the user DB 5 and transmitted to the terminal device such as the mobile phone 1 of the user who made the access request.

Thereby, contact information of a desired user among the users displayed on the user correlation diagram can be acquired, and access (contact) to the desired user can be achieved. it can. The contact information may be set to “public” or “private” at the time of registration. Of course, the correlation information creation server device 3 does not transmit (disclose) the contact information of the user whose contact information is set to “undisclosed” even if an access request is made.

[Effect of the embodiment]
As is clear from the above description, the user correlation diagram creating system of this embodiment registers his / her representative face photograph at the time of user registration, and thereafter the correlation information creating server device 3 at a desired timing. Upload a camera image for. The correlation information creation server device 3 performs face recognition on the camera image uploaded from the user based on the face photograph of the user registered in advance when a user correlation diagram creation request is made from the user. By performing the processing, the user himself / herself shown in each camera image and the co-photographing user shown together with the user himself / herself are recognized.

  In addition, it is determined that the co-photographing user who is photographed together with the user himself / herself is in a direct correlation relationship with the user himself / herself, and the co-photographing users photographed together with the user himself / herself in a single camera image are quasi-directly correlated. It is determined that there is a correlation. Further, it is determined that the co-photographed users who are photographed together with the user himself / herself between different camera images are in an indirect correlation.

  Then, the user information such as the user's face photo, user ID, user name, etc. that made the user correlation diagram request is placed at the center, and the user information of each user is placed on each circle concentric with this center Then, a user correlation diagram in which these pieces of user information are connected by the line type line indicating the determined correlation is created and transmitted to the terminal device such as the cellular phone 1 of the user who has made the creation request.

As a result, the user registers his / her representative facial photograph at the time of user registration, and thereafter performs a simple input operation only to upload a camera image to the correlation information creation server device 3 at a desired timing. By just doing it, the user correlation diagram centered on the user can be automatically acquired when he / she desires.

[Modification]
In the description of each of the embodiments described above, the mobile phone 1 and the personal computer device 2 are used as the terminal device. However, in addition to this, a PHS phone (PHS: Personal Handyphone System), a PDA device (PDA: Personal Digital Assistant). Other terminal devices having a communication function such as portable game machines and digital camera devices may be used. In either case, the same effect as described above can be obtained.

  In the above description of the embodiment, the correlation information creation server device 3 creates a concentric user correlation diagram. However, user correlation such as a tree (tree structure), a list, a table, a graph, etc. A diagram may be created.

  Further, the correlation information creation server device 3 displays each user's face photo on the first circle from the center, displays the user name of each user on the second circle from the center, and starts from the center. You may create the user correlation diagram which changed the user information displayed according to a range like displaying the user ID of each user on the 3rd circle. In this case, the user who is close to the user himself / herself is displayed as a face photo, but the user information displayed as the user's friend's friend, the user's friend's friend friend friend, etc. become distant Since the user information becomes sparse, such as user name → user ID, it is possible to protect the privacy of each user.

  Finally, the present invention is not limited to the above-described embodiment described as an example, and various modifications can be made depending on the design and the like as long as they do not depart from the technical idea according to the present invention. I will add that it is a matter of course.

It is a figure which shows the system configuration | structure of the user correlation diagram production system used as embodiment which applied this invention. It is a block diagram of a mobile phone used in the user correlation diagram creating system of the embodiment. It is a block diagram of the correlation information creation server apparatus which creates a user correlation diagram in the user correlation diagram creation system of an embodiment. 5 is a flowchart showing a flow of uploading a camera image from a terminal device such as a mobile phone to a correlation information creation server device in the user correlation diagram creation system of the embodiment. It is a figure for demonstrating the memory content of the user database of a correlation information creation server apparatus. It is a flowchart which shows the flow until it produces and transmits the user correlation diagram in a correlation information creation server apparatus. It is a flowchart which shows the flow of user correlation diagram creation in a correlation information creation server apparatus. It is a figure for demonstrating the correlation of each user. It is a figure which shows an example of the user correlation diagram produced by the correlation information creation server apparatus.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Mobile phone 2 Personal computer device 3 Correlation information creation server device 4 Network 5 User database 6 Mobile phone base station 11 Mobile phone antenna 12 Communication circuit 13 Speaker unit 14 Microphone unit 15 Display Unit, 16 operation unit, 17 light emitting unit, 18 camera unit, 19 vibration unit, 20 timer, 21 memory, 22 control unit, 31 communication unit, 32 memory, 33 same user discrimination unit, 34 user correlation diagram creation unit, 35 correlation Control unit of information creation server device

Claims (7)

A storage unit for storing at least a face image of each user and a camera image registered for each user;
When creating a user correlation diagram, the camera image registered in the storage unit is referred to by the user who has made the user correlation diagram creation request based on the face image of the user who has made the user correlation diagram creation request. Thus, the user who made the user correlation diagram in the camera image and the co-photographing user who is the user in the camera image together with the user are identified, and the correlation between the users A correlation determination unit for determining the relationship;
A co-photographing user detection unit that detects a face image of the same user as the co-copying user determined by the correlation determination unit from the storage unit;
While displaying the face image of the user who made the user correlation diagram creation request stored in the storage unit, the face image of the co-copy user detected from the storage unit by the co-copy user detection unit, A user correlation diagram creation device comprising: a user correlation diagram creation unit that creates a user correlation diagram displaying correlation information indicating the correlation between the user himself / herself and the co-photographed user identified by the correlation determination unit. The user correlation diagram creation device according to claim 1,
When the face image of the same user as the co-photographing user is not detected from the storage unit by the co-photographing user detection unit, the user correlation diagram creation unit is configured to detect a user correlation diagram that is not detected from the storage unit. A user correlation diagram creation device for hiding a photograph of a copying user's face. The user correlation diagram creation device according to claim 1 or 2,
In the storage unit, contact information of each user is stored together with the face image of each user and the camera image registered for each user,
The user who made the user correlation diagram creation request selects a desired user based on the created user correlation diagram, and when the user contact information acquisition request is made, the acquisition request is made. A user correlation diagram creation device having a contact information transmission unit that reads contact information from the storage unit and transmits the contact information to a terminal device of a user who has made the user correlation diagram creation request. The user correlation diagram creation device according to any one of claims 1 to 3,
The storage unit stores at least the user name and user identification information of each user together with the face image of each user and the camera image registered for each user.
The user correlation diagram creating unit selectively displays the face photograph, the user name, and the user identification information on the user correlation diagram according to the range of the user correlation diagram to be created. A storage control unit storing at least a face image of each user and a camera image registered for each user in the storage unit;
At the time of creating the user correlation diagram, the correlation determining unit is registered in the storage unit by the user who has made the user correlation diagram creation request based on the face image of the user who has made the user correlation diagram creation request. The user who made the user correlation diagram in the camera image and the user who is the user in the camera image together with the user. Determining the correlation between each user;
A step of detecting, from the storage unit, a face image of the same user as the co-copy user determined in the step;
The user correlation diagram creation unit displays the face image of the user who made the user correlation diagram creation request stored in the storage unit, and the face image of the co-photographed user detected from the storage unit in the step And creating a user correlation diagram displaying correlation information indicating the correlation between the user himself / herself and the co-photographed user determined in the step, and a user correlation diagram creation method. Causing the computer to function as a storage control unit that stores at least a face image of each user and a camera image registered for each user in the storage unit;
When creating a user correlation diagram, the camera image registered in the storage unit is referred to by the user who has made the user correlation diagram creation request based on the face image of the user who has made the user correlation diagram creation request. Thus, the user who made the user correlation diagram in the camera image and the co-photographing user who is the user in the camera image together with the user are identified, and the correlation between the users Let the computer function as a correlation determination unit that determines the relationship,
Causing the computer to function as a co-photographing user detection unit that detects a face image of the same user as the co-copying user determined by causing the computer to function as the correlation determination unit;
The face image of the user who made the user correlation diagram creation request stored in the storage unit, and the face of the co-copy user detected from the storage unit by causing a computer to function as the co-copy user detection unit A user correlation diagram creating unit that creates a user correlation diagram that displays correlation information indicating the correlation between the user himself and the co-photographed user determined by causing the computer to function as the correlation determining unit while displaying an image User correlation diagram creation program that causes a computer to function as A terminal device that transmits at least a user's face image and a desired camera image;
A storage unit for storing the user's face image and the desired camera image transmitted via the terminal device, and a user's face image that has made a user correlation diagram creation request at the time of user correlation diagram creation Based on the user who made the request to create the user correlation diagram, the user who made the request to create the user correlation diagram in the camera image by referring to the camera image registered in the storage unit. And a co-photographing user who is a user in the camera image together with the user himself / herself, and a correlation determining unit for determining a correlation between the users, and the co-photographing user determined by the correlation determining unit A co-photographing user detection unit that detects a face image of the same user from the storage unit, and a face image of the user who made the user correlation diagram creation request stored in the storage unit The correlation indicating the correlation between the user himself and the co-printing user determined by the correlation determination unit and displaying the face image of the co-copying user detected from the storage unit by the co-copying user detection unit A user correlation diagram creation system comprising: a server device including a user correlation diagram creation unit that creates a user correlation diagram displaying information and transmits the created user correlation diagram to the terminal device.

Images