JP6725926B1 - Information processing apparatus, information processing system, computer program, and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device, an information processing system, a computer program, and an information processing method capable of recording information about a person who has performed a consent act. An information processing apparatus according to the present embodiment is an information processing apparatus including a display unit and a camera, and includes both consent information related to a consent request and a captured image area for displaying an image captured by the camera. , A display processing unit that performs a process of displaying on the display unit, a face image acquisition unit that acquires a face image of the consenting person displayed in the captured image region, and a face image acquired by the face image acquisition unit And a storage processing unit that performs a process of storing in the unit. An information processing system includes an information processing device and a storage device having the storage unit that stores the face image by processing of a storage processing unit of the information processing device, and the storage processing unit of the information processing device is The face image acquired by the face image acquisition unit is transmitted to the storage device and stored in the storage unit of the storage device. [Selection diagram] Fig. 9

Description

The present invention relates to an information processing device, an information processing system, a computer program, and an information processing method for collecting information about consent from a user.

In recent years, services provided online such as web services and smartphone applications have been widely used. The provider of such a service needs to obtain, for example, consent to the handling of personal information and agreement to the terms of service of the service from the user of the service, the provider of personal information, or their legal representative.

Patent Document 1 describes a content distribution system having a function of transmitting a notification mail for facilitating the confirmation of contract contents and user's contract consent at the time of user registration to a user's information terminal. In this content distribution system, an e-mail for prompting the printing of the terms of use is transmitted to the temporarily registered user who has not yet printed the terms of use.

Japanese Unexamined Patent Publication No. 2001-28469

In the conventional service, there is no method for proving who is the person who handles personal information or agrees to the service usage agreement. For this reason, there is a possibility that the act of consent may be performed by impersonation by a third party.

The present invention has been made in view of such circumstances, and an object thereof is to provide an information processing device, an information processing system, and a computer program capable of recording information about a person who has performed a consent act. And providing an information processing method.

An information processing apparatus according to an embodiment is an information processing apparatus including a display unit and a camera, wherein both consent information regarding a consent request and a captured image area for displaying an image captured by the camera are displayed on the display unit. A display processing unit that performs a display process, a face image acquisition unit that acquires a facial image of the consenting person displayed in the captured image area, and a process that stores the face image acquired by the facial image acquisition unit in a storage unit. The display processing unit scrolls and displays the consent information on the display unit, and, after the consent information in the scroll, displays the captured image area together with the consent information. To display .

According to one embodiment, information about the person who has performed the consent act can be recorded.

It is a schematic diagram for explaining an outline of an information processing system according to the present embodiment. It is a schematic diagram for explaining an outline of an information processing system according to the present embodiment. It is a block diagram which shows the structure of the smart phone which concerns on this Embodiment. It is a block diagram which shows the structure of the server apparatus which concerns on this Embodiment. It is a schematic diagram which shows one structural example of consent information DB. It is a schematic diagram which shows an example of a consent actor selection screen. It is a schematic diagram which shows an example of a consentee information input screen. It is a schematic diagram which shows an example of a consent screen. It is a schematic diagram which shows an example of a consent screen. It is a schematic diagram which shows an example of the consent screen in which the consent button was validated. It is a schematic diagram which shows an example of a consentee delegator information input screen. It is a schematic diagram which shows an example of a consentee photography screen. It is a flowchart which shows the procedure of the consent acquisition process which the smart phone which concerns on this Embodiment performs. It is a flowchart which shows the procedure of the consent acquisition process which the smart phone which concerns on this Embodiment performs. It is a flow chart which shows the procedure of consent acquisition processing which a server concerning this embodiment performs. It is a schematic diagram which shows an example of the search screen displayed on the smart phone 1.

A specific example of the information processing system according to the embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these exemplifications, and is shown by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.

<System overview>
1 and 2 are schematic diagrams for explaining the outline of the information processing system according to the present embodiment. The information processing system according to the present embodiment includes a smartphone 1 which is an information processing device (computer) possessed by a user who uses a service, and a server device 3 which provides a service to this user. In the information processing system according to the present embodiment, the server device 3 acquires, for example, consent to the terms of use or consent to the handling of personal information from the user regarding the service provided to the user, and stores the acquired information about the consent. It has a function to put it.

The service provided by the server device 3 to the user may be any service. Further, a server device that provides a service and a server device that stores information about consent may be provided as separate devices. The information processing device used by the user is not limited to the smartphone 1, and may be a device such as a personal computer, a tablet terminal device, a smart speaker, or a mobile phone.

In the present embodiment, a user who consents to the terms of use or consent to the handling of personal information is referred to as an “acceptor”. In the information processing system according to the present embodiment, information about the terms of use or the handling of personal information is displayed on the smartphone 1, and the consentee who confirms the information taps a predetermined button displayed on the smartphone 1 ( By performing a (touch) operation, it is possible to display an intention of consent to the terms of use or the handling of personal information. In the present embodiment, the act of indicating the intention of consent performed by the consentee using the smartphone 1 is referred to as “agreement act”. In addition, in the present embodiment, a person who has performed a consent act is referred to as a “contractor”. The example illustrated in FIG. 1 is an example in the case where the consenting person and the consenting actor are the same person (user).

In the information processing system according to the present embodiment, as shown in FIG. 2, when the user who should be the consentee cannot perform the consenting act due to, for example, being physically handicapped or being a child. , A person acting on behalf of the consentee is permitted to perform the consenting act on behalf of the consentee. In the present embodiment, a person who performs a consent act on behalf of the consentee is referred to as a "delegator". Further, in this case, the delegator who has performed the consent act is the "contractor". A deputy is a person who is deemed legitimate to give consent on behalf of the consenting individual if the consenting person does not have sufficient consenting ability. Delegates can be, but are not limited to, parents, children, spouses, guardians, parents, supervisors, teachers or coaches.

In the information processing system according to the present embodiment, the smartphone 1 captures an image (hereinafter, referred to as a “face image”) including the face of the person who has performed the consent act, and transmits the image to the server device 3. Memorizes the facial image of the consenting person. In the example shown in FIG. 1, the face image of the consentee is photographed, and in the example shown in FIG. 2, the face image of the substitute is photographed and stored in the server device 3. However, in the example shown in FIG. 2, a face image of the consentee may be captured and stored together with the delegator.

<Device configuration>
FIG. 3 is a block diagram showing the configuration of the smartphone 1 according to the present embodiment. The smartphone 1 according to this embodiment includes a processing unit 11, a storage unit (storage) 12, a camera 13, a communication unit (transceiver) 14, a display unit (display) 15, an operation unit 16, and the like.

The processing unit 11 is configured using an arithmetic processing unit such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The processing unit 11 reads the program 12a stored in the storage unit 12 and executes the program 12a to perform various processes related to the display of information regarding consent, the acceptance of consent action, and the capturing of a facial image.

The storage unit 12 is configured by using a non-volatile memory element such as a flash memory. The storage unit 12 stores various programs executed by the processing unit 11 and various data necessary for the processing of the processing unit 11. In the present embodiment, the storage unit 12 stores a program 12a executed by the processing unit 11. In the present embodiment, the program 12a is distributed by a remote server device or the like, and the smartphone 1 acquires it by communication and stores it in the storage unit 12. However, the program 12a may be written in the storage unit 12 at the manufacturing stage of the smartphone 1, for example. For example, as the program 12a, the smartphone 1 may read the program 12a recorded in a recording medium such as a memory card or an optical disc and store the program 12a in the storage unit 12. For example, the program 12a may be recorded in a recording medium by a writing device and written into the storage unit 12 of the smartphone 1. The program 12a may be provided in the form of distribution via a network, or may be provided in the form of being recorded in a recording medium.

The camera 13 is configured using an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The cameras 13 are provided, for example, at two positions on the front side and the back side of the smartphone 1. The camera 13 gives the image captured by the image sensor to the processing unit 11. In addition, the processing unit 11 can instruct the camera 13 which of the front-side camera 13 and the rear-side camera 13 is used to perform shooting, and switch the shooting direction.

The communication unit 14 communicates with various devices via a network N including a public mobile phone communication network and the Internet. In the present embodiment, the communication unit 14 communicates with the server device 3 via the network N. The communication unit 14 transmits the data given from the processing unit 11 to the server device 3 and gives the data received from the server device 3 to the processing unit 11.

The display unit 15 is configured by using a liquid crystal display or the like, and displays various images and characters based on the processing of the processing unit 11.

The operation unit 16 receives a user's operation through an input device such as a mechanical button or a touch panel provided on the surface of the display unit 15, and notifies the processing unit 11 of the received operation.

Further, in the smartphone 1 according to the present embodiment, the processing unit 11 reads out and executes the program 12a stored in the storage unit 12, whereby the display processing unit 11a, the information input reception unit 11b, the face detection unit 11c, and the consent act. The reception unit 11d, the face image acquisition unit 11e, the screen shot acquisition unit 11f, the storage processing unit 11g, the switching unit 11h, and the like are implemented in the processing unit 11 as software functional units.

The display processing unit 11a performs a process of displaying various characters and images on the display unit 15. In the present embodiment, the display processing unit 11a displays information regarding the terms of use for asking for consent or the handling of personal information, displaying buttons and the like for performing an act of consenting to these, and information regarding consentees or substitutes. The screen for receiving the input of is displayed.

The information input receiving unit 11b performs a process of receiving input of information regarding consent. In the present embodiment, the information input receiving unit 11b receives input of information such as the names or addresses of the consentees or the delegates.

The face detection unit 11c performs a process of detecting the face of a person included in the image captured by the camera 13. The algorithm for face detection from the captured image by the face detection unit 11c is an existing technique, and thus detailed description thereof will be omitted.

The consent act acceptance unit 11d performs a process of accepting an act of consent to information relating to agreement such as the terms of use or the handling of personal information displayed by the display processing unit 11a on the display unit. The consent act acceptance unit 11d accepts the consent act by the consent actor by accepting a tap operation on the button labeled “I agree” displayed on the display unit 15 with the operation unit 16. In the present embodiment, the consent act accepting unit 11d accepts the consent act when the face detecting unit 11c detects a face in the image captured by the camera 13, and accepts the consent act when the face is not detected. Do not accept.

The face image acquisition unit 11e performs a process of acquiring the face image of the consenting person who is taken by the camera 13. In the present embodiment, the face image acquisition unit 11e acquires the image captured by the camera 13 when the consent act acceptance unit 11d accepts the consent act.

The screen shot acquisition unit 11f performs a process of acquiring an entire image including characters and images displayed on the display unit 15 as a screen shot image. In the present embodiment, the screenshot acquisition unit 11f acquires a screen shot image of the display unit 15 when the consent act acceptance unit 11d accepts the consent act.

The storage processing unit 11g stores the information received by the information input reception unit 11b, the face image acquired by the face image acquisition unit 11e, and the screen shot image acquired by the screen shot acquisition unit 11f in association with each other. I do. In the present embodiment, the storage processing unit 11g transmits these pieces of information to the server device 3 via the communication unit 14, and stores the information in a database included in the server device 3. However, the storage processing unit 11g may store such information in its own storage unit 12.

The switching unit 11h performs processing for controlling switching of the shooting direction by the camera 13. In the present embodiment, for example, the cameras 13 are provided at two positions on the front side and the back side of the smartphone 1. The switching unit 11h determines which of the front-side camera 13 and the rear-side camera 13 is used to perform the shooting, and causes the determined one camera 13 to perform the shooting. For example, the switching unit 11h takes a picture with the camera 13 on the front side when photographing the face of the consenting person, and takes a picture with the camera 13 on the rear side when the deputy takes the face of the consenting person. The camera 13 switches the shooting direction so that

FIG. 4 is a block diagram showing the configuration of the server device 3 according to this embodiment. The server device 3 according to this embodiment includes a processing unit 31, a storage unit (storage) 32, a communication unit (transceiver) 33, and the like.

The processing unit 31 is configured using an arithmetic processing device such as a CPU or MPU, a ROM, a RAM, and the like. The processing unit 31 reads the server program 32a stored in the storage unit 32 and executes the server program 32a to execute various processes such as a process of requesting consent from the smartphone 1 and a process of acquiring and storing information regarding consent from the smartphone 1. Process.

The storage unit 32 is configured by using a large-capacity storage device such as a hard disk. The storage unit 32 stores various programs executed by the processing unit 31 and various data necessary for the processing of the processing unit 31. In the present embodiment, the storage unit 32 stores a server program 32a executed by the processing unit 31. Further, the storage unit 32 is provided with a consent information DB (database) 32b that stores information about consent acquired from the smartphone 1, and an image storage unit 32c that stores images captured by the smartphone 1.

In the present embodiment, the server program 32a is provided in a form recorded on a recording medium 99 such as a memory card or an optical disk, and the server device 3 reads the server program 32a from the recording medium 99 and stores it in the storage unit 32. However, the server program 32a may be written in the storage unit 32 at the manufacturing stage of the server device 3, for example. Further, for example, the server program 32a may be acquired by the server device 3 through communication, which is distributed by another remote server device or the like. For example, the server program 32 a may be recorded in the recording medium 99 by a writing device and written in the storage unit 32 of the server device 3. The server program 32a may be provided in the form of distribution via a network, or may be provided in the form recorded on the recording medium 99.

The consent information DB 32b is information regarding consent transmitted from the smartphone 1. The configuration of the consent information DB 32b will be described later.

The image storage unit 32c stores images such as face images and screen shot images taken by the smartphone 1. As described above, the information about the consent transmitted from the smartphone 1 is stored in the consent information DB 32b. However, the substance of the image included in the information from the smartphone 1 is stored in the image storage unit 32c, and the information such as the file name of the image or the address of the storage location is stored in the consent information DB 32b. In the present embodiment, the image storage unit 32c is provided in the server device 3, but the present invention is not limited to this, and a storage device different from the server device 3 may store images.

The communication unit 33 communicates with various devices via a network N including the Internet, a wireless LAN, a mobile phone communication network, and the like. In the present embodiment, the communication unit 33 communicates with the user's smartphone 1 via the network N. The communication unit 33 transmits the data given from the processing unit 31 to another device and gives the data received from the other device to the processing unit 31.

The storage unit 32 may be an external storage device connected to the server device 3. Further, the server device 3 may be a multi-computer including a plurality of computers, or may be a virtual machine virtually constructed by software. Further, the server device 3 is not limited to the above configuration, and may include, for example, a reading unit that reads information stored in a portable storage medium, an input unit that receives an operation input, a display unit that displays an image, or the like. ..

Further, in the processing unit 31 of the server device 3 according to the present embodiment, the consent requesting unit 31a, the information acquisition unit 31b, and the like are provided by the processing unit 31 reading and executing the server program 32a stored in the storage unit 32. It is realized as a software-like functional unit. Note that these functional units are functional units related to the process of obtaining consent from the user, and illustration and description of the functional units other than this, for example, functional units for providing a service to the user, are omitted.

The consent requesting unit 31a performs a process of requesting the user's smartphone 1 to obtain the user's consent when the user's consent is required for the service provision. For example, the consent requesting unit 31a requests a user who downloads the application program to the smartphone 1 and starts using the application program to consent to the terms of use. Further, for example, the consent requesting unit 31a requests consent for handling personal information when the personal information of the user is acquired as the user uses the application program. The consent request unit 31a transmits a consent request message requesting acquisition of the consent to the smartphone 1 of the user at an appropriate time when the consent is required.

The information acquisition unit 31b performs a process of acquiring information transmitted from the smartphone 1 as a response to the consent request message of the consent request unit 31a. The information acquisition unit 31b acquires, from the smartphone 1, information such as the name or address of the person who consented, the face image of the person who consented, and a screenshot image. As shown in FIG. 1, when the contributor is the consentee, the information acquisition unit 31b acquires information such as the name or address of the consentee, the facial image of the consentee, and a screenshot image. When the contributor is a deputy person as shown in FIG. 2, the information acquisition unit 31b causes the information such as the name or address of the consentee, the information of the name or address of the deputy person, or the face image of the deputy person. , And information such as a screenshot image is acquired. In this case, the information acquisition unit 31b may or may not acquire the facial image of the consenting person. The information acquisition unit 31b stores the acquired information in the consent information DB 32b of the storage unit 32.

FIG. 5 is a schematic diagram showing a configuration example of the consent information DB 32b. The consent information DB 32b includes, for example, "application ID", "consent type", "consent date and time", "consent person name", "consent person face image", "delegator name", "delegator face image", and It is configured to include information such as "screenshot image".

The “application ID” is an ID for identifying the application program executed by the smartphone 1. In the present embodiment, the server device 3 provides a plurality of types of application programs, and the application ID is used to determine which application program has performed the consent act. When there is only one application program for which the consent act is performed, the consent information DB 32b need not include the information of the application ID.

The “consent type” distinguishes what kind of condition or agreement has been made, and in the present embodiment, the consent act for two types of “Terms of use” or “Handling of personal information”. Shall be done.

“Agreement date and time” is the date and time when the smartphone 1 accepts the act of consent.

The “sentee name” is information on the name of the consentee whose input is accepted by the smartphone 1. If there is no substitute, this consenter becomes the consenting agent.

The “sentee face image” is a face image of the consentee photographed by the smartphone 1. When the consentee is the consent acter, the consenter face image is required at the time of the consent act, but when the substitute is the consent acter, the consenter face image is always required at the time of the consent act. is not.

“Substitute name” is information of the name of the substitute when the substitute cannot perform the consent and the substitute performs the consent on behalf of the substitute.

The “delegator face image” is a face image of the delegator taken by the smartphone 1. When the consentee performs the consenting action, since the deputy person does not exist, the deputy person name and the deputy person face image are not stored in the consent information DB 32b. If the deputy is a consenting person, the delegator's designation and the deputy's face image are stored.

That is, in the information processing system according to the present embodiment, the smartphone 1 captures a face image of the consenting person who has performed the consenting action, and the server device 3 stores the captured image. Here, in the case where the consenting person is the consenting person himself/herself, a face image of the consenting person who has performed the consenting action is captured by the smartphone 1 and stored as the “sponsor's face image” in the consent information DB 32b. In this case, the information of the "designated delegate" and the "face image of the delegate" is not stored in the consent information DB 32b.

On the other hand, when the deputy performs the consent act on behalf of the consentee, the face image of the delegator who has performed the consent act is photographed by the smartphone 1 and the “delegator face image” is displayed in the consent information DB 32b. Is stored as In this case, the “sentee name” is input by the substitute and is stored in the consent information DB 32b, but the “sentee face image” may or may not be stored in the consent information DB 32b. In this case, whether or not to store the “sentee face image” may depend on, for example, the selection of the substitute, and may be determined at any stage during system design or the like.

The “screenshot image” is a screenshot image obtained when the smartphone 1 makes a consent act.

In the present embodiment, the consent information DB 32b stores information such as a consentee face image, a substituter face image, and a screen shot image such as the file name of the image file or the storage location of the image file, and the substance of the image file is stored. The image is stored in the image storage unit 32c of the unit 32. However, the image file may be stored in the consent information DB 32b. In the figure, these images are in the PNG (Portable Network Graphics) format, but this file format is an example, and various file formats such as GIF (Graphics Interchange Format) or JPEG (Joint Photographic Experts Group). Can be adopted.

<Acquisition process>
In the information processing system according to the present embodiment, when the server device 3 needs the consent of the user when providing the service, the server device 3 transmits the consent request message to the smartphone 1 used by the user. The consent request message includes information such as a command for acquiring consent and the type of consent. The smartphone 1 that has received the consent request message displays a consent actor selection screen for selecting a person who performs the consent act on the display unit 15 in order to obtain the consent from the user.

FIG. 6 is a schematic diagram showing an example of the consent actor selection screen. In the information processing system according to the present embodiment, there are cases where the consent act is performed by the consentee himself and when the consentee is different from the consentee. On the illustrated consent executor selection screen, a button 101 for selecting the consent act by the consent executor himself and a button 102 for selecting the consent act by the substitute. For example, the button 101 is labeled with “I agree with it” and the button 102 is labeled with “I agree with the delegator”. A touch panel that detects a user's touch operation is provided as an operation unit 16 on the display unit 15 of the smartphone 1. The smartphone 1 determines whether the tap operation is performed on the button 101 or the button 102, so that the consentee is selected to perform the consent act, or the delegator performs the consent acter. Determine if a selection was made.

When the tap operation on the button 101 is performed on the consent person selection screen and the consent person performs the consent operation, the smartphone 1 displays a consent person information input screen for inputting the information of the consent person. .. FIG. 7 is a schematic diagram showing an example of the consentee information input screen. On the consentee information input screen of this example, a message "Please enter the name of the consentee (you)." is displayed, an input box 103 is displayed below this message, and a button 104 is further below this message. It is displayed. The input box 103 is for inputting the name of the consentee. When a tap operation or the like is performed on the input box 103, the smartphone 1 displays a software keyboard or the like and accepts input of characters to be input to the input box 103. The button 104 is labeled with “Next”, and when the tap operation is performed on the button 104, the smartphone 1 advances acceptance of the consent act to the next stage. It should be noted that when no character is input in the input box 103, the smartphone 1 may not accept the operation on the button 104. In this example, the name is requested to be input as the information regarding the consentee, but the address, the mail address, the telephone number, or the like may be requested to be input.

When the name of the consentee is input on the consentee information input screen and the tap operation is performed on the button 104, the smartphone 1 displays the consent screen for accepting the consent act. 8 and 9 are schematic diagrams showing an example of the consent screen. In the consent screen of this example, an information display area 105 is provided below the label "Handling of personal information". In the information display area 105, detailed information (hereinafter referred to as consent information) such as a condition for requesting consent or a rule is displayed as characters or images. The detailed consent information has a large amount of information, and there is a high possibility that it will not fit within one screen of the display unit 15 of the smartphone 1. Therefore, the consent screen can be scrolled in the vertical direction by a swipe operation, for example, and a slide bar 106 indicating the scroll position is provided at the right end of the screen. FIG. 8 shows a display example at the top of the consent screen, and FIG. 9 shows a display example at the bottom of the consent screen.

When the consent screen is scrolled to the bottom, a message "Do you agree with the handling of personal information?" and a message "Agreeee name: MMM" are displayed below the information display area 105, A face photographing area 107 is displayed below this message. The consentee name in the message is the name accepted by the consentee information input screen shown in FIG. 7. An image captured by the camera 13 of the smartphone 1 is displayed in the face capturing area 107. A guide mark for accommodating the face of the consenting person (sponsor) is displayed inside the face photographing area 107, and the consenting person positions the smartphone 1 with respect to his/her face so that his/her face fits within the guide mark. adjust. At this time, the smartphone 1 shoots with the camera 13 provided on the front side of the housing of the smartphone 1 and above the display unit 15, and the camera 13 (not shown) provided on the back side No shooting is done. Below the face photographing area 107, a consent button 108 for accepting the consent act is displayed. By performing the tap operation on the consent button 108, it is considered that the consent act has been performed by the consent person.

In addition, in the information processing system according to the present embodiment, the smartphone 1 performs a face detection process of detecting a human face from an image captured by the camera 13. In the present example, the smartphone 1 performs a process of enabling/disabling the consent button 108 depending on whether or not a human face is detected in the guide mark of the face photographing area 107 of the consent screen. In the example shown in FIG. 9, since the human face is not captured (not detected) in the face photographing area 107, the consent button 108 is invalidated, and the tap operation on the consent button 108, that is, the operation of consent action is performed. Is not accepted.

FIG. 10 is a schematic diagram showing an example of a consent screen in which the consent button 108 is activated. In this example, the smartphone 1 activates the consent button 108 because the human face is detected in the guide mark of the face photographing area 107 of the consent screen. In this figure, the consent button 108 is hatched to indicate that the consent button 108 has been activated. When the tap operation on the consent button 108 is performed in this state, the smartphone 1 accepts this tap operation as a consent act, and acquires the image displayed in the face shooting area 107 at this time, that is, the face image of the consent actor. To do. Further, in the present embodiment, the smartphone 1 further acquires the screen shot image at this time. The screenshot image is a whole image displayed on the display unit 15 of the smartphone 1, and an image corresponding to the display example in the display unit 15 illustrated in FIG. 10 is acquired as the screenshot image. ..

When the consent act is made by the consent person by the tap operation on the consent button 108, the smartphone 1 transmits a response message to the consent request message to the server device 3. In the response message, the name of the consenting person who accepted the input on the consenting person information input screen, the face image of the consenting person photographed on the consenting screen, and the information of the screenshot image when the consenting action was made included. Further, the smartphone 1 may include information such as the application ID of the application program for which the consent act has been performed, the consent type, and the consent date and time in the response message. The server device 3 that receives the response message stores these pieces of information included in the received response message in association with the consent information DB 32b of the storage unit 32.

In addition, in the information processing system according to the present embodiment, instead of the consentee who is difficult to perform the consent act, the delegator can perform the consent act. When the tap operation is performed on the button 102 on the consent actor selection screen illustrated in FIG. 6 and the delegate has selected to perform the consent act, the smartphone 1 inputs the information of the consenter and the delegate. Display the information input screen for the substituting person's information.

FIG. 11 is a schematic diagram showing an example of the consentee-delegator information input screen. On the screen for inputting the consentee's-delegator's information in this example, a message "Please enter the name of the agent (you)" is displayed, and the first input box 109 is displayed below this message. Also, below the first input box 109, a message "Please input the name of the consentee." is displayed, below this message is a second input box 110, and below this is a button 111. Is displayed. The first input box 109 is for inputting the name of the delegate, and the second input box 110 is for inputting the name of the consenter. The button 111 is labeled “Next”, and when the tap operation is performed on the button 111, the smartphone 1 advances acceptance of the consent action to the next stage. In addition, when no character is input in the first input box 109 or the second input box 110, the smartphone 1 may not accept the operation on the button 111.

When the name of the deputy person and the name of the consentee are input on the consentee delegator information input screen and the tap operation is performed on the button 111, the smartphone 1 displays the consent screen for accepting the consent act. At this time, the consent screen displayed by the smartphone 1 is the same as the consent screen shown in FIGS. In this case, the face photographed using the face photographing area 107 of the consent screen is the delegator's face. For this reason, not the name of the consenting person, but the name of the legal person who has accepted the input on the consenting person legal person information input screen is displayed above the face photographing area 107 of the consent screen. When the tap operation on the consent button 108 is performed in a state where the face of the substitute is detected in the face capturing area 107, the smartphone 1 accepts this operation as an act of consent on behalf of the substitute, and the smartphone 1 captures the face capturing area. The face image 107 and the screen shot image of the display unit 15 are acquired.

Next, the smartphone 1 displays a consentee photographing screen for the delegator to photograph a face image of the consentee. FIG. 12 is a schematic diagram showing an example of the consentee photographing screen. The consent person shooting screen of this example has substantially the same configuration as the consent screen shown in FIGS. 8 to 10, and the consent button 108 of the consent screen is replaced with a shooting button 112 labeled with “take a picture”. The composition. However, the smartphone 1 takes a picture with the camera 13 on the front side when the consent screen is displayed, but takes a picture with the camera 13 on the back side when the consentee picture is displayed. The consent information does not have to be displayed on the consentee photographing screen.

When the consentee has photographed the consentee by tapping the photographing button 112, the smartphone 1 transmits a response message to the consent request message to the server device 3. In the response message, the names of the consentee and the substitute who accepted the input on the consentee information input screen, the face images of the consentee (substitute) photographed on the consent screen, and when the consent act was made Information of the screenshot image of, and the face image of the consentee photographed on the consentee photographing screen are included. Further, the smartphone 1 may include information such as the application ID of the application program for which the consent act has been performed, the consent type, and the consent date and time in the response message. The server device 3 that receives the response message stores these pieces of information included in the received response message in association with the consent information DB 32b of the storage unit 32.

Note that the image of the face image of the consenting person may not be taken when the deputy person performs the consenting action, and may be performed at another later opportunity. For this reason, the smartphone 1 inquires of the deputy person whether to photograph the facial image of the consentee before displaying the image capturing screen of the consentee, and agrees when a response indicating that the image of the consentee is taken is obtained. The person photographing screen may be displayed.

<Flowchart>
13 and 14 are flowcharts showing the procedure of the consent acquisition process performed by the smartphone 1 according to the present embodiment. The processing unit 11 of the smartphone 1 according to the present embodiment determines whether or not the consent request message has been received from the server device 3 (step S1). When the consent request message is not received (S1: NO), the processing unit 11 waits until the consent request message is received.

When the consent request message is received (S1: YES), the display processing unit 11a of the processing unit 11 displays the consent actor selection screen shown in FIG. 6 on the display unit 15 (step S2).

The processing unit 11 determines whether or not "the person agrees" is selected depending on which of the button 101 and the button 102 is selected on the consent actor selection screen (step S3).

When “the person agrees” is selected on the consent actor selection screen (S3: YES), the display processing unit 11a displays the consent information input screen illustrated in FIG. 7 on the display unit 15 (step S4).

The information input receiving unit 11b of the processing unit 11 determines whether or not the input of information on the consentee information input screen is completed (step S5). Whether or not the input of information is completed is determined by whether or not the name of the consentee is input in the input box 103 of the consentee information input screen and the tap operation is performed on the button 104. When the input of information is not completed (S5: NO), the information input receiving unit 11b returns the process to step S4, maintains the display of the consentee information input screen, and waits until the input of information is completed. ..

When the input of information is completed (S5: YES), the switching unit 11h of the processing unit 11 activates the camera 13 provided on the front surface of the housing of the smartphone 1 to start photographing (step S6).

The display processing unit 11a displays the consent screen shown in FIG. 8 on the display unit 15 (step S7).

The processing unit 11 determines whether or not the consent screen displayed on the display unit 15 has been scrolled to the bottom (step S8). When the consent screen is not scrolled to the bottom (S8: NO), the processing unit 11 returns the process to step S7 and continues to display the consent screen.

When the consent screen is scrolled to the bottom (S8: YES), the face detection unit 11c of the processing unit 11 performs face detection processing on the image captured by the camera 13 (step S9).

The processing unit 11 determines the presence or absence of the face of the consent actor (consent person) at a predetermined position in the face photographing area 107 of the consent screen based on the processing result of the face detection unit 11c (step S10). When there is no face at the predetermined position in the face image capturing area 107 (S10: NO), the processing unit 11 returns the process to step S9 and continues the image capturing by the camera 13 and the face detecting process from the captured image.

When the face is present at the predetermined position in the face image capturing area 107 (S10: YES), the processing unit 11 activates the consent button 108 provided at the bottom of the consent screen (step S11).

The consent act acceptance unit 11d of the processing unit 11 determines whether or not the consent acter has performed the consent act depending on whether or not the consent button 108 on the consent screen is tapped (step S12). When the act of consent has not been made (S12: NO), the processing unit 11 returns the process to step S9.

When the consent act is performed (S12: YES), the face image acquisition unit 11e of the processing unit 11 has performed the consent action, and the consent action was captured by the camera 13 (the face image capturing area 107 was captured). The face image of the person is acquired (step S13).

Further, the screen shot acquisition unit 11f of the processing unit 11 acquires the screen shot image of the image displayed on the display unit 15 when the consent act is performed (step S14).

The storage processing unit 11g of the processing unit 11 stores information such as the name of the consentee who received the input on the consentee information input screen, the face image acquired in step S13, and the screenshot image acquired in step S14. A response message containing the same is transmitted to the server device 3 (step S15), these pieces of information are stored in the server device 3, and the process ends.

On the other hand, when "the person agrees" is not selected in the consent actor selection screen (S3: NO), that is, when "the delegator agrees" is selected, the display processing unit 11a displays the information illustrated in FIG. The consenter-delegator information input screen is displayed on the display unit 15 (step S21).

The information input reception unit 11b of the processing unit 11 determines whether or not the input of information on the consentee-delegator information input screen is completed (step S22). Whether or not the information has been input is determined by inputting the name of the substitute in the first input box 109 and inputting the name of the consent in the second input box 110 on the consent-substitute-information input screen. In addition, it is determined whether or not the tap operation is performed on the button 111. When the input of information is not completed (S22: NO), the information input receiving unit 11b returns the process to step S21, maintains the display of the consentee/legal person information input screen, and completes the input of information. Wait until.

When the input of information is completed (S22: YES), the switching unit 11h of the processing unit 11 activates the camera 13 provided on the front surface of the housing of the smartphone 1 to start shooting (step S23).

The display processing unit 11a displays the consent screen shown in FIG. 8 on the display unit 15 (step S24).

The processing unit 11 determines whether or not the consent screen displayed on the display unit 15 has been scrolled to the bottom (step S25). When the consent screen is not scrolled to the bottom (S25: NO), the processing unit 11 returns the process to step S24 and continues to display the consent screen.

When the consent screen is scrolled to the bottom (S25: YES), the face detection unit 11c of the processing unit 11 performs face detection processing on the image captured by the camera 13 (step S26).

The processing unit 11 determines whether or not there is a face of a consenting actor (delegator) at a predetermined position in the face photographing area 107 of the consent screen based on the processing result of the face detecting unit 11c (step S27). When there is no face at the predetermined position in the face shooting area 107 (S27: NO), the processing unit 11 returns the process to step S26 and continues the shooting by the camera 13 and the face detection process from the shot image.

When the face is located at a predetermined position in the face image capturing area 107 (S27: YES), the processing unit 11 activates the consent button 108 provided at the bottom of the consent screen (step S28).

The consent act acceptance unit 11d of the processing unit 11 determines whether or not the consent act has been performed by the consent actor according to whether or not the tap operation on the consent button 108 on the consent screen has been performed (step S29). If the consent act has not been made (S29: NO), the processing section 11 returns the process to step S26.

When the consent act is performed (S29: YES), the face image acquisition unit 11e of the processing unit 11 is the consent action that was captured by the camera 13 when the consent action was performed (shown in the face image capturing area 107). The face image of the person is acquired (step S30).

Further, the screen shot acquisition unit 11f of the processing unit 11 acquires a screen shot image of the image displayed on the display unit 15 when the consent act is performed (step S31).

Next, the switching unit 11h of the processing unit 11 activates the camera 13 provided on the back surface of the housing of the smartphone 1 to start shooting (step S32).

The display processing unit 11a displays the consentee photographing screen shown in FIG. 12 on the display unit 15 (step S33).

The face detection unit 11c of the processing unit 11 performs face detection processing on the image captured by the camera 13 (step S34).

The processing unit 11 determines the presence/absence of the face of the consentee at a predetermined position in the face imaging area 107 of the consentee imaging screen based on the processing result of the face detection unit 11c (step S35). When there is no face at the predetermined position in the face shooting area 107 (S35: NO), the processing unit 11 returns the process to step S32 and continues the shooting by the camera 13 and the face detection process from the shot image.

When the face is located at the predetermined position in the face image capturing area 107 (S35: YES), the processing unit 11 activates the image capturing button 112 provided at the bottom of the consentee image capturing screen (step S36).

The processing unit 11 determines whether or not the delegator has performed the photographing operation of the consentee according to whether or not the photographing button 112 on the consentee photographing screen has been tapped (step S37). When the photographing operation is not performed (S37: NO), the processing unit 11 returns the process to step S32.

When the shooting operation is performed (S37: YES), the face image acquisition unit 11e of the processing unit 11 acquires the face image of the consenting person displayed in the face shooting area 107 when the shooting operation is performed (step S38). ).

The storage processing unit 11g of the processing unit 11 receives the names of the consentees and the consentees who have received the input on the consentee proxy information input screen, the face image of the authority acquired in step S30, and in step S31. A response message including the acquired screen shot image and information such as the face image of the consentee acquired in step S38 is transmitted to the server device 3 (step S39), and the information is stored in the server device 3, The process ends.

FIG. 15 is a flowchart showing a procedure of consent acquisition processing performed by the server device 3 according to the present embodiment. The consent requesting unit 31a of the processing unit 31 of the server device 3 according to the present embodiment includes a command for acquiring consent and a consent including information such as the type of consent when it is necessary to obtain the user's consent when providing the service. The request message is transmitted to the smartphone 1 (step S51).

The information acquisition unit 31b of the processing unit 31 determines whether or not a response message to the consent request message from the smartphone 1 has been received (step S52). When the response message has not been received (S52: NO), the information acquisition unit 31b waits until the response message is received.

When the response message is received (S52: YES), the information acquisition unit 31b acquires the information such as the name of the consenting actor, the face image, and the screenshot image included in the received response message (step S53).

The information acquisition unit 31b stores the acquired information in association in the consent information DB 32b of the storage unit 32 (step S54), and ends the process.

<Summary>
In the information processing system according to this embodiment having the above configuration, the smartphone 1 includes the camera 13 and the display unit 15. The smartphone 1 is provided with an information display area 105 that displays consent information such as a sentence for asking for consent regarding the handling of personal information or consent to the terms of use, and a face shooting area 107 that displays an image taken by the camera 13. The consent screen is displayed on the display unit 15. The smartphone 1 acquires the face image of the consenting person displayed in the face photographing area 107 and transmits the acquired face image to the server device 3 to store the consent image in the consent information DB 32b of the server device 3. By acquiring and storing the face image of the person displayed in the face shooting area 107 together with the consent information on the consent screen, it is possible to store the face image of the consenting person who has performed the consent act for the consent information. it can.

In addition, the smartphone 1 according to the present embodiment displays an consent button 108 for accepting an operation related to a consent act on the consent screen together with the information display area 105 for displaying the consent information and the face shooting area 107 for shooting the face image. .. The smartphone 1 also performs a face detection process of detecting a face included in the image captured by the camera 13. The smartphone 1 validates the consent button 108 and accepts the operation when a face is detected in the captured image, and invalidates the consent button 108 and does not accept the operation when the face is not detected. Accordingly, the smartphone 1 can reliably acquire the face image of the person who performs the consent act.

Further, the smartphone 1 according to the present embodiment scrolls and displays the consent screen on the display unit 15. The smartphone 1 displays the face photographing area 107 together with the information display area 105 after the information display area 105 displaying the consent information on the consent screen. By displaying the information display area 105 first and then displaying the face shooting area 107, the consent information can be surely confirmed by the consenting person.

In addition, the smartphone 1 according to the present embodiment accepts input of information related to a contributor (a consentor or a substitute), for example, information such as name, date of birth and address, and the received information and face image. It is associated and stored in the consent information DB 32b of the server device 3. As a result, it is possible to store various kinds of information related to the consent executor together with the face image, and it is possible to easily identify who the consent executor is.

In addition, the smartphone 1 according to the present embodiment acquires a screen shot image of the consent screen including the information display area 105 and the face shooting area 107 displayed on the display unit 15, and associates it with the facial image of the consenting person to perform the server operation. It is stored in the consent information DB 32b of the device 3. Accordingly, one image including the consent information displayed in the information display area 105 and the face displayed in the face shooting area 107 can be stored, and the person whose face is shot is the consent actor. You can prove that. Further, the image of the face shooting area 107 included in the screenshot image matches the face image of the consenting person. By comparing the image of the face photographing area 107 included in the screenshot image with the face image of the consent actor, it is confirmed that the face image associated with the information of the consent actor is the face image of the consent actor. We can prove it more clearly.

Further, in the information processing system according to the present embodiment, instead of the consenting person performing the consenting action, the deputy agent (agent) can perform the consenting action on behalf of the agent. The smartphone 1 accepts the input of information related to the consentee and the authority on the consentee-delegator information input screen, and stores the received information and the face image in association with the consent information DB 32b of the server device 3. .. As a result, the substitute can perform the consent act on behalf of a person who is difficult to perform the consent act, such as a physically handicapped person or a young person.

In addition, the smartphone 1 according to the present embodiment switches the shooting direction of the camera 13 when shooting the consentee, depending on whether the consentee performs the consent act or the substitute. The smartphone 1 has cameras 13 mounted on the front and the back of the housing, for example, and when the delegator performs a consent act, the camera 13 on the front side takes a picture on the consent screen, and the delegator agrees. When taking a picture of a person, the camera 13 on the back side takes a picture in the consent person photographing screen. Further, the smartphone 1 may include, for example, a camera 13 whose shooting direction is changeable by movement, and in this case, the camera 13 may be moved in a direction suitable for shooting the consentee to shoot. This facilitates shooting for obtaining the facial image of the consentee.

In this embodiment, the consent screen displayed on the display unit 15 of the smartphone 1 is scrolled in the vertical direction, but the present invention is not limited to this, and may be scrolled in the horizontal direction, for example. When the consent screen is scrolled from right to left, the face shooting area 107 is displayed at the left end of the consent screen, and when the consent screen is scrolled from left to right, the face shooting area 107 is displayed at the right end of the consent screen. To be done. Further, the smartphone 1 may display the consent screen information display area 105 and the face shooting area 107 as separate layers in an overlapping manner, scroll the consent information in the information display area 105, and always display the face shooting area 107. .. The information display area 105 and the face shooting area 107 of the consent screen may be simultaneously displayed in one screen of the display unit 15.

Further, the smartphone 1 may acquire not only the facial image of the consent-executor but also an image of another part of the body of the consent-executor. Moreover, the smartphone 1 may acquire a moving image instead of acquiring a still image of the consenting person. In this case, the smartphone 1 may acquire, for example, a moving image captured by the camera 13 from the time when the consent screen is displayed to the time when the consent button 108 is tapped and stored in the consent information DB 32b of the server device 3. However, it is preferable that the face of the consenting person is photographed in the moving image.

Further, the consent act by the consent acter may be accepted by a device capable of voice input such as the smartphone 1 or a smart speaker. In this case, the device capable of voice input may receive the information such as the name of the consenting person and the consenting operation by voice using a technique such as natural language processing.

Further, the smartphone 1 may acquire the scroll speed of the consent screen displayed on the display unit 15 and store the scroll speed in the consent information DB 32b of the server device 3 together with the face image. When the scroll speed is within a predetermined range, it can be inferred that the consent executor was reading the consent information during the consent act. Further, the smartphone 1 may switch between enabling and disabling the consent button 108 according to the scroll speed of the consent screen. As a result, the scroll speed can be limited to a predetermined range, and the consent executor can more surely confirm the consent information.

<Second Embodiment>
The information processing system according to the second embodiment uses a face image stored in the consent information DB 32b of the server device 3 to search for a consentee or a delegator who has already performed an act of consenting, in other words, still being. This is a system that searches for people who have not performed consent. A user who uses this search function uses the camera 13 of the smartphone 1 to capture an image of the surrounding area. The smartphone 1 according to the second embodiment detects a human face included in the image captured by the camera 13 and transmits the detected face image to the server device 3.

The server device 3 according to the second embodiment compares the face image received from the smartphone 1 with the face image stored in the consent information DB 32b. The server device 3 determines whether the person of the face image received from the smartphone 1 matches the person of the face image stored in the consent information DB 32b based on the comparison result of the both face images, and the determination result is displayed. Send to smartphone 1.

The smartphone 1 displays, based on the determination result received from the server device 3, whether or not the person included in the image captured by the camera 13 is the consentee or the delegator who has already performed the consent act. .. FIG. 16 is a schematic diagram showing an example of a search screen displayed on the smartphone 1. The smartphone 1 displays the image captured by the camera 13 on the display unit 15, and displays the rectangular frames 113a to 113c surrounding the face detected from this image. For example, the smartphone 1 displays a rectangular frame 113a surrounding a face determined to be the same person as the face image of the consentee stored in the consent information DB 32b of the server device 3 in red (represented by a thick line in this figure). The smartphone 1 displays, in blue (represented by a thin line in this figure), a rectangular frame 113b surrounding a face determined to be the same person as the face image of the substitute stored in the consent information DB 32b of the server device 3. The smartphone 1 displays a quadrangular frame 113c surrounding the face that does not match the consentee or the substitute in white (represented by a broken line in this figure). By performing these color-coded displays, the user can easily determine whether or not the person present in the vicinity is the person already stored in the agreement information DB 32b as the consentee or the substitute.

The information processing system according to the second embodiment having the above configuration is based on the facial image already stored in the consent information DB 32b of the server device 3, based on the facial image already captured by the camera 13 of the smartphone 1 or the unidentified person. Detect consent. The face included in the image captured by the smartphone 1 is detected, and the server device 3 determines whether the detected face matches the face image already stored in the consent information DB 32b. If the face images match, the person with this face can be regarded as the consent person, and if they do not match, the person with this face can be regarded as the consent person. As a result, the user can easily find the consented person or the unconfirmed person using the smartphone 1.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above meaning but by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.

1 Smartphone (information processing device, computer)
3 Server device (storage device)
11 processing unit 11a display processing unit 11b information input receiving unit (information receiving unit)
11c Face detection unit 11d Consent act acceptance unit 11e Face image acquisition unit 11f Screen shot acquisition unit 11g Storage processing unit 11h Switching unit 12 Storage unit 12a Program (computer program)
13 camera 14 communication unit 15 display unit 16 operation unit 31 processing unit 31a consent request unit 31b information acquisition unit 32 storage unit 32a server program 32b consent information DB
32c image storage unit 33 communication unit 101 button 102 button 103 input box 104 button 105 information display area (consent information)
106 slide bar 107 face shooting area (shooting image area)
108 Agree button (operation reception part)
109 first input box 110 second input box 111 button 112 shooting button 113a to 113c square frame

Claims (10)

An information processing apparatus including a display unit and a camera,
A display processing unit that performs a process of displaying on the display unit both the consent information relating to the request for consent and the captured image area displaying the image captured by the camera;
A face image acquisition unit for acquiring a face image of the consenting person displayed in the captured image area;
A storage processing unit that stores the face image acquired by the face image acquisition unit in a storage unit ;
The display processing unit,
Scrolling and displaying the consent information on the display unit,
After the consent information in the scroll, the captured image area is displayed on the display unit together with the consent information.
Information processing device. The display processing unit displays an operation accepting unit for accepting an operation relating to a consent act on the display unit, together with the consent information and the captured image area,
A face detection unit that detects a face included in the image captured by the camera,
When the face detection unit detects a face, accepts an operation on the operation accepting unit,
When the face detection unit does not detect a face, the operation to the operation reception unit is not accepted,
The information processing apparatus according to claim 1. An information accepting unit that accepts the input of the information relating to the contributor
The storage processing unit stores the information received by the information receiving unit and the face image in association with each other,
The information processing apparatus according to claim 1 or 2 . A screenshot acquisition unit for acquiring the consent information displayed on the display unit and a screenshot image of the captured image region,
The storage processing unit stores the screen shot image acquired by the screen shot acquisition unit and the face image in association with each other,
The information processing apparatus according to any one of claims 1 to 3 . When the agent acts on behalf of the consentee to perform the consent act, an information reception unit that receives information relating to the consentee and the agent is provided,
The storage processing unit stores the information received by the information receiving unit and the face image in association with each other,
The information processing apparatus according to any one of claims 1 to 4 . A switching unit that switches the shooting direction of the camera when shooting the consentee according to whether the consentee performs the consent act or the agent acts on behalf of the agent;
The information processing device according to claim 5 . Based on the face image stored by the storage processing unit, detecting a consenting person or a non-consentee included in the image captured by the camera,
The information processing apparatus according to any one of claims 1 to 6 . An information processing apparatus according to any one of claims 1 to 7 ,
A storage device having the storage part for storing the face image by processing of the storage processing part of the information processing device,
The storage processing unit of the information processing device transmits the face image acquired by the face image acquisition unit to the storage device and stores the face image in the storage unit of the storage device.
Information processing system. A computer with a display and a camera,
Scroll and display the consent information related to the consent request on the display unit,
After the agreement information in the scroll, together with the agreement information, to display the captured image area for displaying an image in which the camera is taken before Symbol display unit,
Acquiring the facial image of the consenting person displayed in the captured image area,
A computer program that performs the process of storing the acquired face image. On the display unit of the information processing device equipped with a camera, scroll and display the consent information relating to the consent request ,
After the consent information in the scroll, together with the consent information, a captured image area for displaying an image captured by the camera is displayed on the display unit ,
Acquiring the facial image of the consenting person displayed in the captured image area,
Memorize the acquired face image,
Information processing method.

Images