WO2022196042A1

WO2022196042A1 - Support device, system, and method, and computer-readable medium

Info

Publication number: WO2022196042A1
Application number: PCT/JP2022/000279
Authority: WO
Inventors: 久美子高塚; 真由美伊藤; 哲也冬野
Original assignee: 日本電気株式会社
Priority date: 2021-03-19
Filing date: 2022-01-06
Publication date: 2022-09-22
Also published as: JPWO2022196042A1

Abstract

The present invention provides appropriate support according to the potential interests of a person to be supported. A support device (1) comprises a registration unit (11) that registers a person to be supported and a supporting person in association with each other, an acquisition unit (12) that acquires a first captured image taken by the person to be supported, a specification unit (13) that specifies the interests of the person to be supported on the basis of the first captured image, a determination unit (14) that determines one or more support information candidates based on the specified interests, and a presentation unit (15) that presents the determined support information candidate(s) to the supporting person associated with the person to be supported.

Description

SUPPORT DEVICE, SYSTEM AND METHOD, AND COMPUTER-READABLE MEDIUM

The present invention relates to a support device, system, method and program, and more particularly to a support device, system, method and program for supporting a subject by a supporter.

In recent years, the declining birthrate and depopulation have progressed, and the need to watch over children and the elderly has increased. Patent Literature 1 discloses a technology related to a monitoring system. In the watching system, a camera having a face recognition function photographs a person being watched over, and transmits the photographed moving image to an information processing device via the Internet.

JP 2017-111506 A

Here, the supporter who lives away from the target of support (for example, children under elementary school age or the elderly) can grasp the potential interest of the target and provide appropriate support according to the interest. It has been demanded.

The present disclosure is made to solve such problems, and provides a support device, system, method, and program for performing appropriate support according to the potential interest of the support target person. for the purpose.

A support device according to a first aspect of the present disclosure includes:
a registration unit that associates and registers a support target person and a supporter;
an acquisition unit that acquires a first captured image captured by the subject;
a specifying unit that specifies the subject's interest based on the first captured image;
a determination unit that determines one or more candidates for assistance information based on the identified interest;
a presentation unit that presents the determined support information candidate to the supporter associated with the target person;
Prepare.

A support system according to a second aspect of the present disclosure includes:
a first terminal of a support target person;
a supporter's second terminal;
a support device;
The support device is
a registration unit that associates and registers the target person and the supporter;
an acquisition unit that acquires a first captured image captured by the subject from the first terminal;
a specifying unit that specifies the subject's interest based on the first captured image;
a determination unit that determines one or more candidates for assistance information based on the identified interest;
a presentation unit that presents the determined support information candidate to the second terminal of the supporter associated with the target person;
Prepare.

A support method according to a third aspect of the present disclosure includes:
the computer
Corresponding and registering the target of support and the supporter,
Acquiring a first captured image captured by the subject;
Identifying the subject's interest based on the first captured image;
determining one or more candidates for assistance information based on the identified interests;
The determined support information candidate is presented to the supporter associated with the target person.

The support program according to the fourth aspect of the present disclosure is
a registration process for registering a support target person and a supporter in association with each other;
Acquisition processing for acquiring a first captured image captured by the subject;
a specifying process of specifying the subject's interest based on the first captured image;
a determination process of determining one or more candidates for assistance information based on the identified interest;
a presentation process of presenting the determined support information candidate to the supporter associated with the target person;
run on the computer.

With the present disclosure, it is possible to provide a support device, system, method, and program for providing appropriate support according to the latent interest of the support target.

1 is a block diagram showing the configuration of a support device according to the first embodiment; FIG. 4 is a flow chart showing the flow of a support method according to the first embodiment; FIG. 11 is a block diagram showing the overall configuration of a support system according to a second embodiment; FIG. FIG. 11 is a block diagram showing the configuration of a tablet terminal according to the second embodiment; FIG. FIG. 11 is a block diagram showing the configuration of an authentication device according to the second embodiment; FIG. FIG. 11 is a block diagram showing the configuration of a support device according to a second embodiment; FIG. FIG. 11 is a sequence diagram showing the flow of pre-registration processing according to the second embodiment; 9 is a flow chart showing the flow of face information registration processing by the authentication device according to the second embodiment; 10 is a flowchart showing the flow of collection processing according to the second embodiment; 9 is a flow chart showing the flow of face authentication processing by the authentication device according to the second embodiment; 10 is a flow chart showing the flow of support information provision processing according to the second embodiment; FIG. 11 is a diagram showing an example of a screen for presenting support information candidates to a supporter according to the second embodiment; FIG. 11 is a diagram for explaining the concept of presentation of support information candidates, selective purchase, delivery, and notification of a thank-you comment according to the second embodiment; FIG. 11 is a block diagram showing the configuration of a support device according to a third embodiment; FIG. FIG. 11 is a sequence diagram showing the flow of inquiry processing for a captured image according to the third embodiment; FIG. 11 is a diagram showing an example of a selection screen for favorites of captured images according to the third embodiment; FIG. 16 is a sequence diagram showing another example of the flow of inquiry processing for a captured image according to the third embodiment; FIG. 11 is a diagram showing an example of a tag candidate selection screen in a quiz format according to the third embodiment; FIG. 11 is a diagram showing an example of an input screen for attribute information of a captured image according to the third embodiment; FIG. 14 is a sequence diagram showing the flow of storytelling processing according to the fourth embodiment;

Below, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same reference numerals are given to the same or corresponding elements, and redundant description will be omitted as necessary for clarity of description.

<Embodiment 1>
FIG. 1 is a block diagram showing the configuration of a support device 1 according to the first embodiment. The assistance device 1 is an information processing device for assisting a target person to be assisted by a supporter. The support device 1 presents the supporter with support information candidates that meet at least the needs (direct or indirect requests) of the subject. Note that the support device 1 can also be said to be a watching support device that assists a watcher (related person of the target person) to remotely watch over the person being watched (the target person). The target of support is a target of watching over, such as a child or the elderly. For example, if the target person is a child of elementary school age or younger, the supporter is the target person's grandparents or the like. However, the supporter may be an elderly person living alone, or an elderly person who wishes to provide assistance to any subject even if there is no blood relationship. In addition, when the target of support is an elderly person or a person requiring nursing care, the supporter may be a relative of the target (children and couples of the target, etc.), a care worker, an adult guardian, or the like. Here, it is assumed that the support device 1 is connected to the target person's terminal and the supporter's terminal via a communication network (not shown, hereinafter, the communication network is simply referred to as a network) or predetermined wireless communication. . It does not matter whether the network is wired or wireless, and the type of communication protocol does not matter. Here, the terminal may be an information processing terminal equipped with a camera, a microphone, a speaker, a touch panel, etc., such as a tablet terminal.

The support device 1 includes a registration unit 11 , an acquisition unit 12 , an identification unit 13 , a determination unit 14 and a presentation unit 15 . The registration unit 11 associates and registers a support target person and a support person. Acquisition unit 12 acquires a first captured image captured by a subject. It is assumed that the first photographed image includes an object, scenery, or the like in which the subject is interested. The specifying unit 13 specifies the subject's interest based on the first captured image. The determination unit 14 determines one or more support information candidates based on the specified interest. The presentation unit 15 presents the determined support information candidate to the supporter associated with the target person. Note that the identifying unit 13 may analyze the first captured image to extract attribute information (keywords), and identify interests using a model that inputs keywords and outputs interests. Alternatively, the specifying unit 13 and the determining unit 14 may determine support information candidates using a model that inputs the first captured image, attribute information, and the like and outputs support information candidates. It should be noted that a known technique such as an AI (Artificial Intelligence) model can be applied to the model. Also, the model can be machine-learned using stored captured images or attribute information as teacher data and interest or support information candidates as correct answers.

FIG. 2 is a flow chart showing the flow of the support method according to the first embodiment. First, the registration unit 11 associates and registers a support target person and a supporter (S11). For example, the registration unit 11 acquires the identification information of the target person and the supporter from the terminal of the target person, the target person's guardian, or the supporter, and stores the identification information in the storage device in association with each other. . Here, the storage device may be internal or external to the support device 1 and connected to the support device 1 .

Next, the acquisition unit 12 acquires the first captured image captured by the subject (S12). For example, it is assumed that a subject uses a terminal to photograph an object of interest to the subject, and the terminal transmits the photographed image of the object to the support device 1 .

Then, the identifying unit 13 identifies the subject's interest based on the first captured image (S13). The determination unit 14 determines one or more support information candidates based on the specified interest (S14). Thereafter, the presentation unit 15 presents the determined support information candidate to the supporter associated with the target person (S15). For example, the presentation unit 15 transmits the determined support information candidate to the terminal possessed by the supporter, and the terminal displays the received support information candidate. As a result, the supporter can visually recognize the recommended support information candidate through the terminal.

As described above, in this embodiment, it is possible to identify the potential interest of the target person based on the photographed image taken by the target person of the support, and recommend the support information candidate according to the interest to the supporter. Therefore, the supporter can select support information candidates that match the interest of the target person and provide it to the supporter. As a result, it is possible to appropriately and effectively support the target person.

The support device 1 includes a processor, memory, and storage device (not shown). Further, the storage device stores a computer program in which processing of the support method according to the present embodiment is implemented. Then, the processor loads the computer program from the storage device into the memory and executes the computer program. Thereby, the processor implements the functions of the registration unit 11 , the acquisition unit 12 , the identification unit 13 , the determination unit 14 and the presentation unit 15 .

Alternatively, the registration unit 11, the acquisition unit 12, the identification unit 13, the determination unit 14, and the presentation unit 15 may each be realized by dedicated hardware. Also, part or all of each component of each device may be implemented by general-purpose or dedicated circuitry, processors, etc., or combinations thereof. These may be composed of a single chip, or may be composed of multiple chips connected via a bus. A part or all of each component of each device may be implemented by a combination of the above-described circuits and the like and programs. As a processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a quantum processor (quantum computer control chip), or the like can be used.

Further, when a part or all of each component of the support device 1 is realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or distributed. may be placed. For example, the information processing device, circuits, and the like may be implemented as a form in which each is connected via a communication network, such as a client-server system, a cloud computing system, or the like. Also, the functions of the support device 1 may be provided in a SaaS (Software as a Service) format.

<Embodiment 2>
The second embodiment is a specific example of the first embodiment described above. FIG. 3 is a block diagram showing the overall configuration of a support system 1000 according to the second embodiment. The support system 1000 is an information system for supporting a target person (grandchild) U1 by a supporter (grandparent) U3 or a target person's parent U2. In the following, for convenience of explanation, a group of a target person (grandchild) U1, a target person's parent U2, and a supporter (grandparent) U3 will be targeted, but the present invention is not limited to this. That is, the support system 1000 may target two or more groups of one or more target persons and one or more supporters.

In the following description, face authentication, which is an example of biometric authentication, is used as authentication for personal identification, and facial feature information, which is an example of biometric information, is used as personal identification information. However, biometric authentication and biometric information can be applied to other techniques that use captured images. For example, biometric information may be data (feature amounts) calculated from physical features unique to an individual, such as fingerprints, voiceprints, veins, retinas, and iris patterns.

The support system 1000 includes a tablet terminal 101 (first terminal), a smartphone 102, a tablet terminal 103 (second terminal), an authentication device 200, and a support device 300. A tablet terminal 101 , a smartphone 102 , a tablet terminal 103 , an authentication device 200 and a support device 300 are connected via a network N. Here, the network N is a wired or wireless communication line such as the Internet. The tablet terminal 101 is operated by the target person (grandchild) U1 and the target person's parent U2, and the smartphone 102 is possessed and operated by the target person's parent U2. The tablet terminal 103 is operated by a supporter (grandparent) U3. It is assumed that the

tablet terminals

101 and 103 are installed at least in different places (dwellings, etc.).

The tablet terminal 101 is a terminal used by the target person (grandchild) U1 and the target person's parent U2, and is installed in the residence of the target person (grandchild) U1 and the target person's parent U2. The tablet terminal 101 may display a character imitating an animal or a robot by means of installed software, and the character may operate as a user interface with which the target person (grandchild) U1 interacts. In this case, the tablet terminal 101 analyzes the content control information from the support device 300, displays the character, and performs voice output or text display as the character's words. The tablet terminal 101 also collects the speech of the target person (grandchild) U1 and transmits the collected voice information or the voice recognition result to the support device 300 . Thereby, the target person (grandchild) U1 can interact (converse) with the character displayed on the tablet terminal 101 . Moreover, the tablet terminal 101 may transmit the health information of the subject U1 acquired from a health information measuring device (not shown) to the support device 300 via the network N. FIG.

FIG. 4 is a block diagram showing the configuration of the tablet terminal 101 according to the second embodiment. The tablet terminal 101 includes a camera 110 , a microphone 120 , a speaker 130 , a touch panel 140 , a storage section 150 , a communication section 160 , a memory 170 and a control section 180 . The camera 110 is an imaging device that performs imaging under the control of the control unit 180 . The microphone 120 is a sound pickup device that picks up the voice uttered by the target person (grandchild) U1 or the like. The speaker 130 is a device that outputs sound under the control of the controller 180 . Touch panel 140 includes a display device (display unit) such as a screen and an input device. The storage unit 150 is a storage device that stores a program 151 for realizing each function of the tablet terminal 101 . The program 151 implements processing including software that operates as a user interface. The communication unit 160 is a communication interface with the network N. FIG. The memory 170 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily holding information when the control unit 180 operates. The control unit 180 is a processor that controls hardware of the tablet terminal 101 . The control unit 180 loads the program 151 from the storage unit 150 into the memory 170 and executes it. Thereby, the control unit 180 implements the functions of the registration unit 181 and the content processing unit 182 .

The registration unit 181 transmits a pre-registration request including the target person information and the supporter information to the support device 300 via the network N. The target person information includes at least a face image (first biometric information) or identification information of the target person (grandchild) U1, and may further include personal information of the target person (grandchild) U1. The target person information also includes the terminal ID of the tablet terminal 101 . Note that the face image of the subject (grandchild) U1 may be captured by the camera 110 or acquired from an external device such as the smartphone 102 or the like. Further, the target person information may include voice information obtained by picking up the voice of the target person (grandchild) U1, voice recognition results, and health information. Further, the registration unit 181 performs voice recognition on voice information obtained by collecting the utterance of the target person (grandchild) U1 by the microphone 120, and detects the physical condition and desires of the target person (grandchild) U1 from the voice recognition result. The physical condition and requests may be included in the target person information. For example, when the subject (grandchild) U1 utters "I have a headache," the registration unit 181 detects that the subject (grandchild) U1 is in poor physical condition. Further, when the target person (grandchild) U1 utters "I want to go play", the registration unit 181 detects that the target person (grandchild) U1 wants to go to a park, an amusement park, or the like. The supporter information includes at least identification information of the supporters (grandparents) U3, and may further include personal information of the supporters (grandparents) U3, especially payment information. Also, the supporter information may include the terminal ID of the tablet terminal 103 . As the identification information (supporter ID) of the supporter (grandparent) U3, information previously issued in the support device 300 or the like may be used.

Further, the registration unit 181 acquires a photographed image (first photographed image) of the subject (grandchild) U1 photographed by the camera 110, and supports a registration request including the first photographed image via the network N. Send to device 300 . Here, the target person (grandchild) U1 may use the camera 110 to photograph an object or scenery that he or she is interested in. In this case, the registration unit 181 acquires the first captured image from the camera 110 . Note that the subject (grandchild) U1 may capture the first captured image using a digital camera or the like. In this case, it is assumed that the subject's parent U2 transfers the first captured image from the digital camera to the tablet terminal 101 . Then, the registration unit 181 acquires the first captured image from the digital camera and transmits a registration request. Note that the subject (grandchild) U1 may shoot a moving image using the camera 110 or the like. In that case, the registration unit 181 may include part or all of the captured moving images in the registration request as the first captured image group. Note that when the first captured image is acquired, the registration unit 181 automatically transmits a registration request including the first captured image to the support device 300 without depending on the operation of the subject (grandchild) U1 or the like. good too. Further, the registration unit 181 may acquire the position information of the photographing location of the photographed image together with the first photographed image. In that case, the registration unit 181 further includes the acquired location information in the registration request. Further, the registration unit 181 may include the face image (second biometric information) of the target person (grandchild) U1 as the photographer together with the first captured image in the registration request. Alternatively, the registration unit 181 may include the terminal ID of the tablet terminal 101 or the identification information of the subject (grandchild) U1 together with the first captured image.

The content processing unit 182 analyzes content control information from the support device 300, displays characters on the touch panel 140, and provides the user interface described above. In particular, the content processing unit 182 displays the support information on the touch panel 140 when the support information is received from the support device 300 . In this case, the support information may be display content of electronic data, service information, a product shipping notification message, or the like. Note that the content processing unit 182 may display the support information through a character.

In addition, the content processing unit 182 transmits thank-you information for the support information to the support device 300 via the network N in response to the operation of the target person (grandchild) U1 or the target person's parent U2. The thank-you information includes text information such as a message input by the target person (grandchild) U1 or the target person's parent U2 via the touch panel 140, or the target person (grandchild) U1 being photographed when the support information is displayed or when the product is received. A photographed image (second photographed image) may also be included. It is assumed that the second photographed image includes the face area of the subject (grandchild) U1 captured by the user-side camera (camera 110 or the like) of tablet terminal 101 . Alternatively, the subject's parent U2 may capture the second captured image using the smartphone 102 or the like, and transfer the second captured image from the smartphone 102 or the like to the tablet terminal 101 . Then, the content processing unit 182 transmits thank-you information including the second captured image acquired from the smartphone 102 or the like to the support device 300 .

Return to Figure 3 and continue the explanation. The smart phone 102 is an example of a terminal possessed and used by the parent U2 of the subject. Note that the smartphone 102 may be a tablet terminal, a PC (Personal Computer) equipped with or connected to a camera, or the like. The smartphone 102 exchanges text data with the support device 300 via a general-purpose SNS (Social Network Service) application. For example, if the SNS application is a message application, the character is used as the user, and messages are exchanged with the user of the target person's parent U2, thereby realizing message exchange with the supporter (grandparent) U3 via the character. do. In particular, the smartphone 102 may perform the above-described pre-registration request, registration request for the first captured image, display of support information, and transmission of thank-you information.

The tablet terminal 103 is a terminal used by the supporter (grandparent) U3, and is installed in the residence of the supporter (grandparent) U3. Note that the configuration of the tablet terminal 103 is the same as that of the tablet terminal 101 described above, and illustration thereof is omitted. The tablet terminal 103, like the tablet terminal 101, may operate as a user interface that displays a character to be a dialogue partner of the supporter (grandparent) U3. The tablet terminal 103 displays the support information candidates received from the support device 300 on the touch panel. Then, the tablet terminal 103 receives support information selection operation by the support person (grandparent) U3 from among the support information candidates, and transmits a support instruction including the selected support information to the support device 300 . Further, when the tablet terminal 103 receives the thank-you information from the support device 300, the tablet terminal 103 displays the thank-you information on the touch panel. Note that the tablet terminal 103 may register supporter information regarding the supporter (grandparent) U3 in the support device 300 . The supporter information may include identification information (supporter ID) of the supporter (grandparent) U3, personal information, payment information, and terminal ID of the tablet terminal 103 . Alternatively, if the supporter ID has not yet been issued, the tablet terminal 103 may include the face image of the supporter (grandparent) U3 in the supporter information. In that case, the tablet terminal 103 may acquire the supporter ID issued by the support device 300 . Further, the tablet terminal 103 may notify the tablet terminal 101 or the smartphone 102 of the supporter ID and the terminal ID of the tablet terminal 103 .

The authentication device 200 is an information processing device that stores facial feature information of the user. In addition, in response to a face authentication request received from the outside, the authentication device 200 compares the face image or face feature information included in the request with the face feature information of each user, and requests the result of matching (authentication result). Reply to original.

FIG. 5 is a block diagram showing the configuration of the authentication device 200 according to the second embodiment. The authentication device 200 includes a face information DB (DataBase) 210 , a face detection section 220 , a feature point extraction section 230 , a registration section 240 and an authentication section 250 . The face information DB 210 associates and stores a user ID 211 and face feature information 212 of the user ID. The facial feature information 212 is a set of feature points extracted from the facial image.

The face detection unit 220 detects a face area included in a registered image for registering face information and outputs it to the feature point extraction unit 230 . The feature point extraction section 230 extracts feature points from the face area detected by the face detection section 220 and outputs facial feature information to the registration section 240 . Further, the feature point extraction unit 230 extracts feature points included in the facial image received from the support device 300 or the like, and outputs facial feature information to the authentication unit 250 .

The registration unit 240 newly issues a user ID 211 when registering facial feature information. The registration unit 240 associates the issued user ID 211 with the facial feature information 212 extracted from the registered image and registers them in the facial information DB 210 . Authentication unit 250 performs face authentication using facial feature information 212 . Specifically, the authentication unit 250 collates the facial feature information extracted from the facial image with the facial feature information 212 in the facial information DB 210 . If the verification is successful, the authentication unit 250 identifies the user ID 211 associated with the verified facial feature information 212 . The authenticating unit 250 replies to the requester as a face authentication result indicating whether or not the facial feature information matches. Whether the facial feature information matches or not corresponds to the success or failure of the authentication. Note that matching of facial feature information (matching) means a case where the degree of matching is equal to or greater than a threshold. Also, the face authentication result shall include the specified user ID when the face authentication is successful.

Returning to FIG. 3, the description continues.
The support device 300 is an example of the support device 1 described above. The support device 300 is an information processing device that performs pre-registration processing, collection processing, support information provision processing, thank-you information registration and presentation processing, and the like. The support device 300 may be made redundant by a plurality of servers, and each functional block may be realized by a plurality of computers. Note that the support device 300 may generate content control information for controlling characters displayed on the

tablet terminals

101 and 103 and transmit the content control information to the

tablet terminals

101 and 103 .

Next, the support device 300 will be described in detail. FIG. 6 is a block diagram showing the configuration of the support device 300 according to the second embodiment. The support device 300 includes a storage unit 310 , a memory 320 , a communication unit 330 and a control unit 340 . The storage unit 310 is an example of a storage device such as a hard disk or flash memory. The storage unit 310 stores a program 311 , subject management information 312 , supporter information 313 and a support information DB 314 . The program 311 is a computer program in which processing including pre-registration processing, collection processing, support information provision processing, thank-you information registration and presentation processing, etc. according to the second embodiment is implemented.

The subject management information 312 is information for managing the subject (grandchild) U1. The subject management information 312 is information in which a subject ID 3121, a supporter ID 3122, a terminal ID 3123, subject information 3124, a subject image 3125 and a subject image 3126 are associated with each other. The subject ID 3121 is identification information of the subject (grandchild) U1. The subject ID 3121 is information identically or uniquely corresponding to the user ID 211 managed in association with the facial feature information 212 managed in the face information DB 210 of the authentication device 200 . The target person ID 3121 may be identification information of the target person (grandchild) U1 included in the above-described pre-registration request. The supporter ID 3122 is identification information of the supporter (grandparent) U3. The terminal ID 3123 is identification information of the tablet terminal 101 used by the subject (grandchild) U1.

The target person information 3124 is at least part of the target person information included in the pre-registration request described above. The target person information 3124 includes, for example, personal information, health information, physical condition, and requests of the target person (grandchild) U1, but is not limited to these.

The object image 3125 is an example of the first captured image included in the registration request described above. The target object image 3125 is an image including the target object photographed by the target person (grandchild) U1. In addition to the target object image 3125, the target person management information 312 may be associated with the target person ID 3121 as a first photographed image including a landscape image taken by the target person (grandchild) U1.

The target person image 3126 is the second captured image included in the thank-you information described above. That is, the target person image 3126 is an image including the target person (grandchild) U1 photographed when the support information is displayed or when the product is received.

The supporter information 313 is information about supporters (grandparents) U3. The supporter information 313 is information in which a supporter ID 3131, personal information 3132, and terminal ID 3133 are associated with each other. The supporter ID 3131 is identification information of the supporter (grandparent) U3, and is information identical to or uniquely corresponding to the supporter ID 3122 described above. The personal information 3132 includes personal information of the supporters (grandparents) U3, such as payment information. The terminal ID 3133 is identification information of the tablet terminal 103 used by the supporter (grandparent) U3.

The support information DB 314 is a database that manages a plurality of support information candidates 3141 to 314n (n is a natural number of 2 or more). Support information candidates 3141 and the like are information that are candidates for support information for the target person (grandchild) U1. The support information candidates 3141 and the like are presents (products), services (educational contents), travel proposal information, electronic data display contents (electronic books, etc.), and the like.

The memory 320 is a volatile storage device such as RAM (Random Access Memory), and is a storage area for temporarily holding information when the control unit 340 operates. A communication unit 330 is a communication interface with the network N. FIG.

The control unit 340 is a processor that controls each component of the support device 300, that is, a control device. The control unit 340 loads the program 311 from the storage unit 310 into the memory 320 and executes the program 311 . Thereby, the control unit 340 realizes the functions of the registration unit 341 , the acquisition unit 342 , the authentication control unit 343 , the identification unit 344 , the determination unit 345 , the presentation unit 346 and the processing unit 347 .

The registration unit 341 is an example of the registration unit 11 described above. Registration unit 341 receives a pre-registration request from tablet terminal 101 or smartphone 102 and transmits a face information registration request including a face image (of subject (grandchild) U1) included in the pre-registration request to authentication device 200 . Then, the registration unit 341 acquires the user ID issued by the authentication device 200 in accordance with the registration of the face information. Then, the registration unit 341 generates the target person management information 312 by associating the acquired user ID (subject ID 3121) with the target person information and the supporter information included in the pre-registration request. Then, the registration unit 341 registers the generated subject management information 312 in the storage unit 310 . That is, it can be said that the registration unit 341 registers the facial feature information 212 (first biometric information) of the subject (grandchild) U1 in the authentication device 200 in advance. Further, the registration unit 341 can be said to register the supporter ID 3122 , the terminal ID 3123 , the target person information 3124 , etc., and the facial feature information 212 of the target person (grandchild) U1 in association with each other via the target person ID 3121 .

Also, the registration unit 341 sets the photographer of the first captured image specified by the specifying unit 344, which will be described later, as the subject ID 3121, and sets the first captured image as the target object image 3125. As such, the registration unit 341 registers (updates) the subject management information 312 by associating the subject ID 3121 with the subject image 3125 . The registration unit 341 also registers the position information of the shooting location acquired together with the first captured image in association with the object image 3125 .

Also, the registration unit 341 registers the thank-you information for the support information in the target person management information 312 in association with the target person ID 3121 . For example, when the support information is display content and the thank-you information includes the second captured image of the target person (grandchild) U1, the registration unit 341 stores the target person ID 3121, the display content, and the target person image 3126 (second 2) are registered in the subject management information 312 in association with each other. Furthermore, when the content position of the display content displayed on the tablet terminal 101 together with the second captured image is acquired, the registration unit 341 stores the target person ID 3121, the display content, the content position, and the target person image 3126 (the 2) are registered in the subject management information 312 in association with each other. Further, the registration unit 341 registers the second captured image, the display content, and the content when the satisfaction level of the target person (grandchild) U1 measured from the second captured image by the processing unit 347, which will be described later, is equal to or greater than a predetermined value. Positions may be associated with each other and registered.

The acquisition unit 342 is an example of the acquisition unit 12 described above. Acquisition unit 342 acquires a registration request including the first captured image from tablet terminal 101 . Acquisition unit 342 also acquires a second captured image of target person (grandchild) U1 captured when the display content is displayed on tablet terminal 101 (first terminal). The acquisition unit 342 may further acquire the content position (display position) of the display content displayed on the tablet terminal 101 at the time of capturing together with the second captured image. The acquisition unit 342 may acquire the second biometric information (face image, etc.) of the photographer from the tablet terminal 101 together with the first captured image. Alternatively, the acquisition unit 342 may acquire the identification information of the photographer or the terminal ID of the tablet terminal 101 together with the first captured image. Furthermore, the acquisition unit 342 may acquire the position information of the shooting location together with the first captured image.

The authentication control unit 343 controls face authentication for the face image included in the registration request acquired by the acquisition unit 342. Specifically, authentication control unit 343 transmits a face authentication request including a face image to authentication device 200 and receives a face authentication result from authentication device 200 . Note that the authentication control unit 343 may detect the user's face area from the face image and include the image of the face area in the face authentication request. Alternatively, the authentication control unit 343 may extract facial feature information from the face area and include the facial feature information in the face authentication request.

It should be noted that the acquisition unit 342 performs collection processing for collecting the other target person information described above. The other target person information may include the face image of the target person (grandchild) U1, as well as the above-described voice information, voice recognition result, health information, physical condition, requests, and the like. The authentication control unit 343 performs face authentication on the face image included in the target person information and identifies the target person ID 3121 . Then, the registration unit 341 associates the specified target person ID 3121 with the voice information, voice recognition result, health information, physical condition, requests, etc. included in the target person information, and registers them in the target person management information 312 .

The identification unit 344 is an example of the identification unit 13 described above. The identifying unit 344 identifies the photographer of the first captured image. For example, when a face image is included in the registration request for the first captured image, the specifying unit 344 acquires the face authentication result from the authentication control unit 343, and if the face authentication result is successful, the face image is included in the face authentication result. Identifies the user ID that is The identification unit 344 identifies the user ID included in the face authentication result as the photographer, and identifies the user ID as the subject ID 3121 . Note that when the identification information of the photographer is included in the registration request for the first captured image, the identifying unit 344 identifies the photographer using the identification information as the subject ID 3121 . Further, when the terminal ID of the tablet terminal 101 is included in the registration request for the first captured image, the identifying unit 344 identifies the subject ID 3121 associated with the terminal ID 3123 as the photographer.

Also, the identifying unit 344 identifies the interest of the subject based on the subject information 3124 and the subject image 3125 associated with the particular subject ID 3121 . Further, the specifying unit 344 identifies the target person based on the thank-you information associated with the specific target person ID 3121, the target person image 3126, the display content, the content position, the position information of the shooting location of the first captured image, and the like. may identify the interests of As a result, the accuracy of specifying the interest can be improved. Here, the content position is the display position of the display content displayed on the tablet terminal 101 when the second captured image was captured as described above. For example, when the display content includes a plurality of pages and is displayed on the screen of the tablet terminal 101 page by page, the content position (display position) is the displayed page number or the like. The second captured image is an image of the target person (grandchild) U1 captured when the tablet terminal 101 displays (reproduces) a predetermined page (content position) of the display content, which is the support information. For example, when the target person's parent U2 confirms that the target person (grandchild) U1 is interested in a specific page of the display content, the target person's parent U2 uses the inner camera of the tablet terminal 101 to view the target person ( grandchild) U1 may be photographed. The tablet terminal 101 may set the captured image at this time as the second captured image, and may set the page displayed at this time as the content position. Alternatively, the displayed content and the content position may be registered when the above-described degree of satisfaction is equal to or higher than a predetermined value. In these cases, there is a high possibility that the content position indicates the page and content that the target person (grandchild) U1 is interested in within the displayed content. Therefore, the specifying unit 344 can specify the interest of the target person with high accuracy by considering the display content and the content position.

Further, the specifying unit 344 may exclude the face-recognized area from the first captured image and specify the subject's interest based on the excluded image. For example, when the target person (grandchild) U1 takes a picture, there is a case where privacy-related information such as the face of a third party is captured unintentionally. In such a case, the third person's face and the like have nothing to do with the interest of the target person (grandchild) U1. Therefore, the identification unit 344 can improve the accuracy of identification by excluding unintentionally reflected information from the identification target of interest. Also, the specifying unit 344 preferably excludes areas other than the target person from the areas in which the face is recognized in the first captured image. In this case, the registration unit 341 described above may register the excluded image as the target object image 3125 . Alternatively, the registration unit 341 may exclude the face-recognized area from the first captured image and register the excluded image as the target object image 3125 . These can protect the privacy of third parties. Therefore, the target person's parent U2 can allow the target person (grandchild) U1 to use the support system 1000 without worrying about the privacy of a third party.

The determination unit 345 is an example of the determination unit 14 described above. The determination unit 345 determines one or more support information candidates 3141 to 314n from the support information DB 314 based on the interest identified by the identification unit 344 .

The presentation unit 346 is an example of the presentation unit 15 described above. The presentation unit 346 presents the support information candidate to the supporter (grandparent) U3 by transmitting the determined support information candidate to the tablet terminal 103 .

The processing unit 347 receives support instructions including support information selected by the supporter (grandparents) U3 from among the presented support information candidates, and performs processing according to the support information. Specifically, the processing unit 347 performs processing for providing the support information to the target person (grandchild) U1. For example, when the support information is a product, the processing unit 347 performs the purchase and shipping procedures for the product as “processing for providing”. For example, the processing unit 347 transmits a purchase and shipping request to the sales system of the selected product, with the destination being the residence of the target person (grandchild) U1. Further, when the support information is travel, the processing unit 347 performs reservation processing for the selected travel as “processing for providing”. For example, the processing unit 347 sends a request to the server of the travel agency for the selected trip for a trip on a predetermined schedule with the target person (grandchild) U1, the target person's parent U2, and the supporter (grandparent) U3 as participants. Submit a reservation request for Also, in these cases, the processing unit 347 notifies the tablet terminal 101 of the content of the shipment and the content of the reservation. When the support information is an electronic book (display content), the processing unit 347 transmits an electronic book purchase request using the payment information of the supporter (grandparent) U3 to the electronic book sales system. After the purchase, the processing unit 347 notifies the tablet terminal 101 of the download destination of the electronic book. The processing unit 347 performs these purchase requests and notifications as "processing for providing". In these cases, the processing unit 347 makes a payment using the payment information of the supporter (grandparent) U3.

Further, the processing unit 347 measures the degree of satisfaction of the subject person (grandchild) U1 from the second captured image, and if the degree of satisfaction is equal to or higher than a predetermined value, the second captured image is sent to the supporter (grandparent). The tablet terminal 103 of U3 is notified. For example, the processing unit 347 may identify the facial region of the subject (grandchild) U1 included in the second captured image, and measure the degree of satisfaction from the degree of smile using a predetermined smile analysis technique.

FIG. 7 is a sequence diagram showing the flow of pre-registration processing according to the second embodiment. Here, it is assumed that the target person's parent U2 performs pre-registration to associate the target person (grandchild) U1 with the supporter (grandparent) U3. First, the smartphone 102 photographs the face of the target person (grandchild) U1 with the camera according to the operation of the target person's parent U2. Further, the smartphone 102 receives the target person information of the target person (grandchild) U1 and the supporter information of the supporter (grandparent) U3 from the input of the target person's parent U2. Then, the smartphone 102 transmits a pre-registration request including the target person information and the supporter information to the support device 300 via the network N according to the operation of the target person's parent U2 (S101). Here, the pre-registration request includes the face image of the target person (grandchild) U1.

The registration unit 341 of the support device 300 receives a pre-registration request from the smartphone 102 via the network N. Then, the registration unit 341 transmits a face information registration request including the face image of the target person (grandchild) U1 included in the pre-registration request to the authentication device 200 (S102). In response, the authentication device 200 performs face information registration processing (S103).

FIG. 8 is a flow chart showing the flow of face information registration processing by the authentication device according to the second embodiment. Here, the information registration terminal (not shown) photographs the user's body including the face, and transmits a face information registration request including the photographed image (registered image) to the authentication device 200 via the network N. The information registration terminal is, for example, an information processing device such as a personal computer, a smart phone, or a tablet terminal. For example, the information registration terminals may be the tablet terminal 101, the smart phone 102, the tablet terminal 103, and the like. Here, it is assumed that the information registration terminal is the support device 300 that has received a pre-registration request from the smartphone 102 or the like.

First, the authentication device 200 receives a face information registration request (S201). For example, the authentication device 200 receives a face information registration request via the network N from the support device 300 . Next, the face detection unit 220 detects a face area from the face image included in the face information registration request (S202). Then, the feature point extraction unit 230 extracts feature points (facial feature information) from the face area detected in step S202 (S203). Then, the registration unit 240 issues the user ID 211 (S204). Then, the registration unit 240 associates the extracted facial feature information 212 with the issued user ID 211 and registers them in the facial information DB 210 (S205). After that, the registration unit 240 returns the issued user ID 211 to the request source (the information registration terminal, for example, the support device 300) (S206).

Return to Fig. 7 and continue the explanation. The registration unit 341 of the support device 300 acquires the issued user ID from the authentication device 200 via the network N (S104). Then, the registration unit 341 sets the acquired user ID as the subject ID 3121 . Further, the registration unit 341 extracts information (personal information, terminal ID of the tablet terminal 101) other than the face image included in the target person information included in the pre-registration request. The registration unit 341 also extracts the supporter ID included in the supporter information included in the pre-registration request. Then, the registration unit 341 generates the target person management information 312 by associating the target person ID 3121, the supporter ID 3122, the terminal ID 3123, and the target person information 3124 (excluding the face image). Then, the registration unit 341 registers the generated subject management information 312 in the storage unit 310 (S105).

FIG. 9 is a flowchart showing the flow of collection processing according to the second embodiment. First, the tablet terminal 101 takes an image of an object with the camera 110 according to the operation of the object person (grandchild) U1 (S111). Next, the tablet terminal 101 transmits a registration request including the first captured image to the support device 300 via the network N (S112). Here, the registration request includes the face image and identification information of the target person (grandchild) U<b>1 or the terminal ID of the tablet terminal 101 . In the following description, it is assumed that the registration request includes the face image of the subject (grandchild) U1.

The acquisition unit 342 of the support device 300 acquires a registration request including the first captured image (and face image) from the tablet terminal 101 via the network N. The authentication control unit 343 then transmits the face authentication request including the face image included in the registration request to the authentication device 200 via the network N (S113). In response, the authentication device 200 performs face authentication processing (S114).

FIG. 10 is a flow chart showing the flow of face authentication processing by the authentication device according to the second embodiment. First, the authentication device 200 receives a face authentication request from the support device 300 via the network N (S211). Note that the authentication device 200 may receive a face authentication request from the tablet terminal 101 or the like. Next, the authentication device 200 extracts facial feature information from the face image included in the face authentication request, as in steps S202 and S203 described above. Then, the authentication unit 250 of the authentication device 200 collates the facial feature information extracted from the face image included in the face authentication request with the facial feature information 212 of the face information DB 210 (S212), and calculates the matching degree. Then, the authentication unit 250 determines whether or not the degree of matching is equal to or greater than the threshold (S213). If the facial feature information matches, that is, if the degree of matching of the facial feature information is equal to or greater than the threshold, the authentication unit 250 identifies the user ID 211 associated with the facial feature information 212 (S214). Then, the authentication unit 250 returns the result of the face authentication including the result of the successful face authentication and the specified user ID 211 to the support device 300 via the network N (S215). If the degree of matching is less than the threshold in step S213, the authentication unit 250 returns the result of face authentication including failure of the face authentication to the support device 300 via the network N (S216).

Return to Fig. 9 and continue the explanation. The authentication control unit 343 of the support device 300 receives the face authentication result from the authentication device 200 via the network N (S115). Here, it is assumed that the face authentication is successful, and the face authentication result includes the fact that it was successful and the user ID.

Subsequently, the specifying unit 344 determines whether or not the face authentication is successful from the received face authentication results. When determining that the face authentication has succeeded, the specifying unit 344 specifies the user ID included in the face authentication result, and specifies the user ID as the photographer (S116). Incidentally, when the face authentication fails, the support device 300 may reply to that effect to the tablet terminal 101 .

After step S116, the registration unit 341 registers the specified photographer as a target person in association with the first captured image (S117). Specifically, the registration unit 341 sets the user ID identified in step S116 as the subject ID 3121 and sets the first captured image included in the registration request as the subject image 3125 . Then, the registration unit 341 updates the subject management information 312 by associating the subject ID 3121 with the subject image 3125 .

Although FIG. 9 shows an example in which the photographer of the first captured image is identified by face authentication from the support device 300 to the authentication device 200, the present invention is not limited to this. User identification information or the terminal ID of the tablet terminal 101 may be used to identify the photographer in the support device 300 . For example, the tablet terminal 101 may capture a face image of the target person (grandchild) U1 with an internal camera (for the user) and transmit a face authentication request including the face image to the authentication device 200 via the network N. good. Then, the tablet terminal 101 receives the face authentication result from the authentication device 200 via the network N. FIG. When the face authentication is successful, the tablet terminal 101 can specify (acquire) the user ID (identification information) of the target person (grandchild) U1 from the face authentication result. Alternatively, the tablet terminal 101 may identify the person using the ID and password (passcode) entered by the target person (grandchild) U1 when logging into the terminal. In this case, the ID that has successfully logged in may be the same as the user ID managed by the authentication device 200 or can be identified at the position. Alternatively, the tablet terminal 101 may store the face image and user ID of the target person (grandchild) U1 in the storage unit 150 in advance. In this case, when the target person (grandchild) U1 captures the first captured image, the face image of the target person (grandchild) U1 is captured by the inner camera (for the user), and the face image in the storage unit 150 is captured. Face authentication may be performed by matching with a photographed face image. The tablet terminal 101 can identify the user ID when face authentication within the terminal is successful. Accordingly, in step S112, the tablet terminal 101 can transmit a registration request including the specified user ID (of the target person (grandchild) U1) together with the first captured image. In this case, the support device 300 can identify the photographer from the user ID included in the received registration request instead of steps S113 and S115. Alternatively, the tablet terminal 101 may transmit the registration request including the terminal ID together with the first captured image. In this case, instead of steps S113 and S115, the support device 300 can identify the subject ID 3121 associated with the terminal ID 3123 included in the received registration request as the photographer. Alternatively, identification of the photographer by the support device 300 may be performed by biometric authentication other than face authentication.

FIG. 11 is a flowchart showing the flow of support information provision processing according to the second embodiment. The support device 300 starts support information provision processing at a predetermined timing in response to a presentation request from the smartphone 102 or the tablet terminal 103 . In the following description, it is assumed that there is a presentation request from the tablet terminal 103 .

First, the tablet terminal 103 transmits a request for presentation of support information candidates for the target person (grandchild) U1 to the support device 300 via the network N in accordance with the operation of the supporter (grandparent) U3 (S121). Here, the presentation request includes the terminal ID of the tablet terminal 103 . The presentation request may include the supporter ID and face image of the supporter (grandparent) U3.

The acquisition unit 342 of the support device 300 acquires the presentation request from the tablet terminal 103 via the network N. The identifying unit 344 identifies the supporter ID 3131 associated with the terminal ID 3133 included in the acquired presentation request from the supporter information 313, and identifies the supporter ID 3131 associated with the identified supporter ID 3131 (supporter ID 3122) The target person ID 3121 is specified (S122). Then, the acquiring unit 342 acquires the target person information 3124 and the target object image 3125 associated with the identified target person ID 3121 from the target person management information 312 (S123). Then, the specifying unit 344 analyzes the target object image 3125 and specifies the interest of the target person (grandchild) U1 based on the analysis result and the target person information 3124 (S124). Then, the determination unit 345 determines one or more support information candidates 3141 and the like from the support information DB 314 based on the specified interest (S125). An AI model may be used for part or all of steps S124 and S125.

Then, the presentation unit 346 transmits the determined support information candidate to the tablet terminal 103 via the network N (S126). In response, the tablet terminal 103 displays the received support information candidate (group).

FIG. 12 is a diagram showing an example of a presentation screen 51 of support information candidates for supporters according to the second embodiment. The presentation screen 51 includes selection columns 5111 to 5113 , support information candidate display columns 5121 to 5123 , other candidate display button 5131 , purchase button 5132 and cancel button 5133 . Selection columns 5111 to 5113 are columns for receiving a selection operation by a supporter (grandparent) U3. Support information candidate display columns 5121 to 5123 are display columns for support information candidates determined in step S125. For example, it is assumed that the object images 3125 (group) include many flowers as objects. Therefore, in the example of FIG. 12, the support information candidate display column 5121 presents (recommends) a plant illustrated book, the support information candidate display column 5122 presents (recommends) a flower arrangement experience course, and the support information candidate display column 5123 presents (recommends) a sunflower cultivation set. indicate that Another candidate display button 5131 is a button for displaying support information candidates other than the support information candidate display columns 5121 to 5123 . For example, when the display other candidates button 5131 is pressed, the tablet terminal 103 transmits a request for presentation of other candidates to the support apparatus 300, and when receiving other candidates, displays the support information candidate display columns 5121 to 5123. Update with received support information candidates. A purchase button 5132 is a column for accepting a purchase of a candidate selected from the selection columns 5111 to 5113 as support information. The cancel button 5133 is a button for canceling or ending the presentation of the support information candidate (closing the presentation screen 51).

The example of FIG. 12 shows that the supporter (grandparents) U3 has selected the selection field 5111, that is, "botanical picture book" as the support information (S127) and pressed the purchase button 5132. In this case, the tablet terminal 103 transmits a support instruction including the ID (support information) of the support information candidate corresponding to the support information candidate display field 5121 (plant picture book) to the support device 300 via the network N (S128). ).

The processing unit 347 of the support device 300 receives a support instruction from the tablet terminal 103 via the network N. The processing unit 347 performs processing for providing support information included in the support instruction (S129). In addition, as necessary, the processing unit 347 notifies the tablet terminal 101 of information regarding the support information via the network N (S130). Specifically, the processing unit 347 identifies the terminal ID 3123 associated with the subject ID 3121 identified in step S122, and notifies the provision of the support information with the terminal ID 3123 (tablet terminal 101) as the destination. For example, if the support information is an e-book plant illustrated book, the processing unit 347 notifies the tablet terminal 101 of the download destination of the e-book. Note that step S130 may be omitted when presenting (delivering) a surprise gift (product) to the target person (grandchild) U1.

After that, the target person (grandchild) U1 (or the tablet terminal 101) receives support information from the supporter (grandparent) U3 (S131). Then, the tablet terminal 101 transmits the thank-you information input by the target person (grandchild) U1 or the target person's parent U2 to the support device 300 via the network N (S132). At this time, the thank-you information includes the text information of the thank-you message and the second photographed image of the target person (grandchild) U1 at the time of receiving the support information. The thank-you information includes a facial image for authentication of the target person (grandchild) U1, the target person ID, the terminal ID of the tablet terminal 101, and the like.

The acquisition unit 342 of the support device 300 acquires thank-you information from the tablet terminal 101 via the network N. The specifying unit 344 specifies the target person (grandchild) U1 from the face image, the target person ID, the terminal ID of the tablet terminal 101, and the like included in the thank-you information. For example, the authentication control unit 343 may perform face authentication on the face image included in the thank-you information. Then, the registration unit 341 associates the target person ID 3121 of the target person (grandchild) U1 with the thank-you information, and performs registration processing in the target person management information 312 (S133). In addition, the processing unit 347 notifies the obtained thank-you information to the tablet terminal 103 via the network N (S134). In response, the tablet terminal 103 displays the notified thank-you information (S135).

FIG. 13 is a diagram for explaining the concept of presentation of support information candidates, selective purchase, delivery, and notification of thank-you comments according to the second embodiment. It is assumed that the tablet terminal 103 displays the presentation screen 51a, and the supporter (grandparent) U3 has selected and purchased the illustrated book of plants as support information. In response to this, the support device 300 performs the process of purchasing and delivering the pictorial book of plants. Then, it is assumed that the plant illustrated book is delivered to the target person (grandchild) U1. At this time, the target person's parent U2 uses the smartphone 102 to capture a second captured image including the target person (grandchild) U1 who is pleased with the plant illustrated book. Note that the second captured image may be captured by the tablet terminal 101 . Then, the tablet terminal 101 or the smartphone 102 supports thank-you information including the second captured image and the thank-you comment of the target person (grandchild) U1 according to the operation of the target person (grandchild) U1 or the target person's parent U2. Send to device 300 . The support device 300 notifies the tablet terminal 103 of the thank-you information. The tablet terminal 103 displays the thank-you information display screen 51b in response to the thank-you information notification. As a result, the supporter (grandparent) U3 can visually recognize the thank-you comment of the target person (grandchild) U1 and the picture of the target person (grandchild) U1 looking at the botanical illustrated book and enjoying it through the thank-you information display screen 51b.

As described above, the second embodiment has the following effects in addition to the effects of the first embodiment described above. First, since the support information is provided to the target person according to the support instruction from the supporter, the support information that matches the interest of the target person can be provided. In addition, the target person can know the provision from the supporter by the notification of the support information. In addition, since the request for registration of the first photographed image includes the target person's identification information (face image, identification information, terminal ID), it is not necessary for the target person or relatives to input the identification information each time they register. It is possible to specify the photographer of the first captured image accurately without any trouble. In particular, by performing biometric authentication using biometric information as personal identification information, the photographer can be identified with high accuracy. In addition, since the photographing location is registered together with the first photographed image, the precision of specifying the interest is improved.

Note that the tablet terminal 101 may transmit the first captured image to the storage area allocated to the target person (grandchild) U1 on the network N. In this case, the identification unit 344 of the support device 300 can detect that the first captured image has been saved in the storage area and identify the user ID assigned to the storage area as the photographer.

Note that the support device 300 analyzes the photographed image in which the target person (grandchild) U1 is photographed among the photographs taken by the target person's parent U2, identifies the interest of the target person (grandchild) U1, and identifies the interest of the target person (grandchild) U1. grandparents) may be presented to U3. Specifically, the smartphone 102 transmits an image including the target person (grandchild) U1 and the object photographed by the target person's parent U2 to the support device 300 via the network N. At this time, the support device 300 analyzes the received image. For example, if there are many “pictures of a child and an animal (or a specific character) together” in the images taken by the parent, the support device 300 identifies that the child is interested in animals (or a specific character). and present the animal picture book (or picture book of a particular character) to the grandparents. As a result, it is possible to appropriately identify the interest of the supporter and present support information candidates in line with the interest.

<Embodiment 3>
Embodiment 3 is a modification of Embodiment 2 described above. Note that the schematic configuration of the support system according to the third embodiment is the same as that of FIG. 3, so the differences will be mainly described below, and illustration and description of overlapping configurations will be omitted.

FIG. 14 is a block diagram showing the configuration of a support device 300a according to the third embodiment. The support device 300a has a program 311a changed from that of FIG. 6, and an inquiry unit 348 is added. The program 311a is a computer program in which, in addition to the program 311, the query processing according to the third embodiment is implemented.

The inquiry unit 348 inquires of the subject about the attribute information of the first captured image (object image 3125). The acquisition unit 342 acquires attribute information in response to an inquiry from a subject. The specifying unit 344 specifies the subject's interest based on the first captured image and the attribute information.

FIG. 15 is a sequence diagram showing the flow of inquiry processing for a captured image according to the third embodiment. First, the support device 300a starts inquiry processing at a predetermined timing. For example, the support device 300a may start inquiry processing after the first captured image is registered in the collection processing of FIG. 9 .

The inquiry unit 348 reads the target object images 3125 (group) of a specific target person (for example, target person (grandchild) U1) (S301). Then, the inquiry unit 348 generates inquiry content (S302). Here, the inquiry unit 348 generates an inquiry sentence for selecting a favorite image of the subject (grandchild) U1 from the object image group. Then, the inquiry unit 348 transmits the generated inquiry content and the object image group to the tablet terminal 101 via the network N (S303). In response, the tablet terminal 101 displays the content of the inquiry and the object image group received (S304).

FIG. 16 is a diagram showing an example of a selection screen 52 for favorites of captured images according to the third embodiment. The selection screen 52 includes an inquiry message 521 and a captured image group 522 . The inquiry message 521 is a display field for the contents of the inquiry generated in step S302. Here, the message "The registered photos will be displayed. Do you have any favorites?" The photographed image group 522 is a display field for images photographed in the past for the subject (grandchild) U1 and their tags (labels). The tag is information specified as attribute information of the image when the specifying unit 344 analyzes the first captured image. Also, the tag may be information that can be used as teaching data for machine learning of an image recognition engine, which will be described later, by being associated with the image. The identification unit 344 may analyze the image using a predetermined image recognition engine to identify the tag.

The tablet terminal 101 accepts a selection operation by the subject (grandchild) U1 for one or more images from the captured image group 522 (S305). The tablet terminal 101 then transmits the selected image or tag (selection information) to the support device 300a via the network N (S306).

The acquisition unit 342 of the support device 300a acquires the selection information from the tablet terminal 101 via the network N. Then, the identifying unit 344 identifies the interest of the target person (grandchild) U1 based on the selected image indicated by the selection information (S307). Specifically, the identifying unit 344 identifies the tag (attribute information) of the selected image as the interest. Then, the determination unit 345 registers the specified interest in the target person information 3124 (S308). As a result, after this, the determination unit 345 considers that the selected image among the group of target images taken by the target person (grandchild) U1 is more interesting than the other images, and supports the target person (grandchild) U1. Information candidates can be determined. Therefore, it is possible to present support information candidates that match the interest of the target person (grandchild) U1. That is, by having the target person (grandchild) U1 select a favorite image, the accuracy of identifying the interest is increased. Conversely, the group of target images taken by the target person (grandchild) U1 includes photos that are not necessarily favorites. Therefore, by having the target person (grandchild) U1 select a favorite image, it is possible to narrow down the images of interest.

FIG. 17 is a sequence diagram showing another example of the flow of inquiry processing for a captured image according to the third embodiment. Here, it is assumed that one photographed image photographed by the subject person (grandchild) U1 is processed.

First, the inquiry unit 348 reads an arbitrary target object image 3125 of a specific target person (for example, target person (grandchild) U1) (S301a). For example, the inquiry unit 348 reads out an image that is not associated with a tag among the object images 3125 . The identifying unit 344 analyzes the read object image 3125 and estimates tag candidates (S301b). For example, when the target object image 3125 is analyzed by the image recognition engine, the specifying unit 344 estimates a plurality of tags with the highest calculated scores as tag candidates (answer options). Then, the inquiry unit 348 generates inquiry content for selecting a tag candidate (S302a). Here, the inquiry unit 348 generates an inquiry sentence for making the target person (grandchild) U1 select the correct tag from a plurality of tag candidates. The question sentence can be said to be in a quiz format. Then, the inquiry unit 348 transmits the generated inquiry contents, tag candidates, and object images to the tablet terminal 101 via the network N (S303a). In response, the tablet terminal 101 displays the content of the inquiry, the tag candidates, and the object image received (S304a).

FIG. 18 is a diagram showing an example of the tag candidate selection screen 53 in the quiz format according to the third embodiment. The selection screen 53 has an inquiry message 531 and options 532 . The inquiry message 531 is a display field for the contents of the inquiry generated in step S302a. Here, the message "I will display the registered photo. Tell me what kind of photo it is! Do you understand?" An option 532 indicates an option in which a candidate for the name of an object appearing in the image of the object is used as a tag candidate. Choice 532 shows an example in which “gerbera”, “chrysanthemum” and “sunflower” are choices.

The tablet terminal 101 accepts a selection operation by the target person (grandchild) U1 for one option from the options 532 (S305a). The tablet terminal 101 then transmits the selected tag (selected tag) to the support device 300a via the network N (S306a).

The acquisition unit 342 of the support device 300a acquires the selection tag from the tablet terminal 101 via the network N. Then, the processing unit 347 notifies the tablet terminal 103 of the object image and the selection tag (S309). Note that the notification destination may include the smartphone 102 . The tablet terminal 103 displays the notified object image and tag (S310). Thereby, the supporters (grandparents) U3 (and the target person's parent U2) can share the tag answers of the target person (grandchild) U1. Therefore, the supporters (grandparents) U3 and the like can grasp the growth of the target person (grandchild) U1. Further, the supporter (grandparent) U3 or the like may confirm whether the tag for the object image is correct and input the correct tag to the tablet terminal 103 . In that case, the tablet terminal 103 transmits the input correct tag to the support device 300a via the network N (S311). Then, the support device 300a notifies the tablet terminal 101 of the received correct tag (S312). Then, the tablet terminal 101 displays the notified correct tag together with the object image (S313). Thereby, the subject (grandchild) U1 can know the correct answer to the quiz, and learning is promoted. In other words, it is possible to support the education of the target person (grandchild) U1. In addition, it is possible to support interaction between the target person (grandchild) U1 and the supporter (grandparent) U3.

Further, the registration unit 341 of the support device 300a registers the correct tag in association with the object image 3125 (S314). This makes it possible to accumulate accurate tags (attribute information) for the captured image (object image) of the subject (grandchild) U1. Then, the support device 300a may provide a set of the target object image and the correct tag as teacher data for machine learning of the image recognition engine. This improves the accuracy, creation efficiency, and collection efficiency of training data for machine learning. Therefore, the recognition accuracy of the image recognition engine can be efficiently improved.

In FIG. 17, instead of listing options for tag candidates (answers), the target person (grandchild) U1 may be prompted to enter a tag. For example, without performing steps S301b and S302a, the inquiry unit 348 may generate an inquiry that prompts the user to enter a tag (attribute information). In that case, the inquiry unit 348 transmits the generated inquiry content and the object image to the tablet terminal 101 via the network N. FIG. In response to this, the tablet terminal 101 displays the content of the inquiry received, the entry field for the tag, and the object image.

In addition, in FIG. 17, the supporter does not necessarily have to input the correct tag. For example, the support device 300 may determine whether or not the selection tag acquired in step S306a is correct, and notify the tablet terminal 103 and the smartphone 102 of the determination result. This also allows the supporters (grandparents) U3 and the target person's parent U2 to grasp whether the target person (grandchild) U1 is right or wrong with respect to the content of the question. In addition, it is possible to support interaction between the target person (grandchild) U1 and the supporter (grandparent) U3.

FIG. 19 is a diagram showing an example of an input screen 54 for attribute information of a captured image according to the third embodiment. The input screen 54 has an inquiry message 541 , a speech recognition result display field 542 and a character input field 543 . The inquiry message 541 is a display field for the contents of the inquiry generated in step S302a. Here, the message "Please enter the name of the visitor!" is displayed to indicate that the target person (grandchild) U1 is being asked. The voice recognition result display column 542 is a column for displaying text information as a result of voice recognition, which is picked up by the microphone 120 when the target person (grandchild) U1 speaks the name of the target object. Here, it indicates that "gerbera" has been recognized by voice. The character input column 543 is a column for displaying the result of character input by the target person (grandchild) U1. The tablet terminal 101 transmits the character information input in the voice recognition result display field 542 or the character input field 543 as a tag to the support device 300a. The subsequent steps are the same as steps S309 to S314. In this case as well, the support device 300a provides pairs of the target object image and the correct tag as training data for machine learning of the image recognition engine, thereby improving the accuracy, creation efficiency, and collection efficiency of the training data for machine learning. do. Therefore, the recognition accuracy of the image recognition engine can be efficiently improved.

It should be noted that the inquiry unit 348 may target the image associated with the correct tag among the object images 3125 . In that case, the identifying unit 344 may estimate an arbitrary tag candidate, and the inquiry unit 348 may generate inquiry content including the correct tag and the arbitrary tag candidate. This also serves as educational support for the target person (grandchild) U1, and promotes interaction with the supporter (grandparent) U3.

<Embodiment 4>
Embodiment 4 is a modification of Embodiment 2 or 3 described above. In the fourth embodiment, when text information such as a picture book is used as the support information, the supporter (grandparent) U3 further provides the reading voice to the target person (grandchild) U1. Note that the schematic configuration of the support system according to the fourth embodiment is the same as that shown in FIG. 3, so differences will be mainly described below, and illustrations and descriptions of overlapping configurations will be omitted. In the following description, the support information is text information that is an electronic book (display content), but it may be a real book.

That is, when the display content includes text information, the processing unit requests the supporter to read out the text information. Then, the acquisition unit acquires the reading voice for the text information from the supporter. Then, the processing unit notifies the first terminal of the subject as support information including the text information and the reading voice so that the reading voice is reproduced when the text information is displayed on the first terminal of the subject. As a result, the target person can also receive the reading voice of the provided display content as support information, which is most suitable for infants and the like. In addition, when the supporter is an elderly person, the supporter is motivated to read a picture book for his grandchildren and the like as dementia prevention training.

Furthermore, the acquisition unit preferably acquires, as the second captured image, an image captured by the target person when the sentence information is displayed and the reading voice is played back on the first terminal. As a result, it is possible to record the expression of the infant when reading a picture book or the like as an image or moving image, and provide the recorded image or moving image to the supporter. Therefore, the degree of satisfaction of the supporter is further improved.

FIG. 20 is a sequence diagram showing the flow of storytelling processing according to the fourth embodiment. First, in step S127 of FIG. 11 described above, it is assumed that the supporter (grandparents) U3 has selected the picture book of the electronic book as the support information and pressed the purchase button 5132 . At this time, the tablet terminal 103 transmits a support instruction including the ID (support information) of the selected text information (electronic book) to the support device 300 via the network N (S401). In response, the processing unit 347 transmits a read-aloud request for text information included in the support instruction to the tablet terminal 103 via the network N (S402).

Then, the tablet terminal 103 displays text information (S403) and prompts the supporter (grandparent) U3 to read it. When the supporter (grandparent) U3 reads out the displayed text information, the tablet terminal 103 inputs the voice information picked up by the microphone 120 as read-out voice (S404). The tablet terminal 103 then transmits the reading voice to the support device 300 via the network N (S405). In response, the acquisition unit 342 acquires the reading voice.

Then, the registration unit 341 associates the text information acquired in step S401 with the read-out voice acquired and registers them as support information in association with the subject ID 3121 (S406). In addition, the processing unit 347 transmits the support information including text information and reading voice to the tablet terminal 101 via the network N (S407). At this time, the processing unit 347 instructs the tablet terminal 101 to reproduce the reading voice when the text information is displayed.

The tablet terminal 101 displays the received text information and reproduces the reading voice (S408). At this time, the tablet terminal 101 photographs the face of the target person (grandchild) U1 with the camera 110, and specifies the display position (content position) of the text information at the time of photographing (S409). The content position is, for example, the displayed page of the picture book.

The tablet terminal 101 transmits the second captured image captured in step S409 and the specified display position to the support device 300 via the network N (S410). Then, the obtaining unit 342 obtains the second captured image and the display position. Then, the registration unit 341 registers the support information in association with the second captured image and the display position (S411). As a result, as in the above-described second embodiment, the identification unit 344 can identify the interest of the target person by taking into account the content of the displayed page of the picture book. Therefore, the accuracy of specifying the interest is improved. Also, the processing unit 347 transmits the second captured image to the tablet terminal 103 via the network N (S412). In response, the tablet terminal 103 displays the received second captured image (S413). As a result, the supporter (grandparent) U3 can visually recognize the picture book presented by him/herself and the state of the target person (grandchild) U1 at the time of reproducing the read-out voice, thereby improving satisfaction. As an effect of reading aloud as described above, for example, even if it is difficult for parents or grandparents to read directly to their grandchildren, they can identify the content that their grandchildren are interested in and input voice in advance. When the grandchild wants to read a book, the grandmother's voice can be read aloud. In addition, since the grandparents can also check the images and videos during the reading, it will motivate them to read next time.

In step S411, the processing unit 347 measures the degree of satisfaction of the subject (grandchild) U1 from the second photographed image, and if the degree of satisfaction is equal to or higher than a predetermined value, the second photographed image and the display The position may be notified to the tablet terminal 103 of the supporter (grandparent) U3. As a result, the supporters (grandparents) U3 can grasp where in the picture book the target person (grandchildren) U1 was pleased, and the level of satisfaction of the supporters (grandparents) U3 is further improved. Moreover, the processing unit 347 may associate and register the second captured image and the display position when the degree of satisfaction is equal to or higher than a predetermined value. Thereby, the specifying unit 344 can specify the interest of the target person (grandchild) U1 in consideration of the display position with high satisfaction. Therefore, it is possible to improve the accuracy of specifying the interest.

In this embodiment, the

tablet terminals

101 and 103 may perform video and audio communication, and the supporter (grandparents) U3 may read to the target person (grandchild) U1 in real time.

<Other embodiments>
In the second embodiment described above, the support device 300 and the authentication device 200 are described as separate information processing devices, but they may be the same. For example, the support device 300 may register facial feature information in association with the subject ID 3121 of the subject management information 312 . In that case, the control unit 340 may include the face detection unit 220, the feature point extraction unit 230, the registration unit 240, and the authentication unit 250 shown in FIG.

Note that the support device 300 may generate an album image in which a plurality of target person images 3126 are aggregated, and provide the album image to the supporter (grandparent) U3. That is, the acquisition unit acquires a plurality of third captured images in which the target person is captured in response to provision of the support information to the target person. Then, the registration unit associates and registers the subject with the plurality of third captured images. Then, the processing unit may aggregate at least part of the plurality of third captured images associated with the subject to generate a composite image, and transmit the composite image to the supporter. At this time, the third captured image may include an object image 3125 captured by the target person (grandchild) U1 in addition to the target person image 3126 including the target person (grandchild) U1.

Note that the support system 1000 described above may present support information candidates with a third party unrelated to the subject as a supporter. For example, it is also applicable when an elderly person wants to support a child regardless of the presence or absence of a blood relationship. For example, if the target person is interested in flowers, an elderly person familiar with flowers may be associated and registered as a supporter.

In addition, in the above-described embodiment, the hardware configuration has been described, but the configuration is not limited to this. The present disclosure can also implement arbitrary processing by causing a CPU to execute a computer program.

In the above example, the program can be stored and supplied to the computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible discs, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical discs), CD-ROMs (Read Only Memory), CD-Rs, Includes CD-R/W, DVD (Digital Versatile Disc), semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be delivered to the computer on various types of transitory computer readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels.

It should be noted that the present disclosure is not limited to the above embodiments, and can be modified as appropriate without departing from the scope. In addition, the present disclosure may be implemented by appropriately combining each embodiment.

Some or all of the above embodiments may also be described in the following additional remarks, but are not limited to the following.
(Appendix A1)
a registration unit that associates and registers a support target person and a supporter;
an acquisition unit that acquires a first captured image captured by the subject;
a specifying unit that specifies the subject's interest based on the first captured image;
a determination unit that determines one or more candidates for assistance information based on the identified interest;
a presentation unit that presents the determined support information candidate to the supporter associated with the target person;
A support device comprising:
(Appendix A2)
The support device according to appendix A1, further comprising a processing unit that receives support instructions including support information selected by the support person from the presented support information candidates, and performs processing according to the support information.
(Appendix A3)
The support device according to Appendix A2, wherein the processing unit performs processing for providing the support information to the target person.
(Appendix A4)
The support device according to appendix A2 or A3, wherein the processing unit notifies the target person of the support information.
(Appendix A5)
The processing unit notifies the target person's first terminal of the support information including the display content,
The acquisition unit acquires a second captured image of the target person when the display content is displayed on the first terminal,
The support device according to Appendix A4, wherein the registration unit associates and registers the target person, the display content, and the second captured image.
(Appendix A6)
The processing unit measures the satisfaction level of the subject from the second captured image, and notifies the terminal of the supporter of the second captured image when the satisfaction level is equal to or higher than a predetermined value. A support device according to A5.
(Appendix A7)
The acquisition unit further acquires a content position of the display content displayed on the first terminal at the time of capturing together with the second captured image,
The registration unit associates and registers the target person, the content position, and the second captured image,
The support device according to appendix A5 or A6, wherein the specifying unit specifies the interest of the target person based on the content position and the second captured image.
(Appendix A8)
When the display content includes text information, the processing unit requests the supporter to read out the text information,
The acquisition unit acquires a reading voice for the text information from the supporter,
The processing unit notifies the first terminal as the support information including the text information and the reading voice so that the reading voice is reproduced when the text information is displayed on the first terminal. Assistance device according to any one of appendices A5 to A7.
(Appendix A9)
The support device according to Appendix A8, wherein the acquisition unit acquires, as the second captured image, an image captured by the target person during display of the text information and playback of the readout sound on the first terminal.
(Appendix A10)
The acquisition unit acquires a plurality of third captured images in which the target person is captured in response to the provision of the support information to the target person,
The registration unit associates and registers the target person and the plurality of third captured images,
The processing unit aggregates at least part of the plurality of third captured images associated with the subject to generate a composite image, and transmits the composite image to the supporter. The support device according to any one of Claims 1 to 3.
(Appendix A11)
The specifying unit specifies a photographer of the first captured image,
The support device according to any one of Appendixes A1 to A10, wherein the registration unit registers the specified photographer as the target person in association with the first captured image.
(Appendix A12)
The registration unit pre-registers the first biometric information of the subject,
The acquisition unit acquires second biometric information of the photographer together with the first captured image,
The assisting apparatus according to Appendix A11, wherein the specifying unit specifies the photographer as the target person when biometric authentication in the second biometric information is successful using the first biometric information.
(Appendix A13)
further comprising an inquiry unit that inquires of the subject about attribute information of the first captured image;
The acquisition unit acquires the attribute information in response to the inquiry from the subject,
The support device according to any one of Appendixes A1 to A12, wherein the specifying unit specifies the subject's interest based on the first captured image and the attribute information.
(Appendix A14)
The acquisition unit acquires position information of a shooting location together with the first captured image,
The registration unit associates and registers the photographing location with the first photographed image,
The support device according to any one of Appendixes A1 to A13, wherein the specifying unit specifies the subject's interest based on the first captured image and the shooting location.
(Appendix A15)
The identification unit according to any one of appendices A1 to A14, wherein the identification unit excludes a face-recognized area from the first captured image, and identifies the subject's interest based on the excluded image. support equipment.
(Appendix A16)
The assisting apparatus according to Appendix A15, wherein the specifying unit excludes a region other than the target person from among the face-recognized regions of the first captured image.
(Appendix B1)
a first terminal of a support target person;
a supporter's second terminal;
a support device;
The support device is
a registration unit that associates and registers the target person and the supporter;
an acquisition unit that acquires a first captured image captured by the subject from the first terminal;
a specifying unit that specifies the subject's interest based on the first captured image;
a determination unit that determines one or more candidates for assistance information based on the identified interest;
a presentation unit that presents the determined support information candidate to the second terminal of the supporter associated with the target person;
Support system with
(Appendix B2)
The support device is
The support system according to appendix B1, further comprising a processing unit that receives support instructions including support information selected by the support person from the presented support information candidates, and performs processing according to the support information.
(Appendix C1)
the computer
Corresponding and registering the target of support and the supporter,
Acquiring a first captured image captured by the subject;
Identifying the subject's interest based on the first captured image;
determining one or more candidates for assistance information based on the identified interests;
Presenting the determined support information candidate to the supporter associated with the target person;
how to help.
(Appendix D1)
a registration process for registering a support target person and a supporter in association with each other;
Acquisition processing for acquiring a first captured image captured by the subject;
a specifying process of specifying the subject's interest based on the first captured image;
a determination process of determining one or more candidates for assistance information based on the identified interest;
a presentation process of presenting the determined support information candidate to the supporter associated with the target person;
A support program that causes a computer to execute

Although the present invention has been described with reference to the embodiments (and examples), the present invention is not limited to the above-described embodiments (and examples). Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2021-045723 filed on March 19, 2021, and the entire disclosure thereof is incorporated herein.

1 support device 11 registration unit 12 acquisition unit 13 identification unit 14 determination unit 15 presentation unit 1000 support system U1 target person (grandchild)
U2 Subject's parents U3 Supporters (grandparents)
N network 101 tablet terminal 102 smartphone 103 tablet terminal 110 camera 120 microphone 130 speaker 140 touch panel 150 storage unit 151 program 160 communication unit 170 memory 180 control unit 181 registration unit 182 content processing unit 410 camera 420 microphone 430 speaker 440 touch panel 450 storage unit 451 Program 460 Communication Unit 470 Memory 480 Control Unit 481 SNS Application 200 Authentication Device 210 Face Information DB
211 User ID
212 facial feature information 220 face detection unit 230 feature point extraction unit 240 registration unit 250 authentication unit 300 support device 310 storage unit 311 program 312 subject management information 3121 subject ID
3122 Supporter ID
3123 Terminal ID
3124 Subject information 3125 Subject image 3126 Subject image 313 Supporter information 3131 Supporter ID
3132 Personal information 3133 Terminal ID
314 Support Information DB
3141 support information candidate 314n support information candidate 320 memory 330 communication unit 340 control unit 341 registration unit 342 acquisition unit 343 authentication control unit 344 identification unit 345 determination unit 346 presentation unit 347 processing unit 51 presentation screen 5111 selection field 5112 selection field 5113 selection field 5121 Support information candidate display field 5122 Support information candidate display field 5123 Support information candidate display field 5131 Other candidate display button 5132 Purchase button 5133 Cancel button 51a Presentation screen 51b Thank you information display screen 300a Support device 311a Program 348 Inquiry unit 52 Selection screen 521 Inquiry message 522 Photographed image group 53 Selection screen 531 Inquiry message 532 Options 54 Input screen 541 Inquiry message 542 Voice recognition result display field 543 Character input field

Claims

a registration means for registering a support target person and a supporter in association with each other;
Acquisition means for acquiring a first captured image captured by the subject;
identifying means for identifying the subject's interest based on the first captured image;
determining means for determining one or more candidate assistance information based on the identified interest;
presenting means for presenting the determined support information candidate to the supporter associated with the target person;
A support device comprising:
2. The support apparatus according to claim 1, further comprising processing means for receiving a support instruction including support information selected by said supporter from said presented support information candidates and performing a process according to said support information.
3. The support device according to claim 2, wherein the processing means performs processing for providing the support information to the subject.
The support device according to claim 2 or 3, wherein the processing means notifies the target person of the support information.
The processing means notifies the first terminal of the target person of the support information including the display content,
The acquisition means acquires a second captured image of the target person when the display content is displayed on the first terminal,
5. The support device according to claim 4, wherein said registration means associates and registers said target person, said display content, and said second captured image.
The processing means measures the degree of satisfaction of the subject from the second captured image, and notifies the terminal of the supporter of the second captured image when the degree of satisfaction is equal to or higher than a predetermined value. Item 6. The support device according to item 5.
The acquisition means further acquires a content position of the display content displayed on the first terminal at the time of capturing together with the second captured image,
The registration means associates and registers the target person, the content position, and the second captured image,
7. The assisting apparatus according to claim 5, wherein said identifying means identifies the subject's interest based on said content position and said second captured image.
When the display content includes text information, the processing means requests the supporter to read out the text information,
The acquisition means acquires a reading voice for the text information from the supporter,
The processing means notifies the first terminal as the support information including the text information and the reading voice so that the reading voice is reproduced when the text information is displayed on the first terminal. 8. A support device according to any one of claims 5 to 7.
9. The support device according to claim 8, wherein said acquisition means acquires, as said second captured image, an image captured by said target person when said text information is displayed on said first terminal and said reading voice is reproduced.
The acquisition means acquires a plurality of third captured images in which the target person is captured in response to the provision of the support information to the target person,
The registration means associates and registers the target person and the plurality of third captured images,
3. The processing means aggregates at least part of the plurality of third captured images associated with the target person to generate a composite image, and transmits the composite image to the support person. 10. The support device according to any one of 9.
The specifying means specifies a photographer of the first captured image,
11. The support device according to any one of claims 1 to 10, wherein said registration means registers said specified photographer as said target person in association with said first captured image.
The registration means pre-registers the first biometric information of the subject,
The acquisition means acquires second biometric information of the photographer together with the first captured image,
12. The assisting apparatus according to claim 11, wherein said specifying means specifies said photographer as said subject when biometric authentication in said second biometric information is successful using said first biometric information.
further comprising inquiry means for inquiring of the subject about attribute information of the first captured image;
The acquisition means acquires the attribute information in response to the inquiry from the subject,
13. The assisting device according to any one of claims 1 to 12, wherein the identifying means identifies the subject's interest based on the first captured image and the attribute information.
The acquisition means acquires position information of a shooting location together with the first captured image,
The registration means associates and registers the photographing location with the first photographed image,
14. The assisting device according to any one of claims 1 to 13, wherein the specifying means specifies the subject's interest based on the first captured image and the shooting location.
15. The specifying unit according to any one of claims 1 to 14, wherein the identifying means excludes a face-recognized area from the first captured image and identifies the subject's interest based on the excluded image. support device.
16. The support device according to claim 15, wherein the identifying means excludes areas other than the target person from areas in which the face of the first captured image has been recognized.
a first terminal of a support target person;
a supporter's second terminal;
a support device;
The support device is
a registration means for registering the target person and the supporter in association with each other;
Acquisition means for acquiring a first captured image captured by the subject from the first terminal;
identifying means for identifying the subject's interest based on the first captured image;
determining means for determining one or more candidate assistance information based on the identified interest;
presenting means for presenting the determined support information candidate to the second terminal of the supporter associated with the target person;
Support system with
The support device is
18. The support system according to claim 17, further comprising processing means for receiving a support instruction including the support information selected by the supporter from the presented support information candidates and performing a process according to the support information.
the computer
Corresponding and registering the target of support and the supporter,
Acquiring a first captured image captured by the subject;
Identifying the subject's interest based on the first captured image;
determining one or more candidates for assistance information based on the identified interests;
Presenting the determined support information candidate to the supporter associated with the target person;
how to help.
a registration process for registering a support target person and a supporter in association with each other;
Acquisition processing for acquiring a first captured image captured by the subject;
a specifying process of specifying the subject's interest based on the first captured image;
a determination process of determining one or more candidates for assistance information based on the identified interest;
a presentation process of presenting the determined support information candidate to the supporter associated with the target person;
A non-transitory computer-readable medium that stores a support program that causes a computer to execute