JP7367952B2 - Communication devices, information output systems, terminal equipment, and communication programs - Google Patents

Description

The present invention relates to a communication device, an information output system, a terminal device, a communication program, and a communication method.

Unlike grandparents who live with their grandchildren in a two-family house, grandparents who live apart from their grandchildren cannot see their grandchildren grow up on a daily basis.

Patent Document 1 describes a technology that allows two people living in separate locations to easily share images without requiring any special knowledge. With this technology, even grandparents who live far away from their grandchildren can see their grandchildren's growth on a daily basis by viewing posted images of their grandchildren.

International Publication No. 2015/190567

Some two-family homes have separate living spaces for the parent and child households. In such housing, there are relatively few opportunities for parent and child households to meet each other, but because they live next door, they can feel each other's presence on a daily basis.

In the technology described in Patent Document 1, when an image posted by a user in a child household is viewed by a user in the parent household, the user in the child household is notified that the image has been viewed. Therefore, even child households that live apart from the parent household can feel the presence of the parent household when receiving notifications. However, if the images are viewed infrequently, the frequency of notifications is also infrequent, making it difficult for the child household to feel the presence of the parent household on a daily basis.

The purpose of the present invention is to increase the chances that at least one of the two parties will feel the presence of the other.

A communication device according to an embodiment of the present invention includes:
A communications department that sends and receives information;
Receive user input information input by a first user via the communication unit, perform control to output the user input information to the second user in response to an operation by a second user, and perform the control. a control unit configured to transmit, as first action information, information indicating that the second user has taken a first action in contact with the user input information via the communication unit, when the second user and a control unit that recognizes a behavior situation including a second behavior different from the first behavior of the user, and transmits second behavior information indicating a recognition result of the behavior situation via the communication unit.

Another communication device according to an embodiment of the present invention is
A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information to the first user.

A communication program according to an embodiment of the present invention includes:
A computer equipped with a communication unit that sends and receives information,
Receive user input information input by a first user via the communication unit, perform control to output the user input information to the second user in response to an operation by a second user, and perform the control. If the second user has taken a first action in response to the user input information, transmitting information indicating that the second user has taken a first action in response to the user input information as first action information;
Recognizing a behavior situation including a second behavior different from the first behavior of the second user, and transmitting second behavior information indicating a recognition result of the behavior situation via the communication unit. let

Another communication program according to an embodiment of the present invention is
A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. and the steps to
Receive second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user via the communication unit, and transmit the second behavior information to the first user. and a step of controlling output toward the target.

In a communication method according to an embodiment of the present invention,
the terminal device transmits user input information input by the first user;
an information output system receives the user input information;
The information output system outputs the user input information to the second user in response to an operation by a second user, and in that case, the second user takes a first action that contacts the user input information. Send information indicating this as first action information,
The terminal device receives the first behavior information and outputs the first behavior information to the first user,
The information output system recognizes a behavior situation including a second behavior different from the first behavior of the second user, and transmits second behavior information indicating a recognition result of the behavior situation,
The terminal device receives the second behavior information and outputs the second behavior information to the first user.

According to an embodiment of the present invention, it is possible to increase the chances that at least one of the two parties can feel the presence of the other.

1 is a diagram showing an overview of the configuration of a communication system according to an embodiment of the present invention. FIG. 1 is a perspective view of equipment included in an information output system according to an embodiment of the present invention. 1 is a block diagram showing the configuration of a communication system according to an embodiment of the present invention. 1 is a flowchart showing the operation of a communication system according to an embodiment of the present invention. 1 is a flowchart showing the operation of a communication system according to an embodiment of the present invention. 1 is a flowchart showing the operation of a communication system according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of a screen of a terminal device according to an embodiment of the present invention. It is a flowchart which shows the operation of the communication system concerning the 1st modification. It is a flow chart which shows operation of a communication system concerning a 2nd modification.

Embodiments of the present invention will be described below with reference to the drawings.

In each figure, the same or corresponding parts are given the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

An overview of this embodiment will be explained with reference to FIG.

Terminal device 101 is a device used by a first user. The information output system 102 is a system installed in the second user's residence. The first user and the second user may be any users, but in this embodiment, they are a user in a child household and a user in a parent household, respectively, who live in locations apart from each other. A member of the child household family may be regarded as the first user, or the entire child household family may be regarded as the first user. A member of the parent household family may be regarded as the second user, or the entire parent household family may be regarded as the second user. The relationship between the first user and the second user is not limited to a one-to-one relationship, and may be any one-to-many, many-to-one, or many-to-many relationship.

Terminal device 101 transmits user input information 400. Information output system 102 receives user input information 400 . The user input information 400 may be directly transmitted from the terminal device 101 to the information output system 102, but in this embodiment, the user input information 400 is transmitted from the terminal device 101 to the server 103, is temporarily stored in the server 103, and then is transmitted from the server 103. It is transmitted to the information output system 102.

User input information 400 is information input by the first user. In this embodiment, the user input information 400 includes an image such as a photo or video of the first user's child, that is, the second user's grandchild, taken by the first user with the terminal device 101. User input information 400 may include other images or may include data other than images, such as audio or text messages.

The information output system 102 outputs user input information 400 to the second user in response to the second user's operation. In that case, the information output system 102 transmits the first behavior information 410. The terminal device 101 receives the first behavior information 410. The first behavior information 410 may be transmitted from the information output system 102 to the terminal device 101 via the server 103, but in this embodiment, it is directly transmitted from the information output system 102 to the terminal device 101. The terminal device 101 outputs the first behavior information 410 to the first user.

The first action information 410 is information indicating that the second user has taken a first action related to the user input information 400. In this embodiment, the first action information 410 includes a message such as "I saw the image!" indicating that the second user took the action of viewing the image as the first action. If the user input information 400 is audio, the first action information 410 includes a message such as "I heard the audio!" indicating that the second user took the action of listening to the audio as the first action. It's okay. If the user input information 400 is a text message, the first action information 410 includes a message such as "I read the message!" indicating that the second user took the action of reading the text message as the first action. May be included.

The information output system 102 recognizes a behavior situation including a second behavior different from the first behavior of the second user. Information output system 102 transmits second behavior information 420. The terminal device 101 receives the second behavior information 420. The second behavior information 420 may be transmitted from the information output system 102 to the terminal device 101 via the server 103, but in this embodiment, it is directly transmitted from the information output system 102 to the terminal device 101. The terminal device 101 outputs the second behavior information 420 to the first user.

The second behavior information 420 is information indicating the result of recognition of the behavior status of the second user by the information output system 102. The second action information 420 includes information indicating that the second user has taken the second action, or information indicating that the second user has not taken the second action in a situation where the second action should have been taken. . In this embodiment, if the second user's operation to turn on the air conditioner is recognized as the second action, the second action information 420 indicates that the second user has performed the operation to turn on the air conditioner. Messages include things like "I turned on the air conditioner!" If an operation other than the air conditioner or an operation of a home appliance other than the air conditioner is recognized as the second action, the second action information 420 may include a message indicating that the second user performed the operation. good. If an action other than operating a home appliance is recognized as the second action, the second action information 420 may include a message indicating that the second user has taken the action. On the other hand, if the second user is supposed to turn on the air conditioner and such an operation is not recognized as a second action, then the second action information 420 includes the second user's operation to turn on the air conditioner. Messages such as ``The air conditioner is not on!'' are included to indicate that you are not doing this when you should be doing so. If such an operation is not recognized as a second action in a situation where an operation other than the air conditioner or an operation of a home appliance other than the air conditioner is to be performed, the second action information 420 includes information indicating that the second user has performed such an operation. It may also include a message indicating that this is not being done when it should be done. If such an action is not recognized as a second action in a situation in which an action other than operating a home appliance should be taken, the second action information 420 includes information on the action that the second user would take in a situation in which such action should be taken. A message may be included to indicate that there is no such information.

In this embodiment, when an image posted by a first user is viewed by a second user, the first user is notified that the image has been viewed, and a specific action of the second user is recognized. Then, the first user is notified that the action has been taken. Therefore, even child households who live apart from their parents can feel the presence of their parents every time they receive a notification. Even if the frequency of images being viewed is low, there will be notifications for actions other than viewing images, so the frequency of notifications will not decrease and child households will not feel the presence of the parent household on a daily basis. can. In addition to the frequency of image viewing, there are also cases where child households send images infrequently. If the image is not sent, it will not be viewed. However, even in such a case, since there are notifications for actions other than the action of viewing images, the frequency of notifications does not decrease, and the child household can feel the presence of the parent household on a daily basis. Furthermore, in this embodiment, if the second user is in a situation where he or she should take a specific action and the action is not recognized, the first user is notified that the action is not taken in the situation where the action should be taken. Such notifications are likely to be an opportunity for child households living apart from the parent household to visit the parent household or contact the parent household by communication means such as telephone or email, and the child household and the parent household may This will lead to the promotion of communication between

In this way, according to the present embodiment, it is possible to increase the chances that at least the first user can feel the presence of the second user between the first user and the second user.

If the relationship between the first user and the second user is many-to-one, images are transmitted from a plurality of child generation families to a parent generation family, and notifications are sent from the parent generation family to a plurality of child generation families. For example, if the relationship between the first user and the second user is one-to-many, images can be sent from child generation families to multiple parent generation families, and notifications can be sent from multiple parent generation families to child generation families. be exposed. For example, if the relationship between the first user and the second user is many-to-many, images can be sent from multiple child generation families to multiple parent generation families, and from multiple parent generation families to multiple child generation families. A notification will be given.

As a modification of this embodiment, in addition to the notification from the parent household to the child household, the child household may notify the parent household. By performing two-way notification, it is possible to increase the chances that the second user can feel the presence of the first user between the first user and the second user.

For example, a function equivalent to the terminal device 101 according to this embodiment may be added to the information output system 102, and a function equivalent to the information output system 102 according to this embodiment may be added to the terminal device 101. Alternatively, a terminal device having functions equivalent to the terminal device 101 according to this embodiment is provided to the second user, and an information output system having functions equivalent to the information output system 102 according to this embodiment is provided to the first user. It can also be installed in a house.

In this modified example, when an image posted by a second user is viewed by a first user, the second user is notified that the image has been viewed, and a specific action of the first user is recognized. Then, the second user is notified that the action has been taken. Alternatively, if the first user is in a situation where a specific action should be taken and the action is not recognized, the second user is notified that the action is not taken in the situation where the action should be taken. Therefore, even a parent household that lives apart from a child household can feel the presence of the child household every time they receive a notification. Therefore, the parent household can feel the presence of the child household on a daily basis. In other words, like a two-family house, the parent household and the child household can feel each other's presence on a daily basis. As a result, a two-family home in the digital age or a virtual two-family home can be realized.

The configuration of a communication system 100 according to this embodiment will be described with reference to FIGS. 1, 2, and 3.

The communication system 100 includes a terminal device 101, an information output system 102, and a server 103.

The terminal device 101, the information output system 102, and the server 103 can communicate with each other via a network 104 such as a mobile communication network and the Internet.

In this embodiment, the terminal device 101 is configured as a mobile device carried by the first user, such as a smartphone or a tablet, but may be configured as a non-mobile device used by the first user at home or in an office, such as a desktop PC. may be done. "PC" is an abbreviation for Personal Computer.

Terminal device 101 includes communication device 110 .

Communication device 110 is a microcomputer or SoC. "SoC" is an abbreviation for System on a Chip. The communication device 110 includes a control section 111, a storage section 112, a communication section 113, and a positioning section 114.

Control unit 111 is one or more processors. As the processor, a general-purpose processor such as a CPU or a dedicated processor specialized for specific processing can be used. "CPU" is an abbreviation for Central Processing Unit. The control unit 111 may include one or more dedicated circuits, or the one or more processors in the control unit 111 may be replaced with one or more dedicated circuits. For example, an FPGA or an ASIC can be used as the dedicated circuit. "FPGA" is an abbreviation for Field-Programmable Gate Array. "ASIC" is an abbreviation for Application Specific Integrated Circuit. The control unit 111 executes information processing related to the operation of the communication device 110 while controlling each part of the terminal device 101 including the communication device 110.

Storage unit 112 is one or more memories. As the memory, for example, semiconductor memory, magnetic memory, or optical memory can be used. The memory functions as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 112 stores information used for the operation of the communication device 110 and information obtained by the operation of the communication device 110.

Communication unit 113 is one or more communication modules. As the communication module, for example, a communication module compatible with LTE, 4G, or 5G can be used. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th Generation. "5G" is an abbreviation for 5th Generation. When the terminal device 101 is configured as a non-mobile device, a communication module compatible with the LAN standard, for example, can also be used as the communication module. "LAN" is an abbreviation for Local Area Network. The communication unit 113 receives information used for the operation of the communication device 110 and transmits information obtained by the operation of the communication device 110. That is, the communication unit 113 transmits and receives information.

The positioning unit 114 is one or more positioning modules. As the positioning module, for example, a positioning module compatible with GPS, QZSS, GLONASS, or Galileo can be used. "GPS" is an abbreviation for Global Positioning System. "QZSS" is an abbreviation for Quasi-Zenith Satellite System. The QZSS satellites are called quasi-zenith satellites. "GLONASS" is an abbreviation for Global Navigation Satellite System. The positioning unit 114 acquires position information of the terminal device 101.

The functions of the communication device 110 are realized by executing the first communication program according to the present embodiment by a processor included in the control unit 111. That is, the functions of the communication device 110 are realized by software. The first communication program is a program for causing the computer to execute the processing of the steps included in the operation of the communication device 110, thereby causing the computer to realize a function corresponding to the processing of the steps. That is, the first communication program is a program for causing the computer to function as the communication device 110.

The program can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory can be used. Distribution of the program is performed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM on which the program is recorded. "DVD" is an abbreviation for Digital Versatile Disc. "CD-ROM" is an abbreviation for Compact Disc Read Only Memory. The program may be distributed by storing the program in the storage of the server and transferring the program from the server to another computer via a network. The program may be provided as a program product.

For example, a computer temporarily stores a program recorded on a portable recording medium or a program transferred from a server in its memory. Then, the computer uses a processor to read a program stored in the memory, and causes the processor to execute processing according to the read program. A computer may directly read a program from a portable recording medium and execute processing according to the program. The computer may sequentially execute processing according to the received program each time the program is transferred to the computer from the server. Processing may be performed using a so-called ASP type service that implements functions only by issuing execution instructions and obtaining results without transferring programs from the server to the computer. "ASP" is an abbreviation for Application Service Provider. The program includes information similar to a program that is used for processing by an electronic computer. For example, data that is not a direct command to a computer but has the property of regulating computer processing falls under "something similar to a program."

Some or all of the functions of the communication device 110 may be realized by a dedicated circuit included in the control unit 111. That is, some or all of the functions of the communication device 110 may be realized by hardware.

Terminal device 101 further includes one or more input devices 120 and one or more output devices 130.

The input device 120 can be, for example, a physical key, a capacitive key, a pointing device, a touch screen integrated with a display, a camera, or a microphone. The input device 120 receives input of information used for the operation of the communication device 110 from the first user.

As the output device 130, for example, a display or a speaker can be used. As the display, for example, an LCD or an organic EL display can be used. "LCD" is an abbreviation for Liquid Crystal Display. "EL" is an abbreviation for Electro Luminescence. The output device 130 outputs information obtained by the operation of the communication device 110 to the first user.

In this embodiment, the information output system 102 is configured as a combination of multiple devices installed in the second user's residence, such as an STB 201, a remote controller 202, and a television 203. It may be configured as a mobile device that is carried around, or it may be configured as a non-mobile device, such as a desktop PC, that is used by a second user at home or in an office. "STB" is an abbreviation for Set-Top Box.

The outer shell of the STB 201 may have any shape, but in this embodiment, it has a shape that resembles a house. An indicator 204 is provided on the front of the STB 201 to notify that user input information 400 such as a new image has arrived. The indicator 204 may have any shape, but in this embodiment, it has a shape that resembles a window of a house. On the back of the STB 201, a port 205 for HDMI (registered trademark) connection, a port 206 for USB connection, and a port 207 for power connection are provided. "HDMI (registered trademark)" is an abbreviation for High Definition Multimedia Interface (registered trademark). "USB" is an abbreviation for Universal Serial Bus.

The information output system 102 includes a communication device 210.

Communication device 210 is a microcomputer or SoC. The communication device 210 is built into the STB 201 in this embodiment. The communication device 210 includes a control section 211, a storage section 212, a communication section 213, and an I/O section 214. "I/O" is an abbreviation for Input/Output.

Control unit 211 is one or more processors. As the processor, a general-purpose processor such as a CPU or a dedicated processor specialized for specific processing can be used. The control unit 211 may include one or more dedicated circuits, or the one or more processors in the control unit 211 may be replaced with one or more dedicated circuits. For example, an FPGA or an ASIC can be used as the dedicated circuit. The control unit 211 executes information processing related to the operation of the communication device 210 while controlling each part of the information output system 102 including the communication device 210.

Storage unit 212 is one or more memories. As the memory, for example, semiconductor memory, magnetic memory, or optical memory can be used. The memory functions as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 212 stores information used for the operation of the communication device 210 and information obtained by the operation of the communication device 210.

Communication unit 213 is one or more communication modules. As the communication module, for example, a communication module compatible with LTE, 4G, 5G, or LAN standards can be used. The communication unit 213 receives information used for the operation of the communication device 210 and transmits information obtained by the operation of the communication device 210. That is, the communication unit 213 transmits and receives information.

I/O section 214 is one or more I/O modules. As the I/O module, for example, an I/O module compatible with HDMI (registered trademark), USB, or BLE can be used. "BLE" is an abbreviation for Bluetooth (registered trademark) Low Energy. The I/O unit 214 receives information used for the operation of the communication device 210 and transmits information obtained by the operation of the communication device 210.

The functions of the communication device 210 are realized by executing the second communication program according to the present embodiment by a processor included in the control unit 211. That is, the functions of communication device 210 are realized by software. The second communication program is a program for causing the computer to execute the processing of the step included in the operation of the communication device 210, thereby causing the computer to realize a function corresponding to the processing of the step. That is, the second communication program is a program for causing the computer to function as the communication device 210.

Some or all of the functions of the communication device 210 may be realized by a dedicated circuit included in the control unit 211. That is, some or all of the functions of the communication device 210 may be realized by hardware.

Information output system 102 further includes one or more input devices 220, one or more output devices 230, one or more sensors 240, one or more meters 250, and one or more electronic devices 260.

In this embodiment, the remote controller 202 is used as the input device 220, but for example, physical keys, capacitive keys, a pointing device, a touch screen provided integrally with a display, a camera, or a microphone may be used. You may. The input device 220 receives input of information used for the operation of the communication device 210 from the second user.

Although the television 203 is used as the output device 230 in this embodiment, for example, a display other than the television 203 or a speaker may be used. As the display, for example, an LCD or an organic EL display can be used. The output device 230 outputs information obtained by the operation of the communication device 210 to the second user.

The television 203 is connected to the STB 201 via a cable that is inserted into a port 205 provided on the back of the STB 201. Operations on the STB 201 are performed from the remote controller 202 via the television 203. That is, when the second user performs an operation such as pressing a button on the remote controller 202, the remote controller 202 transmits an infrared signal or a wireless signal to the television 203 in accordance with the operation. The television 203 transmits a control signal according to the infrared signal or wireless signal transmitted from the remote controller 202 to the STB 201 through HDMI (registered trademark) communication. The communication device 210 of the STB 201 executes processing according to the control signal transmitted from the television 203. As these series of procedures, the procedure of operating the image output device from a remote control device via a display device in the technique described in Patent Document 1 may be applied.

Sensor 240 is installed in the second user's residence. As the sensor 240, for example, a temperature sensor, a humidity sensor, an illuminance sensor, a door sensor, or a human sensor can be used. Sensor 240 outputs data used for operation of communication device 210. For example, a temperature sensor measures the indoor temperature and outputs temperature data. The humidity sensor measures the indoor humidity and outputs humidity data. The illuminance sensor measures the illuminance in the room and outputs illuminance data. The door sensor detects the open/closed state of the refrigerator door and outputs data on the open/closed state of the door. The human sensor detects people in the surrounding area and outputs data on the presence or absence of people.

Meter 250 is installed in the second user's residence. As the meter 250, for example, an electric power meter, a gas meter, or a water meter can be used. Meter 250 outputs data used for operation of communication device 210. For example, the power meter measures the amount of power used in the first user's residence and outputs data on the amount of power used. The gas meter measures the amount of gas used in the first user's residence and outputs data on the amount of gas used. The water meter measures the amount of water used in the first user's residence and outputs data on the amount of water used.

The sensor 240 and the meter 250 are connected to the STB 201 via a cable inserted into a port 206 provided on the back of the STB 201, or wirelessly connected to the STB 201. When the sensor 240 and the meter 250 obtain output data, they transmit the obtained data to the STB 201 by USB communication, BLE communication, or other wireless communication. The communication device 210 of the STB 201 executes processing using data transmitted from the sensor 240 and meter 250.

Electronic device 260 is installed in the second user's residence. As the electronic device 260, for example, an air conditioner, a humidifier, a lighting device, a refrigerator, or a home appliance such as the television 203 can be used. When the electronic device 260 is operated by the second user, it notifies the communication device 210 of what kind of operation was performed. For example, when an operation such as powering on, powering off, setting temperature, setting humidity, or switching an operating mode is performed on the air conditioner, the communication device 210 is notified of which operation has been performed. When an operation such as turning on the humidifier, turning off the power, setting the intensity, or switching the operating mode is performed, the humidifier notifies the communication device 210 of which operation has been performed. When an operation such as powering on, powering off, or dimming is performed, the lighting device notifies the communication device 210 of which operation has been performed. When an operation such as opening/closing a door or setting the temperature inside the refrigerator is performed, the refrigerator notifies the communication device 210 of which operation has been performed. When an operation such as turning on the power, setting the volume, or switching a channel is performed, the television 203 notifies the communication device 210 of which operation has been performed.

Electronic devices 260 other than the television 203 are connected to the STB 201 via cables inserted into ports 206 provided on the back of the STB 201, or wirelessly connected to the STB 201. When an operation is performed by the second user, the electronic device 260 notifies the STB 201 that the operation has been performed through USB communication, BLE communication, or other wireless communication. The communication device 210 of the STB 201 executes processing according to the notification from the electronic device 260.

Although the server 103 may be installed at any location, in this embodiment, it is installed at a data center. The server 103 is, for example, a cloud server.

The server 103 includes a control section 301, a storage section 302, and a communication section 303.

Control unit 301 is one or more processors. As the processor, a general-purpose processor such as a CPU or a dedicated processor specialized for specific processing can be used. The control unit 301 may include one or more dedicated circuits, or the one or more processors in the control unit 301 may be replaced with one or more dedicated circuits. For example, an FPGA or an ASIC can be used as the dedicated circuit. The control unit 301 executes information processing related to the operation of the server 103 while controlling each unit of the server 103.

Storage unit 302 is one or more memories. As the memory, for example, semiconductor memory, magnetic memory, or optical memory can be used. The memory functions as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 302 stores information used for the operation of the server 103 and information obtained by the operation of the server 103.

Communication unit 303 is one or more communication modules. As the communication module, for example, a communication module compatible with LAN standards can be used. The communication unit 303 receives information used for the operation of the server 103 and transmits information obtained by the operation of the server 103. That is, the communication unit 303 transmits and receives information.

The functions of the server 103 are realized by executing the server program according to this embodiment by a processor included in the control unit 301. That is, the functions of the server 103 are realized by software. The server program is a program that causes the computer to execute the processing of the step included in the operation of the server 103, thereby causing the computer to realize a function corresponding to the processing of the step. That is, the server program is a program for causing a computer to function as the server 103.

Some or all of the functions of the server 103 may be realized by a dedicated circuit included in the control unit 301. That is, some or all of the functions of the server 103 may be realized by hardware.

The operation of the communication system 100 according to this embodiment will be described with reference to FIGS. 4 to 6. The operation of the communication system 100 corresponds to the communication method according to this embodiment.

The processing in steps S11 and S12 in FIG. 4 is executed by the communication device 110 of the terminal device 101.

In step S11, when the first user inputs the user input information 400 via the input device 120, the control unit 111 acquires the user input information 400.

Specifically, when the first user takes an image of the first user's child, that is, the second user's grandchild, with a camera included in the input device 120, the control unit 111 outputs the user input information 400 including the image. get. User input information 400 may include an image identifier. The user input information 400 may include time information indicating the time when the image was captured by the camera. Alternatively, the control unit 111 may add time information indicating the current time to the acquired user input information 400. The control unit 111 may obtain, via the positioning unit 114, the position information of the terminal device 101 at the time indicated by the time information. The control unit 111 may add the acquired position information to the acquired user input information 400. The position information is, for example, two-dimensional coordinates or three-dimensional coordinates of the current position of the terminal device 101 obtained using GPS, QZSS, GLONASS, Galileo, or a combination of two or more of these. The control unit 111 writes the user input information 400 into the storage unit 112.

In step S12, the control unit 111 transmits the user input information 400 acquired in step S11 via the communication unit 113.

Specifically, the control unit 111 reads user input information 400 including an image of the second user's grandchild from the storage unit 112. The control unit 111 transmits the read user input information 400 to the server 103 via the network 104.

The processing from step S13 to step S15 in FIG. 4 is executed by the server 103.

In step S13, the control unit 301 receives, via the communication unit 303, the user input information 400 transmitted in step S12.

Specifically, the control unit 301 receives user input information 400 including an image of the second user's grandchild from the terminal device 101 via the network 104 .

In step S14, the control unit 301 stores the user input information 400 received in step S13.

Specifically, the control unit 301 registers the user input information 400 including the image of the second user's grandchild in the database constructed in the storage unit 302 or external storage.

Although the control unit 301 may store the user input information 400 received in step S13 as is, in this embodiment, the output of the information output system 102 for the image included in the user input information 400 received in step S13. Processing such as enlargement, reduction, and cropping is performed according to the format, and user input information 400 including the processed image is stored.

If the user input information 400 includes meta information such as time information and location information along with the image, the control unit 301 may save the user input information 400 with the meta information included, or may save the user input information 400 with the meta information included. The user input information 400 may be saved without including the meta information, and the meta information may be saved in a separate database in association with the image identifier. When meta information is stored in association with an image identifier, the meta information can be retrieved from the image identifier and used as needed.

In step S15, the control unit 301 transmits the user input information 400 saved in step S14 via the communication unit 303.

Specifically, the control unit 301 acquires user input information 400 including an image of the second user's grandchild from the database. The control unit 301 transmits the acquired user input information 400 to the information output system 102 via the network 104.

The process of step S16 in FIG. 4 is executed by the communication device 210 of the information output system 102.

In step S16, the control unit 211 receives, via the communication unit 213, the user input information 400 transmitted in step S15.

Specifically, the control unit 211 receives user input information 400 including an image of the second user's grandchild from the server 103 via the network 104 . The control unit 211 writes the received user input information 400 into the storage unit 212.

In this embodiment, the image included in the user input information 400 received in step S16 is an image that has been subjected to processing such as enlargement or reduction and cropping in step S14 in accordance with the output form of the information output system 102. .

In step S17, the control unit 211 notifies the second user that the user input information 400 has been received in step S16.

Specifically, the control unit 211 lights or blinks the indicator 204 provided on the front surface of the STB 201.

As described above, in the present embodiment, when the first user inputs new user input information 400, the server 103 notifies the existence of new images using a push method; however, as a modified example of the present embodiment, The information output system 102 may periodically inquire of the server 103 about the presence or absence of new images at intervals such as once every few minutes, and the server 103 may notify that there are new images using a fetch method. Furthermore, in this embodiment, when new user input information 400 is input by the first user, the new user input information 400 is automatically transmitted from the server 103 to the information output system 102; As a modified example, the second user performs an inquiry operation, and in response to the operation, the information output system 102 inquires of the server 103 as to the presence or absence of new user input information 400, and the new user input information 400 is sent to the server 103 as information. It may be sent to output system 102.

The processing from step S21 to step S23 in FIG. 5 is executed by the communication device 210 of the information output system 102.

In step S21, when the second user performs an operation via the input device 220, the control unit 211 outputs the user input information 400 received in step S16 to the second user in accordance with the operation. Take control. According to this control, the output device 230 outputs the user input information 400 to the second user.

Specifically, the control unit 211 performs an operation such as pressing a button on the remote controller 202 included in the input device 220 by the second user, and sends the second user's grandchildren included in the user input information 400 via the television 203. When a request is made to display an image, the image is read from the storage unit 212. The control unit 211 controls displaying the read image on the screen of the television 203 included in the output device 230. According to this control, the television 203 displays the image of the second user's grandchild on the screen. If the user input information 400 includes an image identifier along with the image, the control unit 211 may control displaying the identifier on the screen of the television 203. If time information is included in the user input information 400 along with the image, the control unit 211 may control displaying the time information on the screen of the television 203. If the user input information 400 includes position information together with the image, the control unit 211 may control displaying the position information on the screen of the television 203.

In step S22, the control unit 211 performs control to output the user input information 400 to the second user in response to the second user's operation in step S21. Information indicating that an action has been taken is generated as first action information 410.

Specifically, the control unit 211 sends a message such as "I saw the image!" indicating that the second user took the action of looking at the image of the second user's grandchild included in the user input information 400 as a first action. The first behavior information 410 including the message is generated. The first behavior information 410 may include a message identifier instead of the message itself. The control unit 211 may add time information indicating the current time to the generated first behavior information 410 as viewing time information. If the identifier of the image is included in the user input information 400 along with the image, the control unit 211 may add the identifier to the generated first behavior information 410. If time information is included in the user input information 400 along with the image, the control unit 211 may add the time information to the generated first action information 410 as photographing time information. If the position information is included in the user input information 400 along with the image, the control unit 111 may add the position information to the generated first action information 410. The control unit 211 writes the first behavior information 410 into the storage unit 212.

In step S23, the control unit 211 transmits the first behavior information 410 generated in step S22 via the communication unit 213.

Specifically, the control unit 211 reads the first behavior information 410 including a message such as “I saw the image!” from the storage unit 212. The control unit 211 transmits the read first behavior information 410 to the terminal device 101 via the network 104.

The processing in steps S24 and S25 in FIG. 5 is executed by the communication device 110 of the terminal device 101.

In step S24, the control unit 111 receives, via the communication unit 113, the first behavior information 410 transmitted in step S23.

Specifically, the control unit 111 receives first behavior information 410 including a message such as "I saw the image!" from the information output system 102 via the network 104. The control unit 111 writes the received first behavior information 410 into the storage unit 112.

In step S25, the control unit 111 performs control to output the first behavior information 410 received in step S24 to the first user. According to this control, the output device 130 outputs the first behavior information 410 to the first user.

Specifically, the control unit 111 reads a message such as “I saw the image!” included in the first action information 410 from the storage unit 112. The control unit 111 controls displaying the read message on the screen of a display included in the output device 130. According to this control, the display displays a message such as "I saw the image!" on the screen. If a message identifier is included in the first behavior information 410 instead of the message itself, the control unit 111 identifies a message stored in advance in the storage unit 112 that corresponds to the identifier, and stores the identified message. You may also control what is displayed on a display screen. If the viewing time information is included in the first action information 410 together with the message or message identifier, the control unit 111 may control displaying the viewing time information on the screen of the display. If the first action information 410 includes a message or an image identifier together with the message identifier, the control unit 111 may control displaying the image identifier on the display screen. If the first action information 410 includes the photographing time information together with the message or message identifier, the control unit 111 may control displaying the photographing time information on the display screen. If the first action information 410 includes the position information together with the message or message identifier, the control unit 111 may control displaying the position information on the display screen.

The processes from step S31 to step S33 in FIG. 6 are executed by the communication device 210 of the information output system 102.

In step S31, the control unit 211 recognizes an action situation including a second action different from the first action of the second user.

Specifically, the control unit 211 recognizes the behavioral situation from the data obtained by the sensor 240 installed in the second user's residence. For example, the control unit 211 sequentially receives temperature data from the temperature sensor, and recognizes an operation such as turning on the air conditioner by the second user as the second action by analyzing changes in the values of the received data. The control unit 211 sequentially receives humidity data from the humidity sensor and recognizes an operation such as turning on the air conditioner or humidifier by the second user as a second action by analyzing changes in the values of the received data. . The control unit 211 sequentially receives illuminance data from the illuminance sensor and recognizes an operation such as turning on a lighting device by the second user as a second action by analyzing changes in the values of the received data. The control unit 211 sequentially receives data on the opening/closing state of the door from the door sensor, and recognizes the opening/closing of the refrigerator door by the second user as a second action by analyzing changes in the values of the received data. The control unit 211 sequentially receives data on the presence or absence of a person from the human sensor, and recognizes the entry and exit of the second user into and out of the room as a second action by analyzing changes in the values of the received data. As a technique for recognizing human behavior from the data obtained by the sensor 240, a conventional behavior recognition technique may be used, or a unique behavior recognition technique based on machine learning may be used.

The type of data obtained by the sensor 240 and the behavior recognized as the second behavior do not necessarily have to correspond one-to-one, and the control unit 211 selects one behavior from two or more types of data as the second behavior. It can also be recognized as an action. For example, the control unit 211 may recognize an operation such as turning on the air conditioner by the second user as the second action from the temperature data received from the temperature sensor and the humidity data received from the humidity sensor. Further, the control unit 211 may integrate public data such as real-time weather information with the data obtained by the sensor 240, and recognize one action as the second action from the integrated data. For example, the control unit 211 compares the temperature data received from the temperature sensor with the temperature data of the area where the second user's residence is located, and based on the comparison result, controls the second user to perform operations such as turning on the air conditioner. It may be recognized as a second action.

Alternatively, the control unit 211 recognizes the behavioral situation from data obtained from the meter 250 installed in the second user's residence. For example, the control unit 211 sequentially receives power usage data from a power meter and analyzes fluctuations in the values of the received data, thereby controlling the second user's actions such as waking up, taking a bath, or going to bed. recognized as The control unit 211 sequentially receives gas usage data from the gas meter, and recognizes the actions of the second user, such as waking up, taking a bath, or going to bed, as the second actions by analyzing changes in the values of the received data. . The control unit 211 sequentially receives water usage data from the water meter and analyzes changes in the values of the received data, thereby recognizing the second user's actions such as waking up, taking a bath, or going to bed as the second action. do. As a technique for recognizing human behavior from the data obtained by the meter 250, a conventional behavior recognition technique may be used, or a unique behavior recognition technique based on machine learning may be used.

Alternatively, the control unit 211 recognizes the operation status of the electronic device 260 installed in the second user's residence as the behavior status. For example, the control unit 211 recognizes an operation of the air conditioner notified from the air conditioner as the second action. The control unit 211 recognizes the humidifier operation notified from the humidifier as a second action. The control unit 211 recognizes the lighting device operation notified from the lighting device as a second action. The control unit 211 recognizes the refrigerator operation notified from the refrigerator as a second action. The control unit 211 recognizes the operation of the television 203 notified from the television 203 as a second action.

Alternatively, the control unit 211 receives, via the communication unit 213, visit information input by a visitor to the second user's residence, and from the visit information, determines the behavior of the second user in responding to the visitor. Recognize it as an action. The visit information is, for example, transmitted from a terminal device of a courier, mail, nursing care, or school lunch provider, and includes the time at which the provider visited the second user's residence and the fact that the provider responded to the second user at that time. This is information indicating.

In step S32, the control unit 211 generates second behavior information 420 indicating the recognition result of the behavior situation in step S31.

Specifically, if the control unit 211 recognizes the second action in step S31, it generates information indicating that the second user has taken the second action as the second action information 420. For example, if the control unit 211 recognizes the operation of turning on the air conditioner as the second action in step S31, the control unit 211 displays a message that indicates that the second user has performed the operation of turning on the air conditioner. ” and the like is generated. The operations for turning on the humidifier, lighting equipment, and television 203 are similar to the operations for turning on the air conditioner. If the control unit 211 recognizes the opening and closing of the refrigerator door as the second action in step S31, the control unit 211 generates a message such as "You opened and closed the refrigerator door!" indicating that the second user opened and closed the refrigerator door. The second behavior information 420 including the following is generated. If the control unit 211 recognizes entering the room as the second action in step S31, it generates second action information 420 including a message such as "I have entered the room!" indicating that the second user has entered the room. . If the control unit 211 recognizes getting up as the second action in step S31, it generates second action information 420 including a message such as "I woke up" indicating that the second user has woken up. The control unit 211 may change the expression of the message depending on the accuracy of the recognition result, and may generate the second behavior information 420 including a message such as "It looks like you woke up." If the control unit 211 recognizes bathing as the second action in step S31, it generates second action information 420 including a message such as "I took a bath" indicating that the second user took a bath. If the control unit 211 recognizes going to bed as the second action in step S31, it generates second action information 420 including a message such as "I went to bed" indicating that the second user went to bed. The control unit 211 may change the expression of the message depending on the accuracy of the recognition result, and may generate the second behavior information 420 including a message such as "It looks like you fell asleep", for example. If the control unit 211 recognizes responding to the courier service provider as the second action in step S31, the control unit 211 displays a message such as "I have received the courier service!" indicating that the second user has responded to the courier service provider. The second behavior information 420 including the message is generated. The second behavior information 420 may include a message identifier instead of the message itself. The control unit 211 may add time information indicating the current time to the generated second action information 420 as action time information. The control unit 211 writes the second behavior information 420 into the storage unit 212.

Alternatively, the control unit 211 determines whether the situation is such that the second user should take the second action, based on data obtained by the sensor 240 installed in the second user's residence. When the control unit 211 determines that the situation is such that the second user should take the second action, if the second action is not recognized in step S31, the control unit 211 determines that the second user has not taken the second action. The information shown is generated as the second behavior information 420. For example, the control unit 211 detects a sudden change in temperature by sequentially receiving temperature data from a temperature sensor and analyzing changes in the values of the received data. When the control unit 211 detects a sudden change in temperature, the control unit 211 determines that the situation is such that the second user should perform an operation to turn on the air conditioner. When a sudden change in temperature is detected, the control unit 211 receives data on the presence or absence of a person from the human sensor, refers to the received data, and confirms that the second user is in the room. From this, it may be determined that the situation is such that the second user should perform an operation to turn on the power of the air conditioner. When the control unit 211 determines that the situation is such that the second user should perform an operation to turn on the power of the air conditioner, if such an operation is not recognized in step S31, the control unit 211 causes the second user to turn on the power of the air conditioner. Second behavior information 420 is generated that includes a message such as "The air conditioner is not turned on!" indicating that no operation is being performed. The second behavior information 420 may include a message identifier instead of the message itself. The control unit 211 may add time information indicating the current time to the generated second action information 420 as action time information. The control unit 211 writes the second behavior information 420 into the storage unit 212.

Alternatively, the control unit 211 determines whether the situation is such that the second user should take the second action based on the recognition result of the past action situation. When the control unit 211 determines that the situation is such that the second user should take the second action, if the second action is not recognized in step S31, the control unit 211 determines that the second user has not taken the second action. The information shown is generated as the second behavior information 420. For example, the control unit 211 may include the date, day of the week, time, weather, and data obtained by the sensor 240 when the second user's actions such as getting up, taking a bath, or going to bed were recognized as the second actions in step S31 in the past. The tendency of the second behavior is analyzed according to conditions such as the value of . Any method such as machine learning can be used as the analysis method. In light of the analysis results, the control unit 211 causes the second user to take a specific action as a second action under conditions such as the current date, day of the week, time, weather, and the value of data obtained by the sensor 240. If there is a tendency, it is determined that the situation is such that the second user should take that action. When the control unit 211 determines that the situation is such that the second user should get up, if it does not recognize getting up as the second action in step S31, the control unit 211 displays "not awake" indicating that the second user has not woken up. Second behavior information 420 including a message such as "No." is generated. When the control unit 211 determines that the second user should take a bath, if it does not recognize bathing as the second action in step S31, the control unit 211 displays a message "Not taking a bath" indicating that the second user is not taking a bath. Second action information 420 including a message such as "Not included" is generated. If the control unit 211 determines that the second user is in a situation where he should go to bed, and does not recognize going to bed as the second action in step S31, the control unit 211 displays "not sleeping" indicating that the second user is not going to bed. Second behavior information 420 including a message such as "No." is generated. The second behavior information 420 may include a message identifier instead of the message itself. The control unit 211 may add time information indicating the current time to the generated second action information 420 as action time information. The control unit 211 writes the second behavior information 420 into the storage unit 212.

In step S33, the control unit 211 transmits the second behavior information 420 generated in step S32 via the communication unit 213.

Specifically, the control unit 211 reads the second behavior information 420 including a message such as “I turned on the air conditioner!” or “I did not turn on the air conditioner!” from the storage unit 212. The control unit 211 transmits the read second behavior information 420 to the terminal device 101 via the network 104.

The processing in steps S34 and S35 in FIG. 6 is executed by the communication device 110 of the terminal device 101.

In step S34, the control unit 111 receives, via the communication unit 113, the second behavior information 420 transmitted in step S33.

Specifically, the control unit 111 receives the second action information 420 including a message such as “I turned on the air conditioner!” or “I did not turn on the air conditioner!” from the information output system 102 via the network 104. do. The control unit 111 writes the received second behavior information 420 into the storage unit 112.

In step S35, the control unit 111 performs control to output the second behavior information 420 received in step S34 to the first user. According to this control, the output device 130 outputs the second behavior information 420 to the first user.

Specifically, the control unit 111 reads a message such as “I turned on the air conditioner!” or “I did not turn on the air conditioner!” included in the second action information 420 from the storage unit 112. The control unit 111 controls displaying the read message on the screen of a display included in the output device 130. According to this control, the display displays a message such as "The air conditioner is turned on!" or "The air conditioner is not turned on!" on the screen. If a message identifier is included in the second behavior information 420 instead of the message itself, the control unit 111 identifies a message stored in advance in the storage unit 112 that corresponds to the identifier, and stores the identified message. You may also control what is displayed on a display screen. If the action time information is included in the second action information 420 together with the message or message identifier, the control unit 111 may control displaying the action time information on the screen of the display.

FIG. 7 shows an example of the screen of the terminal device 101 on which the first behavior information 410 and the second behavior information 420 are displayed.

The processes in step S25 and step S35 may be executed separately by the communication device 110 of the terminal device 101, but in this embodiment, they are executed together.

In step S25 and step S35, the first behavior information 410 and the second behavior information 420 may be output separately, but in this embodiment, they are output together.

The control unit 111 performs control to output the first behavior information 410 and the second behavior information 420 in chronological order when performing control to output the first behavior information 410 and the second behavior information 420 at once.

Specifically, the control unit 111 displays the combination of the message and viewing time information included in the first behavior information 410 and the message and behavior time information included in the second behavior information 420 in chronological order. Control the display. That is, the control unit 111 performs control to display the first behavior information 410 and the second behavior information 420 on a timeline. In the example of FIG. 7, the combination of the message "I saw the image!" included in the first behavior information 410 and the viewing time information indicating the time "11:52" and the " A combination of the message "I've turned on the air conditioner!" and action time information indicating the time "13:24" are displayed on the screen in descending order of time. In the example of FIG. 7, an icon corresponding to each message is displayed next to each combination.

The control unit 111 changes output settings for the first behavior information 410 and the second behavior information 420 when performing control to output the first behavior information 410 and the second behavior information 420 together.

Specifically, the control unit 111 displays the combination of the message and the viewing time information included in the first behavior information 410 on the screen in a format that stands out more than the combination of the message and the behavior time information included in the second behavior information 420. control. That is, the control unit 111 performs control to highlight the first behavior information 410. In the example of FIG. 7, the combination of the message "I saw the image!" included in the first action information 410 and the viewing time information indicating the time "11:52" is better than the combination of other messages and time information. It is also displayed on the screen with a frame so that it stands out. The combination of the message and viewing time information included in the first behavior information 410 may be displayed on the screen in a large font size, bold, or in a different color.

As described above, in this embodiment, the first behavior information 410 transmitted in relation to the user input information 400 such as an image input by the first user is the second behavior transmitted regardless of the user input information 400. It is displayed more prominently than the information 420. Therefore, according to the present embodiment, it is possible to improve the first user's motivation to input user input information 400 such as a new image.

The control unit 111 performs control to output the second behavior information 420 in response to the first user's operation. When the second behavior information 420 includes information indicating the recognition results of a plurality of behaviors, the control unit 111 recognizes the behaviors that are within the most recent limit number or within the limit period at the time when the first user's operation was performed. Performs control to output information indicating results.

Specifically, when the first user operates the terminal device 101 to start an application program included in the first communication program, or when the first user further operates the terminal device 101 to start the second communication program, When a mode for displaying a message included in the behavior information 420 is selected, control is performed to display the message on the screen. That is, the control unit 111 performs control to pull-display the second behavior information 420. Then, the control unit 111 adjusts the amount of messages displayed on the screen within a limited range. In the example of FIG. 7, each message is not pushed and displayed on the screen only when there is an explicit request from the first user. Messages from the previous day are omitted, and only messages from the current day are displayed.

As described above, in this embodiment, the amount of information that is output is limited. Therefore, according to the present embodiment, the opportunity for the first user to sense the presence of the second user does not increase excessively, and the first user is less likely to feel bothered.

As a modification of this embodiment, messages may be push-displayed depending on the importance of the messages.

As a variation of this embodiment, not only messages of the current day but also messages of the previous day, messages of several days up to the previous day, or messages of several days in the past that have common attributes such as the current day and day of the week are displayed on the screen. Good too. Messages on days other than the current day may be displayed in a less conspicuous manner, such as by making the message lighter than messages on the current day. Messages on days other than the current day may be displayed in a way that makes them easier to compare, such as aligning them with the messages on that day according to the corresponding time.

According to this modification, when the second user's behavior differs from the usual behavior pattern, the first user can easily notice this. As a result, child households living apart from the parent household are motivated to visit the parent household or contact the parent household through communication means such as telephone or email, and the relationship between the child household and the parent household is increased. Communication is encouraged.

As a modified example of the present embodiment, when the second behavior information 420 includes information indicating recognition results of a plurality of behaviors, the control unit 111 recognizes each behavior based on the frequency with which each behavior is recognized. You may change the output settings for information indicating the results.

Specifically, when a plurality of messages are included in the second behavior information 420, the control unit 111 may omit displaying a message indicating a behavior that is recognized more frequently than a standard, such as once a day. good. That is, the control unit 111 performs control to thin out and display the second behavior information 420. In the example of FIG. 7, since the second user is supposed to wake up and go to bed every day, the display of the message "I woke up" and the message "I went to bed" included in the second behavior information 420 may be omitted.

In this modification, the amount of information output is reduced. Therefore, according to this modification, the opportunity for the first user to sense the presence of the second user does not increase excessively, and the first user is less likely to feel bothered.

As a modified example of the present embodiment, the control unit 111 controls the screen displaying the first behavior information 410 and the second behavior information 420 based on the value of data obtained by the sensor 240 installed in the second user's residence. You may change the display settings.

Specifically, the control unit 111 controls the background color of the screen displaying the first behavior information 410 and the second behavior information 420, and the background color of the screen displaying the first behavior information 410 and the second behavior information 420, and The color of the icon imitating the house of the second user or the window of the house, or the expression or gesture of the avatar imitating the second user displayed on the screen may be changed. For example, if the temperature indicated by the data obtained by the temperature sensor is high, the background color of the screen may be set to a warm color, or the color of the house or window icon on the screen may be set to a warm color. Alternatively, the facial expressions or gestures of the avatar on the screen may be set to convey a sense of warmth. If the temperature indicated by the data obtained by the temperature sensor is low, the background color of the screen may be set to a cool color, or the color of the house or window icon on the screen may be set to a cool color. Alternatively, the facial expressions or gestures of the avatar on the screen may be set to indicate that the avatar is feeling cold.

Alternatively, the control unit 111 controls the background color of the screen displaying the first behavior information 410 and the second behavior information 420, or the second user's house displayed on the screen, based on the value of the data obtained by the illuminance sensor. The color of the imitation icon or the icon imitating a house window may be changed. For example, if the illuminance indicated by the data obtained by the illuminance sensor is high, the background color of the screen may be set to a bright color, or the color of the house or window icon on the screen may be set to a bright color. may be done. If the illuminance indicated by the data obtained by the illuminance sensor is low, the background color of the screen may be set to a dark color, or the color of the house or window icon on the screen may be set to a dark color. .

Alternatively, the control unit 111 controls the display of an icon or avatar imitating the second user displayed on the screen displaying the first behavior information 410 and the second behavior information 420, depending on the value of the data obtained by the human sensor. You may change the presence or absence. For example, the icon or avatar of the second user may be displayed on the screen only when the presence/absence status of a person indicated by the data obtained by the human sensor is "present."

In this modification, the state of the second user's house is visually expressed. Therefore, according to this modification, the first user can experience or feel as if he or she lives next to the second user.

As described above, the control unit 111 of the communication device 110 included in the terminal device 101 transmits the user input information 400 input by the first user via the communication unit 113. When the user input information 400 is output to the second user in response to the second user's operation, the control unit 111 causes the second user to perform a first action in which the second user interacts with the user input information 400 via the communication unit 113. The first action information 410 indicating that the action was taken is received. The control unit 111 performs control to output the first behavior information 410 to the first user. Further, the control unit 111 receives, via the communication unit 113, second behavior information 420 indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user. The control unit 111 performs control to output the second behavior information 420 to the first user.

On the other hand, the control unit 211 of the communication device 210 included in the information output system 102 receives user input information 400 input by the first user via the communication unit 213. The control unit 211 performs control to output the user input information 400 to the second user in response to the second user's operation. When the control unit 211 performs the control, the control unit 211 transmits, via the communication unit 213, first action information 410 indicating that the second user has taken a first action that touches the user input information 400. Further, the control unit 211 recognizes an action situation including a second action different from the first action of the second user. The control unit 211 transmits, via the communication unit 213, second behavior information 420 indicating the recognition result of the behavior situation.

In this embodiment, when the user input information 400 input by the first user is output to the second user, the first user is notified that the second user has touched the user input information 400. In addition, when the behavioral situation including other behaviors of the second user is recognized, the first user is notified of the recognition result. Therefore, the first user can feel the presence of the second user every time the first user receives a notification. Even if the second user comes into contact with the user input information 400 less frequently, the first user is notified of the recognition result of the second user's behavior status, so the frequency of notification can be increased. That is, according to this embodiment, it is possible to increase the chances that at least one of the two parties can feel the presence of the other.

According to this embodiment, a first user such as a child who lives apart from a second user such as a parent has more opportunities to feel the presence of the second user, and a first user who lives next to the second user has more opportunities to feel the presence of the second user. You will be able to get that feeling.

In this embodiment, the information output system 102 not only allows the first user to view the data of the sensor 240 at a distance instead of allowing the first user to view the data of the sensor 240 as it is from a distance; The first user is notified of the second user's behavior estimated from the data or the fact that the second user is not taking any action. Therefore, it is easy for the first user to understand whether the second user is active and what kind of activity the second user is doing.

For parents who are unable or unable to use mobile devices such as smartphones or tablets, the frequency and amount of communication decreases as distance increases. This is the issue with intergenerational communication. In order to deal with this, in this embodiment, the television 203 is used to display images, and even if the frequency of image usage is low, environmental information or the fact that images are not viewed or transmitted itself can be used as information for communication. I'm using it as a trigger. The environmental information is, for example, second behavior information, but may also be data from the sensor 240. That is, the information output system 102 may allow the first user to directly view the data from the sensor 240 from a distance.

As a modified example of this embodiment, a push notification prompting the child household to send content or communicate depending on the parent household's situation, such as whether the parent household is not viewing images or the environmental information is different from normal. The following functions may be added to the information output system 102. Alternatively, a function to notify information to people other than the child household, such as nursing care providers or local governments, depending on the situation of the parent household, such as whether the parent household is not viewing the image or the environmental information is different from normal. may be added to the information output system 102. Alternatively, if the child household views or confirms the status of the parent household less frequently, the information output system 102 may be provided with a push notification function that prompts the child household to send content or communicate.

As a modified example of the present embodiment, the information output system 102 not only outputs the user input information 400 to the second user in response to the second user's operation, but also outputs the user input information 400 to the second user by a certain deadline T1. Even if the user input information 400 is not output to the second user in response to the operation, the first behavior information 410 may be transmitted. However, if the user input information 400 is output to the second user in response to the second user's operation, and if the user input information 400 is output to the second user in response to the second user's operation, and The content of the first behavior information 410 differs depending on the case where the first behavior information 410 is not output.

In other words, the first action information 410 indicates that if the user input information 400 is output toward the second user in response to the second user's operation, the second user took the first action to interact with the user input information 400. This is information indicating that. On the other hand, the first behavior information 410 indicates that the second user has not taken the first behavior if the user input information 400 is not output to the second user in response to the second user's operation by the deadline T1. This is information indicating that. In this modified example, even though the user input information 400 including an image is received by the information output system 102, the image is not displayed to the second user in response to the second user's operation by the deadline T1. If not, the first action information 410 includes a message such as "Not looking at the image!" indicating that the second user has not taken the action of looking at the image as a first action. Even though the user input information 400 including voice is received by the information output system 102, if the voice is not output to the second user in response to the second user's operation by the deadline T1, the The first action information 410 may include a message such as "I am not listening to the audio!" indicating that the second user has not taken the action of listening to the audio as a first action. Even though the user input information 400 including a text message is received by the information output system 102, if the text message is not displayed to the second user in response to the second user's operation by the deadline T1. , the first behavior information 410 may include a message such as "I have not read the message!" indicating that the second user has not taken the action of reading a text message as a first action.

In this modification, even though the first user has posted an image, if a certain period of time passes without the image being viewed by the second user, the first user is notified that the image has not been viewed. Such notifications are likely to be an opportunity for child households living apart from the parent household to visit the parent household or contact the parent household by communication means such as telephone or email, and the child household and the parent household may This will lead to the promotion of communication between

This modification will be further explained as a first modification.

With reference to FIG. 8, the operation of the communication system 100 according to the first modification will be described.

The processes from step S41 to step S43 in FIG. 8 are executed by the communication device 210 of the information output system 102.

In step S41, the control unit 211 determines whether the deadline T1 has arrived. When the control unit 211 determines that the deadline T1 has arrived, it performs the process of step S42.

The control unit 211 may set the deadline T1 based on the time when the user input information 400 is received in step S16, or the time when the first behavior information 410 is transmitted in step S23 and the second behavior in step S33. It may be set based on the later of the time when the information 420 was transmitted, or it may be set based on another time. The control unit 211 may set the deadline T1 based only on the time when the first behavior information 410 is transmitted in step S23, or only on the basis of the time when the second behavior information 420 is transmitted in step S33. You may.

Specifically, the control unit 211 starts counting time when receiving the user input information 400 in step S16. If the control unit 211 performs control to output the user input information 400 to the second user in response to the second user's operation in step S21 while counting the time, the control unit 211 stops and resets the time count. . The control unit 211 determines that the deadline T1 has arrived when the counted time exceeds a certain period, such as one day.

Alternatively, when the control unit 211 transmits the first action information 410 in step S23 and is not in the process of counting time, it starts counting the time. If the control unit 211 is counting time when transmitting the first action information 410 in step S23, it temporarily stops and resets the time count, and then starts counting the time. When the control unit 211 transmits the second action information 420 in step S33, if it is not in the process of counting time, it starts counting the time. If the control unit 211 is in the process of counting time when transmitting the second action information 420 in step S33, it temporarily stops and resets the time count, and then starts counting the time. If the control unit 211 performs control to output the user input information 400 to the second user in response to the second user's operation in step S21 while counting the time, the control unit 211 stops and resets the time count. . The control unit 211 determines that the deadline T1 has arrived when the counted time exceeds a certain period, such as one day.

In step S42, the control unit 211 does not perform control to output the user input information 400 to the second user in response to the second user's operation, even though the deadline T1 has arrived in step S41. Information indicating that the second user has not taken the first action related to the user input information 400 is generated as the first action information 410.

Specifically, the control unit 211 displays the message "Not looking at the image!" indicating that the second user has not taken the action of looking at the image of the second user's grandchild included in the user input information 400 as a first action. The first action information 410 including a message such as "" is generated. The first behavior information 410 may include a message identifier instead of the message itself. The control unit 211 may add time information indicating the current time to the generated first behavior information 410 as viewing time information. If the identifier of the image is included in the user input information 400 along with the image, the control unit 211 may add the identifier to the generated first behavior information 410. If time information is included in the user input information 400 along with the image, the control unit 211 may add the time information to the generated first action information 410 as photographing time information. If the position information is included in the user input information 400 along with the image, the control unit 111 may add the position information to the generated first action information 410. The control unit 211 writes the first behavior information 410 into the storage unit 212.

In step S43, the control unit 211 transmits the first behavior information 410 generated in step S42 via the communication unit 213.

Specifically, the control unit 211 reads the first behavior information 410 including a message such as “You are not looking at the image!” from the storage unit 212. The control unit 211 transmits the read first behavior information 410 to the terminal device 101 via the network 104.

The processing in steps S44 and S45 in FIG. 8 is executed by the communication device 110 of the terminal device 101.

In step S44, the control unit 111 receives, via the communication unit 113, the first behavior information 410 transmitted in step S43.

Specifically, the control unit 111 receives, from the information output system 102, first behavior information 410 including a message such as "You are not looking at the image!" via the network 104. The control unit 111 writes the received first behavior information 410 into the storage unit 112.

In step S45, the control unit 111 performs control to output the first behavior information 410 received in step S44 to the first user. According to this control, the output device 130 outputs the first behavior information 410 to the first user.

Specifically, the control unit 111 reads a message such as “You are not looking at the image!” included in the first behavior information 410 from the storage unit 112. The control unit 111 controls displaying the read message on the screen of a display included in the output device 130. According to this control, the display displays a message on the screen such as "You are not viewing the image!". If a message identifier is included in the first behavior information 410 instead of the message itself, the control unit 111 identifies a message stored in advance in the storage unit 112 that corresponds to the identifier, and stores the identified message. You may also control what is displayed on a display screen. If the viewing time information is included in the first action information 410 together with the message or message identifier, the control unit 111 may control displaying the viewing time information on the screen of the display. If the first action information 410 includes a message or an image identifier together with the message identifier, the control unit 111 may control displaying the image identifier on the display screen. If the first action information 410 includes the photographing time information together with the message or message identifier, the control unit 111 may control displaying the photographing time information on the display screen. If the first action information 410 includes the position information together with the message or message identifier, the control unit 111 may control displaying the position information on the display screen.

As a modified example of the present embodiment, the information output system 102 performs a second action when the user input information 400 is not output to the second user in response to the second user's operation by a certain deadline T2. Information 420 may also be transmitted. That is, the frequency of notification of the recognition result of the second user's behavioral situation may be adjusted depending on the frequency with which the second user comes into contact with the user input information 400.

In this modified example, even though the first user posted an image, a certain period of time passes without the image being viewed by the second user, and a specific action of the second user is recognized at that time or immediately after that. If so, the first user is notified that the action has been taken. Therefore, even child households who live apart from their parents can feel the presence of their parents every time they receive a notification. Even if the frequency of images being viewed is low, there will be notifications for actions other than viewing images, so the frequency of notifications will not decrease and child households will not feel the presence of the parent household on a daily basis. can. In addition, in this modified example, even though the first user posted an image, a certain period of time passes without the image being viewed by the second user, and the second user takes a specific action at or immediately after that. If the action is not recognized in the situation where it should be taken, the first user is notified that the action is not taken in the situation where it should be taken. Such notifications are likely to be an opportunity for child households living apart from the parent household to visit the parent household or contact the parent household by communication means such as telephone or email, and the child household and the parent household may This will lead to the promotion of communication between

This modification will be further explained as a second modification.

With reference to FIG. 9, the operation of the communication system 100 according to the second modification will be described. This operation replaces the operation shown in FIG.

The processes from step S51 to step S56 in FIG. 9 are executed by the communication device 210 of the information output system 102.

The process in step S51 is the same as the process in step S31 in FIG. 6, so the explanation will be omitted.

In step S52, the control unit 211 determines whether the deadline T2 has arrived. When the control unit 211 determines that the deadline T2 has arrived, it performs the process of step S53.

The detailed processing of step S52 is the same as the detailed processing of step S41 in the first modification, except that the deadline T1 is changed to the deadline T2, so the explanation will be omitted. The process by which the control unit 211 sets the deadline T2 is also the same as the process by which the control unit 211 sets the deadline T1 in the first modification, and therefore the description thereof will be omitted.

The process in step S52 may be performed before the process in step S51. That is, the control unit 211 does not need to perform the process of step S51 until it determines that the deadline T2 has arrived.

The processing in step S53 and step S54 is the same as the processing in step S32 and step S53 in FIG. 6, respectively, so the description thereof will be omitted.

The processing in steps S55 and S56 in FIG. 9 is executed by the communication device 110 of the terminal device 101.

The processing in step S55 and step S56 is the same as the processing in step S34 and step S55 in FIG. 6, respectively, so the description thereof will be omitted.

The invention is not limited to the embodiments described above. For example, a plurality of blocks shown in the block diagram may be integrated, or one block may be divided. Instead of performing the steps in the flowchart in chronological order as described, they may be performed in parallel or in a different order depending on the processing power of the device performing each step or as needed. Other changes can be made without departing from the spirit of the invention.

100 Communication System 101 Terminal Equipment 102 Information Output System 103 Server 104 Network 110 Communication Device 111 Control Unit 112 Storage Unit 113 Communication Unit 114 Positioning Unit 120 Input Device 130 Output Device 201 STB
202 Remote controller 203 Television 204 Indicator 205 Port 206 Port 207 Port 210 Communication device 211 Control section 212 Storage section 213 Communication section 214 I/O section 220 Input device 230 Output device 240 Sensor 250 Meter 260 Electronic device 301 Control section 302 Storage section 303 Communication department 400 User input information 410 First behavior information 420 Second behavior information

Claims (26)

A communications department that sends and receives information;
Receive user input information input by a first user via the communication unit, perform control to output the user input information to the second user in response to an operation by a second user, and perform the control. a control unit configured to transmit, as first action information, information indicating that the second user has taken a first action in contact with the user input information via the communication unit, when the second user a control unit that recognizes a behavior situation including a second behavior different from the first behavior of the user and transmits second behavior information indicating a recognition result of the behavior situation via the communication unit ;
If the control unit does not perform control to output the user input information to the second user in response to an operation by the second user by a certain deadline, the control unit may output the user input information to the second user via the communication unit. A communication device that transmits information indicating that a second user has not taken the first action . A communications department that sends and receives information;
Receive user input information input by a first user via the communication unit, perform control to output the user input information to the second user in response to an operation by a second user, and perform the control. a control unit configured to transmit, as first action information, information indicating that the second user has taken a first action in contact with the user input information via the communication unit, when the second user a control unit that recognizes a behavior situation including a second behavior different from the first behavior of the user and transmits second behavior information indicating a recognition result of the behavior situation via the communication unit ;
If the control unit does not perform control to output the user input information to the second user in response to an operation by the second user by a certain deadline, the control unit may output the user input information to the second user via the communication unit. 2. A communication device that transmits behavior information . The communication device according to claim 1 or 2 , wherein the control unit sets the deadline based on a time point when the user input information is received. 3. The control unit sets the deadline based on the later of the time when the first behavior information is transmitted and the time when the second behavior information is transmitted . Communication device. The communication device according to any one of claims 1 to 4 , wherein the control unit recognizes the behavioral situation from data obtained by a sensor installed in the second user's residence. The communication device according to any one of claims 1 to 5 , wherein the control unit recognizes the behavioral situation from data obtained from a meter installed in the second user's residence. The communication device according to any one of claims 1 to 6 , wherein the control unit recognizes, as the behavior status, an operation status of an electronic device installed in the second user's residence. The control unit receives, via the communication unit, visit information input by a visitor to the second user's residence, and determines the behavior of the second user in responding to the visitor based on the visit information. The communication device according to any one of claims 1 to 4 , which is recognized as the second action. The control unit determines whether the situation is such that the second user should take the second action, and when it is determined that the situation is such that the second user should take the second action, the control unit The communication according to any one of claims 1 to 4 , wherein if the action is not recognized, information indicating that the second user has not taken the second action is transmitted as the second action information. Device. The communication system according to claim 9 , wherein the control unit determines whether the situation is such that the second user should take the second action, based on data obtained by a sensor installed in the second user's residence. Device. The communication device according to claim 9 , wherein the control unit determines whether the situation is such that the second user should take the second action based on a recognition result of the past action situation. A communications department that sends and receives information;
Receive user input information input by a first user via the communication unit, perform control to output the user input information to the second user in response to an operation by a second user, and perform the control. a control unit configured to transmit, as first action information, information indicating that the second user has taken a first action in contact with the user input information via the communication unit, when the second user a control unit that recognizes a behavior situation including a second behavior different from the first behavior of the user and transmits second behavior information indicating a recognition result of the behavior situation via the communication unit ;
The control unit receives, via the communication unit, visit information input by a visitor to the second user's residence, and determines the behavior of the second user in responding to the visitor based on the visit information. A communication device that recognizes the second action . A communication device according to any one of claims 1 to 12,
An information output system comprising: an output device that outputs the user input information to the second user according to control by the communication device. A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information toward the first user ,
The control unit is a communication device that performs control to output the first behavior information and the second behavior information in chronological order when controlling to output the first behavior information and the second behavior information collectively. A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information toward the first user ,
The control unit is a communication device that changes output settings between the first behavior information and the second behavior information when performing control to output the first behavior information and the second behavior information together. A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information toward the first user ,
When the second behavior information includes information indicating recognition results of a plurality of behaviors, the control unit changes the output setting of the information indicating the recognition results of each behavior depending on the frequency with which each behavior is recognized. communication equipment. A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information toward the first user ,
The control unit performs control to output the second behavior information in response to an operation of the first user, and when the second behavior information includes information indicating recognition results of a plurality of behaviors, the control unit outputs the second behavior information according to the operation of the first user. A communication device that performs control to output information indicating the recognition result of actions that are within the most recent limit number or within the limit period at the time when the user's operation was performed. A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information toward the first user ,
The control unit performs control to display the first behavior information and the second behavior information on the screen as a control to output the first behavior information and the second behavior information, and is installed in the second user's residence. The background color of the screen, the color of an icon simulating the second user's house displayed on the screen, or the color of the icon simulating the window of the house displayed on the screen are changed depending on the value of data obtained by the sensor. communication equipment. A communications department that sends and receives information;
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. The control unit receives, via the communication unit, second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user, and and a control unit that performs control to output the second behavior information toward the first user ,
The control unit performs control to display the first behavior information and the second behavior information on the screen as a control to output the first behavior information and the second behavior information, and is installed in the second user's residence. A communication device that changes the expression or gesture of an avatar imitating the second user displayed on the screen according to the value of data obtained by a sensor. The communication device according to any one of claims 14 to 19 ,
A terminal device comprising: an output device that outputs the first behavior information and the second behavior information to the first user according to control by the communication device. A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. and the steps to
Receive second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user via the communication unit, and transmit the second behavior information to the first user. a step of controlling output toward the
When performing control to output the first behavior information and the second behavior information collectively, performing control to output the first behavior information and the second behavior information in chronological order;
A communications program that runs A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. and the steps to
Receive second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user via the communication unit, and transmit the second behavior information to the first user. a step of controlling output toward the
When performing control to output the first behavior information and the second behavior information together, changing output settings for the first behavior information and the second behavior information;
A communications program that runs A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. and the steps to
Receive second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user via the communication unit, and transmit the second behavior information to the first user. a step of controlling output toward the
when the second behavior information includes information indicating the recognition results of a plurality of behaviors, changing the output setting of the information indicating the recognition results of each behavior depending on the frequency with which each behavior is recognized;
A communications program that runs A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive, as first action information, information indicating that the second user has taken a first action in contact with the user input information, and output the first action information to the first user. and the steps to
Second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user is received via the communication unit, and the second behavior information is received in response to the operation of the first user . Control is performed to output second behavior information toward the first user, and when the second behavior information includes information indicating recognition results of a plurality of behaviors, the first user's operation is performed. A communication program that executes a step of controlling to output information indicating a recognition result of actions that are within the most recent limit number or within a limit period at a time . A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive information indicating that the second user has taken a first action related to the user input information as first action information, and to display the first action information on the screen. A step of controlling to output behavioral information toward the first user;
Receiving second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user via the communication unit, and displaying the second behavior information on the screen. performing a control to output the second behavior information toward the first user ;
The background color of the screen, the color of an icon imitating the second user's house displayed on the screen, or the window of the house, depending on the value of data obtained by a sensor installed in the second user's house. The step of changing the color of the icon that imitates
A communications program that runs A computer equipped with a communication unit that sends and receives information,
When transmitting user input information input by a first user via the communication unit and outputting the user input information to the second user in response to an operation by a second user, the communication unit control to receive information indicating that the second user has taken a first action related to the user input information as first action information, and to display the first action information on the screen. A step of controlling to output behavioral information toward the first user;
Receiving second behavior information indicating a recognition result of a behavior situation including a second behavior different from the first behavior of the second user via the communication unit, and displaying the second behavior information on the screen. performing a control to output the second behavior information toward the first user ;
changing the facial expressions or gestures of an avatar imitating the second user displayed on the screen according to the value of data obtained by a sensor installed in the second user's residence;
A communications program that runs

Images