CN110268212B

CN110268212B - Network system, server, information processing method and refrigerator

Info

Publication number: CN110268212B
Application number: CN201780085366.5A
Authority: CN
Inventors: 岩田圭
Original assignee: Sharp Corp
Current assignee: Sharp Corp
Priority date: 2017-02-17
Filing date: 2017-08-31
Publication date: 2021-06-22
Anticipated expiration: 2037-08-31
Also published as: JP2018132285A; CN110268212A; JP6854146B2; WO2018150612A1

Abstract

A network system (1) is provided. A network system (1) is provided with a plurality of refrigerators (300) and a server (100A), wherein the server (100A) determines the timing for enabling each refrigerator in the plurality of refrigerators (300) to output each message of a plurality of types by voice based on the previous opening and closing action of the door of each refrigerator in the plurality of refrigerators (300).

Description

Network system, server, information processing method and refrigerator

Technical Field

An aspect of the present invention relates to a technology of a network system, a server, an information processing method, and a refrigerator that output a voice message.

Background

Conventionally, a technique for outputting a message suitable for a situation is known. For example, japanese patent laying-open No. 2016-. According to patent document 1, the control device includes: a retrieval section that retrieves advertisement information, which is information on an advertisement to be output as a response message, based on voice characteristics in a conversation between a conversation agent system and a user; and an output control unit that controls to output the advertisement related to the advertisement information retrieved by the retrieval unit as a response message. Therefore, the control device can output a message suitable for the user.

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2016-24636

Disclosure of Invention

(problems to be solved by the invention)

An object of an aspect of the present invention is to provide a network system, a server, an information processing method, and a refrigerator capable of outputting a message at a timing suitable for a situation.

Means for solving the problems

According to an aspect of the present invention, a network system is provided. The network system includes a plurality of refrigerators and a server that determines timings for causing each of the plurality of refrigerators to output a plurality of types of messages by voice based on past door opening and closing operations of each of the plurality of refrigerators.

Effects of the invention

As described above, according to an aspect of the present invention, there are provided a network system, a server, an information processing method, and a refrigerator capable of outputting a message at a timing suitable for a situation.

Drawings

Fig. 1 is a schematic diagram showing an overall configuration and an outline of an operation of a network system 1 according to a first embodiment.

Fig. 2 is a block diagram showing the configuration of a server 100A constituting the network system 1 according to the first embodiment.

Fig. 3 is a block diagram showing the configuration of a refrigerator 300 according to the first embodiment.

Fig. 4 is a block diagram showing a functional configuration of the network system 1 according to the first embodiment.

Fig. 5 is a diagram showing trigger condition data 121 according to the first embodiment.

Fig. 6 is a diagram showing the speech data 122 according to the first embodiment.

Fig. 7 is a schematic diagram showing the operation/dialogue log database 123 according to the first embodiment.

Fig. 8 is a schematic diagram of a graph 123B showing changes in the number of times of operation/session recording according to the first embodiment.

Fig. 9 is a schematic diagram showing a graph 124B for timing prediction according to the first embodiment.

Fig. 10 is a schematic diagram showing the timing prediction data 124 according to the first embodiment.

Fig. 11 is a diagram showing user information data 125 according to the first embodiment.

Fig. 12 is a flowchart showing record accumulation processing in the server 100A according to the first embodiment.

Fig. 13 is a flowchart showing a message providing process in the server 100A according to the first embodiment.

Fig. 14 is a flowchart showing a message output process in the refrigerator 300 according to the first embodiment.

Fig. 15 is a diagram showing event management data 126 according to the second embodiment.

Fig. 16 is a schematic diagram showing timing prediction data 124C according to the second embodiment.

Fig. 17 is a flowchart showing a message providing process in the server 100A according to the second embodiment.

Fig. 18 is a diagram showing response data 127 according to the third embodiment.

Fig. 19 is a diagram showing summarized data 128 according to the third embodiment.

Fig. 20 is a flowchart showing a message providing process in the server 100A according to the third embodiment.

Fig. 21 is a flowchart showing a message output process of the refrigerator 300 according to the third embodiment.

Fig. 22 is a diagram showing speech data 122D according to the fourth embodiment.

Fig. 23 is a schematic diagram showing timing prediction data 124D according to the fourth embodiment.

Fig. 24 is a diagram showing speech data 122E according to the fourth embodiment.

Fig. 25 is a schematic diagram showing timing prediction data 124E according to the fifth embodiment.

Fig. 26 is a flowchart showing a message providing process in the server 100A according to the fifth embodiment.

Fig. 27 is a diagram showing first call data 122F according to the sixth embodiment.

Fig. 28 is a diagram showing second speech data 122G according to the sixth embodiment.

Fig. 29 is a schematic diagram showing timing prediction data 124H according to the seventh embodiment.

Fig. 30 is a diagram showing trigger condition data 121H according to the seventh embodiment.

Fig. 31 is a schematic diagram showing the overall configuration and operational overview of the network system 1 according to the eighth embodiment.

Fig. 32 is a flowchart showing a record accumulation process in the refrigerator 300 according to the eighth embodiment.

Fig. 33 is a flowchart showing a message output process of the refrigerator 300 according to the eighth embodiment.

Detailed Description

Embodiments of the present invention will be described below with reference to the drawings. In the following description, the same members are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

< first embodiment >

< overall configuration of network System 1 >

First, the overall configuration of the network system 1 according to the present embodiment will be described with reference to fig. 1. The network system 1 mainly includes: a server 100A for implementing a message providing service;

other servers

100B and 100C that provide weather forecast, news, recommended food, calendar information of the user, and the like to the server 100A; a refrigerator 300 that can communicate with the server 100A via the router 200, the internet, and the like.

< summary of operation of network System 1 >

The following describes an outline of the operation of the network system 1 according to the present embodiment. Referring to fig. 1(a), when a user opens a door of the refrigerator 300 immediately after getting up, the refrigerator 300 outputs a message suitable for getting up. Referring to fig. 1(b), when the user opens the refrigerator 300 while preparing breakfast, the refrigerator 300 outputs a message suitable for preparing breakfast. Referring to fig. 1(c), when a user opens a door of the refrigerator 300 before work, the refrigerator 300 outputs a message suitable for work.

As described above, in the network system 1 according to the present embodiment, a message suitable for the situation where the user using the refrigerator is located is output. Hereinafter, a specific configuration of the network system 1 for realizing such a function will be described in detail.

< construction of Server 100A >

With reference to fig. 2, the configuration of the server 100A constituting the network system 1 will be described. First, the configuration of the server 100A will be described. The server 100A includes, as main components: a CPU (Central Processing Unit) 110, a memory 120, an operation section 140, and a communication interface 160.

The CPU110 controls various parts of the refrigerator 300 by executing a program stored in the memory 120 or an external storage medium.

The Memory 120 is implemented by various RAMs (Random Access memories), various ROMs (Read-Only memories), and the like. The memory 120 stores programs executed by the CPU110, data generated by the CPU110 executing the programs, data input via the operation unit 130, and the like.

The operation unit 130 receives a command from a service manager or the like and inputs the command to the CPU 110.

The communication interface 160 is implemented by a communication module such as a wireless LAN or a wired LAN. The communication interface 160 exchanges data with the refrigerator 300 or other servers through wired or wireless communication.

< construction of Server 300 >

The configuration of the refrigerator 300 is explained with reference to fig. 3. As main constituent elements, the refrigerator 300 includes: CPU310, storage 320, display 330, operation section 340, communication interface 360, speaker 370, microphone 380, device driver 390, and door sensor 391.

The CPU310 controls various parts of the refrigerator 300 by executing a program stored in the memory 320 or an external storage medium.

The memory 320 is implemented by various RAMs or various ROMs and the like. The memory 320 stores a program executed by the CPU310, data generated by the CPU310 executing the program, and the like.

The display 330 outputs text, images, and the like based on a signal from the CPU 310. The display 330 may also be a mere light. The operation unit 340 receives various user operations and inputs an operation command to the CPU 310. The display 330 and the operation unit 340 may constitute a touch panel.

The communication interface 360 is implemented by a communication module such as a wireless LAN or a wired LAN. The communication interface 360 exchanges data with other devices such as a server or the like through wired communication or wireless communication.

Speaker 370 outputs voice based on signals from CPU 310. Microphone 380 accepts speech and passes the speech signal to CPU 310.

The device driving unit 390 controls each unit (motor, heater, etc.) of the electric device based on a signal from the CPU 110.

The door sensor 391 detects the opening and closing operation of the door of the refrigerating compartment or the freezing compartment, and inputs the detection result to the CPU 310.

< functional configuration of network System 1 >

Next, the functional configurations of the server 100A and the refrigerator 300 constituting the network system 1 will be described with reference to fig. 4. The server 100A includes a recording storage unit 111, a learning unit 112, and a speech control unit 113. The recording storage unit 111, the learning unit 112, and the speech control unit 113 are realized by the CPU110 executing programs of the memory 120. The memory 120 of the server 100A stores event trigger condition data 121, speech data 122, an operation/session record database 123, and event timing prediction data 124.

First, as shown in fig. 5, the trigger condition data 121 includes, for each expected standard event: event ID, event name, time period condition, and opening and closing condition of the door.

Further, as shown in fig. 6, the speech data 122 includes, for each expected standard event: event ID, event name, type of speech content, and main body of speech content.

As shown in fig. 7, the operation/dialogue log database 123 is data of the operation history of the refrigerator 300 by the user. The operation/dialogue log database 123 includes, for each operation by the user: for determining the correspondence between the ID of the appliance, the cumulative number of times the door of the refrigerator is opened and closed, the cumulative number of times the door of the freezer is opened and closed, and the date and time at which the user operation is detected.

In the present embodiment, every time the record storage unit 111 receives information on the operation of the refrigerator by the user, the data is accumulated in the operation/dialogue record database 123. Then, the learning unit 112 generates a graph 123B of changes in the number of times of opening and closing for each user or for each refrigerator per day as shown in fig. 8 based on the operation/dialogue record database 123.

The learning unit 112 creates a graph 124B of the timing prediction shown in fig. 9 by referring to the trigger condition data 121 based on the graph 123B of the change. That is, the learning unit 112 specifies the timing at which the event is likely to occur for each user or for each refrigerator. As shown in fig. 10, the timing prediction data 124 includes, for each event: a correspondence between the device ID, the event name, the date, the start time of the time period in which the event is likely to occur, and the end time of the time period in which the event is likely to occur. For example, the learning unit 112 associates the time when the door is opened for the first time at 6 am with the event of getting up, associates the time when the door is opened for the second time with the event of starting preparation of breakfast, and associates the time when the door is opened twice or more in three minutes or three or more times or less in five minutes with the event of the time when breakfast is prepared.

The learning unit 112 can specify the predicted time of the event on the next day based on the operation/dialogue log data on one day. Alternatively, the learning unit 112 can determine the predicted time of the next week event based on the operation/dialogue recorded data during the week. Alternatively, the learning unit 112 can specify the predicted time of the event of the next month based on the operation/dialogue log data of one month. Alternatively, the learning unit 112 can specify the predicted time of the event of each N weeks (each N week is one week to one week) based on the past operation/dialogue recorded data of each N weeks. Alternatively, the learning unit 112 determines weekdays and after holidays based on calendar information, user operations, and the like for each user, determines the predicted time of an event on a weekday based on operation/dialogue log data on the weekdays, and determines the predicted time of an event on a holiday based on operation/dialogue log data on the holidays.

The predicted timing for each event may be one time or an average time of a plurality of times, and as described in the present embodiment, may be a period corresponding to a deviation in the measurement period, for example, a period from the average time to a standard deviation corresponding to a deviation from the switching timing in the front-rear direction.

The speech control unit 113 refers to the timing prediction data 124, and transmits a message of the speech data 122, a period to be outputted by voice, and the like to the refrigerator 300, thereby causing the refrigerator 300 to output a voice message at a timing suitable for the situation.

Also, the storage 120 stores user information data 125 as shown in fig. 11. The user information data 125 includes, for each user or each refrigerator: user ID, user name, device ID, model of device, user's residence, address of device, etc. Thus, the CPU110 can output a voice message at a timing suitable for the surrounding situation for each of the refrigerators 300 to be targeted.

< information processing in Server 100A >

Next, the creation update process of the timing prediction data 124 of the server 100A according to the present embodiment will be described with reference to fig. 12. When the CPU110 receives data from the refrigerator 300 via the communication interface 160, the following processing is performed.

The CPU110 as the record storage section 111 acquires the device ID of the refrigerator 300 from the received data (step S102). The CPU110 reads the operation record and the session record from the received data, and accumulates them in the operation/session record database 123 in association with the device ID (step S104). The CPU110 as the learning unit 112 creates or updates the timing prediction data 124 by referring to the trigger condition data 121 based on the latest operation/session record database 123 (step S106).

Next, a message output process of the server 100A according to the present embodiment is described with reference to fig. 13. First, the CPU110 reads the timing prediction data 124 for each refrigerator 300 to be targeted (step S112). The CPU110 determines an event whose start time is the earliest among the events whose processing has not been completed (step S114).

The CPU110 reads the speech content of the event (step S122).

The CPU110 refers to the user information data 125, and transmits the speech start time, the speech end time, and the speech data of the speech content to the target refrigerator 300 via the communication interface 160 (step S138).

The CPU110 receives the speech execution recording from the refrigerator and stands by until the expiration date of the speech data of the speech content is ended (step S142).

The CPU110 refers to the timing prediction data 124 to determine whether or not there is any event that has not been processed (step S148). If the target event is found (yes in step S148), CPU110 repeats the processing from step S114. If the target event is not found (no in step S148), CPU110 ends the series of processing on this day.

< information processing in refrigerator 300 >

Next, a message output process of the refrigerator 300 according to the present embodiment will be described with reference to fig. 14. The CPU310 receives the voice data of the speech content from the server 100A via the communication interface 360 (step S6152). CPU310 waits until the expiration date start time of the voice data (step S154).

When the start timing is reached, the CPU310 prepares voice data and waits for an operation by the user (step S156). In the present embodiment, the operation by the user is to open the door of the refrigerating compartment or the freezing compartment, but may be an operation to open another door, close the doors, input a command to the touch panel, or the like.

CPU310 determines whether or not the operation by the user is accepted (step S160). When receiving the operation by the user (yes in step S160), CPU310 causes speaker 370 to output a message to be saved as a voice. The CPU110 notifies the server 100A of the utterance execution record via the communication interface 160 (step S174). CPU310 waits for the next data from server 100A via communication interface 360.

If the operation by the user is not accepted (no in step S160), CPU310 determines whether or not the valid period of the audio data has ended (step S180). If the validity period of the voice data has not ended (no in step S180), CPU310 repeats the processing from step S156. When the valid period of the voice data ends (yes in step S180), CPU310 waits for the next data from server 100A via communication interface 360.

< second embodiment >

In the first embodiment, the start time and the end time of the validity period are set for each event. In the present embodiment, the interval between events is considered in addition to this.

Specifically, the storage 120 of the server 100A stores event-related data 126 as shown in fig. 15. For events related to other events, the event correlation data 126 includes: a correspondence relationship between the event ID, the event ID of the associated destination, the minimum elapsed time of the event from the associated destination, and the maximum elapsed time of the event from the associated destination.

Thus, in the present embodiment, the learning unit 112 creates timing prediction data 124C shown in fig. 16 by referring to the trigger condition data 121 based on the graph 123B of the change in the number of door openings for each user or for each refrigerator for one day as shown in fig. 8. The temporal prediction data 124C includes, for each event: a correspondence between the device ID, the event name, the date, the start time of the event-prone time period, the end time of the event-prone time period, the associated event, the minimum elapsed time from the associated event, and the maximum elapsed time from the associated event.

Next, a message output process of the server 100A according to the present embodiment is described with reference to fig. 17. First, the CPU110 reads the timing prediction data 124C for each refrigerator 300 to be targeted (step S112). The CPU110 determines an event whose start time is the earliest among the events whose processing has not been completed (step S114).

The CPU110 reads the speech content of the event (step S122). The CPU110 determines whether there is an event associated with the event based on the timing prediction data 124C (step S128). In the case where there is no event associated with the event (in the case of no in step S128), the CPU110 executes the processing from step S138.

If there is an event associated with the event (yes in step S128), CPU110 determines whether or not the processing of the associated event has been completed (step S130). In a case where the associated event has not been issued (no in step S130), the CPU110 executes the processing from step S138.

When the processing of the related event is completed (yes in step S130), CPU110 calculates the speech start time, that is, the start time of the possible speech period, based on the following expression (1), and calculates the speech end time, that is, the end time of the possible speech period, based on the following expression (2) (step S134).

Speech start time (speech time of associated event + minimum elapsed time of self-associated event … (1))

Speech end time (speech time of associated event + maximum elapsed time of self-associated event … (2))

The CPU110 receives the utterance execution recording or waits until the expiration of the valid period of the voice data of the utterance content from the refrigerator 300 via the communication interface 160 (step S142).

The CPU110 refers to the timing prediction data 124C to determine whether or not there is any event that has not been processed (step S148). If the target event is found (yes in step S148), CPU110 repeats the processing from step S114. If the target event is not found (no in step S148), CPU110 ends the series of processing on this day.

< third embodiment >

In the present embodiment, a response message to the user of the message output from refrigerator 300 is acquired and used in the subsequent processing.

Specifically, the storage 120 of the server 100A according to the present embodiment stores the response data 127 as shown in fig. 18. The response data 127 includes, for each user's response: the correspondence between the device ID, the inquiry contents of the refrigerator, the kind of response of the user, and the date and time at which the response record was acquired. In the present embodiment, the type of the response of the user includes whether the response of the user is positive or negative.

In addition, storage 120 of server 100A stores summary data 128 as shown in FIG. 19. For each event, summary data 128 includes: the correspondence between the device ID, the event name, the number of positive responses from the user, the number of negative responses from the user, and information indicating which response is more.

In the present embodiment, when the CPU110 serving as the learning unit 112 receives a response from the user via the communication interface 160, a response record is added to the response data 127. Also, CPU110 updates summary data 128 based on latest answer data 127.

Next, a message output process of the server 100A according to the present embodiment is described with reference to fig. 20. First, the CPU110 reads the timing prediction data 124 for each refrigerator 300 to be a target (step S112). The CPU110 determines an event whose start time is the earliest among the events whose processing has not been completed (step S114).

In the present embodiment, the CPU110 refers to the summary data 128 to determine whether the event is an ng (no good) utterance (step S116). If the event is an NG speech (yes in step S116), the CPU110 repeats the processing from step S114.

If the event is not an NG utterance (no in step S116), CPU110 determines whether the event is an OK utterance (step S118). If the event is an OK utterance (yes in step S118), CPU110 reads the utterance content of the event from utterance data 122 (step S120). The CPU110 executes the processing from step S138.

If the event is not OK speech (no in step S118), CPU110 reads the speech content of the event from speech data 122 and reads the confirmation phrase of the speech of the event from response data 127.

The CPU110 refers to the user information data 125, transmits the speech start time, the speech end time, and the speech data of the speech content to the refrigerator 300 as the object via the communication interface 160, and also transmits the speech data of the confirmation phrase if necessary (step S138B).

The CPU110 receives the utterance execution recording from the refrigerator 300 via the communication interface 160 or stands by until the expiration date of the voice data of the utterance content is finished (step S142). When the confirmation phrase is transmitted to refrigerator 300, CPU110 receives an affirmative answer or a negative answer from the user, and updates answer data 127 and summary data 128.

The CPU110 refers to the timing prediction data 124 to determine whether or not there is any event that has not been processed (step S148). If the target event is found (yes in step S148), CPU110 repeats the processing from step S114. If the target event is not found (no in step S148), CPU110 ends the series of processing.

Next, a message output process of the refrigerator 300 according to the present embodiment will be described with reference to fig. 21. The CPU310 receives the voice data of the speech content from the server 100A via the communication interface 360 (step S152). CPU310 waits until the expiration date start time of the voice data (step S154).

When the start time of the validity period is reached, the CPU310 prepares voice data and waits for an operation by the user (step S156). CPU310 determines whether or not the operation by the user is accepted (step S160). When the operation by the user is accepted (yes in step S160), CPU310 determines whether or not the utterance confirmation phrase is included in the received data (step S162).

If the speech confirmation data is included in the received data (yes in step S162), CPU310 causes speaker 370 to output a speech confirmation phrase (step S164). The CPU310 determines whether an affirmative response is received from the user via the microphone 380 (step S166). When determining that an affirmative response has been received from the user (yes in step S166), the CPU310 executes the processing from step S172.

If an affirmative response is not received from the user (no in step S166), the CPU310 transmits an NG utterance record to the server 100A via the communication interface 360 (step S168). CPU310 waits for the next data from server 100A via communication interface 360.

If the speech confirmation data is not included in the received data (no in step S162), CPU310 causes speaker 370 to output the speech of the speech content (step S172). The CPU110 notifies the server 100A of the OK speech record and the speech execution record via the communication interface 160 (step S174B). CPU310 waits for the next data from server 100A via communication interface 360.

If CPU310 has not received the operation by the user (no in step S160), it determines whether or not the valid period of the voice data has ended (step S180). If the validity period of the voice data has not ended (no in step S180), CPU310 repeats the processing from step S156. When the valid period of the voice data ends (yes in step S180), CPU310 waits for the next data from server 100A via communication interface 360.

The response data 127 and the summary data 128 may be data common to a plurality of users, or may be information indicating that a response content, an event name, the number of positive responses, the number of negative responses, or a large number of responses are provided for each user. That is, in step S116 and step S118 in fig. 20, the determination may be made in common for a plurality of users, or may be made for each user.

< fourth embodiment >

Further, it is also possible to determine a time period during which the user is out, or to output a message for chatting while the refrigerator is operated during the time period. More specifically, in the present embodiment, as shown in fig. 22, the speech data 122D stores messages for a time zone during which the user is frequently out. Also, as shown in fig. 23, the timing prediction data 124 stores various data on an event of going out for each user or each refrigerator.

For example, the CPU110 as the learning unit 112 determines a period in which the possibility that the user does not perform an operation for several hours or more is high as the user's outgoing period, for example, 3 hours or more, based on the operation/dialogue record database 123. The CPU110 associates the time period with the outgoing event and stores the time period in the timing prediction data 124D. Thus, when a user who should have gone out operates the refrigerator 300, the CPU110 causes the refrigerator 300 to output "Yita, which is today rest? "such a voice message.

< fifth embodiment >

Alternatively, in addition to the event of getting up, a time period later than usual may be set, or a chatting message may be output while the refrigerator is operated during the time period. Of course, a time zone earlier than usual may be set, or a chat message may be output when the refrigerator is operated in the time zone.

More specifically, in the present embodiment, as shown in fig. 24, the speech data 122E stores a message for a later time zone. Also, as shown in fig. 25, the timing prediction data 124E stores various data on events that come late for each user or each refrigerator. In the present embodiment, a related event is set for an event for chat, but the minimum elapsed time and the maximum elapsed time from the related event are not set.

In this case, as shown in fig. 26, when the processing of the related event is completed (yes in step S130), CPU110 determines whether or not the elapsed time from the related event is set (step S132). If the elapsed time is not set (no in step S132), the CPU110 executes the processing from step S148. This prevents both the normal message at the time of getting up and the message at the time of late from being output.

When the elapsed time is set (yes in step S132), CPU110 calculates the speech start time, that is, the start time of the possible speech period, and calculates the speech end time, that is, the end time of the possible speech period (step S134D).

< sixth embodiment >

In the first to fifth embodiments, when the refrigerator 300 is operated within the valid period of the message, the message is immediately voice-output. However, in the present embodiment, the interval from the operation of the refrigerator 300 to the output of the message is different for each event.

For example, when the door is opened at the time of the user's getting up, a message is immediately output. For example, in a food for breakfast or the like, a message is output after several seconds from opening the door.

In the present embodiment, as shown in fig. 27, the speech data 122F includes, for each expected standard event: event ID, event name, type of speech content, body of speech content, and time from operation of the refrigerator by the user to voice output. Thus, for example, referring to fig. 13, in step S138, CPU110 refers to user information data 125, and transmits the speech start time, the speech end time, and the speech data of the speech content and the time from the user operation to the speech output to refrigerator 300 as the target via communication interface 160. Referring to fig. 14, in step S172, CPU310 waits for this time and then causes speaker 370 to output a voice.

Alternatively, it may be specified for each event whether the message is output when the door is opened or when the door is closed. For example, information with a low possibility of being troubled by the user even if forgotten is output when the door is opened, such as information about cooking, news, a call, and the like, while information with a high possibility of being troubled by the user if forgotten is output when the door is closed, such as carried items, a schedule, and the like.

In the present embodiment, as shown in fig. 28, the speech data 122G includes, for each expected standard event: event ID, event name, type of speech content, speech content body, and type of user operation to output speech. Thus, for example, referring to fig. 13, in step S138, CPU110 refers to user information data 125, and transmits the speech start time, the speech end time, the speech data of the speech content, and the type of user operation to refrigerator 300 as the target via communication interface 160. Referring to fig. 14, in step S172, CPU310 causes speaker 370 to output a voice when detecting an operation of the type of the user operation.

< seventh embodiment >

The configuration of the various databases is not limited to the configuration of the first to sixth embodiments. For example, as shown in fig. 29, the timing prediction data 124H may be replaced with timing prediction data 124C according to the second embodiment of fig. 16 in which the correlated event is correlated with all events other than the first event, and the elapsed time from the correlated event is correlated with all events other than the first event.

Alternatively, as shown in fig. 30, the trigger condition data 121H of the event may include the width of the time period of each event, that is, the waiting period of the user operation, instead of the trigger condition data 121 of the event of the first embodiment shown in fig. 5. Thus, when determining the period of the prediction timing for each event, the learning unit 112 can determine the start time and the end time based on the waiting time of the trigger condition data 121H, centering on the average time for each event.

< eighth embodiment >

Some or all of the functions of the refrigerator 300 in the above embodiments may be performed by the server 100A, a communication terminal such as a smartphone, a remote control device, or the like. Alternatively, a part or all of the functions of the server 100A may be performed by a plurality of servers, a communication terminal such as the refrigerator 300 or a smartphone, a remote control device, or the like.

For example, in the first to seventh embodiments, server 100A provides the voice data and the valid period of the message to refrigerator 300 for each message, but server 100A may provide the message and the valid period of one hour to refrigerator 300 in a summary manner, server 100A may provide the message and the valid period of one day to refrigerator 300 in a summary manner, server 100A may provide the message and the valid period of one week to refrigerator 300 in a summary manner, server 100A may provide the message and the valid period of one month to refrigerator 300 in a summary manner, and server 100A may provide the message and the valid period of all periods to refrigerator 300 in a summary manner.

Alternatively, the refrigerator 300 may have a function of most of the server 100A. Fig. 31 is a schematic diagram showing an outline of the operation of the refrigerator 300 constituting the network system 1 according to the present embodiment. Referring to fig. 31(a), when the user opens the door of the refrigerator 300 immediately after getting up, the refrigerator 300 outputs a message suitable for getting up. Referring to fig. 31(b), when the user opens the refrigerator 300 while preparing breakfast, the refrigerator 300 outputs a message suitable for preparing breakfast. Referring to fig. 31(c), when the user opens the door of the refrigerator 300 before work, the refrigerator 300 outputs a message suitable for work.

In the present embodiment, CPU310 of refrigerator 300 realizes recording storage unit 111, learning unit 112, and speech control unit 113. The memory 320 also stores an operation/session record database 123, event trigger condition data 121, event timing prediction data 124, and speech data 122.

Referring to fig. 32, CPU310 of refrigerator 300 functions as record storage unit 111, and accumulates operation records and session records in operation/session record database 123 based on user operations and user voices (step S204). The CPU310 as the learning unit 112 creates or updates the timing prediction data 124 by referring to the trigger condition data 121 based on the latest operation/session record database 123 (step S206).

Referring to fig. 33, the CPU310 reads the timing prediction data 124 (step S212). The CPU310 determines an event whose start time is the earliest among the events whose processing has not been completed (step S214).

The CPU310 reads the speech content of the event (step S222). CPU310 waits until the expiration date start time of the voice data (step S254).

The CPU310 prepares voice data and waits for an operation by the user (step S256). CPU310 determines whether or not the user operation is accepted (step S260). When accepting the operation by the user (yes in step S260), CPU310 causes speaker 370 to output a voice (step S272). The CPU310 executes the processing from step S288.

If the operation by the user is not accepted (no in step S260), CPU310 determines whether or not the valid period of the audio data has ended (step S280). If the validity period of the voice data has not ended (no in step S280), CPU310 repeats the processing from step S256.

When the validity period of the speech data is over (yes in step S280), CPU110 refers to timing prediction data 124 and determines whether or not there is an event that has not been processed (step S288). If the target event is found (yes in step S288), the CPU110 repeats the processing from step S214. If the target event is not found (no in step S288), CPU110 ends the series of processing.

< other application example >

Of course, the embodiments of the present invention can be applied to a case where the embodiments are implemented by providing a program to a system or an apparatus. Further, the advantageous effects of the embodiments of the present invention can be enjoyed also by supplying a storage medium (or a memory) storing a program expressed by software for realizing the embodiments of the present invention to a system or an apparatus, and reading out and executing the program code stored in the storage medium by a computer (or a CPU or MPU) of the system or the apparatus.

In this case, the program code itself read out from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes an embodiment of the present invention.

It goes without saying that the computer executes the read program codes, not only can the functions of the above-described embodiments be realized, but also, based on instructions of the program codes, a part or all of actual processing can be performed by an OS (operating system) or the like operating on the computer, and the functions of the above-described embodiments can be realized by this processing.

It goes without saying that the program code read out from the storage medium is written to a function expansion board inserted into the computer or another storage medium provided in a function expansion unit connected to the computer, and then, based on an instruction of the program code, a part or all of actual processing is performed by a CPU or the like provided in the function expansion board or the function expansion unit, and the functions of the above-described embodiments are realized by the processing.

It should be understood that the embodiments disclosed herein are illustrative in all respects, and not restrictive. The scope of the present invention is indicated by the scope of the claims, is not indicated by the above description, and includes all modifications within the meaning and scope equivalent to the claims.

Description of the reference numerals

1: network system

100A to 100C: server

110：CPU

111: recording storage unit

112: learning part

113: speaking control part

120: memory device

121: trigger condition data

122: data of speaking

123: dialogue record database

124: timing prediction data

125 user information data

126: event associated data

127: response data

128: summary data

140: operation part

160: communication interface

200: router

300: refrigerator with a door

310：CPU

320: memory device

330: display device

340: operation part

360: communication interface

370: loudspeaker

380: microphone (CN)

390: device driving section

391: door sensor

Claims

1. A network system is characterized by comprising:

a plurality of refrigerators; and a server for storing the received data,

the server is provided with:

a memory storing a plurality of kinds of messages, a time period condition for voice output of the messages, and a record database in which opening and closing operations of a door of each of the plurality of refrigerators are recorded;

a learning section that creates timing data for causing each of the plurality of refrigerators to speech-output a plurality of kinds of messages based on information of the record database and the time period condition,

a speech control section that transmits the timing data and the message to each of the plurality of refrigerators.

2. The network system of claim 1,

the timing data of each of the plurality of kinds of messages includes a period during which a message should be outputted in voice when the door of the refrigerator is opened.

3. The network system according to claim 1 or 2,

the time zone condition is set with an interval from the output of a first message to the output of a next second message among the plurality of types of messages.

4. The network system according to claim 1 or 2,

the server obtains the user's reaction to the message from the plurality of refrigerators, and decides whether to voice-output the message thereafter based on the reaction.

5. The network system according to claim 1 or 2,

the server determines a period during which the opening and closing of the door is rarely performed based on the information of the record database of each of the plurality of refrigerators,

causing the refrigerator to output a prescribed message when the refrigerator is opened during the period.

6. The network system of claim 3,

the server causes the refrigerator to output the second message different from the first message in place of the first message when a door is opened at a prescribed timing, which is before or after a timing associated with the first message of the plurality of kinds of messages.

7. The network system according to claim 1 or 2,

the message of each of the plurality of types is set with an interval from opening the door to starting the voice output or a type of user operation that is a timing to start the voice output.

8. A server is characterized by comprising:

a communication interface for communicating with a plurality of refrigerators;

a memory storing a plurality of kinds of messages and a time period condition for voice outputting the messages; and

a processor that acquires time data of an opening and closing action of a door of each of the plurality of refrigerators via the communication interface, creates timing data for causing each of the plurality of refrigerators to speech-output respective messages of a plurality of kinds based on the time data of the opening and closing action and the time period condition, and transmits the created timing data and the messages to each of the plurality of refrigerators via the communication interface.

9. An information processing method of a server, comprising:

a step in which the processor acquires, via the communication interface, time data of opening and closing operations of the door of each of the plurality of refrigerators;

a step in which the processor creates timing data for causing each of the plurality of refrigerators to speech-output respective messages of a plurality of kinds based on the acquired timing data of the opening and closing action of the door of each of the plurality of refrigerators and the time period condition stored in the memory; and

a step in which the processor transmits the created opportunity data and the message to each of the plurality of refrigerators via the communication interface.

10. A refrigerator is characterized by comprising:

a door;

a speaker;

a memory for storing a plurality of kinds of messages, a time period condition for outputting the messages by voice, and a record database in which opening and closing operations of the door are recorded; and

a learning unit that creates timing data for causing the speaker to output each of a plurality of types of messages in a voice manner based on information in the recording database and the time zone condition, and stores the timing data in the memory;

and a speech control unit that, when a user operation is received within a period during which the timing data is valid for speech output, causes the speaker to speech output a message corresponding to the timing data.