JP6900089B2 - Personal assistant control system - Google Patents

Description

The present invention relates to a personal assistant control system that controls a personal assistant such as a robot, and more particularly to a personal assistant control system that provides an appropriate service according to the growth of a person.

Until now, most robots have been so-called industrial robots mainly for the purpose of automation of production, but nowadays, robots that communicate with an unspecified number of people such as reception / customer service and product introduction are becoming a society. Has begun to be introduced in. In the future, robots will be owned by individuals, and it is thought that their functions as personal assistants will be strengthened in order to provide appropriate services to those individuals. Development is in progress.

As a watching robot for infants, for example, a communication robot that realizes voice chat between parents and children and friends for small children (infants) who do not have a smartphone or the like is known. On the other hand, regarding watching robots for the elderly, the Ministry of Economy, Trade and Industry and the Ministry of Health, Labor and Welfare have set "watching and communication" as a priority field in the use of robot technology for long-term care, and specifically, "use in nursing care facilities" as a priority field. , "Platform of equipment using robot technology equipped with sensors and external communication functions", "Life support equipment using robot technology for communication with elderly people, etc."

Whereas conventional watching devices simply detect and notify the state of the watching target, the watching robot not only detects the state of the watching target and notifies an alert, but also predicts the state of the watching target. In addition, it has been improved in that it is possible to analyze the situation when an alert occurs, and by connecting the watching robot to the network, at the same time as watching, the behavior and life data of the watching target are accumulated and the data is utilized. By doing so, for example, in watching over the elderly, it is said that it can lead to improvement of the care plan and improvement of the quality of care.

As a technique for providing a robot as a personal assistant, for example, in a computer-based method of actively and automatically providing a personal assistant using a robot device or platform connected to at least one sensor, communication unit and output device. There is a step of detecting first data associated with at least one person near at least one of the at least one sensor using the at least one sensor, and using the at least one sensor. The step of detecting a second data related to at least one nearby object, organism, event, place, environment or a combination thereof of the at least one sensor, and using the processor to obtain the training data, said first. The step of selectively storing the detection data, the second detection data, the pre-programmed algorithm stored in the database, or a combination thereof in the database, and the communication unit connected to or communicable with the processor. The step of passively outputting the first output data in response to a request received by the processor or a predetermined or scheduled event stored in the database using the output device, and the communication unit or the said. A step of actively and automatically outputting the second output data based on the first detection data, the second detection data, the training data, the pre-programmed algorithm, or a combination thereof, using the output device. And computer-based methods, including, are known. (Patent Document 1)

According to Patent Document 1, data related to a user and the environment surrounding the user is detected and processed, and a personal assistant is actively and automatically provided by using a robot platform / device based on the detected and analyzed data. It is said that it can provide a method / system for.

Japanese Unexamined Patent Publication No. 2014-176963

However, the techniques described in Patent Document 1 provide reminders to users, advice during medical examinations at medical institutions, support for users with Alzheimer's disease, support for parental control of parents / caregivers' children, and wheelchairs. Although the situation where the personal assistant intervenes in each scene such as the support of the user who uses the medical care is disclosed, the specific control mode of the personal assistant when the robot as the personal assistant is replaced as the person grows. Nothing is disclosed.

That is, the process of an infant growing into an adult and the process of gradually aging after an adult are continuous events on a daily basis, while personal assistants such as robots are used by users at some point in the growth and aging of humans. It is considered that it is usually replaced with a drastic one by an act such as purchase. The technique described in Patent Document 1 does not mention what kind of processing should be performed in a system including a robot when such a drastic replacement of a personal assistant occurs.

The present invention has been devised to solve the problems of the prior art, and even if a personal assistant such as a robot is replaced as a person grows, the personal assistant is not just a tool but a person's growth. At the same time, it is to provide a personal assistant control system that can appropriately support the user as a partner.

The present invention made to solve the above problems is used by a user in a first period, and a first personal assistant for acquiring information from a first sensor group including a plurality of sensors, and the first personal. It includes a server that is connected to the assistant via a network and estimates the user's situation based on the information acquired from the first sensor group, and the server is the first sensor in the first period. Among the sensors included in the group, the situation of the user is estimated based on the information acquired from the first attribute detection sensor that detects at least the first attribute, and in the pre-period which is a period earlier than the first period. , The user's situation is estimated based on the information acquired from the second attribute detection sensor that detects at least the second attribute different from the first attribute among the sensors included in the first sensor group. It is a personal assistant control system characterized by.

As a result, for example, when the user in the first period is an infant and the user in the pre-period is a foetation existing in the mother, the first personal assistant as a personal assistant common to the first period and the pre-period is used. It is possible to accurately estimate the situation of the user as an infant and the user as a foetation, and the personal assistant appropriately supports the user as a partner as the person grows, including the period when the person is a foetation. It becomes possible.

The present invention also includes a first personal assistant that is used by the user in the first period to acquire information from a first sensor group that includes a plurality of sensors, and a user that is earlier than the first period. A pre-period personal assistant that is used during the pre-period and acquires information from a second sensor group including a plurality of sensors, and the first personal assistant and the pre-period personal assistant are connected via a network. The server includes a server that estimates the user's situation based on the information acquired from the first sensor group or the second sensor group, and the server is the first sensor in the first period. Among the sensors included in the group, the user's situation is estimated based on the information acquired from the first attribute detection sensor that detects at least the first attribute, and the sensors included in the second sensor group during the pre-period. Among them, the personal assistant control system is characterized in that the situation of the user is estimated based on the information acquired from the second attribute detection sensor that detects at least the second attribute different from the first attribute.

As a result, for example, when the user in the first period is an infant and the user in the pre-period is a foetation existing in the mother, the first personal assistant is used in the first period and the pre-period personal assistant is used in the pre-period. By using it, it is possible to accurately estimate the situation of the user as an infant and the user as a foetation, and the personal assistant appropriately makes the user as a partner as the person grows, including the period when the person is a foetation. It will be possible to support.

As described above, according to the present invention, even if a personal assistant such as a robot is replaced as a person grows, the personal assistant is not a mere tool, but can appropriately support the user as a partner as the person grows. Become.

Explanatory drawing which shows the outline of the personal assistant control system S1 in the 1st period T1 and the 2nd period T2 in 1st Embodiment of this invention. (A) to (d) are explanatory views showing the relationship between the first period T1, the second period T2, the third period T3, and the fourth period T4 in the first embodiment of the present invention. The block diagram which shows the structure of the personal assistant control system S1 in the 1st period T1 in 1st Embodiment of this invention. The block diagram which shows the structure of the personal assistant control system S1 in the 2nd period T2 in 1st Embodiment of this invention. Explanatory drawing which shows the outline of the personal assistant control system S1 in the 1st period T1 and the 2nd period T2 in the 1st modification of 1st Embodiment. The block diagram which shows the structure of the personal assistant control system S1 in the 2nd period T2 in the 1st modification of 1st Embodiment. Explanatory drawing which shows the outline of the personal assistant control system S1 in the 1st period T1 and the 2nd period T2 in the 2nd modification of 1st Embodiment. The block diagram which shows the structure of the personal assistant control system S1 in the 2nd period T2 in the 2nd modification of 1st Embodiment. Explanatory drawing which shows the outline of the personal assistant control system S1 in the 1st period T1 and the pre-period T0 in the 2nd Embodiment of this invention. (A), (b) is an explanatory diagram showing the relationship between the pre-period T0, the first period T1, the second period T2, the third period T3, and the fourth period T4 in the second embodiment of the present invention. The block diagram which shows the structure of the personal assistant control system S1 in the pre-period T0 in the 2nd Embodiment of this invention. Explanatory drawing which shows the outline of the personal assistant control system S1 in the 1st period T1 and the pre-period T0 in the modification of 2nd Embodiment of this invention.

(First Embodiment)
Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing an outline of a personal assistant control system S1 in the first period T1 and the second period T2 in the first embodiment of the present invention.

As shown in FIG. 1, the personal assistant control system S1 includes a first personal assistant (hereinafter, may be abbreviated as "first PA1") and a second personal assistant (hereinafter, simplified). It may be referred to as "second PA2", and when the first PA1 and the second PA2 are not distinguished, it may be simply referred to as "PA"), and the first PA1 and A network 4 to which a second PA 2 is connected, a server 5 connected to the network 4, and an information terminal 6 connected to the network 4 are included.

Here, as the PA, for example, a robot is preferably used. The PA may be configured to enable at least some kind of interaction with the user 3, and may or may not have a displacement mechanism such as an arm or a movement mechanism. When a moving mechanism is provided, the moving mechanism may be composed of a bipedal walking mechanism, wheels, or the like, or may be configured to be movable in the air such as a drone.

In FIG. 1, the first PA1 is used by the user 3 in the first period T1 and the second PA2 is used by the same user 3 in the second period T2. In the first embodiment, the user 3 sets the period from the infant (for example, 0 years old to 1 year old) to the first half of the infant (for example, 1 year old to 2 years old) as the first period T1, and the user 3 sets it as the second period T2. The period of the latter half of infants (for example, from 3 years old to before entering elementary school) is assumed.

Considering the degree of development of the user 3 during the period in which each PA is used, the first PA1 used in the first period T1 has, for example, a deformed appearance of a small animal, and the user 3 in the first period T1 It is configured to have a sense of intimacy and security. On the other hand, the second PA2 used for the second period T2 has a configuration in which more functionality is emphasized in accordance with the user 3 who has grown from the first period T1. That is, the second PA2 includes, for example, a display 1u, and is configured to be able to provide visual information to the user 3.

As will be described later, the first period T1 and the second period T2 may overlap in time series, and even if the first period T1 and the second period T2 are separated in time series. Good. That is, although both the first PA1 and the second PA2 are shown in FIG. 1, the personal assistant control system S1 has either the first PA1 or the second PA2 as the user 3 grows. It is possible that only one of them is included.

The first PA1 acquires various information from the first sensor group 41 (see FIG. 3) described later, and at least a part of the information is transmitted to the server 5 via the network 4. The server 5 estimates the situation of the user 3 based on the output of the first sensor group 41 received.

The server 5 outputs a control command to the first PA1 via the network 4 based on the estimated situation of the user 3. Upon receiving this, the first PA1 outputs a voice or the like that calms the user 3 in a situation where the user 3 is crying, for example. Then, the reaction or response of the user 3 to the voice or the like is acquired by the first sensor group 41, and this information is also transmitted to the server 5. In this way, interaction is achieved between the user 3 and the first PA1. The server 5 learns how to respond to the user 3 to be more calm, for example, depending on the situation of the user 3, and if a similar situation occurs in the user 3, the server 5 is based on the learning result. You will respond appropriately.

The second PA2 acquires various information from the second sensor group 42 (see FIG. 4) described later, and at least a part of the information is transmitted to the server 5 via the network 4. The server 5 estimates the situation of the user 3 based on the output of the second sensor group 42 received. When the user 3 grows up and reaches the second period T2, the learning result acquired in the first period T1 is inherited and used in the interaction between the second PA2 and the user 3. In the second period T2, the language ability of the user 3 is significantly improved. Therefore, in this period, the interaction (two-way conversation) mainly through the language is performed between the user 3 and the second PA2. ) Is planned.

In FIG. 1, the information terminal 6 is a smartphone, a tablet, a smart watch, a personal computer, or the like, and is possessed by, for example, a third party 15 as a mother of the user 3. The server 5 transmits information about the situation of the user 3 estimated based on the information obtained from the first PA1 or the second PA2 and the environment in which the user 3 is placed to the information terminal 6, and the third party 15 transmits the information about the user 3 to the information terminal 6. Even if it is far from the user 3, the situation of the user 3 and the surrounding environment can be grasped. Then, the third party 15 inputs the voice to the information terminal 6, for example, and the voice is reproduced by the PA.

Further, in FIG. 1, the vital sensor 7 has a wristband shape, for example, and is worn on the wrist or the like of the user 3.

2 (a) to 2 (d) are explanatory views showing the relationship between the first period T1, the second period T2, the third period T3, and the fourth period T4 in the first embodiment of the present invention. .. As described above, in the first embodiment, as the first period T1 in which the first PA1 is used, the period from the baby to the first half of the infant is set as the second period T2 in which the second PA2 is used. User 3 assumes a period in the latter half of the infant. That is, the first period T1 is a period including the past than the second period T2.

As shown in FIGS. 2A to 2D, the first period T1 and the fourth period T4 start after a predetermined time has elapsed from the birth of the user 3, but of course the first period T1 And the fourth period T4 may start at the time when the user 3 is born. In this way, the start of the first period T1 and the fourth period T4 changes, for example, at what timing the guardian (third party 15) of the user 3 introduces the first PA1 (purchases). This is because it depends on whether it is used for use.

As shown in FIG. 2A, the first period T1 and the second period T2 may be separated in the time axis T direction. This case indicates that the use of the first PA1 has ended and the use of the second PA2 has started after a while.

Further, as shown in FIG. 2B, the end of the first period T1 and the start of the second period T2 may be simultaneous. This case indicates that the use of the first PA1 was stopped when the second PA2 was purchased.

Further, as shown in FIGS. 2 (c) and 2 (d), a part of the latter half of the first period T1 and a part of the first half of the second period T2 may overlap. This case shows that while using the first PA1, the second PA2 is also used at the same time.

Here, the third period T3 refers to a period straddling the first period T1 and the second period T2, and as shown in FIG. 2A, in the third period T3, the first period T1 And the second period T2 may be separated in time series, and as shown in FIG. 2B, in the third period T3, the end of the first period T1 and the beginning of the second period T2. As shown in FIGS. 2 (c) and 2 (d), the latter half of the first period T1 and the first half of the second period T2 are superimposed in the third period T3. Alternatively, as shown in FIG. 2D, the end of the third period T3 may coincide with the end of the second period T2.

Further, as shown in FIGS. 2A to 2D, the fourth period T4 means a period of the first period T1 that is not included in the third period T3. That is, the fourth period T4 is a period earlier than the third period T3. Hereinafter, as will be described in detail, in the first embodiment, the server 5 is set according to each period of the first period T1, the second period T2, the third period T3, and the fourth period T4. The type (combination) of the sensor to be referred to when estimating the situation of the user 3 is changed.

FIG. 3 is a block diagram showing the configuration of the personal assistant control system S1 in the first period T1 according to the first embodiment of the present invention. The personal assistant control system S1 is composed of at least a first PA1 and a server 5, and the first PA1 and the server 5 are connected to each other via a network 4. Further, the personal assistant control system S1 may include an information terminal 6 and a vital sensor 7.

Hereinafter, the first PA1 will be described. The first PA1 is a PA that connects the environment sensor 50, the first sensor group 41, the output interface 53, the input unit 1k, the first PA1 and the network 4, and inputs / outputs information to and from the server 5. It includes a network interface 1p and a PA control unit 1v.

Each of these components and various sensors constituting the first sensor group 41 described later (or a conversion module for A / D conversion of the output (analog signal in this case)) are connected by a bus (not shown). The PA control unit 1v includes, for example, a PA storage unit 1q composed of a ROM (Read Only Memory), a RAM (Random access memory), or the like, and a PA calculation unit 1s composed of, for example, a CPU (Central Processing Unit) or the like. .. The PA calculation unit 1s controls the PA network interface 1p to transmit the information acquired from the first sensor group 41 to the server 5 based on the program stored in the PA storage unit 1q, and conversely transmits the information from the server 5. The output interface 53 is controlled based on the command or the like.

The PA storage unit 1q stores, for example, a date of birth, a name, a nickname, etc. as code information for identifying the first PA1, a user ID, and personal information of the user 3. The personal information of the user 3 is transmitted from the information terminal 6 to the first PA1 via the network 4 by, for example, a third party 15 such as a guardian of the user 3 operating and inputting the information terminal 6. When the PA1 receives the personal information, it stores the personal information in the PA storage unit 1q and transmits the personal information together with the code information stored in the PA storage unit 1q in advance to the server 5. Upon receiving these, the server 5 issues a user ID as a unique identifier, associates the code information of the first PA1 transmitted from the information terminal 6 with the user ID and personal information, and stores the user ID in the database 5k. Then, the server 5 transmits the generated user ID to the first PA1. The first PA1 stores the received user ID in the PA storage unit 1q. This user ID is used as a search key when searching the database 5k of the server 5, for example.

Hereinafter, the environment sensor 50 will be described. The first PA1 includes a human sensor 1a composed of a charcoal sensor, an image sensor, etc., an illuminance sensor 1b composed of a phototransistor with a built-in optical filter, and a temperature sensor composed of a resistance temperature detector. 1c, a humidity sensor 1d that detects changes in resistance value and capacitance in response to changes in humidity is provided as a so-called environment sensor 50 that measures environmental information, and measures information about the environment in which the user 3 is placed.

Here, as the motion sensor 1a, a so-called image-type sensor capable of detecting the position and number of people is preferably used. The motion sensor 1a detects that the user 3 (see FIG. 1) exists in the vicinity of the first PA1 or that the third party 15 (or another person other than the third party 15) exists. In the first embodiment, the motion sensor 1a is built in the first PA1, but in the case of the image type sensor, it is provided as a separate body from the first PA1, for example, on the ceiling, and the position of a person. The result of measuring the number may be wirelessly transmitted to the first PA1. The illuminance sensor 1b detects the illuminance of the environment in which the first PA1 is placed (usually, it is also the environment in which the user 3 is placed). The temperature sensor 1c detects the temperature of the environment in which the first PA1 is placed, and the humidity sensor 1d detects the humidity of the environment in which the first PA1 is placed. The output of these environment sensors 50 is transmitted to the server 5.

Next, the first sensor group 41 will be described. Among the first sensor group 41, the camera 1e composed of an image sensor and the like, the microphone 1f as a sensor for acquiring sound, the odor sensor 1g, the pressure sensor 1h, the acceleration sensor 1i, the angular velocity sensor 1j, and the vital sensor 7 are included. It is used as a sensor to detect the situation of the user 3.

In the first period T1, the first PA1 is usually placed in the vicinity of the crib or the like where the user 3 is placed. In this aspect, the camera 1e mainly captures the user 3 as a subject. The camera 1e captures a moving image or a still image, and the captured image is transmitted to the server 5. When the image is a still image, the timing of imaging can be arbitrarily determined. For example, imaging may be performed periodically, or when a sound exceeding a predetermined sound pressure is detected by the microphone 1f (for example, a case where the user 3 starts crying), imaging may be performed by the camera. Imaging may be performed when the smile of the user 3 is detected based on the output of 1e. Further, the PA control unit 1v may detect the position of the user 3 based on the acquired image, and also has a pan / tilt function of the camera 1e as the actuator 1o, or a mechanism for rotating at least the first PA1 in a plane. The PA control unit 1v may be provided, and the position of the user 3 may be traced based on the captured image, and the actuator 1o may be controlled so that the user 3 is included in the imaging range of the camera 1e.

The microphone 1f mainly acquires the voice emitted by the user 3. The acquired sound information may be periodically transmitted to the server 5 in a digitized state, or only when it is determined that the sound pressure of the voice or the like acquired by the PA control unit 1v exceeds a predetermined value. You may send it. When the PA control unit 1v has a frequency analysis function, only sound information including a predetermined frequency may be transmitted.

The odor sensor 1g is a sensor that applies a phenomenon in which the resistance value of a semiconductor is lowered by adsorbing a specific odor molecule on the surface of a metal oxide semiconductor, for example. Normally, the odor sensor 1 g is often used as the environmental sensor 50, but as described above, the first period T1 assumes that the user 3 is an infant, and here, it is based on the baby's urination and defecation. From the viewpoint of detecting the odor, the odor sensor 1g is included in the sensor that detects the situation of the user 3. The PA control unit 1v may directly send the output of the odor sensor 1g to the server 5, and when the output becomes larger than a predetermined value, the PA control unit 1v determines that the user 3 has urinated or defecation, and to that effect. May be sent to the server 5.

The pressure sensor 1h detects, for example, the deformation of the semiconductor strain gauge formed on the surface of the diaphragm. The pressure sensor 1h is used in situations where the user 3 holds or holds the first PA1 in his arm, or how the user 3 handles the first PA1 (whether it is handled carefully or roughly, etc.), that is, , Detects the user 3's direct action with respect to the first PA1. For example, when the output of the pressure sensor 1h exceeds a predetermined value, the PA control unit 1v transmits the value to the server 5. From the viewpoint that the pressure sensor 1h measures a value directly caused by the behavior of the user 3, a plurality of pressure sensors 1h may be provided in the first PA1.

The accelerometer 1i detects a change in capacitance using, for example, MEMS technology (three axes). Further, the angular velocity sensor 1j detects, for example, the Coriolis force (three axes). The acceleration sensor 1i and the angular velocity sensor 1j detect the user 3's direct action with respect to the first PA1 in the same manner as the pressure sensor 1h.

The vital sensor 7 is configured in, for example, a wristband shape (see FIG. 1). The vital sensor 7 is provided with, for example, a near-infrared light receiving / emitting element, and is attached to the wrist or the like of the user 3 to measure a pulse wave or the like. Of course, the vital sensor 7 may be used to measure not only the pulse wave but also the body temperature of the wristband wearing portion of the user 3. If the vital sensor 7 can measure acceleration and angular velocity, these may be measured. As for the pulse wave, the face image of the user 3 or the like may be captured as a moving image by the above-mentioned camera 1e, and the pulse wave of the user 3 may be measured using the captured image (particularly the output of the G channel).

In the first embodiment, among the first sensor group 41, all but the vital sensor 7 are built in the first PA1. The vital sensor 7 existing outside the first PA1 is connected to the first PA1 by radio such as BLE, and the pulse wave measurement result is transmitted to the server 5 via the first PA1. Since the information acquired by the first sensor group 41 is used for inputting the learning model described later, synchronization is achieved when acquiring the information from the first sensor group 41. Of course, the trigger for the synchronization is not particularly limited, and the information may be acquired periodically, or for example, a state in which the output of the microphone 1f becomes larger than a predetermined value may be acquired as a trigger.

Next, the input unit 1k will be described. The input unit 1k is provided at a predetermined position of the first PA1, for example, and detects an external touch or pressure. As an application of the input unit 1k, a third party 15 who is in contact with the user 3 (here, an infant) determines the situation of the user 3, and for example, the user 3 such as "calm" or "irritated". Enter the situation (corresponding to the "sensitivity index" described later). As the input unit 1k, a plurality of pressing switches corresponding to the various situations of the user 3 may be provided, or an input interface of a method of confirming after selecting a predetermined item may be configured.

Next, the output interface 53 (first user interface) will be described. The transmission unit 1l is a film-like member provided on a part of the surface of the first PA1 and made of, for example, flexible polypropylene (PP) or the like. Assuming the user 3 in the first half of the infant period, for example, when the user 3 holds the first PA1, a relatively low frequency vibration that imitates the heartbeat of the mother is transmitted through the transmission unit 1l. Therefore, it is possible to lead the user 3 to a more relaxed state.

Further, the output interface 53 includes a speaker 1 m that outputs sound and the like, and a light emitting unit 1n that is provided in an easily visible portion of the first PA1 and is composed of, for example, an LED (Light Emitting Diode) or an OLED (Organic Light Emitting Diode). At least one of the actuators 1o (vibrator as a kind thereof) that causes the first PA1 to perform a predetermined mechanical operation is included. These are driven based on the information and instructions transmitted from the server 5 via the network 4.

Next, the server 5 will be described. The server 5 includes a first PA1, a server network interface 5a that inputs / outputs information to / from the information terminal 6, and a server control unit 5n via the network 4. The server control unit 5n includes, for example, a server storage unit 5b composed of a ROM, a RAM, or the like, and a server calculation unit 5c composed of, for example, a CPU or the like. The server calculation unit 5c controls other components of the server 5 based on a program or the like stored in the server storage unit 5b.

Further, the server 5 includes an image recognition unit 5d, a voice recognition unit 5e, a speaker analysis unit 5f, a user situation estimation unit 5g, a vital data analysis unit 5h, a PA command generation unit 5i, a user maturity determination unit 5j, a database 5k, and a user. Includes an action range recognition unit 5 m.

The image recognition unit 5d extracts the face region of the user 3 from the image information transmitted from the first PA1 and extracts a predetermined feature amount. The voice recognition unit 5e extracts "phonemes" from the sound information transmitted from the first PA1, converts them into text, and then specifies the vocabulary information. The speaker analysis unit 5f performs, for example, frequency analysis on the sound information to distinguish at least the user 3 as a speaker and the third party 15 (which may include other than the mother). The vital data analysis unit 5h estimates the physical condition of the user 3 based on the pulse wave information of the user 3 transmitted from the first PA1. The PA command generation unit 5i generates and transmits a predetermined command or the like for the first PA1 based on the instruction of the server control unit 5n. The user maturity determination unit 5j determines, in particular, the number of vocabularies recognized by the voice recognition unit 5e, the difficulty level of the word, and the degree of certainty of recognition. The database 5k is composed of so-called large-capacity storage.

As described above, the database 5k stores the user ID and the personal information of the user 3 transmitted from the first PA1 to the server 5. Further, the database 5k stores a learning model in which a series of learning has been completed in advance. The user situation estimation unit 5g inputs the information acquired by the first sensor group 41 into the learning model (as shown below, some information converted into feature quantities and the like is input). A predetermined index (sensitivity index) is derived as the output of the learning model. Here, as the learning model, for example, SVM (Support Vector Machine), which is a pattern recognition model, can be preferably applied. Of course, a model that has been trained for each of a plurality of Kansei indexes may be applied by deep learning.

What is input to this learning model is, for example, a feature amount generated by the image recognition unit 5d based on a camera image acquired by the camera 1e (a still image is desirable, and a still image is captured in the case of a moving image). , Voice tone based on sound information acquired by microphone 1f, vocabulary information generated by voice recognition unit 5e, odor information acquired by odor sensor 1g (particularly odor related to urination and defecation), user 3 first. This is the information acquired by the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j by the direct action of the PA1. Of course, the output of the vital data analysis unit 5h may be input together with this.

Here, the Kansei index is an index showing the situation of the user 3, and is a situation that tends to appear in facial expressions such as "laughing", "angry", and "crying", "calm", and "irritated". The user situation estimation unit 5g, which can simultaneously include internal situations such as "being", "excited", "resting", and "concentrating", applies a learning model and "crying". Derivation of Kansei indicators in multiple aspects such as "I am" and "I am excited". Then, the situation of the user 3 is estimated by the combination of the sensitivity indexes. As described above, in the first embodiment, the learning model outputs a plurality of Kansei indexes for a group of inputs, but of course, a single Kansei index may be used.

Further, since the sensitivity index is also affected by the environment in which the user 3 is placed, the information acquired by the environment sensor 50 may be referred to when constructing (learning) the learning model. At this time, the input of the learned learning model also includes the information acquired from the environment sensor 50. Similarly, since the sensitivity index is also affected by the physical or physical condition related to the physical condition and health of the user 3, the image acquired by the vital sensor 7 or captured by the camera 1e was used. A value based on pulse wave information may be added. At this time, the server 5 may measure the stress degree of the user 3 by referring to the measurement result of the pulse wave obtained from the vital sensor 7, and use this as an input of the user situation estimation unit 5g.

The user situation estimation unit 5g transmits the derived sensitivity index to the server control unit 5n. The server control unit 5n that received the sensitivity index learned from the PA command generation unit 5i in the past (or initially learned) in a situation where the user 3 is "excited" and "crying", for example. Of the response contents, the music and voice (for example, mother's words) that are the response contents when the user 3 is changed from the "excited" and "crying" situation to the "calm" situation. Instructed to extract, the PA command generation unit 5i searches the database 5k, selects content data that is presumed to be appropriate, and outputs an audio reproduction instruction to the first PA1. Of course, there is some correlation (here, for example, user 3 is relaxed) between the sensitivity index and the light emission pattern of the light emitting unit 1n of the first PA1 and the operation pattern of the first PA1 by driving the actuator 1o (vibrator). If there is such an effect), not only the sound but also the reproduction instruction regarding the light emission pattern of the light emitting unit 1n and the operation pattern (vibration pattern) of the first PA1 may be output.

By doing so, an interaction occurs between the first PA1 and the user 3 by using the sensitivity index derived based on the output of the first sensor group 41. For example, when the number of pulse waves (that is, heart rate) acquired by the vital sensor 7 at a predetermined time is larger than usual and the user 3 hits or throws the first PA1 is detected (at this time, the pressure sensor). 1h, the output of the acceleration sensor 1i and the angular velocity sensor 1j fluctuates greatly), and the user situation estimation unit 5g determines, for example, that the user 3 is "irritated". Based on this, the server control unit 5n determines that it is most effective to reproduce, for example, the mother's words "○○ -chan, Yoshiyoshi. Oriko-san" from among a plurality of response candidates, and generates a PA command. A command is given to the unit 5i to transmit an instruction to reproduce the content to the first PA1, and the PA command generation unit 5i transmits data related to a predetermined command and the content to the first PA1 based on the command. To do.

The sound, light emission, or motion pattern reproduced in the first PA1 in this way causes the user 3 to experience new emotional and behavioral changes. The emotional and behavioral changes are acquired by the first sensor group 41 and transmitted to the server 5 to derive a new sensitivity index. For example, when there is no improvement in the emotional index of "irritated", the server control unit 5n commands the use of other candidates among the contents considered to be effective, and the user 3 and the first PA1 Interaction continues. Then, through this interaction, the server control unit 5n modifies the priority of the content to respond to the emotional index. This response content is accumulated in the database 5k, and the relationship between the sensitivity index and the response content is learned.

As described above, in the first embodiment, the server 5 receives the information acquired from the first sensor group 41 in the first period T1 (the period including the past than the second period T2 as described above). The situation (sensitivity index) of the user 3 is estimated based on the situation, and the first user interface provided in the first PA1 (here, for example, the speaker 1 m included in the output interface 53) is estimated based on the estimated situation of the user 3. While responding to the user 3 via the above, the response content and the situation (sensitivity index) of the user 3 are associated with each other for learning. Further, as will be described later, in the second period T2, the situation of the user 3 is estimated based on the information acquired from the second sensor group 42 provided in the second PA2, and the estimated situation of the user 3 is obtained. Based on this, the response content learned in the first period T1 is referred to, and the user 3 is referred to through the second user interface provided in the second PA2 (for example, the speaker 1 m or the display 1u (see FIG. 4)). Respond to.

What is important here is that the first sensor group 41 that acquires information in the first period T1 and the second sensor group 42 that acquires information in the second period T2 include different sensors. The user situation estimation unit 5g derives the situation of the user 3 as a sensitivity index, which is a common index, even when information acquired from different sensors is included as an input. By using this common sensitivity index, the response contents learned when the user 3 is an infant can be utilized even when the user 3 grows up to be an infant.

It is also possible to directly associate the output of the first sensor group 41 with the content to be responded to and learn. However, in this case, the degree of black boxing of the learning model increases. By providing an "sensitivity index" as a common index that does not depend on the age of the user 3 between the output of the first sensor group 41 and the content to be output, black boxing is kept to a minimum, and the user 3 Appropriate support can be provided regardless of age.

In addition, there is a discrepancy between the input result for the input unit 1k provided in the first PA1 (the subjective situation of the user 3 held by the third party 15) and the sensitivity index estimated by the user situation estimation unit 5g. In that case, it is possible to reduce the divergence by retraining the learning model described above or performing additional learning.

Further, in the database 5k, the image information captured by the camera 1e is stored (archived) together with the imaging date, the imaging time, and the sensitivity index. Of course, the information based on the other first sensor group 41 obtained at the time of imaging may be accumulated in association with the image information.

Further, the sound information recorded by the microphone 1f may be stored in the database 5k together with the recording date, the recording time, and the sensitivity index. Of course, the information based on the other first sensor group 41 obtained at the time of recording may be accumulated in association with the sound information. For example, in the case where an abnormal sound is detected, it goes back to the past several tens of seconds at the time of detection. The image information may be accumulated.

The accumulated image information and sound information can be reproduced by the user 3 or the third party 15 later instructing the first PA 1 or the information terminal 6 by voice or the like. For example, user 3 can make a request after reaching adulthood, "I want to see a laughing picture when I was 3 years old." The voice recognition unit 5e of the server 5 interprets the request, and the server control unit 5n searches the database 5k by adding the sensitivity index as a key in the age, and transmits, for example, the information desired by the user 3 to the information terminal 6.

Further, the server control unit 5n is, for example, a third party 15 (here, a mother) and a user based on the measurement result of the environment sensor 50 (human sensor 1a) of the first PA1 and the analysis result of the speaker analysis unit 5f. Detect the interaction with 3. The voice when the third party 15 speaks to the user 3 is stored in the database 5k as sound information, and is extracted as a vocabulary by the voice recognition unit 5e, and the sensitivity index of the user 3 at that time is acquired. The status of this interaction is also accumulated in the database 5k, and learning is performed by associating the actions of the third party 15 (speaking here) with the status of the user 3 (sensitivity index) based on the information acquired by the first sensor group 41. Will be done.

Through this learning, for example, the mother's speech "○○ -chan is cute" often changed the sensitivity index of user 3 from "excited" to "calm down" and "laughing". In the situation where the sensitivity index of user 3 is "excited", among the sound information stored in the database 5k, the mother's utterance of "OO-chan, cute" is the first PA1. The probability of being reproduced / reproduced is high. This function is effective especially when a third party 15 is absent near the user 3, for example, in a scene where the user 3 starts crying.

As described above, in the first embodiment, the server 5 performs an action (for example, talking) of the third party 15 to the user 3 based on the information acquired from the first sensor group 41 in the first period T1. It is extracted and learned by associating the actions of the third party 15 with the situation (sensitivity index) of the user 3 based on the information acquired by the first sensor group 41. As will be described later, in the second period T2, at least the actions of the third party 15 in the first period T1 are based on the situation of the user 3 estimated from the information acquired by the second sensor group 42. Instructions are given to the second PA2 to reproduce part of it.

The interaction between the third party 15 and the user 3 is not limited to the direct one described above. In the first embodiment, the information acquired by the environment sensor 50, the image information captured by the camera 1e, the sound information recorded by the microphone 1f, and the like are transmitted to the server 5 via the network 4, and the server 5 performs these. It is possible to transmit the information of the above to the information terminal 6 possessed by the third party 15. At this time, the above-mentioned sensitivity index may be transmitted at the same time. Further, the server 5 receives the sound information recorded by the second microphone (not shown) provided in the information terminal 6 and transmits the voice reproduction instruction to the PA1 of 1 via the PA command generation unit 5i. It is possible to do.

For example, when the third party 15 grasps that the user 3 is crying based on the image information and the sound information received by the information terminal 6, the third party 15 is the second microphone of the information terminal 6 (not shown). Can be used to indirectly speak to the user 3. Further, when the third party 15 determines that the temperature and humidity of the room are high, the room is too bright, or the like, the information terminal 6 can be operated to make the environment in which the user 3 is placed more comfortable. As a result, such actions of the third party 15 with respect to the user 3 are also measured by the environment sensor 50 and the first sensor group 41, and the sensitivity index is derived by the server 5. Then, the actions of these third parties 15 and the sensitivity index are associated and used for learning. Then, for example, when the user 3 becomes a specific situation, the server 5 tells the third party 15 via the information terminal 6, for example, "If the room light is slightly dimmed, XX will calm down." Advice may be provided, and if the server 5 has a function to control the lighting of the room, it may be controlled to dim the lighting of the room.

As described above, the first period T1 assumes that the user 3 is a period from an infant (for example, 0 years old to 1 year old) to the first half of an infant (for example, 1 year old to 2 years old). In the first period T1, the growth of the user 3 is said to be remarkable especially in the development of the language. Infants begin so-called "cooling" such as "ah" and "uh" after two months of age. After that, as the interaction with the third party 15 progressed, the baby gradually began to respond to "hearing the sound" by voice, and after four months of age, "Well-" , "Da" and other babbling words including consonants will be emitted. After eight months of age, various consonants can be uttered clearly, and imitation of the vocabulary produced by mothers and the like begins. According to materials from the Ministry of Health, Labor and Welfare, about 95% of infants can speak meaningful words on their own initiative by the time they are 20 months old. On the other hand, it is also known that there are large individual differences in language development from infants to infants.

In the first period T1, as the baby or toddler grows up, the number of vocabularies recognized by the speech recognition unit 5e increases, the difficulty of the words used also increases, and as the baby grows up, As the pronunciation becomes clearer, the recognition rate by the voice recognition unit 5e also improves.

The user maturity determination unit 5j provided on the server 5 determines the maturity of the user 3 based on at least one of the number of vocabularies recognized by the voice recognition unit 5e, the difficulty level of the word, and the degree of certainty of recognition. To do. That is, the user maturity determination unit 5j evaluates the language ability of the user 3 based on the corpus or the like, and outputs this as a maturity index. When the maturity index becomes larger than a predetermined value, the server control unit 5n determines that the user 3 has reached a stage where more advanced interaction using the language is sufficiently possible, and the user 3 or the third party 15 is notified. On the other hand, for example, through the first PA1 or the information terminal 6, the time when the role of the first PA1 is about to end is approaching, and more advanced interaction such as providing information by characters or images to the user 3 can be executed. A proposal is made to encourage replacement (new purchase, etc.) with the configured second personal assistant (second PA2 described later). The user maturity determination unit 5j determines the maturity of the user 3 by referring to the sensitivity index output by the user situation estimation unit 5g, for example, by referring to the fact that the rate of change in emotions and emotions decreases in one day. It may be determined, or it may be referred to that the action range of the user 3 estimated by the user action range recognition unit 5 m is expanded beyond a predetermined value, and further, the individual user stored in the PA storage unit 1q may be referred to. Of the information, the information on the date of birth may be referred to.

FIG. 4 is a block diagram showing the configuration of the personal assistant control system S1 in the second period T2 in the first embodiment of the present invention. Since the server 5 is the same as that described with reference to FIG. 3 in FIG. 4, the description is omitted after assigning the same reference numerals, and the components included in the second PA2 are also described with reference to FIG. The same reference numerals are given to the elements common to the above, and the description thereof will be omitted.

Hereinafter, the second PA2 used in the second period T2 will be described with reference to FIG. 3 in combination with FIG. As shown, the second PA2 includes a second sensor group 42. In the second sensor group 42, the odor sensor 1g, the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j are removed as compared with the first sensor group 41 described above, while the first position information sensor 1t and the second sensor group 42 are removed. A position information sensor 8 is added.

As for the output interface 53, the transmission unit 1l and the light emitting unit 1n are removed as compared with the first PA1, while the display 1u is added. That is, the second PA2 includes at least one of the speaker 1m, the display 1u, and the actuator 1o as the output interface 53 (second user interface).

As described above, the second period T2 assumes that the user 3 is in the latter half of the infant (for example, from 3 years old to before entering elementary school). In general, it is a guideline for infants to be able to "remove diapers" during the daytime by about one and a half to two years old. Therefore, the second PA2 used in the second period T2 includes The odor sensor 1g that detects the presence or absence of urination or defecation on a daily basis is not installed. Further, in the second period T2, the user 3 can express his / her emotions in words, and a rational aspect also appears in the behavior, so that the user 3 directly acts on the second PA2. It is more appropriate to derive an emotional index based on the facial expression based on the image information acquired by the camera 1e and the vocabulary recognized by the voice recognition unit 5e using the sound information acquired by the microphone 1f, rather than deriving the emotional index. It is thought that it can be done. Of course, the outputs of the environmental sensor 50 and the vital sensor 7 that affect the emotional index may be referred to.

The first position information sensor 1t included in the second sensor group 42 is a sensor that measures the position of the second PA2 indoors, and for example, three points due to differences in radio wave strength and arrival time from a plurality of WiFi access points. WiFi positioning that measures the position by calculating the positioning, and beacon positioning that measures the position by calculating the three-point positioning using the radio wave strength based on the BLE (Bluetooth (registered trademark) Low Energy) standard, etc. It can be applied.

Further, the second position information sensor 8 is a sensor that measures the position of the user 3 indoors, and for example, the same sensing means as the first position information sensor 1t is used. In this case, the user 3 carries, for example, a Beacon transmitter based on the BLE standard. Of course, a positioning sensor using GPS (Global Positioning System) may be used as the first position information sensor 1t and the second position information sensor 8, or a positioning technology such as IMES (Indoor MEssaging System) may be applied. Often, using the image information output by the camera 1e, the relative position of the user 3 may be measured with reference to the position information of the second PA2 indoors, and this may be used as the position of the user 3. In this case, the camera 1e is preferably a so-called stereo camera in that depth information can be obtained.

In the second period T2, the user 3 can freely move indoors, and it is considered that, for example, the moving speed and the moving range also affect the emotional index. That is, when calm, the moving speed is relatively slow, while when irritated, the moving speed is generally high. Further, the separation distance between the second PA2 and the user 3 is considered to be one of the parameters indicating the relationship between the two. In particular, in the case where the second PA2 is configured to be movable by the actuator 1o, the second PA2 can be moved closer to the user 3 based on the sensitivity index by acquiring the positional relationship between the two, or vice versa. It is possible to separate them from each other.

Also in the second period T2, the information acquired by the second sensor group 42 is transmitted to the server 5 via the network 4, the user situation estimation unit 5g outputs the sensitivity index, and the server control unit 5n outputs the sensitivity index. The situation of the user 3 is estimated based on the above, the PA command generation unit 5i is made to select the operation by the second PA2, and the PA command generation unit 5i generates a command to execute the operation. This command also includes a command to move the position of the second PA2 described above.

The second PA2 is equipped with a display 1u that was not mounted on the first PA1, and the content of text or an image (still image, moving image) is displayed on the display 1u. The server 5 determines that the display 1u is mounted on the second PA2 based on the code information (the code that identifies the PA as described above) transmitted from the second PA2, and sets the sound content. Alternatively, or along with the sound content, the image content is transmitted to the second PA2.

In the second period T2, since the user 3 has significantly improved image recognition ability as well as language ability, it is extremely important to provide the image content to the user 3. For example, when a specific Kansei index (here, for example, "concentrated") is superior to the display of other contents when the image of the vehicle is displayed to the user 3. The server 5 will make a proposal to the user 3, for example, "Would you like to see a foreign bus now?" This is not limited to image information, and similar proposals can be made for sound information. This fosters the intellectual curiosity of user 3, and further, information terminals such as "OO-chan seems to be very interested in mobility in the city. Should we increase the frequency of providing related basic knowledge?" Through 6, it becomes possible to give advice to the third party 15 about the learning guideline of the user 3.

In the second PA2, the input unit 1k may be a touch panel or the like provided so as to be superimposed on the image surface of the display 1u. The user 3 can select the desired content by operating the input unit 1k. Of course, like the first PA1, the input unit 1k is also used as a means for inputting the situation of the user 3 by being operated by the third party 15. That is, in the second period T2, when the third party 15 feels at ease while listening to a specific song, for example, the third party 15 inputs subjective information such as "calm". To do.

After that, as the first period T1 to the second period T2 elapses, the first sensor group 41 from which the first PA1 acquires information and the second sensor group 42 from which the second PA2 acquires information. 3 and 4 will be described with reference to FIG. 2 as to how the is used.

In the first period T1, the first PA1 is used by the user 3, and the first PA1 acquires information from the first sensor group 41. In the first period T1, the situation of the user 3 is estimated as a sensitivity index based on the output of the first sensor group 41, but the contribution of each sensor in deriving the sensitivity index changes as the user 3 grows. .. In the fourth period T4, which occupies at least the first half of the first period T1, the sensitivity index is derived based on the outputs of all the sensors included in the first sensor group 41, but as the user 3 grows, For example, after "disengagement of the diaper", the contribution of the odor sensor 1g gradually decreases, and after the behavior of the user 3 has a rationale, the contribution of the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j is similarly increased. It will decrease.

The term "decrease in contribution" as used herein means that the frequency of characteristic inputs to the learning model decreases, while learning based on other inputs such as image information, sound information, and information acquired from the vital sensor 7. It means that the weight of the information provided by the odor sensor 1g or the like is substantially reduced as the additional learning of the model progresses.

In the latter half of the first period T1, the language ability of the user 3 is improved, the facial expression is enriched, and the user 3 is further moved around. Therefore, during this period, among the information acquired by the first sensor group 41, information such as image information acquired by the camera 1e, sound information acquired by the microphone 1f, and pulse waves acquired by the vital sensor 7 is obtained. It will affect the derivation of the sensitivity index.

Since the growth of the user 3 is continuous every day, it can be considered that there is no significant change in the language ability of the user 3 at least from the end of the first period T1 to at least the beginning of the second period T2. .. Therefore, even in the initial stage of the second period T2, among the information acquired by the second sensor group 42 that provides information to the second PA2 used by the user 3, the image information acquired by the camera 1e, the microphone 1f The sound information acquired by the camera, the pulse wave acquired by the vital sensor 7, and the like affect the derivation of the sensitivity index.

That is, in the third period T3 that straddles the first period T1 and the second period T2, the attributes common to the first period T1 and the second period T2 (here, image, sound, biological information). The situation of the user 3 is estimated based on the information obtained from the camera 1e, the microphone 1f, and the vital sensor 7 as the "common sensor". The common sensor may include at least a camera 1e and a microphone 1f.

That is, the personal assistant control system S1 of the first embodiment is the same as the first PA1 which is used by the user 3 for the first period T1 and acquires information from the first sensor group 41 including a plurality of sensors. The second PA2, which is used by the user 3 for the second period T2 and acquires information from the second sensor group 42 including a plurality of sensors, is connected to the first PA1 and the second PA2 via the network 4. The server 5 includes a server 5 that estimates the situation of the user 3 based on the information acquired from the first sensor group 41 or the second sensor group 42, and the server 5 has a first period T1 and a second period T2. Obtained from a "common sensor" that acquires at least common attributes (at least images and sounds) among the sensors included in the first sensor group 41 and the second sensor group 42 in the third period T3 straddling the above. The situation of the user 3 is estimated based on the information. In addition, in FIG. 3 and FIG. 4, the “common sensor” is marked with (1).

Here, in the fourth period T4 of the first period T1 which is not included in the third period T3, the camera 1e, the microphone 1f, and the vital sensor as "common sensors" included in the first sensor group 41. Information acquired by at least one of the sensors other than 7, that is, the odor sensor 1g, the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j as "non-common sensors" is also reflected in the derivation of the sensitivity index. Further, it is not necessary for the non-common sensor to include all the above-mentioned sensors, and for example, the pressure sensor 1h may be excluded as appropriate. As described above, in the personal assistant control system S1, in the fourth period T4, the server 5 refers at least the information acquired from the non-common sensor other than the common sensor in the first sensor group 41 to the user 3. Estimate the situation. That is, as a common index, the server 5 refers to the information acquired from the common sensor in the third period T3 and the information acquired from the non-common sensor in the fourth period T4. Derivation of the sensitivity index of.

By the way, in the second period T2 in which the second PA2 is used, in addition to the camera 1e, the microphone 1f, and the vital sensor 7 as common sensors in the second sensor group 42, the first position information sensor 1t and the second The sensitivity index is derived by taking into account the information acquired by the position information sensor 8. The sensitivity index is an index commonly used in both the first period T1 and the second period T2. Therefore, when responding to the user 3 based on the sensitivity index derived in the second period T2, the response content learned in the first period T1 is referred to (reflected) in the response. However, since the second PA2 is equipped with a display 1u that was not mounted on the first PA1, for example, in the server control unit 5n, the response performed by the first PA1 is via the speaker 1m. Even if it is, a response may be made using, for example, text information via the display 1u, or may be further responded together with the sound information from the speaker 1 m. Then, learning is performed from the viewpoint of what kind of combination of output interfaces 53 is used to make the user 3 more calm.

Further, as described above, in the first period T1, the actions of the third party 15 and the user's situation (sensitivity index) based on the information acquired by the first sensor group 41 are learned in association with each other. In the second period T2, the sensitivity index of the user 3 is derived based on the information acquired by the second sensor group 42. Then, based on the derived sensitivity index, the actions of the third party 15 learned in the first period T1 (for example, talking to the user 3) are reproduced.

FIG. 5 is an explanatory diagram showing an outline of the personal assistant control system S1 in the first period T1 and the second period T2 in the first modification of the first embodiment. In FIG. 1, the baby to the first half of the infant is assumed as the user 3 in the first period T1, and the latter half of the infant is assumed as the user 3 in the second period T2. As 3, the latter half of the infant is assumed, and as the user 3 in the second period T2, an adult is assumed.

Further, in the first embodiment, a third party 15 is assumed as a person who uses the information terminal 6, but in the first modification, the user 3 who has reached an adult mainly in the second period T2 is the information terminal. 6 is used. The second PA2 is a movable form (for example, a bipedal walking robot or a moving mechanism) that can autonomously interact with the user 3 and accurately support the user 3 who has reached adulthood. However, if it is assumed that the user 3 will carry the robot, for example, the robot may not have a moving mechanism.

FIG. 6 is a block diagram showing a configuration of the personal assistant control system S1 in the second period T2 in the first modification of the first embodiment. Hereinafter, using FIG. 4 in combination with FIG. 6, the configuration and function of the second PA2 used in the second period T2 in the first modification, and the first PA1 used in the first period T1 The relationship between In the first modification, the second PA2 shown in FIG. 4 corresponds to the first PA1 from the viewpoint of being used in the first period T1.

As shown in the figure, in the first modification, the second PA2 includes a third sensor group 43 (corresponding to the second sensor group 42 in FIG. 4). A first life sensor 9 is added to the third sensor group 43 as compared with the second sensor group 42 (see FIG. 4) described above. Here, the first life sensor 9 is a sensor that measures, for example, the amount of electric power used, the amount of gas used, the amount of water used, and the like of the user 3, and is also called a so-called smart meter. Usually, the user 3 often starts living independently of the parent when he / she becomes an adult. Since the value measured by the smart meter reflects the life pattern of the user 3, it also affects the sensitivity index as an index showing the situation of the user 3. With the addition of the first life sensor 9, the user situation estimation unit 5g will allow the user 3 to "tend to be blocked" or "in a situation where the life pattern for one month is almost unchanged, including Saturdays, Sundays, and holidays, for example." It is possible to presume that there is no energy.

Further, in the first modification, the physical situation such as "tired" and "dull" and the mental situation such as "lack of energy", which are less likely to be a problem if the user 3 is an infant or the like, are sensitive. Included in the index. That is, in the first modification, the sensitivity index is expanded. When expanding these Kansei indexes, the learning model may be retrained, or different learning models may be used in parallel. Further, from the viewpoint of expanding the sensitivity index, the first life sensor 9 may include a measuring device such as a weight scale or a sphygmomanometer. Further, the user 3 may be able to input the daily meal menu, amount, and calorie value via the input unit 1k.

By the way, when the user 3 is an infant or an infant, the input unit 1k is a means for a third party 15 such as a mother to input the situation of the user 3 based on the subjectivity. The 1k is provided as a means for the user 3 to notify the second PA2 (that is, the server 5) of his / her situation. As described above, the learning model is retrained or additional learning is performed based on the input of the input unit 1k. When the response of the second PA2 deviates from the subjective situation of the user 3, the user 3 presses the input unit 1k based on, for example, "I want you to understand me more". Manipulate. Then, the server control unit 5n retrains the learning model or executes additional learning so that the interaction between the user 3 and the second PA2 becomes more preferable based on the input of the input unit 1k.

As shown in FIGS. 4 and 6, the "common sensor" in the first modification is a camera 1e, a microphone 1f, a first position information sensor 1t, a vital sensor 7, and a second position information sensor 8. In addition, in FIG. 4 and FIG. 6, the “common sensor” is marked with ▲.

Also in the first modification, in the third period T3 that straddles the first period T1 and the second period T2, the attributes common to the first period T1 and the second period T2 (here, the image attribute). , Sound attribute, biological attribute, position attribute) based on the information obtained from the camera 1e, microphone 1f, first position information sensor 1t, vital sensor 7, and second position information sensor 8 as "common sensors". The situation of user 3 is estimated.

FIG. 7 is an explanatory diagram showing an outline of the personal assistant control system S1 in the first period T1 and the second period T2 in the second modification of the first embodiment. In the first modification, the latter half of the infant is assumed as the user 3 in the first period T1, and the adult is assumed as the user 3 in the second period T2. However, in the second modification, the user 3 in the first period T1 is assumed. Assuming an adult as, and as a user 3 in the second period T2, an elderly person.

In FIG. 7, since the user 3 in the second period T2 is an elderly person, the information terminal 6 is mainly used by the third party 15 who cares for the user 3 (of course, even if the user 3 himself uses it). Good). Further, the second PA2 may be in a movable form as in the first modification, or may be in a form such as an electric wheelchair on which the user 3 is boarded and moved. Of course, if it is assumed that the user 3 carries it, the mode may not have a moving mechanism.

FIG. 8 is a block diagram showing a configuration of the personal assistant control system S1 in the second period T2 in the second modification of the first embodiment. Hereinafter, in the second modification, the configuration and function of the second PA2 used in the second period T2 and the first PA1 used in the first period T1 will be used in combination with FIG. The relationship with is explained. In the second modification, the second PA2 shown in FIG. 6 corresponds to the first PA1 from the viewpoint of being used in the first period T1.

As shown, the second PA2 includes a fourth sensor group 44 (corresponding to the second sensor group 42 in FIG. 4). In the fourth sensor group 44, an odor sensor 1 g and a second life sensor 10 are added as compared with the third sensor group 43 (see FIG. 6) described above. Here, the odor sensor 1g detects the odor based on the incontinence of the user 3. The second life sensor 10 corresponds to a door sensor that detects a situation in which the user 3 is moving between rooms, and a mat sensor that is provided on the bed and detects the situation in which the user 3 wakes up. The mat sensor is composed of a sensor that detects pressure and acceleration. That is, the second life sensor 10 is a sensor that focuses on watching over the elderly.

For example, since the value measured by the mat sensor reflects the life pattern of the user 3 as an elderly person, it also affects the sensitivity index as an index showing the situation of the user 3. With the addition of the second life sensor 10, the user situation estimation unit 5g can change the situation where the user 3 moves extremely little with the door sensor or the user 3 lies in the bed almost all day with the mat sensor, for example. In the case of detection, it is possible for the user 3 to presume that he / she tends to be blocked or has no energy. In particular, a situation in which the information such as pulse waves acquired from the vital sensor 7 is not abnormal but "not motivated" is directly linked to so-called "frailing" especially in the elderly, so the server control unit 5n is the first. Through the dialogue with the user 3 via the PA2 of 2, for example, it is possible to achieve an interaction such as encouraging going out.

As shown in FIGS. 6 and 8, the “common sensor” in the second modification is the camera 1e, the microphone 1f, the first position information sensor 1t, the vital sensor 7, the second position information sensor 8, and the first life sensor 9. Is. In addition, in FIG. 6 and FIG. 8, the “common sensor” is marked with ▼.

Also in the second modification, in the third period T3 that straddles the first period T1 and the second period T2, the attributes common to the first period T1 and the second period T2 (here, the image attribute). , Sound attribute, biological attribute, position attribute, life-related attributes such as power consumption), camera 1e as "common sensor", microphone 1f, first position information sensor 1t, vital sensor 7, second position information sensor 8. The situation of the user 3 is estimated based on the information obtained from the first life sensor 9.

When the second PA2 is provided with a mechanism for transporting the user 3 like an electric wheelchair, the user 3 may board the second PA2 and go out. At this time, it is desirable that the first position information sensor 1t is a positioning sensor using GPS or the like. When the user 3 is outdoors together with the second PA 2, the output of the first life sensor 9 composed of, for example, a smart meter or the like is not used for deriving the sensitivity index. Even in such a case, the learning model can derive the Kansei index by clustering in a multidimensional space. Of course, the learning model itself may be exchanged between indoors and outdoors.

As described in detail above, in the personal assistant control system S1 according to the present invention, the personal assistant continuously provides services to the user 3 across generations. From this point of view, the personal assistant control system S1 may be rephrased as a personal assistant platform or a personal robot platform as a so-called base.

(Second Embodiment)
Hereinafter, the second embodiment of the present invention will be described with reference to the drawings. FIG. 9 is an explanatory diagram showing an outline of the personal assistant control system S1 in the first period T1 and the pre-period T0 in the second embodiment of the present invention. In the first embodiment, all the users 3 are assumed to be after birth, but in the second embodiment, the user 3 is different from the first embodiment in that the prenatal fetal period is also included. In the following description, the first PA1, the server 5, and the information terminal 6 already described in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 9, the personal assistant control system S1 includes a first PA1, a pre-period personal assistant (hereinafter, may be abbreviated as “pre-period PA20”), and a first PA1. A network 4 to which the PA 1 and the pre-period PA 20 are connected, a server 5 connected to the network 4, and an information terminal 6 connected to the network 4 are included.

Here, for the first PA1 and the pre-period PA20, for example, a robot is preferably used. The PA may be configured to enable at least some kind of interaction with the user 3, and may or may not have a displacement mechanism such as an arm or a movement mechanism.

In FIG. 9, the first PA1 is used by the user 3 in the first period T1 (postnatal), and the pre-period PA20 is used in the pre-period T0, which is the prenatal period for the user 3. Therefore, the personal assistant control system S1 may not include the first PA1 and the pre-period PA20 at the same time. Although it is the third party 15 who directly uses the pre-period PA20 in the pre-period T0, the pre-period PA20 and the first PA1 provide services to the same user 3 from prenatal to postnatal. Therefore, in the pre-period T0, the user 3 indirectly uses the pre-period PA20.

The PA20 for the pre-period in the pre-period T0 is designed with a lot of three-dimensional curved surfaces and is made of a flexible material so that the third party 15 who directly touches it reminds the image of safety and security. That is, it is assumed that the pre-period PA 20 is used in a manner held by a third party 15.

The pre-period PA 20 acquires various information from the second sensor group 42 (see FIG. 11), which will be described later, and at least a part of the information is transmitted to the server 5 via the network 4. The server 5 estimates the situation of the user 3 existing in the body of the third party 15 based on the output of the second sensor group 42 received.

The server 5 outputs a control command to the pre-period PA 20 via the network 4 based on the estimated situation of the user 3. Upon receiving this, the pre-period PA 20 outputs a sound that calms the user 3, for example. Then, the reaction or response of the user 3 to this sound is acquired by the second sensor group 42, and this information is also transmitted to the server 5. The server 5 learns how to respond to the user 3 to be more calm, for example, depending on the situation of the user 3, and if a similar situation occurs in the user 3, the server 5 is based on the learning result. You will respond appropriately. When the user 3 is born and reaches the first period T1, the learning result obtained in the pre-period T0 is inherited and used in the interaction between the first PA1 and the user 3.

In the second embodiment, the information terminal 6 is possessed by a third party 15. The server 5 transmits the status of the user 3 estimated based on the information obtained from the pre-period PA 20 to the information terminal 6, and the third party 15 can grasp the status of the user 3 existing in the body.

Further, in FIG. 9, the vital sensor 7 has a wristband shape, for example, and is worn on the wrist or the like of a third party 15.

10 (a) and 10 (b) show the relationship between the pre-period T0, the first period T1, the second period T2, the third period T3, and the fourth period T4 in the second embodiment of the present invention. It is explanatory drawing.

As shown in FIGS. 10A and 10B, the pre-period T0 corresponds to a period earlier than the first period T1. The beginning of the pre-period T0 is usually after conception, after the third party 15 notices pregnancy, and the end is at birth. The first period T1 may be started immediately after the end of the pre-period T0 as shown in FIG. 10 (a), and the interval between the pre-period T0 and the first period T1 as shown in FIG. 10 (b). May be vacated. As described with reference to FIGS. 2A to 2D, there may be various aspects of the relationship between the first period T1 and the second period T2.

FIG. 11 is a block diagram showing a configuration of the personal assistant control system S1 in the pre-period T0 according to the second embodiment of the present invention. The personal assistant control system S1 is composed of at least a pre-period PA 20 and a server 5, and the pre-period PA 20 and the server 5 are connected to each other via a network 4.

Hereinafter, the PA20 for the pre-period will be described. The pre-period PA 20 includes a second sensor group 42 and an output interface 53. Since the configurations of the PA network interface 1p, the PA calculation unit 1s, the PA storage unit 1q, the PA control unit 1v, and the like are the same as those of the first PA1 already described, the description thereof will be omitted. Further, although the environmental sensor 50 (see FIG. 3 and the like) mounted on the first PA1 is not mounted on the pre-period PA20, the emotions of the third party 15 also affect the user 3. The environment sensor 50 may be included for the purpose of monitoring the environment in which the third party 15 is located.

In the second embodiment, the microphone 1f acquires the heartbeat sound of the user 3 as a foetation, the heartbeat sound of the third party 15, and the voice emitted by the third party 15. The pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j detect the heartbeat and fetal movement of the user 3. The acquired information on voice, heartbeat, heartbeat, and fetal movement is transmitted to the server 5 and stored in the database 5k.

In the second embodiment, the vital sensor 7 is used to measure the situation of the third party 15 instead of the user 3 (see FIG. 9). Hormones such as dopamine and β-endorphin, which are secreted when the mother, a third party 15, feels “happy” or “good feeling”, are supplied to the user 3 who is a foet in the bloodstream, and the foetation is relaxed. It is known that the situation of the third party 15 is grasped indirectly, which leads to grasping the situation of the user 3.

The transmission unit 1l is composed of a film-like member made of, for example, flexible polypropylene (PP) provided on a part of the surface of the pre-period PA20. The sound acquired by the microphone 1f is amplified by an amplifier (not shown), and the sound output by the speaker 1m enters the body of the third party 15 via the transmission unit 1l which is in close contact with the body of the third party 15. introduce. The transmission unit 1l provides the user 3 with a scene in which the third party 15 or its partner (here, for example, a father) speaks to the user 3 before birth, and sound information (for example, music) of the environment in which the third party 15 is placed. It is used in the scene to be transmitted.

Next, the function of the server 5 will be described. Since the configuration of the server 5 is the same as that already described in the first embodiment, the description thereof will be omitted. Also in the second embodiment, the user situation estimation unit 5g inputs the information acquired by the second sensor group 42 into the learning model to derive the situation (sensitivity index) of the user 3.

Information about the heartbeat, heartbeat, and fetal movement acquired by, for example, the microphone 1f, the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j is input to the learning model. Since the heartbeat slows down when the user 3 is calm, the heartbeat affects the sensitivity index, and it is said that a moderate fetal movement indicates good communication between the user 3 and the third party 15. Therefore, it is considered that it affects the sensitivity index.

Since the information such as the heartbeat of the third party 15 is superimposed on this information, the server control unit 5n analyzes the frequency of the input information and obtains the information between the user 3 and the third party 15. Distinguish. Since it is considered that the user 3 and the third party 15 are emotionally synchronized, the information regarding the heartbeat of the third party 15 affected by the emotions of the third party 15 or the information acquired by the vital sensor 7 is input. May be added. However, in the second embodiment, since the user 3 exists in the body of the third party 15, it is not possible to estimate the situation in which the user 3 is likely to appear in the facial expression as shown in the first embodiment. Internal situations such as "are" and "excited" are presumed.

The user situation estimation unit 5g transmits the derived sensitivity index to the server control unit 5n. The server control unit 5n that has received the sensitivity index tells the PA command generation unit 5i, for example, in a situation where the user 3 is "excited", the music that the user 3 has heard when the user 3 is "calm" in the past. And voice (for example, mother's words) are instructed, and in response to this, the PA command generation unit 5i selects content data that is presumed to be appropriate, and issues the content data and voice playback instruction to the pre-period PA20. Output to. Similar to the first embodiment, this response content is accumulated in the database 5k, and the relationship between the sensitivity index and the response content is learned.

Here, the attributes of the information acquired by the first sensor group 41 (see FIG. 3) of the first PA1 used by the user 3 after birth and the information acquired by the second sensor group 42 of the pre-period PA20. Compare with attributes.

In the first period T1, among the first sensor group 41 of the first PA1 shown in FIG. 3, at least the camera 1e and the microphone 1f (first attribute detection sensor 51) are used to display the facial expression of the user 3. Acquire attributes (first attribute) that can be said to be "mental attributes" such as voice tone and vocabulary information. On the other hand, in the pre-period T0, the user can use the microphone 1f, the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j (second attribute detection sensor 52) constituting the second sensor group 42 of the pre-period PA20. Acquire attributes (second attribute) that can be said to be "physical attributes" such as heartbeat and fetal movement of 3. Then, the user situation estimation unit 5g derives the sensitivity index of the user 3 in the first period T1 based on the first attribute, and derives the sensitivity index of the user 3 in the pre-period T0 based on the second attribute. Although the pressure sensor 1h and the like are also included in the first PA1, the contribution of the pressure sensor 1h and the like decreases in deriving the sensitivity index as the user 3 grows, and the mental situation is estimated. "Mental attributes" suitable for the above will be the main factor in determining the sensitivity index.

As described above, the personal assistant control system S1 of the second embodiment is used by the user 3 for the first period T1, and the first PA1 that acquires information from the first sensor group 41 including a plurality of sensors and the user. The pre-period PA20, which is used by 3 for the pre-period T0, which is a period earlier than the first period T1 and acquires information from the second sensor group 42 including a plurality of sensors, and the first PA1 and the pre-period. The server 5 is connected to the PA 20 for use via the network 4, and includes a server 5 that estimates the status of the user 3 based on the information acquired from the first sensor group 41 or the second sensor group 42. In the period T1, the situation of the user 3 is estimated based on the information acquired from the first attribute detection sensor 51 that detects at least the first attribute among the sensors included in the first sensor group 41, and the pre-period T0 In, the situation of the user 3 is estimated based on the information acquired from the second attribute detection sensor 52 that detects at least the second attribute different from the first attribute among the sensors included in the second sensor group 42. ..

What is important here is that the first attribute detection sensor 51, which acquires information in the first period T1, and the second attribute detection sensor 52, which acquires information in the pre-period T0, detect different attributes. The user situation estimation unit 5g derives the situation of the user 3 as a sensitivity index, which is a common index, even when different attributes are input. By using this common sensitivity index, the response contents learned when the user 3 is a foetation can be utilized even when the user 3 grows up to be an infant.

Further, the server control unit 5n, for example, speaks to the user 3 of the third party 15 based on the result of analyzing the voice acquired by the microphone 1f included in the second sensor group 42 by the speaker analysis unit 5f. Extract actions. The voice (response content) when the third party 15 speaks to the user 3 is stored in the database 5k as sound information, and is extracted as a vocabulary by the voice recognition unit 5e. Further, at this time, the sensitivity index of the user 3 when the third party 15 performs an action is acquired. The status of this interaction is also accumulated in the database 5k, and the actions of the third party 15 (speaking here) and the status of the user 3 (sensitivity index) based on the information acquired by the second sensor group 42 are learned in association with each other. To.

Further, the server control unit 5n is based on, for example, the operation of the input unit 1k by the third party 15 (including the generation of a command based on the recognition result by the voice recognition unit 5e, the operation of the information terminal 6 and the like), for example, the microphone 1f. It is possible to record the heartbeat sounds of the third party 15 acquired in. Further, the server control unit 5n can also instruct the pre-period PA 20 to reproduce the recorded heartbeat sound based on the operation of the input unit 1k by the third party 15. Of course, at this time as well, the sensitivity index of the user 3 when the third party 15 performs an action (here, an operation of the input unit 1k, etc.) is acquired. This situation is also accumulated in the database 5k, and the action of the third party 15 (here, reproduction of the heartbeat sound) is associated with the situation (sensitivity index) of the user 3 based on the information acquired by the second sensor group 42. Be learned.

Through this learning, for example, the sensitivity index of user 3 changes from "excited" to "calm" by talking to the mother "○○ -chan, let's go" and listening to the heartbeat sound of a third party 15. In a situation where the user 3's sensibility index becomes "excited" as the number of changes increases, the mother's "OO-chan, Orikoune" is uttered from the sound information stored in the database 5k. The probability that the action or heartbeat sound will be reproduced / reproduced (without waiting for the instruction of the third party 15) will increase.

Then, the reproduction of the vocalization action, the heartbeat sound, and the like is inherited from the pre-period T0 to the first period T1 via a common sensitivity index. In the pre-period T0, the server 5 extracts the actions of the third party 15 with respect to the user 3 based on the information acquired from the second sensor group 42 (second attribute detection sensor 52), and the actions of the third party 15 And the situation of the user 3 based on the information acquired by the second attribute detection sensor 52 are learned, and the user 3 acquires the information by the first attribute detection sensor 51 (see FIG. 3) in the first period T1 after birth. Based on the situation of the user 3 estimated from the information obtained, the first PA1 is instructed to reproduce at least a part of the actions of the third party 15 in the pre-period T0. That is, based on the situation of user 3. For example, the content such as the heartbeat sound recorded in the pre-period T0 will be reproduced in the first period T1 as well.

FIG. 12 is an explanatory diagram showing an outline of the personal assistant control system S1 in the first period T1 and the pre-period T0 in the modified example of the second embodiment of the present invention. In the second embodiment described above, the first PA1 used in the first period T1 and the pre-period PA20 used in the pre-period T0 are different robots, but in the modified example, they are different robots. , The same (single) PA (first PA1) is used in both the first period T1 and the pre-period T0. Hereinafter, a modified example of the second embodiment will be described with reference to FIG.

As shown in FIG. 3, the first PA1 acquires information from the first sensor group 41. Among the first sensor group 41, the camera 1e and the microphone 1f correspond to the above-mentioned first attribute detection sensor 51, and the microphone 1f, the pressure sensor 1h, the acceleration sensor 1i, and the angular velocity sensor 1j correspond to the second attribute detection sensor 52. Applicable. That is, the microphone 1f, which is the same sensor, is also used to acquire the "physical attribute" in the pre-period T0 and the "mental attribute" in the first period T1.

Also in the modified example, as described in the second embodiment, the personal assistant control system S1 is used by the user 3 for the first period T1 and acquires information from the first sensor group 41 including a plurality of sensors. The first PA1 is connected to the first PA1 via the network 4, and the server 5 includes a server 5 that estimates the situation of the user 3 based on the information acquired from the first sensor group 41. In the period T1, the situation of the user 3 is estimated based on the information acquired from the first attribute detection sensor 51 that detects at least the first attribute among the sensors included in the first sensor group 41, and the first Acquired from the second attribute detection sensor 52 that detects at least the second attribute different from the first attribute among the sensors included in the first sensor group 41 in the pre-period T0, which is a period earlier than the period T1. The situation of the user 3 is estimated based on the information obtained. Also in the modified example, the sensitivity index, which is a common index, is based on the information acquired by the first attribute detection sensor 51 and the information acquired by the second attribute detection sensor 52, which detects an attribute different from that of the first attribute detection sensor 51. Is derived.

Further, the server 5 extracts the actions of the third party 15 against the user 3 based on the information acquired from the first sensor group 41 in the pre-period T0, and the actions of the third party 15 and the second attribute detection sensor. Learning is performed in association with the situation of the user 3 based on the information acquired in 52, and in the first period T1, the pre-period T0 is based on the situation of the user 3 estimated from the information acquired by the first attribute detection sensor 51. Instruct the first PA1 to reproduce at least a part of the actions of the third party 15 in.

Then, the first PA1 has a microphone 1f, an amplifier (not shown) that amplifies the sound acquired by the microphone 1f, a speaker 1m that reproduces the audio signal amplified by this amplifier, and a sound reproduced by the speaker 1m. It is provided with a transmission unit 1l that transmits to the body of the three parties 15.

The personal assistant control system S1 according to the present invention has been described in detail based on specific embodiments, but these are merely examples, and the present invention is not limited to these embodiments. For example, the sensors included in the first sensor group 41 and the second sensor group 42 can be appropriately selected, and are not limited to the sensors described in the present embodiment. Further, in the first embodiment, the personal assistant applied to infants, toddlers, adults, and the elderly has been described, but the process of human growth is divided into smaller periods, and an appropriate personal assistant is applied to each. It's clear that you can.

In the personal assistant control system S1 according to the present invention, even if a personal assistant such as a robot is replaced with the growth of a person, the robot as a personal assistant is not a mere tool, but is adjusted to the growth of the person including the fetal stage. Since it becomes possible to appropriately support the user 3 as a partner, it can be suitably applied as, for example, a platform for a personal assistant.

1 1st PA (1st personal assistant)
1e Camera 1f Microphone 1u Display 2 Second PA (second personal assistant)
3 User 4 Network 5 Server 6 Information terminal 7 Vital sensor 15 Third party 20 Pre-period PA (pre-period personal assistant)
41 First sensor group 42 Second sensor group 43 Third sensor group 44 Fourth sensor group 50 Environmental sensor 51 First attribute detection sensor 52 Second attribute detection sensor S1 Personal assistant control system

Claims (8)

A first personal assistant that is used by the user during the first period and acquires information from a first sensor group that includes a plurality of sensors.
It is provided with a server connected to the first personal assistant via a network and estimating the situation of the user based on the information acquired from the first sensor group.
The server
In the first period, the user's situation is estimated based on the information acquired from the first attribute detection sensor that detects at least the first attribute among the sensors included in the first sensor group.
A second attribute detection sensor that detects at least a second attribute different from the first attribute among the sensors included in the first sensor group in a pre-period that is a period earlier than the first period. A personal assistant control system characterized in that the situation of the user is estimated based on the information obtained from. The server
In the pre-period
Based on the information acquired from the first sensor group, the actions of a third party against the user are extracted.
Learn by associating the actions of the third party with the situation of the user based on the information acquired by the second attribute detection sensor.
In the first period
To the first personal assistant to reproduce at least a part of the third party's actions during the pre-period based on the user's situation estimated from the information acquired by the first attribute detection sensor. The personal assistant control system according to claim 1, wherein the instruction is given. The first personal assistant is
A microphone as a sensor that acquires sound,
An amplifier that amplifies the sound acquired by the microphone, and
A speaker that reproduces the audio signal amplified by the amplifier, and
The personal assistant control system according to claim 2, further comprising a transmission unit that transmits the sound reproduced by the speaker into the body of the third party. A first personal assistant that is used by the user during the first period and acquires information from a first sensor group that includes a plurality of sensors.
A pre-period personal assistant that is used by the user during a pre-period, which is a period earlier than the first period, and acquires information from a second sensor group including a plurality of sensors.
A server that is connected to the first personal assistant and the pre-period personal assistant via a network and estimates the user's situation based on the information acquired from the first sensor group or the second sensor group. With
The server
In the first period, the user's situation is estimated based on the information acquired from the first attribute detection sensor that detects at least the first attribute among the sensors included in the first sensor group.
The situation of the user based on the information acquired from the second attribute detection sensor that detects at least the second attribute different from the first attribute among the sensors included in the second sensor group during the pre-period. A personal assistant control system characterized by estimating. The server
In the pre-period
Based on the information acquired from the second sensor group, the actions of a third party against the user are extracted.
Learn by associating the actions of the third party with the situation of the user based on the information acquired by the second attribute detection sensor.
In the first period
To the first personal assistant to reproduce at least a part of the actions of the third party during the pre-period based on the user's situation estimated from the information acquired by the first attribute detection sensor. The personal assistant control system according to claim 4, wherein the instruction is given. The pre-period personal assistant
A microphone as a sensor that acquires sound,
An amplifier that amplifies the sound acquired by the microphone, and
A speaker that reproduces the audio signal amplified by the amplifier, and
The personal assistant control system according to claim 5, further comprising a transmission unit that transmits the sound reproduced by the speaker into the body of the third party. The server derives a common index based on the information acquired by the first attribute detection sensor and the information acquired by the second attribute detection sensor that detects an attribute different from the first attribute detection sensor. The personal assistant control system according to any one of claims 1 to 6, wherein the personal assistant control system is characterized. The personal assistant according to any one of claims 1 to 7, wherein the pre-period is a prenatal period by the user and the first period is a postnatal period by the user. Control system.

Images