WO2022208905A1 - Information processing device, information processing method, information processing program, and information processing system - Google Patents

Abstract

[Problem] To provide an information processing device, an information processing method, an information processing program, and an information processing system that appropriately control output to a user regardless of situation. [Solution] An information processing device comprising: a user's state estimation unit for estimating a user's state; an environment estimation unit for estimating an environmental state to be presented to the user on the basis of the user's state; and an output control unit for controlling output on the basis of the environmental state.

Description

Information processing device, information processing method, information processing program, and information processing system

The present disclosure relates to an information processing device, an information processing method, an information processing program, and an information processing system that control output to a user.

There is a technology that recognizes utterances and environmental sounds, and selects and outputs content such as music based on the recognized sounds (Patent Document 1).

U.S. Pat. No. 1,089,1948 U.S. Pat. No. 9,398,361

　The technology to recognize speech and environmental sounds is applicable only to environments with sounds. Therefore, a user who does not want to make noise or a situation where he/she does not want to make noise may not be able to select appropriate content. Also, natural language processing requires high computational power, making it difficult to process locally.

In view of the circumstances as described above, an object of the present disclosure is to provide an information processing device, an information processing method, an information processing program, and an information processing system that appropriately control output to the user regardless of the situation.

An information processing device according to one aspect of the present disclosure includes:
a user state estimation unit that estimates a user state;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
an output control unit that controls output based on the environmental state;
Equipped with

According to this embodiment, it is possible to appropriately control the output to the user regardless of the situation.

a user position estimating unit that estimates a user position based on a detection value of a sensor unit of the wearable device worn by the user;
a location attribute estimating unit that estimates a location attribute, which is an attribute of a location where the user is located, based on the user location;
further comprising
The user state estimation unit may estimate the user state based on the location attribute.

The user position estimation unit may estimate the user position using PDR (Pedestrian Dead Reckoning).

According to this embodiment, it is possible to appropriately control the output to the user based on the user's location in the house and other user contexts, regardless of situations such as situations in which users do not want to make noise.

The environment estimation unit may estimate the environmental state based on the location attribute.

For example, when the user is at the desk during teleworking, the output for the user can be controlled so that the user can focus on the work, and when the user is at the resting place, the output for the user can be controlled so that the user can relax.

The sensor unit of the wearable device may include at least one of an acceleration sensor, a gyro sensor, a compass, and a biosensor.

　In contrast to outdoors, indoor spaces are relatively small, and external equipment such as high-precision beacons and cameras are generally required to estimate a specific position. On the other hand, according to the present embodiment, it is possible to specify the position inside the house using the acceleration sensor, the gyro sensor and/or the compass attached to the wearable device without any external equipment.

The user position estimation unit
an angle correction unit that calculates a correction value of the azimuth angle of the user based on the detection value of the sensor unit of the wearable device worn by the user;
an angle estimation unit that estimates an azimuth angle of the user based on the detection value of the sensor unit of the wearable device worn by the user and the correction value;
a user position estimation unit that estimates the user position using the azimuth angle;
may have

The angle at which the wearable device is worn differs for each user. Therefore, the angles of the sensor axes of the acceleration sensor and the gyro sensor are different for each user. Therefore, the user position estimation unit can estimate the angle of the sensor axis of the sensor unit for each user, and use this as a correction value to estimate the direction (angle) with high accuracy without depending on individual differences.

The user position estimation unit estimates a moving route of the user position,
The location attribute estimation unit may estimate the location attribute after movement based on the movement route.

According to this embodiment, for example, when the user is at the desk during telework, the output can be controlled so that the user can focus on the work, and when the user is at the resting place, the output can be controlled so that the user can relax.

The location attribute estimating unit may store a plurality of movement routes, and may estimate the location attribute after movement by matching the estimated movement route with the plurality of held movement routes.

According to the present embodiment, it is possible to store the pattern of moving places and their order, and specify the place after the user moves from the most recent moving pattern.

The location attribute estimator may output a warning when matching fails a predetermined number of times.
As a result, when the user moves from his/her home to a different indoor space (for example, a coworking space) and continues to detect a movement route that is completely different from the plurality of movement routes held, location attributes after movement can be detected. is estimated from the new moving route.

The location attribute estimation unit may perform the matching using DTW (dynamic time warping).

The location attribute estimation unit may estimate the location attribute by determining the user's stay time at the location where the user is.

By determining the length of stay in addition to the route of travel, it is possible to estimate location attributes with higher accuracy.

further comprising a context acquisition unit that acquires the context of the user;
The user state estimation unit may estimate the user state based on the acquired context.
The context may include at least one of location information of the user and terminal information of the information processing device.

By estimating the user state based not only on location attributes but also on the user's context, it is possible to estimate the user state more accurately.

The user state estimation unit may estimate the user state based on the detected value of the sensor unit of the wearable device and/or the location attribute.

This makes it possible to estimate the user's state more accurately.

The user state may indicate a plurality of activity states of the user.

For example, the user state indicates four levels of activity: break time, neutral, DND (Do Not Disturb), and offline. Break time is the most relaxed activity state, Neutral is the normal activity state, DND is the relatively busy activity state, and Offline is the busiest activity state.

The output control unit is
A content control unit that reproduces content selected based on the environmental state, and/or a notification control unit that controls the number of notifications to the user based on the environmental state.

The content control unit may reproduce content that allows the user to focus and content that allows the user to relax. The notification control unit may reduce or eliminate the number of notifications so that the user can focus, or keep the number of notifications normal if the user is relaxing.

An information processing method according to one aspect of the present disclosure includes:
Estimate the user state,
estimating an environmental state to be presented to a user based on the user state;
The output is controlled based on the environmental conditions.

An information processing program according to one aspect of the present disclosure includes
the processor of the information processing device,
a user state estimation unit that estimates a user state;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
It operates as an output control unit that controls the output based on the environmental conditions.

An information processing system according to one aspect of the present disclosure includes:
wearable devices and
a user state estimation unit that estimates a user state of a user wearing the wearable device;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
an output control unit that controls output based on the environmental state;
an information processing device having
Equipped with

1 shows a configuration of an information processing system according to an embodiment of the present disclosure; 1 schematically shows a worn wearable device; Schematically shows individual differences in wearing wearable devices. The concept of angle correction is shown schematically. 4 shows an operation flow of an angle correction unit; Schematically shows a user's movement. The concept of angle correction is shown schematically. Specific processing of the angle correction unit will be shown. A specific calculation example is shown. Shows the relationship between initial frames. Shows how to specify the natural front. It is a figure for demonstrating the process of a place estimation part. 4 shows an application example of the processing of the location estimator. 4 shows a recognition example of the processing of the place estimating unit. 4 shows an operation flow of a location estimation unit; 4 shows a supplemental operational flow of the location estimator; The following shows the operation when different walking styles are identified for the same route. 4 shows a modification of a method for estimating a place by a place estimating unit; It is a flow for estimating the environmental state presented to the user from the context. The operation of the user state estimator is shown. It shows the mapping relationship between context and user state. Fig. 3 shows how the user state estimator determines the user state. The operation of the environment estimator is shown. 4 shows the operation of the content control unit of the output control unit; The operation of the notification control unit of the output control unit is shown. 1 shows the configuration of a content reproduction system according to the present embodiment; 4 shows an example of a GUI of a preset application. 4 shows an operational flow of a content playback control application; 1 shows an example of a table used to select a content providing application;

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

1. Information processing system configuration

FIG. 1 shows the configuration of an information processing system according to one embodiment of the present disclosure.

The information processing system 10 has an information processing device 100 and a wearable device 200 .

The information processing device 100 is a terminal device used by an end user, such as a smartphone, tablet computer, or personal computer. Information processing apparatus 100 is connected to a network such as the Internet.

The wearable device 200 is a device worn on the user's head. The wearable device 200 is typically a wireless earphone (FIG. 2), but may be a wireless headphone, a wired headphone, a wired earphone, an HMD (Head Mount Display) for AR (Augmented Reality) or VR (Virtual Reality), or the like. There may be. In addition, although FIG. 2 shows an open-ear earphone that does not completely cover the ear canal, it may be a canal-type earphone, a hearing aid, or a sound collector that closes the ear canal. The information processing apparatus 100 and the wearable device 200 are connected to various types of proximity such as Bluetooth (registered trademark) (specifically, BLE (Bluetooth Low Energy) GATT (Generic Attribute Profile)) and Wi-Fi (registered trademark). They are communicably connected to each other by long-distance wireless communication. Wearable device 200 has sensor section 210 . The sensor unit 210 includes an acceleration sensor 211 that detects acceleration, a gyro sensor 212 that detects angular velocity, and a compass 213 that detects azimuth. The sensor unit 210 further includes a biosensor 214 such as a heartbeat sensor, blood flow sensor, electroencephalogram sensor, or the like. The wearable device 200 supplies the detection value of the sensor unit 210 to the information processing device 100 .

The information processing apparatus 100 has a context acquisition unit 110 and a PDR (Pedestrian Dead Reckoning) unit 120 (user position estimating unit), location estimating unit 130 (location attribute estimating unit), user state estimating unit 140, environment estimating unit 150, and output control unit 160.

The context acquisition unit 110 acquires the user's context. The user's context includes location information and terminal information. Here, the context is, for example, a sensor value obtained from the sensor unit 210, user's schedule information obtained from a calendar application, or the like. The context acquisition unit 110 has a device such as a GPS sensor 111 and a beacon transmitter/receiver 112 that acquires location information as a context. Context acquisition section 110 further includes terminal information acquisition section 113 that acquires terminal information as a context. The terminal information acquisition unit 113 acquires screen lock information (locked, unlocked), user behavior information (run, bicycle, stationary, walking, riding, etc.), location (specific location such as home, office, etc.) as terminal information that is context. , unspecified place), calendar application information (scheduled, no meeting), time information (during work time, outside work time), phone application information (during a call), voice recognition application information (during speaking), automatic DND (Do Not Disturb) setting (within time frame, outside time frame), manual DND setting (on, offline), etc.

The PDR section 120 (user position estimation section) estimates the user position based on the detection values (acceleration, angular velocity and azimuth angle) of the sensor section 210 of the wearable device 200 worn by the user. Specifically, PDR section 120 has angle correction section 121 , angle estimation section 122 , and user position estimation section 123 . The angle correction unit 121 calculates a correction value for the user's azimuth angle based on the detection values (acceleration, angular velocity, and azimuth angle) of the sensor unit 210 of the wearable device 200 worn by the user. The angle estimation unit 122 estimates the azimuth angle of the user based on the detection values (acceleration, angular velocity, and azimuth angle) of the sensor unit 210 of the wearable device 200 worn by the user and the correction value. The user position estimation unit 123 estimates the user position using the corrected azimuth angle. PDR (Pedestrian Dead Reckoning) is a technique for positioning relative to a certain reference point based on detection values from a plurality of autonomously operating sensors. In this example, the PDR unit 120 detects changes in the user position from room to room, that is, movement of the user position, based on acceleration, angular velocity, and azimuth angle detected by the acceleration sensor 211, gyro sensor 212, and compass 213. Estimate a route.

The location estimation unit 130 (location attribute estimation unit) estimates the attribute of the user's location (location attribute) based on the change in the user's position estimated by the PDR unit 120 . In other words, based on the moving route estimated by the PDR unit 120, the location attribute after the user moves is estimated. A location attribute is, for example, a division within a building that is even finer than the building itself. For example, the location attribute is living room, bedroom, toilet, kitchen, washroom, etc. within one house. Alternatively, the location attribute is a desk, conference room, etc. within one co-working space. However, the location attribute is not limited to this, and the location attribute may indicate the building itself or the like, or may indicate both the building itself and the section within the building.

User state estimation unit 140 is based on the context acquired by context acquisition unit 110, detection values (acceleration, angular velocity, and azimuth angle) of sensor unit 210 of wearable device 200, and location attributes estimated by location estimation unit 130. , to estimate the user state. A user state indicates a user's multi-level activity state. For example, the user state indicates four levels of activity: break time, neutral, DND (Do Not Disturb) and offline. Break time is the most relaxed activity state, Neutral is the normal activity state, DND is the relatively busy activity state, and Offline is the busiest activity state. In addition to the four levels described above, it may be possible to set an arbitrary number of levels on the system, or allow the user to set the number of levels as appropriate.

The environment estimation unit 150 estimates the environmental state to be presented to the user based on the user state estimated by the user state estimation unit 140 . The environment estimation unit 150 may further estimate the environmental state presented to the user based on the location attributes estimated by the location estimation unit 130 . The environmental state presented to the user is, for example, an environmental state in which the user can focus (concentrate) or an environmental state in which the user can relax.

The output control unit 160 controls output based on the environmental state estimated by the environment estimation unit 150 . Specifically, the output control unit 160 has a content control unit 161 and a notification control unit 162 . The content control unit 161 reproduces content (music, video, etc.) selected based on the environmental state estimated by the environment estimation unit 150 . For example, the content control unit 161 notifies the DSP (Digital Service Provider) of the environmental state via the network, and the DSP selects content based on this environmental state (for example, content that the user can focus on, content that the user can relax, etc.) content, etc.) may be received and reproduced. The notification control unit 162 controls the number of notifications to the user based on environmental conditions. For example, the notification control unit 162 reduces or eliminates the number of notifications (e.g., notifications of new arrivals of applications or messages) so that the user can focus, or sets the number of notifications to normal if the user is relaxing. may be processed.

2. Operation of the angle correction section of the PDR section

Fig. 2 schematically shows the worn wearable device.

The wearable device 200 is typically a wireless earphone. A wearable device 200, which is a wireless earphone, has a speaker 221, a driver unit 222, and a sound conduit 223 connecting them. The speaker 221 is inserted into the ear canal to position the wearable device 200 against the ear, and the driver unit 222 is located behind the ear. A sensor section 210 including an acceleration sensor 211 and a gyro sensor 212 is built in a driver unit 222 .

Fig. 3 schematically shows individual differences in wearable devices worn.

The angle of the driver unit 222 of the wearable device 200 with respect to the front of the face differs for each user. Therefore, the angles of the sensor axes of the acceleration sensor 211 and the gyro sensor 212 of the sensor unit 210 built in the driver unit 222 with respect to the front of the face differ for each user. For example, (a) shows the case where the user wears the wearable device 200 shallowly hooked on the ear, and (b) shows the case where the user wears the wearable device 200 deeply fixed to the ear. The difference between the angle of the user's sensor axis with respect to the front face of (a) and the angle of the user's sensor axis with respect to the front of the face of (b) may be 30° or more. Therefore, the PDR unit 120 estimates the angle of the sensor axis of the sensor unit 210 with respect to the front of the face for each user, and uses this as a correction value to accurately estimate the orientation (angle) of the face without depending on individual differences.

FIG. 4 schematically shows the concept of angle correction.

In the azimuth angle (azimuth) of the wearable device 200, the updated value Azimuth _E based on the sensor value acquired by the sensor unit 210 and the difference Azimuth _Offset in the orientation from the front of the face when worn are expressed as "Azimuth = Azimuth _E + Azimuth _Offset ” has a relationship. Here, Azimuth _E is obtained from the three-dimensional posture obtained by integrating sensor values obtained by the gyro sensor 212 that detects angular velocity. On the other hand, the Azimuth _Offset differs for each user and cannot be measured just by wearing the device, so it is necessary to estimate the Azimuth _Offset for each user.

To estimate the pose, we define two coordinate systems with both ears horizontal. Coordinate system (1) is a global frame (fixed), and is composed of a vertical Z-axis extending overhead, an X-axis connecting both ears and positive in the right direction, and a Y-axis orthogonal to the X-axis and Z-axis. is a coordinate system. A coordinate system (2) is a sensor frame, and is a coordinate system (X _E , Y _E , Z _E ) that is fixed with respect to the sensor unit 210 of the wearable device 200 . Azimuth _Offset , which is a correction value, indicates the amount of rotation of the coordinate system (2) with respect to the coordinate system (1).

FIG. 5 shows the operation flow of the angle corrector. FIG. 6 schematically shows user movements. FIG. 7 schematically shows the concept of angle correction. FIG. 8 shows specific processing of the angle corrector. FIG. 9 shows a specific calculation example.

The user wears the wearable device 200 and moves the head downward so as to look diagonally downward from the front ((a) of FIG. 6) ((b) of FIG. 6) (step S101). The angle correction unit 121 calculates Pitch and Roll with respect to the global frame coordinate system (X, Y, Z) from the acceleration value when moving the head downward (step S102). The angle correction unit 121 starts collecting angular velocity values of the gyro sensor 212 . Let the time at this time be t0 (step S103) (process (2) in FIG. 8). Next, the user slowly moves his or her head up so as to look up diagonally from the front without blurring left and right ((c) in FIG. 6) (step S104). The angle correction unit 121 continues collecting angular velocity values of the gyro sensor 212 (step S105). When the user raises his or her head to the limit, the angle corrector 121 stops collecting the angular velocity values of the gyro sensor 212 . The time at this time is set to t1 (step S106, YES).

The angle correction unit 121 obtains the rotation axis [α _X , α _y , α _z , ] ^T from the collected angular velocity values of the gyro sensor 212 . This rotation axis is based on the sensor axis. Next, the angle correction unit 121 defines a rotation matrix (RotMat) at t0 as "RotMat at t0=R _Z (yaw)*R _X (pitch)*R _Y (roll)". This RotMat is based on the front face. R _Z (·), R _X (·), and R _Y (·) are the rotation matrices of the Z-axis, Y-axis, and X-axis, respectively. Pitch and roll with reference to the front of the face can be obtained from the accelerometer, but yaw is unknown. The angle correction unit 121 can calculate yaw from the relationship of RotMat*axis=[1;0;0] (process (4) in FIG. 8). First, _RotMat *axis is set to [ _rX ,ry, _rz ] ^T (step S107). If r _Z deviates from the threshold value (if the difference from 0 is large), the angle correction unit 121 fails and redoes the process (step S108, NO). If r _Z is within the threshold, the process proceeds to the next step (step S108, YES). The angle corrector 121 obtains a correction value (Azimuth _Offset ) from _rX and _rY (step S109) (process (5) in FIG. 8). The angle correction unit 121 obtains a rotation matrix (RotMat) from Azimuth _Offset , Pitch and Roll (step S110). This RotMat is based on the front face axis.

Next, I will explain how to specify the natural front instead of the face looking down.

FIG. 10 shows the relationship between initial frames.

Let the initial posture with the head lowered (Head Center Pose) be I _3x3 (Identity Matrix). Let R _t0 (RotMat in FIG. 8) be the posture of the sensor (Right Sensor Pose assuming that it is attached to the right ear).

Fig. 11 shows a method of specifying a natural front view.

If R _t0 which is the posture of the right sensor (Right Sensor Pose) is obtained by the method of FIG. The relational expression _Rt2 in the new attitude can be obtained from _Rt0 and the acceleration sensor value in the new attitude by the method of FIG.

3. How the location estimator works

FIG. 12 is a diagram for explaining the processing of the location estimation unit.

The user moves daily while wearing the wearable device 200 in the house. The place estimating unit 130 stores the movement pattern and order. In many cases, a finite number of places in a house where a user moves and the order in which they move are determined. The place estimation unit 130 identifies a place from the most recent N (for example, N=3) movement patterns.

In FIG. 12, (1) is the route from the living room to the bedroom, (2) is the route from the bedroom to the living room, (3) is the route from the living room to the toilet, and (4) is A route from the toilet to the living room, (5) from the living room to the kitchen, and (6) from the kitchen to the living room.
For example, the user wears the wearable device 200 and starts working in the living room. After a while, after going to the toilet, I returned to my seat after washing my hands in the washroom. After a while, I moved to the kitchen, got a drink, and returned to the living room. The movement pattern here is as follows. From the living room to the toilet (route (3)). From the toilet to the living room (route (4)). From the living room to the kitchen (route (5)). From the kitchen to the living room (route (6)).

The place estimation unit 130 stores these four patterns and their order. The next time the user moves, the movement pattern is matched with the stored pattern. If the matching is successful, the place estimating unit 130 can specify the post-movement place, and if the matching is unsuccessful, the place estimating unit 130 adds it to the route list as a new pattern. The route list (on the right side of FIG. 12) is a learned list of the most recent N (eg, N=3) movement patterns. For example, the route list includes movement patterns (top row) of "(1) living room to bedroom, (2) bedroom to living room, (5) living room to kitchen", and "(2) bedroom to living room, (5) living room The most recent N (for example, N=3) movement patterns are registered such as "from kitchen to kitchen, (6) from kitchen to living room" (second row from the top).

In this way, the location estimation unit 130 holds a plurality of movement routes, and matches the movement routes estimated by the PDR unit 120 with the plurality of held movement routes to obtain location attributes after movement (living room , bedroom, toilet, kitchen, washroom, etc.) can be estimated. Also, the location estimation unit 130 may estimate location attributes by determining how long the user stays at the location where the user is. By determining the staying time in addition to the moving route, the location attribute can be estimated more accurately.

FIG. 13 shows an application example of the processing of the location estimation unit.

Even if the starting point and destination point are the same, if the walking style is different, matching may not be possible, and learning is performed by adding it to the memory pattern. Therefore, multiple patterns are learned even when moving between the same locations. The coordinate system of FIG. 13 shows the transition of the user position with the origin as the starting point and the user position plotted periodically (eg, every second) as it progresses from the origin (starting point) to another room. The axis (1) indicates the moving route from the living room (origin) to the bedroom. The axis (2) indicates the movement path (distance) from the bedroom (origin) to the living room. The axis (3) indicates the moving route from the living room (origin) to the toilet. The axis (4) indicates the moving route from the toilet (origin) to the living room.

FIG. 14 shows a recognition example of processing by the location estimation unit.

The location estimation unit 130 attaches labels indicating attributes when learning routes. As a result, the label indicating the attribute can be automatically displayed when the matching is successful. Next, the operation of the location estimation unit 130 will be described more specifically.

FIG. 15 shows the operation flow of the location estimation unit.

The PDR unit 120 estimates the change of the user position from room to room, that is, the movement route of the user position (step S201). The place estimating unit 130 detects that the user has stopped based on the change in the user's position detected and estimated by the PDR unit 120 (step S202, YES). The location estimation unit 130 increments (+1) the stop counter (step S203). When there are N (eg, N=3) or more movements from room to room (step S204, YES), the location estimating unit 130 holds the most recent N (eg, N=3) routes. Matching is performed with a plurality of moving routes (step S205). If the matching is successful (step S206, YES), the place estimating unit 130 identifies the post-movement place (step S207). On the other hand, if the matching fails (step S206, NO), the location estimating unit 130 adds it to the route list as a new pattern (step S208).

FIG. 16 shows a supplementary operation flow of the location estimation unit.

By the way, it is conceivable that the user moves from his/her home to a different indoor space (for example, a coworking space) and continues to detect a movement route that is completely different from the multiple movement routes held. In this case, the location estimation unit 130 continues to fail in matching (step S206, NO) for a while (step S209, YES). On the other hand, if enough new travel routes are accumulated in the route list to the extent that matching is successful (step S208), matching is successful (step S206, YES), and the location after travel can be identified ( step S207). When the matching failure continues for a predetermined number of times (step S209, YES), the place estimation unit 130 outputs a warning indicating that there is a possibility of another place not registered in the route list (step S210). This makes it possible to notify the user that the location attribute after movement will be estimated from the new movement route.

FIG. 17 shows the operation when different walking styles are identified for the same route.

As mentioned above, if the starting point and destination point are the same but the walking style is different, matching may not be possible, and learning is performed by adding it to the memory pattern. Here's how. The distances between the latest N paths and the N patterns stored in the database are calculated by DTW (dynamic time warping) and compared with a threshold. DTW (dynamic time warping) is a technique used to measure the distance and similarity between time-series data. A different walking style may exceed the DTW threshold, in which case it is stored as separate data.

FIG. 18 shows a modification of the method for estimating the location by the location estimating unit.

The location estimation unit 130 may estimate the attribute of the location where the user is located (location attribute), especially outdoors, based on the location information acquired by the GPS sensor 111 and the beacon transmitter/receiver 112 . The place estimation unit 130 may estimate the attribute of the place where the user is (place attribute) based on the biometric information acquired by the biosensor 214 . For example, if it is known that the user is falling asleep based on the biometric sensor 214 (heartbeat sensor or the like), the location estimation unit 130 may estimate the bedroom as the location attribute.

4. Operation of User State Estimator

FIG. 19 is a flow for estimating the environmental state presented to the user from the context.

The context acquisition unit 110 acquires the user's context. User state estimation unit 140 is based on the context acquired by context acquisition unit 110, detection values (acceleration, angular velocity, and azimuth angle) of sensor unit 210 of wearable device 200, and location attributes estimated by location estimation unit 130. , to estimate the user state. Based on the user state estimated by the user state estimation unit 140, the environment estimation unit 150 estimates the environmental state (focus (concentration), relaxation, etc.) to be presented to the user.

FIG. 20 shows the operation of the user state estimation unit.

User state estimation unit 140 is based on the context acquired by context acquisition unit 110, detection values (acceleration, angular velocity, and azimuth angle) of sensor unit 210 of wearable device 200, and location attributes estimated by location estimation unit 130. , to estimate the user state. The user's context includes location information and terminal information. Terminal information includes screen lock information (lock, unlock), user behavior information (run, bicycle, stationary, walking, riding, etc.), location (specific location such as home or office, unspecified location), calendar application information ( Scheduled meeting, no meeting), time information (during work time, outside work time), phone application information (during a call), voice recognition application information (during speaking), automatic DND (Do Not Disturb) setting (during time frame, time out of frame), manual DND settings (on, offline), etc. A user state indicates a user's multi-level activity state. For example, the user state indicates four levels of activity: break time, neutral, DND (Do Not Disturb) and offline. Break time is the most relaxed activity state, Neutral is the normal activity state, DND is the relatively busy activity state, and Offline is the busiest activity state.

FIG. 21 shows the mapping relationship between context and user state.

The user state estimation unit 140 estimates the user state by mapping the context to the user state. For example, if the screen lock information as the context is unlocked, the user state estimation unit 140 estimates that the user state is DND, and if the screen lock information is locked, the user state is estimated to be neutral. The user state estimating unit 140 also estimates user states for other contexts. Also, the context is not limited to that shown in FIG. 21, and any context may be used as long as it represents some kind of context.

FIG. 22 shows how the user state estimation unit determines the user state.

The user state estimation unit 140 estimates the user state as offline if even one of the contexts includes offline. The user state estimation unit 140 estimates the user state as DND if there are no offline contexts and at least one context includes DND. The user state estimation unit 140 estimates the user state as neutral if there is no offline, DND and break time for a plurality of contexts. The user state estimating unit 140 estimates the user state as the break time if there is no offline or DND and the break time is included.

5. Operation of environment estimator

FIG. 23 shows the operation of the environment estimation unit.

The environment estimation unit 150 estimates the environmental state to be presented to the user based on the user state estimated by the user state estimation unit 140 and the location attribute estimated by the location estimation unit 130 . The environmental state presented to the user is, for example, an environmental state in which the user can focus (concentrate) or an environmental state in which the user can relax.

For example, (1) the environment estimating unit 150 estimates that the environmental state presented to the user is the focus when the time period is at work, the user state is neutral, the action is stay, and the location is desk. (2) If the time zone is working and the user state is break time, the environment estimation unit 150 estimates that the environmental state to be presented to the user is relaxed. (3) If the time zone is non-work and the user state is break time, the environment estimation unit 150 estimates that the environmental state to be presented to the user is relaxed.

6. Operation of the output controller

FIG. 24 shows the operation of the content control section of the output control section.

The content control unit 161 of the output control unit 160 reproduces content (music, video, etc.) selected based on the environmental state estimated by the environment estimation unit 150 . For example, the content control unit 161 notifies the DSP (Digital Service Provider) of the environmental state via the network, and the DSP selects content based on this environmental state (content that allows the user to focus, content that allows the user to relax). content) is received and played back. For example, if the user is at work and the user state is focused, the content control unit 161 plays music that helps the user concentrate, and if the user state is relaxed, the content control unit 161 plays music that helps the user relax. For example, when the user falls asleep, the content control unit 161 reproduces sleep-promoting music if the user state is relaxed, and stops the music when the user falls asleep.

FIG. 25 shows the operation of the notification control section of the output control section.

The notification control unit 162 of the output control unit 160 controls the number of notifications to the user based on the environmental conditions. For example, the notification control unit 162 may reduce or eliminate the number of notifications (notifications of new arrivals of applications or messages) so that the user can focus, or may keep the number of notifications normal if the user is relaxing. For example, if the user is at work and the user state is focused, the notification control unit 162 reduces the number of notifications, and if the user state is relaxed, the notification control unit 162 issues the normal number of notifications.

7. Brief Summary

There is technology that recognizes speech and environmental sounds, and selects and outputs content such as music based on the recognized sounds. Technology that recognizes utterances and environmental sounds is applicable only to environments with sounds. Therefore, a user who does not want to make noise or a situation where he/she does not want to make noise may not be able to select appropriate content. Also, natural language processing requires high computational power, making it difficult to process locally.

In contrast, according to the present embodiment, it is possible to output content that encourages focus (concentration) and relaxation based on the user's location in the house and other user contexts. It is possible to appropriately control the output to the user regardless of the situation such as a situation where it is desired not to make a sound. For example, based on user context, if the user is at their desk while teleworking, we can output focusable content, and if they are at their resting place, we can play relaxing music.

　In contrast to outdoors, indoor spaces are relatively small, and external equipment such as high-precision beacons and cameras are generally required to estimate a specific position. On the other hand, according to the present embodiment, it is possible to identify the position inside the house using the sensor unit 210 (the acceleration sensor 211, the gyro sensor 212, and the compass 213) attached to the wearable device 200 without any external equipment. can. Specifically, by storing the pattern of moving places and their order, it is possible to identify the place after the user moves from the N patterns of the most recent moves.

Telework has become commonplace, and users are spending more time at home, not only relaxing, but also focusing on work. At this time, it is thought that there are more users who do not want to make noise and situations in which they do not want to make noise than in the past when telework was not widespread. Therefore, as in the present embodiment, it will be more and more useful to specify the location in the house, estimate the environmental state to be presented to the user, and control the output to the user without the need to speak. .

Also, according to this embodiment, the user state is estimated by mapping the context obtained from each sensor information to the user state, so the user state can be estimated without speaking and making a sound. According to this embodiment, since the context obtained from each sensor information is mapped to the user state, the amount of calculation is much smaller than that of natural language processing, and local processing is easy.

8. Content playback system

FIG. 26 shows the configuration of a content reproduction system according to this embodiment.

The content reproduction system 20 has an information processing device 100 and a wearable device 200 .

The information processing apparatus 100 loads a content reproduction control application 300, a content providing application 400, and a preset application 500, in which a processor such as a CPU of a control circuit is recorded in a ROM, into a RAM and executes them. Note that the content reproduction control application 300 may be installed in the wearable device 200 instead of the information processing apparatus 100 and executed by the wearable device 200 .

The wearable device 200 is, as described above, wireless earphones (see FIG. 2), wireless headphones, wired headphones, wired earphones, or the like. The wearable device 200 has a sensor section 210 and an input device 220 . The sensor unit 210 includes an acceleration sensor 211, a gyro sensor 212, a compass 213, and a biosensor 214 such as a heart rate sensor, a blood flow sensor, an electroencephalogram sensor (see FIG. 1). Wearable device 200 inputs the detection value of sensor unit 210 to content reproduction control application 300 and content providing application 400 . The input device 220 is a touch sensor, a physical button, a non-contact sensor, or the like, and inputs a contact or non-contact operation by the user. The input device 220 is provided on the outer surface of the driver unit 222 (see FIG. 2) of the wearable device 200, for example.

The content providing application 400 provides content. A content providing application 400 is an application group including a plurality of different content providing applications 401 and 402 . For example, a plurality of different content providing applications 401 and 402 respectively provide content (specifically, audio content) of different genres such as music, environmental sounds, healing sounds, and radio programs. The content providing application 400 is simply referred to when the different content providing applications 401 and 402 are not distinguished.

The content reproduction control application 300 includes the context acquisition unit 110, the PDR (Pedestrian Dead Reckoning) unit 120 (user position estimation unit), the location estimation unit 130 (location attribute estimation unit), and the user state estimation unit 140. , the environment estimation unit 150, and the content control unit 161 of the output control unit 160 (see FIG. 1). The content control unit 161 selects the content providing application 400 based on the environmental state estimated by the environment estimation unit 150 or based on different operations input by the user to the input device 220 of the wearable device 200 . The content control unit 161 generates a cue for the content providing application 400 to select content based on the environmental state, outputs the generated cue to the selected content providing application 400, and instructs the content providing application 400 to provide the content based on the cue. The content is reproduced from the wearable device 200 by making the selection.

The preset application 500 pre-assigns a plurality of different operations input by the user to the input device 220 of the wearable device 200 to a plurality of different functions related to services provided by the content providing application 400 . For example, the preset application 500 pre-assigns a selection of different content providing applications 401,402. A plurality of different operations input by the user to the input device 220 of the wearable device 200 (for example, single-tap, double-tap, triple-tap, press of a radio button, etc.) are assigned in advance to selection of a plurality of different content providing applications 401 and 402. . Preset application 500 may be independent of content reproduction control application 300 or may be included in content reproduction control application 300 .

FIG. 27 shows an example of the GUI of the preset application.

The preset application 500 has, for example, a playback control GUI 710, a volume control GUI 720, and a quick access control GUI 730. Note that the GUI provided by the preset application 500 and the combination of settable functions and operations differ depending on the model of the wearable device 200 .

The user can use the playback control GUI 710 to assign a plurality of different operations input by the user to the input devices 220 of the left and right wearable devices 200 to each function during content playback. For example, the user assigns a single-tap operation of the wearable device 200 on the right side to play and pause, assigns a double-tap operation to play the next song, assigns a triple-tap operation to play the previous song, and assigns a long press operation to the voice assistant. Can be assigned to activate a function. Note that the functions assigned to each operation may be functions other than those described above, and the functions may be assigned to each operation by default.

The user can use the volume control GUI 720 to assign a plurality of different operations that the user inputs to the input devices 220 of the left and right wearable devices 200 to each function of the volume control. For example, the user can assign a single-tap operation of the left wearable device 200 to volume up and a long press operation to volume down.

The user uses the quick access control GUI 730 to convert a plurality of different operations that the user inputs to the input devices 220 of the left and right wearable devices 200 into a quick access function that selects and activates a plurality of different content providing applications 401 and 402. can be assigned. For example, the user can assign a double tap operation on the left wearable device 200 to launch the content providing application 401 and a triple tap operation to launch the content providing application 402 .

In this way, the preset application 500 can perform a plurality of different operations input by the user to the input devices 220 of the left and right wearable devices 200 not only through playback control and volume control while the content providing application 400 is running, but also through the content providing application 400 . can be assigned to the selection and activation of

FIG. 28 shows the operational flow of the content playback control application.

In the content reproduction control application 300, the context acquisition unit 110 acquires the user's context. User state estimation unit 140 is based on the context acquired by context acquisition unit 110, detection values (acceleration, angular velocity, and azimuth angle) of sensor unit 210 of wearable device 200, and location attributes estimated by location estimation unit 130. , to estimate the user state (four-level activity state: break time, neutral, DND (Do Not Disturb) and offline). Here, as an example, there are four levels of user states, but there may be more or less. Alternatively, the user may be allowed to set any number of user states. Based on the user state estimated by the user state estimation unit 140, the environment estimation unit 150 estimates the environmental state (focus (concentration), relaxation, etc.) to be presented to the user (see FIG. 19). The content control unit 161 of the output control unit 160 detects an appropriate timing to start reproducing content based on the environmental state estimated by the environment estimation unit 150 (step S301).

In the content reproduction control application 300 , the content control unit 161 of the output control unit 160 selects the content providing application 400 . For example, the content control unit 161 selects the content providing application 400 based on different operations input by the user to the input device 220 of the wearable device 200 . For example, the content control unit 161 selects the content providing application 401 if the operation input by the user to the input device 220 of the wearable device 200 is a double tap, and selects the content providing application 402 if it is a triple tap. Alternatively, the content control unit 161 selects the content providing application 400 based on the environmental state (scenario described later) estimated by the environment estimation unit 150 (step S302). Alternatively, the content control unit 161 can be set by the user (for example, by setting the content providing application 400 in advance according to the situation) such that the scenario will not fire even under the same conditions if the refusal is repeated. Based on this, the content providing application 400 may be selected.

FIG. 29 shows an example of a table used for selecting content providing applications.

For example, the content control unit 161 refers to the table 600 and selects the content providing application 400 . Table 600 has ID 601 , scenario 602 , user context 603 and queue 604 . A scenario 602 corresponds to the environmental state estimated by the environment estimation unit 150 . The user context 603 corresponds to the user state estimated by the user state estimation unit 140 based on the user's context acquired by the context acquisition unit 110 . A queue 604 is a queue for the content providing application 400 to select content. In table 600, selection flag 605 of content providing application 401 and selection flag 606 of content providing application 402 are recorded in nine records of Music_01 to 09 with ID 601, respectively. A record in which only the selection flag 605 is recorded means that the content providing application 401 is selected in the scenario 602 (environmental state). On the other hand, both of the selection flags 605 and 606 mean that either one of the content providing applications 401 and 402 is selected under different conditions in the scenario 602 (environmental state). For example, the content control unit 161 may learn in advance and select the content providing application 400 that is frequently executed at the current time, the content providing application 400 that is frequently used, and the like.

In the content reproduction control application 300, the content control unit 161 of the output control unit 160 generates a cue 604 for the selected content providing application 400 to select content based on the scenario 602 (environmental state) (step S303). . The content control unit 161 outputs the generated cue to the selected content providing application 400, causes the content providing application 400 to select content based on the cue, and reproduces the content from the wearable device 200 (step S304). For example, the content providing application 400 selects a plurality of content candidates based on the cue from the content reproduction control application 300, and based on the detected value input from the sensor unit 210 of the wearable device 200, the content to be reproduced from the plurality of candidates. may be selected. Also, the content providing application 400 may select content with a fast tempo that matches the user's running speed based on the detected value input from the sensor unit 210 .

After starting reproduction, the content control unit 161 of the content reproduction control application 300 detects the timing to start reproducing another content based on the environmental state (step S301), selects the content providing application 400 (steps S302, This step can be omitted), the queue 604 is generated (step S303), and the content is reproduced from the wearable device 200 (step S304). In other words, the content reproduction control application 300 has user information (that is, user context 603 (user state), scenario 602 (environmental state)) that the content providing application 400 cannot know. Therefore, the content reproduction control application 300 can know cases where it is desirable to change the content being reproduced by the content providing application 400 . For example, it is possible to induce a change in the user's feelings by changing the content that is being reproduced using the fact that the user is commuting or the fact that he or she has finished work as a trigger. When the content reproduction control application 300 knows (that is, the user context 603 (user state) and the scenario 602 (environmental state)), it sends a cue to the content providing application 400 to change the content being reproduced. By transmitting, it is possible to provide the user with more desirable contents (music, healing sounds, etc.).

Furthermore, the content control unit 161 of the content reproduction control application 300 generates a cue for the content providing application 400 to stop (rather than change) the reproduction of the content based on the scenario 602 (environmental state) (step S303). is output to the content providing application, and the content providing application 400 is caused to stop the reproduction of the content based on the cue (step S304). For example, there are cases where it is better to stop the music due to a state change such as the start of a meeting. The content playback control application 300 detects these states and sends a stop command to the content providing application 400 .

Also, during content reproduction, the content providing application 400 generates content with a fast tempo that matches the running speed of the user based on the detected values input from the sensor unit 210, for example, according to predetermined values of heart rate and acceleration. You can select and play. In other words, during content reproduction, the content providing application 400 actively reproduces the content based on the detection values input from the sensor unit 210 without receiving a cue from the content control unit 161 of the content reproduction control application 300. Attributes of content (tempo, pitch, etc.) can be selected and the selected content can be played back. In short, during content playback, the content providing application 400 can actively change the content to be played back.

9. Conclusion

According to the content reproduction system 20 according to this embodiment, the content reproduction control application 300 selects the content providing application 400 and outputs a cue to the content providing application 400 . Therefore, it is not necessary for the content providing application 400 to consider content reproduction conflicts between a plurality of different content providing applications 401 and 402 .

Also, the content reproduction control application 300 generates a cue for the content providing application 400 to select content based on the environmental state, which is the user's sensitive information. Therefore, the content providing application 400 does not share the environmental state, which is the user's sensitive information, with the content providing application 400 from the content reproduction control application 300. The reflected content can be played back. Therefore, it is possible to improve the user experience while reducing the security risk.

Also, the content reproduction control application 300 selects the content providing application 400, and the selected content providing application 400 reproduces the content. Furthermore, the preset application 500 allows the content reproduction control application 300 to select the content providing application 400 based on different operations input by the user to the input device 220 of the wearable device 200 . This makes it possible to provide a user experience that integrates the services of a plurality of different content providing applications 401 and 402 without requiring active selection by the user.

The present disclosure may have the following configurations.

(1)
a user state estimation unit that estimates a user state;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
an output control unit that controls output based on the environmental state;
An information processing device comprising:
(2)
The information processing device according to (1) above,
a user position estimating unit that estimates a user position based on a detection value of a sensor unit of the wearable device worn by the user;
a location attribute estimating unit that estimates a location attribute, which is an attribute of a location where the user is located, based on the user location;
further comprising
The information processing apparatus, wherein the user state estimation unit estimates the user state based on the location attribute.
(3)
The information processing device according to (2) above,
The information processing apparatus, wherein the user position estimation unit estimates the user position using PDR (Pedestrian Dead Reckoning).
(4)
The information processing device according to (2) or (3) above,
The information processing apparatus, wherein the environment estimation unit estimates the environmental state based on the location attribute.
(5)
The information processing device according to any one of (2) to (4) above,
The information processing apparatus, wherein the sensor unit of the wearable device includes at least one of an acceleration sensor, a gyro sensor, a compass, and a biosensor.
(6)
The information processing device according to any one of (3) to (5) above,
The user position estimation unit
an angle correction unit that calculates a correction value of the azimuth angle of the user based on the detection value of the sensor unit of the wearable device worn by the user;
an angle estimation unit that estimates an azimuth angle of the user based on the detection value of the sensor unit of the wearable device worn by the user and the correction value;
a user position estimation unit that estimates the user position using the azimuth angle;
An information processing device.
(7)
The information processing device according to any one of (3) to (6) above,
The user position estimation unit estimates a moving route of the user position,
The information processing apparatus, wherein the location attribute estimation unit estimates the location attribute after movement based on the movement route.
(8)
The information processing device according to (7) above,
The location attribute estimation unit stores a plurality of movement routes, and estimates the location attribute after movement by matching the estimated movement route with the plurality of held movement routes.
(9)
The information processing device according to (8) above,
The information processing apparatus, wherein the location attribute estimation unit outputs a warning when matching fails a predetermined number of times.
(10)
The information processing device according to (8) or (9) above,
The information processing device, wherein the location attribute estimation unit performs the matching using DTW (dynamic time warping).
(11)
The information processing device according to any one of (1) to (10) above,
The information processing apparatus, wherein the location attribute estimation unit estimates the location attribute by determining the length of stay of the user at the location where the user is.
(12)
The information processing device according to any one of (1) to (11) above,
further comprising a context acquisition unit that acquires the context of the user;
The information processing apparatus, wherein the user state estimation unit estimates the user state based on the acquired context.
(13)
The information processing device according to (12) above,
The context includes at least one of location information of the user and terminal information of the information processing device.
(14)
The information processing device according to any one of (1) to (13) above,
Information processing apparatus, wherein the user state estimation unit estimates the user state based on the detection value of the sensor unit of the wearable device and/or the location attribute.
(15)
The information processing device according to any one of (1) to (14) above,
The information processing apparatus, wherein the user state indicates a plurality of activity states of the user.
(16)
The information processing device according to any one of (1) to (15) above,
The output control unit is
An information processing apparatus comprising: a content control unit that reproduces content selected based on the environmental state; and/or a notification control unit that controls the number of notifications to the user based on the environmental state.
(17)
Estimate the user state,
estimating an environmental state to be presented to a user based on the user state;
controlling output based on the environmental conditions;
Information processing methods.
(18)
the processor of the information processing device,
a user state estimation unit that estimates a user state;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
An information processing program operated as an output control unit that controls output based on the environmental state.
(19)
wearable devices and
a user state estimation unit that estimates a user state of a user wearing the wearable device;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
an output control unit that controls output based on the environmental state;
an information processing device having
An information processing system comprising
(20)
the processor of the information processing device,
a user state estimation unit that estimates a user state;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
A non-transitory computer-readable recording medium recording an information processing program operated as an output control unit that controls output based on the environmental state.

Furthermore, the present disclosure may have the following configurations.

(1)
wearable devices and
a user state estimation unit that estimates a user state of a user wearing the wearable device;
an environment estimation unit that estimates an environmental state of the user based on the user state;
A content providing application that provides content generates a cue for selecting content based on the environmental state, outputs the cue to the content providing application, and causes the content providing application to select content based on the cue. a content control unit that reproduces the content;
a content playback control application having
an information processing device having a control circuit that executes
A content playback system comprising:
(2)
The content reproduction system according to (1) above,
the control circuit of the information processing device executes a plurality of different content providing applications;
The content reproduction system, wherein the content control unit selects a predetermined content providing application for reproducing the content based on the environmental state.
(3)
The content reproduction system according to (1) or (2) above,
the control circuit of the information processing device executes a plurality of different content providing applications;
The wearable device has an input device,
The content reproduction system, wherein the content control unit selects a predetermined content providing application for reproducing the content based on different operations input by a user to the wearable device.
(4)
The content reproduction system according to any one of (1) to (3) above,
The content reproduction system, wherein the control circuit of the information processing device executes a preset application that assigns the plurality of different operations to selection of the plurality of different content providing applications.
(5)
The content reproduction system according to (4) above,
The content reproduction system, wherein the preset application is included in the content reproduction control application.
(6)
The content reproduction system according to any one of (1) to (5) above,
The wearable device has a sensor unit,
The content playback control application is
a user position estimation unit that estimates a user position based on a detection value input from a sensor unit of the wearable device worn by the user;
a location attribute estimating unit that estimates a location attribute, which is an attribute of a location where the user is located, based on the user location;
further having
The content reproduction system, wherein the user state estimation unit estimates the user state based on the location attribute.
(7)
The content reproduction system according to (6) above,
The content reproduction system, wherein the sensor unit of the wearable device includes at least one of an acceleration sensor, a gyro sensor, a compass, and a biosensor.
(8)
The content reproduction system according to (6) or (7) above,
The content providing application selects a plurality of content candidates based on the cue, and selects content to be played back from the plurality of candidates based on the detection value input from the sensor unit.
(9)
The content reproduction system according to any one of (6) to (8) above,
A content reproduction system wherein the content providing application selects attributes of content to be reproduced based on the detection value input from the sensor unit and reproduces the selected content during reproduction of the content.
(10)
The content reproduction system according to any one of (1) to (9) above,
The content control unit generates a cue for the content providing application to stop playing the content based on the environmental state, outputs the cue to the content providing application, and instructs the content providing application to stop the reproduction of the content based on the cue. A content reproduction system that stops the reproduction of the content.
(11)
The content reproduction system according to any one of (1) to (10) above,
The content playback control application is
further comprising a context acquisition unit that acquires the context of the user;
The content reproduction system, wherein the user state estimation unit estimates the user state based on the acquired context.
(12)
a user state estimation unit that estimates a user state of a user wearing the wearable device;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
A content providing application that provides content generates a cue for selecting content based on the environmental state, outputs the cue to the content providing application, and causes the content providing application to select content based on the cue. a content control unit that reproduces the content;
a content playback control application having
An information processing device comprising a control circuit for executing
(13)
The control circuit of the information processing device,
a user state estimation unit that estimates a user state of a user wearing the wearable device;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
A content providing application that provides content generates a cue for selecting content based on the environmental state, outputs the cue to the content providing application, and causes the content providing application to select content based on the cue. A content reproduction control application that operates as a content control unit that reproduces the content.
(14)
The control circuit of the information processing device,
a user state estimation unit that estimates a user state of a user wearing the wearable device;
an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
A content providing application that provides content generates a cue for selecting content based on the environmental state, outputs the cue to the content providing application, and causes the content providing application to select content based on the cue. A non-transitory computer-readable recording medium recording a content reproduction control application that operates as a content control unit that reproduces the content.

Although the embodiments and modifications of the present technology have been described above, the present technology is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology. Of course.

10 information processing system 100 information processing device 110 context acquisition unit 111 GPS sensor 112 beacon transceiver 113 terminal information acquisition unit 120 PDR unit 121 angle correction unit 122 angle estimation unit 123 user position estimation unit 130 location estimation unit 140 user state estimation unit 150 Environment estimation unit 160 Output control unit 161 Content control unit 162 Notification control unit 200 Wearable device 210 Sensor unit 211 Acceleration sensor 212 Gyro sensor 213 Compass 214 Biosensor

Claims (19)

1. a user state estimation unit that estimates a user state;
   an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
   an output control unit that controls output based on the environmental state;
   An information processing device comprising:
2. The information processing device according to claim 1,
   a user position estimating unit that estimates a user position based on a detection value of a sensor unit of the wearable device worn by the user;
   a location attribute estimating unit that estimates a location attribute, which is an attribute of a location where the user is located, based on the user location;
   further comprising
   The information processing apparatus, wherein the user state estimation unit estimates the user state based on the location attribute.
3. The information processing device according to claim 2,
   The information processing apparatus, wherein the user position estimation unit estimates the user position using PDR (Pedestrian Dead Reckoning).
4. The information processing device according to claim 2,
   The information processing apparatus, wherein the environment estimation unit estimates the environmental state based on the location attribute.
5. The information processing device according to claim 2,
   The information processing apparatus, wherein the sensor unit of the wearable device includes at least one of an acceleration sensor, a gyro sensor, a compass, and a biosensor.
6. The information processing device according to claim 3,
   The user position estimation unit
   an angle correction unit that calculates a correction value of the azimuth angle of the user based on the detection value of the sensor unit of the wearable device worn by the user;
   an angle estimation unit that estimates an azimuth angle of the user based on the detection value of the sensor unit of the wearable device worn by the user and the correction value;
   a user position estimation unit that estimates the user position using the azimuth angle;
   An information processing device.
7. The information processing device according to claim 3,
   The user position estimation unit estimates a moving route of the user position,
   The information processing apparatus, wherein the location attribute estimation unit estimates the location attribute after movement based on the movement route.
8. The information processing device according to claim 7,
   The location attribute estimation unit stores a plurality of movement routes, and estimates the location attribute after movement by matching the estimated movement route with the plurality of held movement routes.
9. The information processing device according to claim 8,
   The information processing apparatus, wherein the location attribute estimation unit outputs a warning when matching fails a predetermined number of times.
10. The information processing device according to claim 8,
    The information processing device, wherein the location attribute estimation unit performs the matching using DTW (dynamic time warping).
11. The information processing device according to claim 1,
    The information processing apparatus, wherein the location attribute estimation unit estimates the location attribute by determining the length of stay of the user at the location where the user is.
12. The information processing device according to claim 1,
    further comprising a context acquisition unit that acquires the context of the user;
    The information processing apparatus, wherein the user state estimation unit estimates the user state based on the acquired context.
13. The information processing device according to claim 12,
    The context includes at least one of location information of the user and terminal information of the information processing device.
14. The information processing device according to claim 1,
    Information processing apparatus, wherein the user state estimation unit estimates the user state based on the detection value of the sensor unit of the wearable device and/or the location attribute.
15. The information processing device according to claim 1,
    The information processing apparatus, wherein the user state indicates a plurality of activity states of the user.
16. The information processing device according to claim 1,
    The output control unit is
    An information processing apparatus comprising: a content control unit that reproduces content selected based on the environmental state; and/or a notification control unit that controls the number of notifications to the user based on the environmental state.
17. Estimate the user state,
    estimating an environmental state to be presented to a user based on the user state;
    controlling output based on the environmental conditions;
    Information processing methods.
18. the processor of the information processing device,
    a user state estimation unit that estimates a user state;
    an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
    An information processing program operated as an output control unit that controls output based on the environmental state.
19. wearable devices and
    a user state estimation unit that estimates a user state of a user wearing the wearable device;
    an environment estimation unit that estimates an environmental state to be presented to the user based on the user state;
    an output control unit that controls output based on the environmental state;
    an information processing device having
    An information processing system comprising

Images