KR101817661B1 - Contextualizing seonsor, service and device data with mobile devices - Google Patents

Abstract

Method and system for user data contextualization and presentation. And collecting information including service activity data and sensor data from one or more electronic devices. The information is organized based on the time associated with the collected information. One or more of the content information for the potential interest and the service information, and at least one of the user context and the user activity.

Description

{CONTEXTUALIZING SEONSOR, SERVICE AND DEVICE DATA WITH MOBILE DEVICES}

One or more embodiments generally involve gathering, contextualizing, and presenting user activity data, collecting sensor and service activity information, in particular with recommended content and services, storing information, contextualizing information And presenting organized user activity data.

Many individuals who have mobile electronic devices (e.g., smartphones) have access to information such as photos, appointments, life events (e.g. walking, attendance, birth, birthday, The information is manually entered and organized by the user.

According to the present disclosure, there is provided a method for gathering information comprising service activity data and sensor data from one or more electronic devices, organizing information based on time associated with the collected information, and based on one or more of user context and user activity And providing one or more electronic devices with one or more of content information and service information regarding potential interests.

One or more general embodiments relate to collecting, contextualizing, and presenting user activity data. In one embodiment, the method includes collecting information including service activity data and sensor data from one or more electronic devices. The information can be organized based on the time associated with the collected information. In addition, one or more of the content information and service information regarding the potential interest may be presented to one or more electronic devices based on one or more of the user context and the user activity.

The method may further include filtering information organized based on the one or more selected filters.

The user context may be determined based on one or more of location information, motion information, and user activity.

The organized information may be provided in a chronological order on a graphical timeline.

Providing one or more of the content and services for potential interests may include providing one or more of the electronic devices with one or more of alerts, suggestions, events, and communication.

The content information and service information may be user subscribable for use in one or more electronic devices.

Organized information can be delivered dynamically to one or more electronic devices.

The service activity data, sensor data, and content may be captured as events that are flagged based on user actions.

Wherein service activity data and sensor data from one or more electronic devices may be provided to one or more of a cloud based system and a network system to determine a user context and wherein the user context comprises one of mode activation and notification on one or more electronic devices May be provided to one or more electronic devices for controlling the above.

Organized information may be provided continuously and may include life event information collected over a timeline, and life event information may be stored in one or more of the cloud-based system, the network system, and one or more electronic devices.

The one or more electronic devices include mobile electronic devices, and the mobile devices may include one or more of a mobile phone, a wearable computing device, a tablet device, and a mobile computing device. In one embodiment, a system may be provided that includes an activity module for collecting information, including service activity data and sensor data. The system may also include an organizational module configured to organize information based on the time associated with the collected information. The information analyzer module may provide one or more of the content information and service information for potential interests to one or more electronic devices based on one or more of a user context and a user activity.

The organizational module may provide filtering to information organized based on one or more selected filters.

The user context is determined by the information analyzer module based on at least one of location information, movement information and user activity, and the organized information can be presented in a specific time order on the graphical timeline of one or more electronic devices.

One or more of the content information and service information for a potential interest may include one or more of alerts, suggestions, events, and communications.

The content information and service information may be user subscripted for use in one or more electronic devices.

The one or more electronic devices may include a plurality of haptic components for providing a haptic signal.

In response to receiving a recognized user action at one or more electronic devices, service activity data, sensor data, and content may be captured into the flagged event.

Service activity data and sensor data from one or more electronic devices may be provided to an information analyzer module running on one or more of a cloud based system and a network system to determine a user context, And may be provided to one or more electronic devices for controlling one or more of mode activation and notification.

Organized information can be presented continuously and organizational information can include life event information collected through a timeline, and life event information can be stored on a cloud-based system, a network system, and on one or more of the one or more electronic devices .

The one or more electronic devices may comprise mobile electronic devices, and the mobile electronic devices may include one or more of a mobile phone, a wearable computing device, a tablet device, and a mobile computing device.

In one embodiment, a computer-readable non-transitory recording medium having recorded thereon a program for execution on a computer includes means for collecting information comprising service activity data and sensor data from one or more electronic devices . The information can be organized based on the time associated with the collected information. In addition, one or more of the content information and service information for potential interests may be provided to one or more electronic devices based on one or more of user context and user activity.

The computer readable non-volatile recording medium may further comprise filtering information organized based on the one or more selected filters, wherein the user context may be determined based on one or more of location information, movement information, and user activity have.

Organized information can be presented in a specific time order on the graphical timeline, and providing one or more of the content information and service information for a potential interest can include one or more of alerts, suggestions, events, Lt; / RTI >

The content information and service information may be user subscribed for use in one or more electronic devices, the organized information may be dynamically delivered to one or more electronic devices, the service activity data, sensor data, Can be captured as an event flagged based on an action.

The service activity data and sensor data from one or more electronic devices may be provided to one or more of the cloud-based system and the network system to determine a user context and may be configured to control one or more of mode activation and notification on one or more electronic devices A user context may be provided to one or more electronic devices.

Organized information may be provided continuously and organizational information may include life event information collected via a timeline, and life event information may be stored in a cloud-based system, a network system, and one or more of the one or more electronic devices .

One or more of the electronic devices may comprise mobile electronic devices, and the mobile electronic devices include one or more of a mobile phone, a wearable computing device, a tablet device, and a mobile computing device.

In one embodiment, a graphical user interface (GUI) displayed on a display of the electronic device may include one or more timeline events associated with information including service activity data and sensor data collected from the at least one electronic device. The GUI may further include one or more of selectable service categories and content information for the potential interest of the user based on one or more of user activity and user context associated with one or more timeline events.

One or more icons may be selected to display one or more categories associated with one or more timeline events, and one or more of the proposed content information and service information that the user is interested in may be provided on the GUI.

In one embodiment, the display architecture of the electronic device may include a plurality of content components and a timeline that includes one or more content components for potential user interests. In one embodiment, the plurality of time-based components include one or more of event information, communication information, and context alert information, and a plurality of time-based components are displayed in a specific time order. In one embodiment, the plurality of time-based components are extensible to provide extended information based on the received and recognized user actions.

In one embodiment, a wearable electronic device includes a processor, a memory coupled with the processor, a curved display, and one or more sensors. In one embodiment, context information is determined and additional information received from one or more of the service activity data and the supplementary sensor data from the paired host electronic device, and context information determined based on the sensor data, To the analysis module, which provides one or more of the content information and service information to the timeline module of the wearable electronic device, the sensors provide sensor data. In one embodiment, the timeline module organizes content related to the timeline interface of the curved display.

These and other aspects and advantages of one or more embodiments will become more apparent through the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of one or more embodiments.

The following detailed description is to be read in conjunction with the accompanying drawings, in order to provide a thorough understanding of the advantages and the nature of the examples, as well as a full understanding of the preferred methods of use. The accompanying drawings are as follows.
1 shows a schematic diagram of a communication system according to one embodiment.
2 illustrates a block diagram architecture of a system including a server and one or more electronic devices in accordance with one embodiment.
FIG. 3 illustrates an exemplary system environment in accordance with one embodiment.
FIG. 4 illustrates an example of organizing data in an archive according to one embodiment.
FIG. 5 illustrates an exemplary timeline view in accordance with one embodiment.
6 illustrates exemplary instructions for gesture search in accordance with one embodiment.
Figures 7A-D illustrate examples of extending events on a timeline graphical user interface (GUI) according to one embodiment.
FIG. 8 illustrates an example of flagging events according to one embodiment.
Figure 9 illustrates an example of dashboard detail views according to one embodiment.
10 illustrates an example of service and device management according to one embodiment.
11A-D illustrate an example of service management for application / service discovery according to one embodiment.
12A-D illustrate an example of service management for application / service streams in accordance with one embodiment.
13A-D illustrate an example of service management for an application / service user interest.
14 is an exemplary schematic diagram of mode sensing in accordance with one embodiment.
Figure 15 illustrates an exemplary process for aggregating / collecting and displaying user data in accordance with one embodiment.
16 illustrates an exemplary process for service management through an electronic device according to one embodiment.
Figure 17 illustrates an exemplary timeline and slides in accordance with one embodiment.
18 illustrates an exemplary process information architecture in accordance with one embodiment.
FIG. 19 illustrates exemplary active tasks in accordance with one embodiment.
Figure 20 illustrates an example of timeline logic for an input slide and an active task in accordance with one embodiment.
Figures 21A-B illustrate exemplary detail timelines according to one embodiment.
22A-B illustrate examples of timeline logic associated with an exemplary slide category according to one embodiment.
23 illustrates examples of timeline push notification slide categories according to one embodiment.
24 illustrates examples of timeline pull notifications in accordance with one embodiment.
25 illustrates an exemplary process for input slide routing in accordance with one embodiment.
26 illustrates an exemplary wearable device block diagram according to one embodiment.
27 illustrates an exemplary notification function in accordance with one embodiment.
28 illustrates an exemplary input gesture that interacts with a timeline in accordance with one embodiment.
29 illustrates an exemplary process for creating a slide in accordance with one embodiment.
30 shows an example of creating a slide using a template according to an embodiment.
31 illustrates an example of a context voice command based on a slide displayed on a screen according to an embodiment.
32 illustrates an exemplary block diagram of a wearable device and a host device / smartphone, according to one embodiment.
33 illustrates an exemplary process for receiving an instruction in a wearable device in accordance with one embodiment.
34 illustrates an exemplary process for a motion-based gesture of a mobile / wearable device in accordance with one embodiment.
35 illustrates an exemplary smart alert using haptic components in accordance with one embodiment.
Figure 36 illustrates an exemplary process for writing customized haptic patterns in accordance with one embodiment.
37 illustrates an example of a wearable device flow diagram for receiving a haptic recording in accordance with one embodiment.
38 illustrates an exemplary diagram of haptic recording in accordance with one embodiment.
39 illustrates an exemplary single axis force sensor for recording a haptic input in accordance with one embodiment.
40 illustrates an exemplary touch screen for haptic input in accordance with one embodiment.
41 illustrates an exemplary block diagram of a wearable device system in accordance with one embodiment.
42 illustrates a block diagram flow diagram for contextualizing and presenting user data in accordance with one embodiment.
43 is a high-level block diagram illustrating an information processing system including a computing system for performing one or more embodiments.

The following description is intended to illustrate the general principles of one or more embodiments and is not intended to be limited to the inventive concepts claimed in this document. In addition, certain features described herein may be used in combination with other described features and with various possible combinations and permutations. Unless specifically defined in this document, all terms are given in the broadest possible sense of the term, including terms which are understood by ordinary artisans and / or defined by dictionaries, treaties, etc., as well as to the analogy of the specification.

Embodiments may collect sensor and service activity information from one or more electronic devices (e.g., smart phones, wearable devices, tablet devices, cameras, etc.), store information, contextualize the information , And providing / displaying organized user activity data along with recommendation content information and service information. One embodiment is a method of collecting information comprising service activity data and sensor data from one or more electronic devices. The above information can be organized based on the time associated with the collected information. Based on one or more of the user context and user activity, one or more of the content information and service information of the potential interest may be provided to the one or more electronic devices described in the document.

One or more embodiments may be implemented in a system that is capable of capturing from the ecosystem of electronic devices into a timeline life log of event data that can be filtered through various "lenses ", filters, captures and organizes individual "life events". In one embodiment, the captured life events are large in scope and rich in content. In one embodiment, a wide variety of services (e.g., third party services, cloud-based services, etc.) and other electronic devices belonging to a personal ecosystem (e.g., smartphones, wearable devices, Electronic devices used by users such as tablet devices, smart TV devices, other computing devices, etc.) are collected and organized.

      In one embodiment, the life data (e.g., user activity on devices, sensor data from used devices, third party services, cloud-based services, etc.) (E. G., Smart phones) and wearable electronic devices as well as services (e. G., Using services such as e-mail services, restaurant referral services, review services, financial services and guidance services) Can be automatically and dynamically visualized in a dashboard GUI based on a particular area of interest. One or more embodiments provide a large set of modes within which life events (e.g., walking, driving, flying, cycling, buses, trains, etc.) can be organized. The embodiments above may not only rely on sensor data from a handheld device, but may also leverage sensor information from a wearable companion device.

      One or more embodiments may be implemented as a wearable application that may take the form of a companion application to help manage how different kinds of content are shown to the user and how different kinds of content are viewed through which touch points on the GUI To a main service to accompany the device. These embodiments may be used to synthesize a variety of different life events ranging from using services (e.g., service activity data) and user activity (e.g., sensor data, electronic device activity data) It is possible to provide a unique journey view to an electronic device that places events within a large context. The above embodiments may group various different information into a single view by utilizing sensor information to add service information and content information / data (e.g., text, pictures, links, video, audio, etc.) .

      One or more embodiments enable users to learn about themselves by emphasizing insights into the user ' s life based on the user ' s actual activity. One embodiment shows a central touchpoint for managing services and how services are experienced. One or more embodiments may be implemented as a method of proposing content with various types of services that an electronic device user may subscribe to (e.g., provided by a third party, provided by cloud-based services, etc.) . In one embodiment, the users can select, for example, where they are checking in (places, facilities, etc.) based on various types of user input, and what activities the users are doing (e.g., ) Can be seen.

      1 illustrates a schematic diagram of a communication system 10, in accordance with one embodiment. The communication system 10 may include a communication device 110 that initiates an outgoing communication operation 12 (transmission device 12) and a transmission device 12 that communicates within the communication network 110 with other communication devices May be used to initiate and perform communications operations with the base station. For example, the communication system 10 may include a communication device (receiving device 11) that receives a communication operation from the transmitting device 12. [ Although communication system 10 may include a plurality of transmission devices 12 and reception devices 11, only one is shown in FIG. 1 for simplicity of illustration.

      A suitable electrical circuit, apparatus, system, or combination thereof (e.g., a wireless communication infrastructure including communication towers and telecommunication servers) for generating a communication network may be used to create the communication network 110 . The communication network 110 may be capable of providing communication using an appropriate communication protocol. In some embodiments, the communication network 110 may be, for example, a conventional telephone line, a cable TV, a Wi-Fi (e.g., an IEEE 802.11 protocol), a Bluetooth®, a high frequency system , 2.4 GHz, and 5.6 GHz communication systems), infrared, or other relatively localized wireless communication protocols, or a combination thereof. In some embodiments, communication network 110 may support protocols used in wireless and cellular telephones and personal e-mail devices. These protocols may include, for example, GSm, GSM plus EDGE, CDMA, quadband, and other mobile phone protocols. In another example, the long-distance communication protocol may include protocols of outgoing or incoming calls using Wi-Fi and VOIP, LAN, WAN or other TCP-IP based communication protocols. When the transmitting device 12 and the receiving device 11 are located in the communication network 110, the transmitting device 12 and the receiving device 11 can communicate either across the bidirectional communication path such as the path 13, Lt; / RTI > communication paths. Both the transmitting apparatus 12 and the receiving apparatus 11 may be able to initiate a communication operation and receive the communication operation initiated.

      The transmitting device 12 and the receiving device 11 may comprise suitable devices for transmitting and receiving communication operations. For example, the transmitting device 12 and the receiving device 11 may be used in various applications such as mobile telephone devices, TV systems, cameras, camcorders, devices with audiovisual functions, tablets, wearable devices, (E. G., With the aid of an enabling auxiliary system) or other devices capable of communicating (e. G. Using conventional telephone lines) via a wired path. The communication operation may be, for example, voice communication (e.g., telephone calls), data communication (e.g., emails, text messages, media messages), video communication, ≪ / RTI > meetings).

      FIG. 2 is a flow chart illustrating a method for collecting sensor and service activity information, storing the information, contextualizing the information, and using the one or more electronic devices 120 and wearable device 140 to < RTI ID = 0.0 > Which may be used to provide a service or application for presenting a message to a user. Both the transmitting device 12 and the receiving device 11 may include some or all of the features of the electronic device 120 and / or the features of the wearable device 140. In one embodiment, the electronic device 120 and the wearable device 140 can communicate with one another and can synchronize data, information, content, etc. with one another and provide complementary or similar features.

      In one embodiment, the electronic device 120 includes a display 121, a microphone 122, an audio output 123, an input mechanism 124, a communication circuit 125, a control circuit 126, an application 1- N (127), a camera module 128, a Bluetooth® module 129, a Wi-Fi module 130 and sensors 1 through N (N is a positive integer), an activity module 132, RTI ID = 0.0 > 133 < / RTI > and other suitable components. In one embodiment, applications 1-N 127 may be provided and obtained from a cloud or server 150, communications network 110, etc. where N is a positive integer greater than or equal to one. In one embodiment, the system 100 collects evidence and context information of the display 121, queries the evidence and context information, and provides a query request and a solution to the query on the display 121 And context aware query applications that operate in conjunction with cloud-based or server-based subscription services. In one embodiment, the wearable device 140 may include part or all of the features, components, and modules of the electronic device 120.

       In one embodiment, all applications used by the audio output 123, the display 121, the input mechanism 124, the communication circuit 125 and the microphone 122 are interconnected by the control circuit 126 and managed . In one embodiment, a portable music player that is capable of transmitting music to other tuning devices may be combined with the electronic device 120 and the wearable device 140. [

      In one embodiment, the audio output 123 may include suitable audio components that provide audio to the users of the electronic device 120 and the wearable device 140. For example, the audio output 123 may include one or more speakers (e.g., mono or stereo speakers) embedded in the electronic device 120. In one embodiment, the audio output 123 may include an audio component that is remotely connected to the electronic device 120 or the wearable device 140. For example, the audio output 123 may communicate with a communication device and a wired (e.g., connected to an electronic device 120 / wearable device 140 and jack) or wireless (e.g., A headset, a headset, and an earphone.

      In one embodiment, the display 121 may include a suitable screen or projection system to provide a display to the user. For example, the display 121 may include a screen (e.g., an LCD screen) included in the electronic device 120 or the wearable device 140. As another example, the display 121 may include a mobile display or projection system for displaying content on a surface remote from the electronic device 120 or the wearable device 140 (e.g., a video projector ). Display 121 may operate to display content (e.g., information about communication operations or information about available media selections) under the direction of control circuitry 126.

      In one embodiment, the input mechanism 124 may be a suitable mechanism or user interface for providing user inputs or commands to the electronic device 120 or the wearable device 140. The input mechanism 124 may take various forms, such as, for example, a button, a keypad, a dial, a click wheel, or a touch screen. The input mechanism 124 may include a multi-touch screen.

      In one embodiment, communication circuitry 125 may be any suitable communication circuitry for accessing communication networks and for communicating communication operations and media from electronic device 120 or wearable device 140 For example, communication network 110, Figure 1). The communication circuit 125 may be, for example, a Wi-Fi (e.g., 802.11 protocol), Bluetooth, a high frequency system (e.g., 900 MHz, 2.4 GHz and 5.6 GHz communication systems) plus EDGE, CDMA, quandband and other cellular telephone protocols, VOIP, TCP-IP or other suitable protocols.

      In one embodiment, the communication circuitry 125 may be used to create a communication network using an appropriate communication protocol. For example, communication circuitry 125 may create a short-range communication network that uses a short-range communication protocol to connect to another communication device. For example, the communication circuitry 125 may be used to create a local communication network using the Bluetooth® protocol to connect the Bluetooth® headset with the electronic device 120.

      In one embodiment, the control circuitry 126 may be utilized to control operations and execution of the electronic device 120 or the wearable device 140. [ The control circuitry 126 may be implemented as, for example, a processor, a bus (e.g., for transmitting instructions to an electronic device 120 or other components of the wearable device 140), a memory, 120 or other suitable components for controlling the operation of the wearable device 140. [ In one embodiment, the processor can drive the display and process inputs from the user interface. Memory and storage may include, for example, cache, flash memory, ROM and / or RAM / DRAM. In one embodiment, the memory may be dedicated to storing firmware (e.g., device applications such as an operating system, user interface functions, and processor functions). In one embodiment, a memory may be used to store information relating to other devices for which the electronic device 120 or the wearable device 140 performs communication operations (e. G., Contact information associated with communication operations) Store or store information related to different media types and media items selected by the user).

      In one embodiment, the control circuitry 126 may be utilized to perform one or more applications that are executed in the electronic device 120 or the wearable device 140. Appropriate numbers or types of applications may be executed. It will be appreciated that while the following discussion will list other applications, some or all applications may be combined with one or more applications. For example, the electronic device 120 and the wearable device 140 may be implemented using any one or more of the following technologies: an automatic speech recognition (ASR) application, a chat application, a map application, a media application (e.g., QuickTime, MobileMusic.app, , Etc.), social network applications (e.g., Facebook®, Twitter®, etc.), Internet search applications, and the like. In one embodiment, the electronic device 120 and the wearable device 140 may include one or more applications that may be utilized to perform communications operations. For example, the electronic device 120 and the wearable device 140 may be implemented as a messaging application, a mail application, a voice mail application, an instant messaging application (e.g., for a chat), a video conferencing application, Lt; RTI ID = 0.0 > applications. ≪ / RTI >

      In one embodiment, the electronic device 120 and the wearable device 140 may include a microphone 122. As an alternative, for example, during a communication operation or as a means of establishing a communication operation or using a physical user interface, a user may select audio (e.g., voice audio) for voice control and navigation of applications 1-N 127 The electronic device 120 and the wearable device 140 may include a microphone 122 to enable transmission. The microphone 122 may also be coupled to the electronic device 120 and the wearable device 140 and to the electronic device 120 and the wearable device 140 remotely. For example, the microphone 122 may be coupled to wired headphones, the microphone 122 may be coupled to a wireless headset, and the microphone 122 may be coupled to a remote control device or the like.

      In one embodiment, the camera module 128 may include one or more cameras (e.g., cameras) that configure interopertabililty for transmission, sharing, and the like, for stopping and capturing video images, editing functions, Device.

      In one embodiment, the Bluetooth (R) module 129 includes a program for processing processes and / or Bluetooth (R) information and may configure a receiver, transmitter, transceiver, and the like.

      In one embodiment, the electronic device 120 and the wearable device 140 may include a plurality of sensors, such as, for example, an accelerometer, gyroscope, microphone, temperature, light, barometer, magnetometer, compass, Lt; RTI ID = 0.0 > 1 < / RTI > In one embodiment, the plurality of sensors 1 to N 131 provide information to the user module 132.

      In one embodiment, the electronic device 120 and the wearable device 140 may include other components suitable for performing communication operations. For example, electronic device 120 and wearable device 140 may include a power supply, interfaces to ports or host devices, ancillary input mechanisms (e.g., ON / OFF switches), or other suitable components can do.

      FIG. 3 illustrates an exemplary system 300, in accordance with one embodiment. In one embodiment, block 310 shows collecting and understanding the collected data. Block 320 shows data (e.g., life data) being presented to electronic devices such as electronic device 120 (FIG. 2) and wearable device 140. Block 330 shows storing collected data in a life hub (i.e., a cloud-based system / server, network, storage device, etc.). In one embodiment, the system 300 is configured to determine how the user's data (e.g., life data) proceeds through the system 300, in three aspects, namely, the collection and understanding at block 310, , And a storage process at block 330.

      At block 310, the gathering and understanding process may include providing third party service information from user activity, user device (s) (e.g., electronic device 120, and / or wearable device 140) (e. g., life data) from other devices in the ecosystem. In one embodiment, data may be collected in the electronic device 120 and / or the activity module 132 (FIG. 2) of the wearable device 140. The service activity information may include information about what the user is viewing, reading, searching, and viewing. For example, if a user uses a travel service (for example, a travel guide service / application, a travel recommendation service, an application, etc.), the service activity information may include information about hotels / motels, Dates, car rental information, read reviews, entered search criteria (e.g., price, rating, date, etc.), comments left over, pricing scores, and the like. In one embodiment, the collection and analysis can be managed by user facing touchpoints in a mobile device (e.g., electronic device 120, wearable device 140, etc.). In one embodiment, management can configure the service integration and device integration described below.

      In one embodiment, a process in system 300 may provide the user with appropriate data (e. G., Via electronic device 120) via wearable devices (e. G., Wearable device 140) or mobile devices , Life data) can be transmitted intelligently. These devices together with other devices constitute a device environment (ecosystem). The presentation at block 320 may include notifications that may be handled through graphics, text, sound, speech, vibration, light, etc. in the form of slides, cards, data or content time base elements, Events, communications, and the like. Data constituting the presentation form can be communicated through a variety of methods of communication interfaces, such as, for example, Bluetooth (R), Near Field Communications (NFC), WiFi, cellular,

      In one embodiment, the archiving process in block 330 may utilize data from third parties and user activities that are presented to the user and interact with the user. In one embodiment, the process can edit and process the data and then create a dashboard in a timeline presentation (shown at block 330) You can create dashboards. The data may be stored / stored in the cloud / server 150, the electronic device 120 (and / or the wearable device 140) or any combination thereof.

      FIG. 4 illustrates an example of organizing 400 data in a repository according to one embodiment. In one embodiment, data processing into the archived timeline format 420 may occur within the cloud 150 and out of the electronic device 120 and the wearable device 140. Optionally, the electronic device 120 may process data and create a repository or any combination of one or more of the electronic device 120, the wearable device 140, and the cloud 150 may process the data and create a repository . As shown, the data may be retrieved from the active service 410, the electronic device 120 (e.g., data, content, sensor data, etc.) and the wearable device 140 (e.g., data content, sensor data, etc.) Can be collected.

      FIG. 5 illustrates an exemplary timeline view 450 in accordance with one embodiment. According to one embodiment, the timeline view 450 includes an illustrative journal or archive timeline view. The user ' s stored daily activities may be organized on the timeline 420. As described above, a repository that provides an integrated view of user life data is populated with activities or places that the user has actually interacted with. In one embodiment, an action bar located above the timeline 420 provides navigation to a home / timeline view or a specific interest view, which will be described below.

      In one embodiment, the header points to the current date being viewed and is captured by the user, or constitutes the image provided by the third party based on user activity or location. In one embodiment, the context is a mode (e.g., walking). In one embodiment, " now "or logged current life events may be extended to display additional information such as event title, progress and consumed or captured media. As can be seen in view 450, in one embodiment, the user is walking around the city.

      In one embodiment, past events include logged events from the current date. In one embodiment, as shown in view 450, the user interacts with two events while in the Ritz Carlton. Either of the events may be selected and expanded to view more specific information (described below). Optionally, other contexts such as a location may be used. In one embodiment, the wearable device 140 achievement events are highlighted with different icons and symbols in the timeline. In one embodiment, the user may continue to scroll to the previous date of the life events for timeline 420 information. Optionally, when the bottom of the timeline 420 is reached, more content is automatically loaded into the view 450 and kept visible.

      FIG. 6 illustrates an exemplary instruction 600 for gesture search, according to one embodiment. As shown, the user-paced touch point of the example timeline 620 may be searched through the gesture input 610 interpretation from the user. In one embodiment, these inputs may be interpreted as scrolling, movement between regions of interest, expansion, and so on. In one embodiment, gestures such as pinch-in or out using a plurality of fingers can provide a search that crosses category layers. In one embodiment, the finch gesture in the daily display screen can be converted to a weekly view, and can be converted back to a monthly view or the like. Similarly, opposite motions (e.g., multiple finger gestures for zooming in) can be zoomed in from weekly views, monthly views, and the like.

      7A-D illustrate examples for 710, 711, 712, and 713, respectively, for extending events (e.g., slides / time based components) on a timeline GUI, in one embodiment. In one embodiment, examples 710-713 show how details for events on the archived timeline can be shown. In one embodiment, these extensions may show additional details related to events such as recorded and analyzed sensor data, applications / services / content suggestions, and the like. The user can extend the event to view detailed content by receiving a recognized input (e.g., a momentary force, tapping touch, etc.) or activating a user-paced touchpoint on the life data event on the timeline. In one embodiment, example 710 shows the result of recognizing an activation command for a received input or "good morning" event. In example 711 a good morning event is shown in the expanded view. At example 712, the timeline is scrolled down through the recognized input or activation command and another event is expanded through the received recognized input or touch point activation. In the example 713, the extended event is displayed.

      FIG. 8 illustrates an example 800 of flagging events in accordance with one embodiment. In one embodiment, wearable device 140 (FIG. 2) may predetermine user actions and gestures (e.g., squeezing bands), wherein user input actions and gestures are received and flagged by user Can be registered in the event. For example, the user may squeeze 810 the wearable device 140 to initiate flagging. In one embodiment, flagging captures the various data points into a single event 820, such as places, pictures or other images, friends or family in the vicinity, and additional events issued at the same location. System 300 can determine that data points can be incorporated into events through contextual relationships such as photos taken during the activity, activity data (time spent, distance traveled, number of steps, etc.), activity locations, In one embodiment, the flagged events may be stored in the timeline 420 (FIG. 4) and highlighted events 830 (e.g., specific colors, symbols, icons, moving symbols / colors / ).

      FIG. 9 illustrates an example 900 of dashboard detail views, according to one embodiment. In one embodiment, examples 910, 911, and 912 illustrate example detail views of a dashboard that can be navigated by a user through the timeline 420 (FIG. 4) GUI. The Dashboard Detail view allows users to view aggregated information about a particular concern. In one embodiment, a particular interest may be selected from the user interface on the timeline 420 by selecting an appropriate icon, link, symbol, or the like. In one embodiment, concerns may include financial, health, mobility, and the like. The user may select a financial symbol or icon as shown in the example view 910. [ In one embodiment, budgets may be tailored for different time periods (e.g., day, week, month, custom time, etc.). In one embodiment, the dashboard may show graphical breakdowns or other topics related to expenditure history or finances.

      In one embodiment, the health dashboard in the exemplary view 912 is shown based on a health icon or symbol selected by the user. In one embodiment, the health view includes details of performed activities, a measure of various activities (e.g., number of steps, distance traveled, time spent, calories consumed, etc.) . ≪ / RTI > In other embodiments, the movement detail may be displayed based on a movement icon or symbol that can show places such as near or far distances that the user has visited. In one embodiment, the categories of interest may be stretched or customized. For example, interest categories may be applied to specific interests such as climbing, golf, exploration, sports, hobbies, etc., so that interest categories may be broken up into further steps to include display or materialized data.

      10 illustrates an example 1000 of service and device management, in accordance with one embodiment. As described later in this document, in one embodiment, the user-facing touch point provides for the management of services and devices. In one embodiment, a sidebar icon or symbol on the example timeline 1010 is displayed. Option management view 1011 opens and displays other services and devices that can be managed by the user.

      11A-D illustrate exemplary views 1110, 1120, 1130, and 1140 of service management for application / service discovery, in accordance with one embodiment. The illustrated examples illustrate exemplary embodiments that enable discovery of related applications and services. In one embodiment, the timeline 420 (FIG. 4) GUI may display recommendations for services to be combined with the virtual dashboard streams described above. Recommendations can be broken down into multiple categories. In one embodiment, one category may be a personal recommendation based on context (e.g., user activity, existing applications / services, location, etc.). In another example, the categories added to the streams may be the most popular applications / services. In yet another example, the third category may include new noteworthy applications / services. These categories can display the application in various forms including an example format similar to how the application / service will be displayed in the timeline, coordinate view, list view, and so on.

      In one embodiment, in selecting a category, a service or application may display preview details with additional information about the service or application. In one embodiment, if an application or service is already installed, service management may combine the application with virtual dashboards. In one embodiment, an example 1110 shows a user touching a draw on a timeline 420 space GUI to open a drawer. In one embodiment, one section is prepared for the user to access actions such as Discover, Discover Manager, and the like. In one embodiment, tapping the "cover" causes the user to move to a new screen (e.g., move from the example 1110 to the example 1120).

      In one embodiment, example 1120 shows a "Discover" screen that includes recommendations for streams that can be classified by a plurality of categories, such as For You, Popular, and What's New. In one embodiment, the Apps icons / symbols that allow users to "see " streams are configured similar to the journey view. In one embodiment, users may tap the "Add" button on the right to add a stream. As shown in the example, the categories may be related to users similar to the examples provided above.

      In one embodiment, the example 1120 shows that a user can tap a tab to move directly to a tab or to swipe one tab at a time. As described above, categories can display applications in various forms. At example 1130, the Popular tab displays the available streams on a coordinate format and provides a preview when the icon or symbol is tapped. At example 1140, the What's New tab displays a list of available services or applications, along with a short description and respective list items accompanied by a "Add" button.

      12A-D illustrate service management examples 1210, 1220, 1230, and 1240 for application / service streams, according to one embodiment. In one embodiment, examples 1210-1240 show that users can edit virtual dashboards or streams. The user-facing touch point may provide the user with the option of enabling or disabling applications viewed through the virtual dashboard. The touch point also allows the user to select which details the application will show on the associated device (e.g., electronic device 120, wearable device 140, etc.) within the virtual dashboard and device environment .

      In one embodiment, the activation on the input or drawer icon received and recognized in the example 1210 is received and recognized. Optionally, the drawer icon may be a full-width tool bar that loads an option menu. At example 1220, the options menu may be displayed with, for example, Edit My stream, Edit My Interests, and so on. In one embodiment, the Edit My Stream of example 1220 is selected based on received and recognized actions (e.g., temporal forces on the touch point, received and recognized user input, etc.). In example 1230 (streams screen), the user may be provided with a list of services after a selection to edit the streams. In one embodiment, the user may tap the switch to toggle the service on or off. In one embodiment, the features / content provided at this level may be pre-canned. Optionally, details of the list item may be displayed (e.g., the user taps on the touch point) upon receipt of the list item on the touch point and upon receipt of the recognized input, command or activation instruction. In one embodiment, the displayed items may include an area that allows each displayed item to be "grabbed" and dragged to rearrange a list (e.g., a top priority item) . In example 1230, the grabbable area is located to the left of each item.

      In one embodiment, example view 1240 shows a detail view of an individual stream and allows users to customize the stream. In one embodiment, the user can select which features / content he / she wants to see and which users want to view in some device (e.g., electronic device 120, wearable 140, FIG. 2). In one embodiment, features / content that can not be turned off are displayed but can not operate.

      13A-D illustrate examples of service management 1310, 1320, 1330, and 1340 for application / service user interests, in accordance with one embodiment. One or more embodiments provide for management of user interests on the timeline 420 (FIG. 4). In one embodiment, users can add, delete, rearrange, modify, etc. list categories. Optionally, users can also customize to be displayed on visual dashboards of interest (e.g., applications / services to be displayed with details). Additionally, the management described as described can include a portion of user feedback for calibration.

      In one embodiment, the input (e.g., transient force, force applied to the touch point movement, etc.) received and recognized in the example 1310 is passed to a drawer icon or code (e.g., a received tap or a direct swipe) . Optionally, icons or symbols within the full-width toolbar may be used to invoke the options menu. As an example, in example 1320, the options menu appears with Edit My Streams, Edit My Interests, and so on. In an exemplary embodiment, the selection menu "Edit My Interests" as shown in example 1320 is selected based on the received and recognized input. In one embodiment, a display appears in an example 1330 that includes a list of interests (previously selected when the user first used it). In one embodiment, interests may be rearranged, deleted, and added based on the received and recognized input. As an example of an embodiment, the user can rearrange interests based on preferences, swipe to erase interests, and tap the "+" symbol to add interest.

      In one embodiment, the detail view of an individual stream of example 1340 allows the user to customize the stream. In one embodiment, a user may select features / content that the user would like to see in some device (e.g., electronic device 120, wearable device 140, etc.). In one embodiment, features / content that can not be turned off are displayed but can not operate. In one embodiment, the selector may be grayed or other similar displays indicating that the feature is locked.

      14 illustrates an exemplary schematic for mode sensing in accordance with one embodiment. In one embodiment, the overview shows a user mode detection system 1400. In one embodiment, the system 1400 utilizes a wearable device 140 (e.g., a wristband associated with a host device, e.g., electronic device 120). In one embodiment, wearable device 140 may provide sensor data 1440 that is mounted, for example, an accelerometer, gyroscope, magnetometer, and the like. In one embodiment, the data may be provided from various communication interface methods, for example, Bluetooth, WiFi, NFC, telephone, and the like. In one embodiment, the electronic device 120 may include data from the wearable device 140 and, for example, time, location (via GPS, telephone triangulation, beacons or other similar methods), accelerometer, gyroscope, Data from the wearable device internal sensors such as a magnetometer can be synthesized. In one embodiment, the aggregated data set 1430 to be analyzed may be provided to the context search system 1410 in the cloud 150.

      In one embodiment, the context search system 1410 may be located in the cloud 150 or other network. In one embodiment, the context search system 1410 may receive data 1430 through various methods of communication interfaces. In one embodiment, after the data from the learning learning data set 1420 or from the learning learning data set 1420 is trained, the context search system 1410 analyzes the received data Lt; RTI ID = 0.0 > and / or < / RTI > In one exemplary embodiment, the algorithm may be a machine learning algorithm that may be tailored to user feedback. In one embodiment, the training data set 1420 may constitute initial general data about the various modes stacked from various sources. The new data may be added to the learning data set in response to providing feedback for better mode determination. In one embodiment, the context search system 1410 may then generate an output of analyzed data 1435, which indicates the mode of the user, and return it to the electronic device 120.

      In one embodiment, the smartphone may provide mode 1445 back to wearable device 140, or may return determined mode 1445 to the life hub application (e.g., activity module 132, FIG. 2) May be utilized within a logging application (e.g., organizational module 133) or may even be used to throttle messages pushed to wearable device 140 based on the context. In an exemplary embodiment, if the user participates in activities such as driving, cycling, the electronic device 120 can receive the mode 1445 and block messages sent to the wearable device 140, May provide non-intrusive notifications so that they are not interrupted. In one embodiment, this essentially considers user activity instead of relying on another method, such as geofencing. In an exemplary embodiment, another example includes a pedometer mode that is automatically activated to show the distance traveled when the user is detected as running.

      FIG. 15 illustrates an exemplary process 1550 for aggregating / collecting and displaying user data in accordance with one embodiment. In one embodiment, process 1500 begins at block 1501 (e.g., automatically, manually, etc.). The activity module 132 (Figure 2) in block 1510 receives (e.g., from the electronic device 120, and / or the wearable device 140) third party service data. At block 1530, the collected data is provided to one or more connected devices (e.g., electronic device 120, and / or wearable device 140) to be displayed to a user. At block 1540, the user interaction data is received by the activity module 132.

      At block 1550, the associated data is identified and associated with interest categories (e.g., context search system 1410, FIG. 14). At block 1560, the relevant data is collected into events (e.g., context search system 1410 or organizational module 133). At block 1570, a virtual dashboard of events is generated and arranged in reverse chronological order (e.g., by organizational module 133). At block 1580, a virtual dashboard of interest categories is generated utilizing events containing associated related data. In one embodiment, at block 1590, one or more virtual dashboards are displayed using the timeline 420 (FIG. 4) GUI. At block 1592, process 1500 ends.

      16 illustrates an exemplary process for service management over an electronic device, in accordance with one embodiment. In one embodiment, the process 1600 begins at a start block 1601. At block 1610, the process 1600 determines whether to search for applications. If process 1600 searches for applications, then process 1600 proceeds to block 1611. If process 1600 does not search for applications, process 1600 proceeds to block 1620 where it is determined whether to edit dashboard applications. If the dashboard applications are determined to be edited, the process 1600 proceeds to block 1621 where a list of associated applications and current state details is displayed. If it is determined not to edit the dashboard applications, the process 1600 proceeds to block 1630 where it is determined whether or not to edit the categories of interest. If it is determined not to edit the categories of interest, the process 1600 proceeds to block 1641.

      After block 1611, the process 1600 proceeds to block 1612 where suggestions based on the user context of one or more categories are displayed. At block 1613, a user selection for one or more applications associated with the virtual dashboard is received. At block 1614, one or more applications are downloaded to an electronic device (e.g., electronic device 120, FIG. 2). At block 1615, the download application is associated with a virtual dashboard.

      At block 1622, user changes are received. At block 1623, the associated applications according to the received input are changed.

      If it is determined to edit interest categories, at block 1631 a list of interest categories and applications associated with each of the categories are displayed. At block 1632, a user change is received regarding the categories and associated applications. At block 1633, the categories and / or associated applications are changed according to the received input.

      Process 1600 proceeds after block 1633, block 1623 or block 1615 and ends at block 1641. [

      Figure 17 illustrates an example 1700 of a timeline overview 1710 and slides / time-based components 1730 and 1740, in accordance with one embodiment. In one embodiment, the wearable device 140 (FIG. 2) may include a wrist band type device. In an exemplary embodiment, the wristband device may constitute a strap forming a bracelet-like structure. In an exemplary embodiment, the bracelet-like structure may be circular or oval shaped to fit the wearer's wrist.

      In one embodiment, the wearable device 140 may comprise a curved OLED (organic light emitting diode) touch screen or a similar type of display screen. In an exemplary embodiment, the OLED screen can be bent in a convex manner to fit the curve of the bracelet structure. In one embodiment, as described above, the wearable device 140 may further include a processor, memory, communication interface, power source, and the like. Alternatively, a wearable device, as illustrated in FIG. 42, may include the components described below.

      In one embodiment, the timeline overview 1710 includes data cases (shown through slides / data or time-based components) and three general categories: Past, Now, ) And Future (suggestions). Past cases may include previous notifications or recorded events shown on the left side of the timeline overview 1710. Now instances may include time, weather, or other input slides 1730 or suggestions 1740 associated with the current user. In one embodiment, the input slides (data or content time-based components) 1730 include current life events (e.g., health records, payment, etc.), input communications Phone, etc.), personal notifications (e.g., sports scores, current traffic, police, emergency, etc.). Future cases may include relevant helpful suggestions and predictions. In one embodiment, the predictions or suggestions may be based on user profiles or previous actions / preferences of the user. In one embodiment, proposed slides 1740 may include recommendations such as coupon provision near the intended place, upcoming activities near the place, flight delay notices, and the like.

      In one embodiment, input slides 1730, which are described in greater detail below, are under the control of push or pull notifications. In one embodiment, timeline search 1720 is provided through a touch-based interface (or voice commands, motion, action recognition, etc.). Various user drives or gestures can be received and interpreted as search commands. As an example of one embodiment, a horizontal gesture or swipe can be used to navigate left and right horizontally, a tab displays a date, and an up or vertical swipe can invoke an action menu or the like.

      18 illustrates an exemplary process information architecture 1800, according to one embodiment. In one embodiment, exemplary architecture 1800 shows a typical information architecture of a timeline that a user experiences through a timeline search 1810. In one embodiment, Past slides (data or content time-based components) 1811 are stored for a predetermined period of time, or under other conditions before the accessible bank is deleted . In an exemplary embodiment, these conditions may constitute the size of the cache that stores past slides. In one embodiment, Now slides constitute home / time 1813 with recent notification (s) (slides, date or content time-based components) 1812 along with active tasks.

      In one embodiment, recent notifications 1812 are received from user input 1820 (voice input 1821, Payments 1822, Check-Ins 1823, touch gestures, etc.) . In one embodiment, external input 1830 from device environment 1831 or third party services 1832 may be received via timeline logic 1840 provided from the host device. In one embodiment, the most recent notification 1812 may also communicate data (e.g., ignore or cancel a notification) in communication with timeline logic 1840 presenting user actions. In one embodiment, recent notifications 1812 may remain until the user sees notifications and may then be moved to the past 1811 stack or removed from the wearable device 140 (FIG. 2) .

      In one embodiment, timeline logic 1840 may insert new slides as new slides arrive from the left of the most recent notification slide 1812, for example, from a point remote from the groove 1813, Slices can be inserted to the right of tasks. Alternatively, there may be exceptions where the input slides are located just to the right of the active tasks.

      In one embodiment, the home 1813 may be a default slide capable of displaying time (or other possible user settable information). In one embodiment, various modes 1850 can be accessed from a home 1813 slide, such as health 1851, alarms 1852, settings 1853, and the like.

      In one embodiment, proposals 1814 (future, future) slides / time-based components can interact with timeline logic 1840 similar to recent notifications 1812, as described above. In one embodiment, the suggestions 1814 are contextual and may be based on time, location, user interest, user schedule / calendar, and so on.

      FIG. 19 shows exemplary active tasks 1900, according to one embodiment. In an exemplary embodiment, the two active tasks are displayed with a music remote 1910 and a search 1920, each with a separate set of rules. In one embodiment, active tasks 1900 do not disappear into timelines (e.g., timeline 420, FIG. 4), such as categories of other slides. In one embodiment, 1900 remains readily available and can be displayed instead of groove 1813 until the task of active slides is completed or ignored.

      20 shows a timeline logic example 2000 with input slides 2030 and active tasks 2010 according to one embodiment. In one embodiment, the new slides / time-based components 2030 enter the left side of the active task slides 2010 and become past slides and disappear into the timeline 2020 when they are replaced by new content . In one embodiment, the music remote 2040 active task slide is activated when the headphones are connected. In one embodiment, the search 2050 slides are activated when the user requests a turn-by-turn search. In one embodiment, the home slide 2060 may be permanently fixed in the timeline 2020. In one embodiment, as described above, the home slide 2060 may be temporarily replaced by a visible slide by an active task.

      Figures 21A and 21B illustrate exemplary detail timelines 2110 in accordance with one embodiment. In one embodiment, a more detailed description of executing past notifications, current / recent notifications, input notifications and suggestions is described. In one embodiment, timeline 2110 shows a touch or gesture example based on user experience (UX) interacting with slides / time-based components. In one embodiment, the UX timeline 2110 may include features that increase the speed at which the wearable device 140 (FIG. 2) discovery may use the host device (e.g., electronic device 120). In one embodiment, if the user navigates from the notification to a second layer of information (e.g., an event also extends the slide / time based component), the application of the paired host device may respond to more complex user input can be opened to the corresponding screen.

      A representative term dictionary of user actions (e.g., symbols, icons, etc.) is shown in the second row from the left side of Figure 21A. In one embodiment, these user actions facilitate limited input interaction of the wearable device 140. In one embodiment, recent slide 2120, home slide 2130, and suggested slides 2140 are displayed on timeline 2100.

      In one embodiment, the timeline UX may include a suggestion engine that can learn the user's preferences. In one embodiment, the suggestion engine is first trained through the initial categories selected by the user, and then based on the feedback, such as an action on the user's suggestion or a user deletion of the offered offer, Corrected. In one embodiment, the engine may also replace old suggestions or provide new suggestions when the user deletes the suggestions.

      FIGS. 22A and 22B show exemplary slide / time-based component categories 2200 for timeline logic, according to one embodiment. In one embodiment, representative categories also indicate how long the slide (or card) can be stored in the wearable device 140 (FIG. 2). In one embodiment, timeline slides 2110 include event slides, warning slides, communication slides, Now slides 2210, Always slides (e.g., home slides) Proposed slides 2140 are shown.

      23 shows examples of timeline push notification slide categories 2300 according to one embodiment. In one embodiment, the events 2310, communication 2320, and context alerts 2330 categories are assigned to push notifications by timeline logic. In one embodiment, the slide duration for events 2310 is a predetermined number of days (e.g., two days), the maximum number of slides reached, or the earliest of user abandonments. In an exemplary embodiment, the communication 2320 may be viewed or ignored in the electronic device 120 (FIG. 2) or a predetermined number of days (e.g., two days) The slides stay in the timeline until the slides receive a response such as reaching the number, which is the slice duration. In an exemplary embodiment, the slides in the context alerts 2330 remain until they are no longer relevant (e.g., when the user is no longer in the same place, or when conditions or time change) This is the duration of the slides.

      FIG. 24 illustrates examples of timeline pull notifications 2400, in accordance with one embodiment. In one embodiment, proposed slides 2410 are considered pool notifications and provided to a user request via a swipe (e.g., swipe left). In one embodiment, the user does not have to explicitly subscript the service to receive the offer 2410 from the service. The suggestions may be based on time, place and user interest. In one embodiment, the initial user interest categories may be defined within the wearable devices setting application located in the electronic device 120 or the wearable device 140 (at a future stage, user interest may be automatically adjusted by use ). In one embodiment, examples of suggestions 2410 include location-based coupons, popular recommendations about places, places, entertainment and events, proposed health or lifestyle goals, traffic during non-travel time Events that will occur later, such as updates, weather forecasts, or planned events.

      In one embodiment, a predetermined number of proposals (e.g., three as shown in the example) can be preloaded if the user indicates that they wish to receive suggestions (e.g., swipe left). In one embodiment, additional suggestions 2410 (when available) may be loaded depending on the situation then, if the user continues to swipe left. In one embodiment, suggestions 2410 are refreshed when the user changes location or a particular time of day. In one example, a coffee shop may be offered in the morning while a movie is offered in the late afternoon.

      25 illustrates an example of an exemplary process 2500 for routing input slides, according to one embodiment. In one embodiment, process 2500 begins at start block 2501. At block 2510, a timeline slide is received at the paired device (e.g., electronic device 120, FIG. 2). At block 2520, the timeline logic determines if the received timeline slide is a requested proposal. If the received timeline slide is a requested proposal, process 2500 proceeds to block 2540. [ At block 2540, the proposed slides are arranged on the home slide in the timeline or to the right of the most recent suggested slide.

      At block 2550, it is determined whether user overrides have occurred or the slides are no longer associated. If the user has not abandoned the slide or if the slide is still relevant, the process 2500 proceeds to block 2572. If the user abandons the slide or if the slide is no longer associated, the process 2500 proceeds to block 2560 where the slide is deleted. Process 2500 then proceeds to block 2572 and the process ends. At block 2521 the slides are arranged in the timeline to the left of the home slide or to the left of the active slide. At block 2522, it is determined whether the slide is a notification type slide. At block 2530, it is determined if the duration for the slide has reached. If the duration has been reached, the process 2500 proceeds to block 2560 where the slide is deleted. If the duration has not been reached, the process 2500 proceeds to block 2531 where the slide is located in the past slides bank. Process 2500 proceeds to block 2572 and ends.

      Figure 26 illustrates an exemplary wearable device 140 block diagram, in accordance with one embodiment. In one embodiment, the wearable device 140 includes a processor 2610, a memory 2620, a touch screen 2630, a communication interface 2640, a microphone 2665, a timeline logic module 2670, OLED, etc.) module 2650 and an actuator module 2660. In one embodiment, the timeline logic module includes a suggestion module 2671, a notification module 2672, and a user input module 2673.

      In one embodiment, the modules in wearable device 140 are instructions stored in memory and may be executable by processor 2610. In one embodiment, communication interface 2640 may be configured to connect to the host device via various communication methods such as Bluetooth LE, WiFi, and the like. In one embodiment, the optional LED module 2650 may be monochrome or multicolored, and the actuator module 2650 may include one or more actuators. Alternatively, the wearable device 140 may be configured to use the optional LED module 2650 and the actuator module 2660 may be used to transmit unnecessary notifications through each of the specifically programmed displays or vibrations .

      In one embodiment, timeline logic module 2670 may control the overall logic and architecture of how timeline slides are organized in past, now and suggestions. Timeline logic module 2670 can accomplish this by controlling the rules for how long the slides are available for user interaction through slide categories. In one embodiment, the timeline logic module 2670 may or may not include submodules such as the suggestion module 2671, the notification module 2672, or the user input module 2673.

      In one embodiment, the suggestion module 2671 may provide context-based suggestions such as user preferences, location, and the like. Alternatively, the suggestion module 2671 may include a suggestion engine that adjusts and learns the user's preferences through the user's interaction with the proposed slides. In one embodiment, the suggestion module 2671 can remove old or irrelevant suggestion slides and replace them with new, more relevant suggestions.

      In one embodiment, the notification module 2672 may control the display of throttling and notifications. In one embodiment, as described below, the notification module 2672 may have general rules for all notifications. In one embodiment, the notification module 2672 may also distinguish between two types of notifications: critical and non-critical. In an exemplary embodiment, important notifications may be immediately visible on the display and may be accompanied by vibration from the actuator module 2660 and / or LED module 2650 activity. In one embodiment, the screen may be off based on user preferences and the critical notifications may be delivered via vibration and LED activation. In one embodiment, the non-critical notifications may only activate the LED module 2650. In one embodiment, the wearable device 140 may further include other modules described with reference to the wearable device 140 shown in Fig.

      FIG. 27 illustrates exemplary notification functions 2700 in accordance with one example. In one embodiment, the notifications include important notifications 2710 and non-critical notifications 2720. [ The user input module 2673 may recognize user gestures on the touch screen 2630, sensed user actions, or physical buttons that interact with the slides. In one example embodiment, if the user activates the touch screen 2630 in accordance with the new notification, the notification is shown on the touch screen 2630. In one embodiment, the LED of the LED module 2650 is subsequently turned off, indicating a "read" state. In one embodiment, when the notification arrives, if the content was being viewed at the wearable device 140, the touch screen 2630 would not change (to avoid disturbing) And if the message is significant, vibration will also be received from the actuator module 2660. [ In one embodiment, the wearable device 140 touch screen 2630 may be activated after a certain number of idle seconds (e.g., 15 seconds) or after another period of time (e.g., 5 seconds).

      28 illustrates an example of exemplary input gestures 2800 that interact with a timeline architecture, in accordance with one embodiment. In one embodiment, the user may swipe 2820 left or right to search for timelines and suggestions on the timeline 2810. In one embodiment, the tap gesture 2825 on the slide shows additional details 2830. [ In one embodiment, another tab 2825 returns to its original state. In one embodiment, a swipe up 2826 on the slide represents actions 2840. [

      FIG. 29 illustrates an exemplary process 2900 for creating slides in accordance with one embodiment. In one embodiment, the process 2900 begins at a start block 2901. At block 2910, third party data including characters, images or specific actions is received. At block 2920, the image is ready to be displayed on a wearable device (e.g., wearable device 140, Figure 2, Figure 26). At block 2930, the characters are arranged in the designated template area. At block 2940, a dynamic slide is created for the unique actions. At block 2950, the slide is provided to the wearable device. At block 2960, the interaction response is received from the user. At block 2970, the user response is provided to the third party. Process 2900 proceeds to end block 2982.

      30 shows an example of a slide creation 3000 using a template, according to one embodiment. In one embodiment, the timeline slides provide data to the interaction model. In one embodiment, the model allows third party services to interact with users without much resources being required to create the slides. Third party services may provide data as part of external input 1830 (Figure 18). In one embodiment, the third party data may include characters, images, image pointers (e.g., URLs), or specific actions. In one embodiment, such third party data may be provided via other similar methods, such as third party applications, APIs or HTTP. In one embodiment, the wearable device 140 (Figs. 2 and 26) logic, host device (e.g., electronic device 120) on the wearable device 140 to display third- Card or other suitable presentation format for a particular device (e.g., based on screen size or device type) within the cloud 150 (FIG. 2).

      In one embodiment, the data sent to the interaction model can sense the target device and determine the presentation type (e.g., slides / cards, suitable areas, etc.) regarding the display. In one embodiment, the image may be prepared by cropping using pre-adjusted design rules tailored to feature detection and display. For example, the design rules may present a portion of a picture of an object (e.g., an airplane, a human face, etc.) associated with the focus of the display.

      In one embodiment, the template may include designated locations (e.g., pre-adjusted image, text regions, designs, etc.). In a strict sense, an image may be embedded in a background and appropriate characters provided in various areas (e.g., the first or second area). The third party data may also include combined data at an additional level. Additional levels may be prepared using details or action slides. Some actions are default actions that can be included on all slides (eg, remove, bookmark, etc.). In one embodiment, the unique actions provided by the third party service may be located in the dynamic slide created by the template. For example, the unique action shown in representative slide 30 may be an indication that a user has seen an airplane. Dynamic slides are accessible from the default action slides.

      In one embodiment, the prepared slide may be provided to the wearable device 140 in which the timeline logic module 2670 (Fig. 26) indicates the display. In one embodiment, the user response may be received from an interaction. The results may be provided back to third parties in a manner similar to when third party data is first provided, e.g., via a third party application, an API, or through other methods such as HTTP.

      31 illustrates examples of contextual voice commands 3100 based on display slides, in accordance with one embodiment. In one embodiment, wearable device 140 uses a gesture 3110 that includes a long press from slide 3120 to receive voice prompt 3130, for example. Such a push may be a long touch sensed on the touch screen or a long press on the physical button. In one embodiment, general voice commands 3140 and slide-specific voice commands 3150 are interpreted as actions. In one embodiment, a combination of voice commands and gesture interaction on a wearable device 140 (e.g., a wristband) may be used to search for an event-based architecture. As an example of one embodiment, the combination of these voice commands and gesture inputs may be used to register specific gestures for operating the voice prompt 3130 on user input via internal sensors (e.g., accelerometer, gyroscope, etc.) ≪ / RTI >

      In one embodiment, the combined voice and gesture interaction provides a feedback prompt interaction to improve the user experience with visual prompts. In addition, limited gesture / touch based input significantly complements voice commands to support actions on an event based system, such as searching for specific slides / cards, fast filtering and sorting, and the like. In one embodiment, the diagram illustrates an example of context voice commands based on general voice commands 3140 from a slide (e.g., slide specific voice commands 3150) or a display displayed on a touch screen .

      In an exemplary embodiment, a user may perform a long press 3120 actuation of a hard button to activate the voice command function when the slide is displayed. In another embodiment, the voice command function may be operated through touch gestures through embedded sensors or recognized user motions. In an exemplary embodiment, the wearable device 140 configures the voice input when the user turns their wrists while the user is lifting it up to speak to the wristband or when the user performs a sharp wrist swing / .

      In one embodiment, the wearable device 140 displays a visual prompt on the screen to inform the user that it is ready to accept the spoken commands. In another example, the wearable device 140 may include a speaker to provide an audio prompt, or if the wearable is located at a base station or a docking station, the base station may provide an audio prompt Speakers can be configured to do so. In one embodiment, wearable device 140 may provide a haptic notification (such as a specific vibration sequence) to signal that the user is in a listening mode.

      In one embodiment, the user indicates a verbal command from a preset list that can be recognized by the device. In one embodiment, an example of generic voice commands 3140 is shown in example 3100. [ In one embodiment, the instructions may be generic (thus usable on any slide) or contextual, and may be applied to a particular slide displayed. In one embodiment, in certain situations, general instructions 3140 may be contextually associated with the slide currently being displayed. In an exemplary embodiment, the command "check-in" can be checked in to the location if the place slide is displayed. In addition, if a slide contains content of a large list, the command may be used to select a particular content from the list.

      In one embodiment, wearable device 140 may provide system responses that request a description or more information and may wait for a response from the user. In an exemplary embodiment, it may be that the user does not understand the user's commands from the wearable device 140, recognizes the commands as invalid / non-preset commands, or the command requires additional user input have. In one embodiment, once the entire command is ready to be executed, wearable device 140 may perform the next action that the user has identified. In one embodiment, wearable device 140 may request an acknowledgment and then prepare an instruction for execution.

      In one embodiment, the user may also interact with the wearable device 140 through actuation of the touch screen simultaneously with, or together with, voice commands. In an exemplary embodiment, the user may use a finger swipe to scroll up and down to review the command. Other gestures can be used to either clear the commands or tap / taps the virtual confirmation button to accept commands (e.g., tapping the screen to indicate a virtual clear button) . In other embodiments, physical buttons may be used. In an exemplary embodiment, a user may ignore / clear voice commands and other actions by pressing a physical button or switch (e.g., a home button).

      In one embodiment, a wearable device 140 with sensors (e.g., gyroscopes, accelerometers, etc.) is used to register motion gestures as well as finger gestures on the touch screen. In an exemplary embodiment, using registered motions or gestures can be used to cancel or clear commands (e.g., swinging the wearable device 140 once). In other embodiments, a wrist can be tilted for scrolling, a wrist clockwise to move to the next slide, or a counterclockwise search to move to the previous slide can be used. In one embodiment, there may be context motion gestures recognized by particular categories of slides.

In one embodiment, the wearable device 140, in contrast to the application, may utilize an apples process in which the primary display for information includes cards or slides. One or more embodiments allow the user to navigate the system architecture based on the event without parsing each of the slides. In an exemplary embodiment, the user may request a particular slide (e.g., "Show at 6 o'clock this morning") and the slide may be displayed on the screen. These commands may pull back archived slides that are no longer stored in the wearable device 140. In one embodiment, certain commands may present choices that can be presented on the display and selections that can be navigated through the sliding-selection mechanism. In one exemplary embodiment, the voice command "check-in" can display various places such that the user can select one and check in or select one to request a check-in.

In one embodiment, an interesting display for card-based navigation through fast filtering and sorting that facilitates access to the appropriate events can be used. In one exemplary embodiment, the command "What did I do at 3 pm yesterday" provides a display of a subset of available cards by the indicated time. In one embodiment, wearable device 140 may display a visual notification indicating the number of slides that make up a subset or reference. If the number of constituent subsets exceeds a predetermined limit (for example, a number of cards of 10 or more), the wrist band may prompt the user whether they wish to perform further filtering or sorting have. In one embodiment, the user may use the touch input to search for a subset of cards, or may utilize voice commands to further filter or sort the subset (e.g., "arranged according to relevance "," Achievements first seen "etc.).

      In one embodiment, another embodiment includes voice commands to perform actions in third party services on a paired device (e.g., electronic device 120, FIG. 2) can do. In an exemplary embodiment, a user can check in to a place that can be reflected through third party applications such as Yelp, Facebook, etc., without opening a third party service on the paired device. Another example embodiment constitutes a social update command that allows a user to update a status on a social network, such as the Twitter® update, Facebook® status update, etc. shown above.

      In one embodiment, voice commands (e.g., general voice commands 3140 and specific slide voice commands 3150) may be processed by the host device paired with the wearable device 140. [ In one embodiment, the instructions may be communicated to the host device. Alternatively, the host device may provide instructions to help the cloud 150 (Fig. 2) to interpret the commands. In one embodiment, some commands, such as "go to" commands, general actions, etc., may be left exclusively in wearable device 140.

      In one embodiment, the wearable device 140 interacts with external devices or servers primarily through the host device, although in some embodiments the wearable device 140 may be a user of the user, such as a TV, tablet, You can establish a direct communication connection with other devices in the device environment (ecosystem). In one embodiment, other examples of devices may include a temperature controller (e.g., Nest), a balance, a camera, or other connected devices in the network. In one embodiment, such control may include activating or controlling devices or may assist in communicating the various devices.

      In one embodiment, wearable device 140 may recognize a predetermined motion gesture for operating a particular condition of listening, such as a filtered search for a particular category or a particular slide type. For example, the device can recognize the sign language motion about "suggest" and limit the search to proposed category cards. In one embodiment, wearable device 140 based voice commands may utilize a microphone for sleep tracking. In addition, such monitoring may utilize various sensors including an accelerometer, a gyroscope, a photodetector, and the like, which constitute the wearable device 140. Data associated with light, sound, and movement, along with other details about the sleep pattern, can provide a more accurate determination in the analysis that determines when the user is asleep and awake.

      32 illustrates an exemplary block diagram 3200 for a wearable device 140 and host device (e.g., electronic device 120), according to one embodiment. In one embodiment, the wearable device 140 with the voice command module 3210 mounted thereon is coupled to the touch display 2630, the microphone 2665, the sensor 4340, and the communication module 2640 components To provide an output to communication module 2640 for receiving inputs and for communicating commands to a touch display 2630 for prompting / confirmation or to a host device (e.g., electronic device 120) . In one embodiment, the voice command module 3210 may include a gesture recognition module 3220 to process touch or motion inputs from the touch display 2630 or each of the sensors 3230.

      In one embodiment, a host device (e.g., electronic device 120) mounted on voice command processing module 3240 can process execution commands and communicate with communication modules (e.g., communication module 2640) And 125) to the voice command module 3210 of the wearable device 140. In one embodiment, this voice command processing module 3240 may configure the companion application programmed to operate with the wearable device 140 or with a user-friendly background program.

      In one embodiment, the voice command processing module 3240 of the host device (e.g., electronic device 120) may process audio or voice data transmitted from the wearable device 140 only, And provide the data to the voice command module 3210 of the wearable device 140 in the form of command instructions. In one embodiment, the voice command processing module 3240 performs various functions such as distinguishing the cards that are no longer available in the wearable device 140 and providing them with the processed commands to the wearable device 140 Which may include a search command recognition submodule 3250.

      Figure 33 shows an exemplary process 3300 for receiving instructions on a wearable device (e.g., wearable device 140, Figure 2, Figure 26, Figure 32), in accordance with one embodiment. In one embodiment, at any point within the process 3300, the user may interact with the scrolling touch screen to review the commands. In one embodiment, the user at process 3300 may cancel the physical button or use a specific canceled touch / motion gesture. In one embodiment, the user may tap the screen to provide a firmation to accept when the command is presented.

      In one embodiment, process 3300 begins at start block 3301. At block 3310, an instruction to enter the listening mode is received by the wearable device (e.g., wearable device 140, Figures 2, 26, 32). At block 3320, a voice command from the wearable device is prompted to the user. At block 3330, the wearable device receives an audio / voice command from the user. Block 3340 determines whether the voice command received is valid or not. If it is determined that the voice command is not valid, the process 3300 proceeds to block 3335 where the user alerts the user of an invalid command received from the wearable device.

      If it is determined that the voice command is valid, the process 3300 proceeds to block 3350 where it is determined whether a description is required or not. If a description of the voice command is needed for the received voice command, the process 3300 proceeds to block 3355. [ At block 3355, the user is prompted for a description from the wearable device.

      At block 3356, the wearable device receives a description from the user via another voice command. If a determination is made that the voice command is not needed, the process 3300 proceeds to block 3360. At block 3360, the wearable device prepares an instruction for an execution and confirmation request. At block 3370 an acknowledgment is received from the wearable device. At block 3380, process 3300 executes the command and the command is sent to the wearable device for execution. Process 3300 then proceeds to block 3392 and the process ends.

      34 illustrates an exemplary process 3400 for motion based gestures related to a mobile / wearable device, in accordance with one embodiment. In one embodiment, the process 3400 receives instructions including a motion-based gesture on a wearable device (e.g., wearable device 140, Figure 2, 26, 32) And a wearable device (e.g., a cuff band) that senses a predetermined movement or movement of the wearable device 140 in response to the movement. In one embodiment, at any point within the process 3400, the user can scroll and interact with the touch screen to review commands. In yet another embodiment, scrolling may be accomplished through recognized motion gestures such as turning a cuff or other gestures that tilt or rotate the wearable device. In one embodiment, the user can cancel voice commands through various methods that allow the process 3400 to be restarted from where the command was canceled, such as forcing a recently canceled command. In addition, after the displayed prompt, if voice commands or other inputs are not received for a predetermined period of time (e. G., An idle period), the process is stopped and automatically canceled.

In one embodiment, the process 3400 begins at a start block 3401. At block 3410, the motion gesture indication for entering the listening mode is received by the wearable device. At block 3411, a visual prompt about the voice command is displayed on the wearable device. At block 3412, an audio / voice command to search for an event-based architecture is received from the user by the wearable device. At block 3413, audio / speech is provided to the wearable device (e.g., electronic device 120) for processing.

      At block 3414, the processed instruction is received. At block 3420, it is determined whether the command is valid or not. If it is determined that the voice command is not valid, the process 3400 proceeds to block 3415 where a visual indication of the invalid command is displayed. At block 3430, a determination is made as to whether or not a description of the received voice command is required. If the description is determined to be required, process 3400 proceeds to block 3435 where the wearable device is prompted for explanation from the users.

      In block 3436 the voice description is received by the wearable device. At block 3437, audio / voice is provided for processing the wearable device. At block 3438, a process command is received. If it is determined that no description is required, process 3400 proceeds to optional block 3440. At optional block 3440, the instruction is prepared for execution and is also ready for a request for confirmation. At decision block 3450 a confirmation is received. An instruction is executed at optional block 3460, or sent to the wearable device for execution. The process 3400 then proceeds to the last block 3472.

      35 illustrates embodiments 3500 of smart alert wearable device 3510 using haptic components 3540, according to one embodiment. In one embodiment, the haptic array or plurality of haptic components 3540 may be embedded in a wearable device 3510, such as a wristband. In one embodiment, the array is configured to rotate other portions (e.g., portions 3550, portions 3545, or all haptic components 3540) around the bands of the haptic components 3540 And may be customized by the user for unique notifications. In one embodiment, cycled notifications may be presented as one instance of a chasing pattern around a haptic array where users may feel motion around the wrist. In one embodiment, other portions of the wearable device 3510 band may vibrate with a pattern, for example, clockwise or counterclockwise around the wrist. The other patterns are such that the opposite sides of the band are regularly oscillating at the same time (e.g., haptic portions 3550) and then the opposite set of other haptic motor components is vibrated (e.g., Portions 3545) rotation pattern. In an exemplary embodiment, the upper and lower portions may vibrate at the same time, the back side portions thereof may vibrate, and the like. In one exemplary embodiment, the haptic components 3550 of the smart alert wearable device 3510 show opposite sides that vibrate for an alert. In another exemplary embodiment, the haptic components 3545 of the smart alarm wearable device 3510 show four points of vibration on an alarm on a band. In one embodiment, the haptic components 3540 of the smart alarm wearable device 3510 oscillate around the band and vibrate.

      In one embodiment, the pulsing of the haptic components can be limited to allowing the user to feel only a portion of the band pulse at a time. This may be done using adjacent haptic component 3540 motors to offset the vibrations of other parts of the band.

      In one embodiment, the wearable device may include customizable periodic notifications, as well as haptic languages in which certain pulses or patterns have defined meanings. In one embodiment, vibrational patterns or pulses may be used to indicate the new state of the wearable device 3510. In one exemplary embodiment, when important notices or phones are received, when identifying notifications, identifying message senders' identities through unique haptic patterns, and the like.

      In one embodiment, the wearable device 3510 may include more conductive material so that the user can feel the effects of the haptic array. Such a material may be a softer device to enhance the local feel. In one embodiment, a more rigid device may be used for a more uniform vibration feel or mixture of vibrations caused by the haptic array. In one embodiment, the interior of the wearable device 3510 can be customized as shown in the wearable device 3520 to have other types of material (e.g., softer, harder, more elastic, etc.) .

As described above, in one embodiment, the haptic feedback array can be tailored to specific patterns and can be programmed. Programming can be input using a physical force resistance sensor or a touch interface. In one embodiment, wearable device 3510 initiates or records a haptic pattern using one of the mentioned input methods. In yet another embodiment, the wearable device 3510 may be configured to receive a non-verbal message from a particular person (e.g., via pressure, slowly wrapping vibration, etc.) of a touch using a tactile sense, such as a wrist clasp. In one embodiment, the non-verbal message may be a unique vibration or pattern. In an exemplary embodiment, a user may hold their wearable device 3510 to send a programmed unique vibration to a pre-selected recipient, such as by squeezing a band and sending special notifications to a family member. In one embodiment, the customized vibration pattern may be accompanied by a displayed text message, an image, or a specific slide.

      In one embodiment, various methods for haptic pattern writing may be used. In one embodiment, a multidimensional haptic pattern constituting the arrangement, amplitude, phase, vibration, etc. may be recorded. In one embodiment, these components of the pattern may be recorded or interpreted separately from the user input. In one embodiment, an alternative method may utilize the touch screen with a GUI that includes touch input locations corresponding to the various actuators. In an exemplary embodiment, the touch screen may position the x and y axes in accordance with an array of haptic actuators in response to a force input. In one embodiment, a multidimensional pattern algorithm or module may be used to compile the user input into a haptic pattern (e.g., utilizing an array, amplitude, phase, vibration, etc.). Still another embodiment may consider performing haptic pattern writing on a device (e.g., electronic device 120) that is separate from wearable device 3510 using a recording program. In an embodiment, predetermined patterns may be utilized or the program may utilize an intelligent algorithm to help the user generate haptic patterns effortlessly.

      Figure 36 illustrates an example of an exemplary process 3600 for recording a customized haptic pattern, in accordance with one embodiment. In one embodiment, process 3600 may be performed on an external device (e.g., electronic device 120, cloud 150, etc.) and may be executed on a wearable device (e.g., wearable device 140 or 3510, 2, 26, 32, 35). In one embodiment, the flow receives input indicative of initiation of the haptic input recording mode. In one embodiment, the initiation may display or include a GUI or other UI for accepting input commands for customized recording. In one embodiment, the recording mode for the haptic input is continued until no input is detected for a preset limit or time reached or a specified number of seconds (e.g., idle time). In one embodiment, haptic recording proceeds later. The process may involve applying an algorithm to compile the haptic input in a unique pattern. In one exemplary embodiment, the algorithm may convert a single force input for a period of time into a unique pattern that includes variations in amplitude, vibration, and position (e.g., around the wristband). In one embodiment, the processing may include applying one or more filters to transform the input into a rich play-back experience by enhancing or creatively changing the characteristics of the haptic input. In an exemplary embodiment, the filter may remove a haptic sample or apply a fading effect to the input. The processed record may be transmitted or transferred to the recipient. Transfers can be accomplished through a variety of communication interface methods such as Bluetooth®, WiFi, mobile phones, and HTTP. In one embodiment, the transmission of the processed record may involve moving a small size message that is routed to the cloud backend, routed to the phone, and then routed to the wearable device via Bluetooth.

      In one embodiment, the interaction of a wearable device with a human is defined in 3610. At block 3620, recording of the haptic input is initiated. At block 3630, a haptic sample is recorded. At block 3640, it is determined whether a recording limit has been reached or no input has been received for a particular period of time (e.g., a few seconds). If the recording limit is not reached and an input is received, the process 3600 proceeds back to block 3630 and proceeds. If the recording limit has been reached or no input has been received for a certain time, the process 3600 proceeds to block 3660. [ At block 3660, the haptic recording is processed. At block 3670, the haptic recording is transmitted to the recipient. In one embodiment, the process 3600 proceeds back to block 3610 and repeats and flows to or ends the process shown below.

      Figure 37 illustrates an exemplary process 3700 for a wearable device (e.g., wearable device 140 or 3510, Figure 2, 26, 32, 35) receiving and playing a haptic record, in accordance with one embodiment do. In one embodiment, the input record 3710 may be preprocessed at block 3720 to verify that it can be played on the wearable device, i.e., in a suitable format, to confirm that there is no loss and alteration in transmission, and so on. In one embodiment, the recording may be played in a wearable device within block 3730 that allows the user to experience the generated recording.

      In one embodiment, the recording, processing and playback can occur entirely in one device. In the above embodiment, transmission may not be required. In one embodiment, preprocessing in block 3720 may also be omitted. In one embodiment, a filtering block may be used. In one embodiment, the filtering block may be used to eliminate signal interference. Other filters can be used to uniquely add effects to transform a simple input into a rich play-back experience. In an exemplary embodiment, a filter may be applied to selectively fade the recording or enhance recording as the recording travels around the wearable device band.

      38 illustrates an exemplary diagram 3800 of haptic recording, in accordance with one embodiment. In one embodiment, example diagram 3800 illustrates exemplary haptic recording of force over time. In one embodiment, other variables may be used to allow the generation of customized haptic patterns. In one embodiment, diagram 3800 shows a simplified haptic recording. The haptic value is not only dependent on the force, but also is a complex combination of frequency, amplitude and position. In one embodiment, haptic recording can be filtered according to different filters to enhance or uniquely change the characteristics of the signal.

      Figure 39 is a diagrammatic view of a wearable device 3920 (e.g., wearable device 140 or 3510, similar to Figures 2, 26, 32, 35) with respect to haptic input 3930 writing FIG. 39 shows an embodiment 3900 of a force sensor 3910. FIG. In one embodiment, the haptic sensor 3910 can recognize a single type of input, such as a force from the finger 3940, on the sensor. In an exemplary embodiment, the haptic recording with a single haptic input 3930 can be viewed as a force diagram over time (diagram 3800, similar to FIG. 39).

      Figure 40 illustrates a touch relating to haptic input to a wearable device 4010 (e.g., wearable device 140, Figure 2, 26, 32, 3510, Figure 35, 3920, An embodiment 4000 of screen 4020 is shown. In one embodiment, a plurality of methods for recognizing haptic inputs may be used. In an exemplary embodiment, the single type of recognized haptic input may be force 4030 on the sensor by the user's finger. In an exemplary embodiment, another type of haptic input 4040 may include utilizing both touchscreen 4020 and force 4030 on the sensor. The x and y positions on the touch screen 4020 as well as the force 4030 can be recognized as a haptic input. This allows the algorithm to take up space and make freeform approaching to construct the haptic signal. In an exemplary embodiment, the third party type 4050 of the haptic input may be performed using the GUI solely on the touch screen 4020. The input type may include the use buttons displayed by the GUI for different signals, tonalities or effects. In one embodiment, the GUI may configure a mix of buttons and trackpad for additional haptic input combinations.

      41 shows an exemplary block diagram of a wearable device 140 system 4100, according to one embodiment. In one embodiment, touch screen 2630, force sensor 4110, and haptic arrangement 4130 can perform the functions described above. In one embodiment, communication interface module 2640 may be coupled to other devices via various communication interface methods, such as Bluetooth, NFC, WiFi, cellular, etc., to enable transmission or reception of data. In one embodiment, the haptic pattern module 3210 may control the recording of the haptic input with the playback of the haptic input on the start and haptic arrangement 4130. In an exemplary embodiment, the haptic pattern module 4210 may also perform processing of the recorded inputs described above. In one embodiment, the haptic pattern module 4120 may include an algorithm for uniquely configuring the haptic signal, i. E., An algorithm that translates the position and force into a haptic signal around or around the wearable device 140 band. In one embodiment, the haptic pattern module 4120 can also transmit haptic patterns to other devices via the communication interface module 2640 or receive haptic patterns for playback on the wearable device 140.

      Figure 42 illustrates a process block diagram 4200 for contextualizing and presenting user data in accordance with one embodiment. In one embodiment, the process at block 4210 includes collecting information that includes service activity data and sensor data from one or more electronic devices. Block 4220 provides for organizing the information based on the time associated with the collected information. Is provided based on at least one of user context and user activity of at least one of content information and service information of potential interest for one or more electronic devices at block 4230.

      In one embodiment, process 4200 includes filtering organized information based on one or more selected filters. In one example, the user context is determined based on at least one of location information, movement information, and user activity. Organized information can be presented in a specific time order on the visualized timeline. In an exemplary embodiment, providing one or more of the content and services relating to potential interests includes providing one or more of the alerts, suggestions, events, and communications to the one or more electronic devices.

      In one example, the content information and service information may be user subscripted using one or more electronic devices. In one embodiment, the organized information is dynamically delivered to one or more electronic devices. In one embodiment, service activity data, sensor data, and content may be captured as events flagged based on user actions. Service activity data and sensor data from one or more electronic devices may be provided to one or more of a cloud-based system and a network system to determine a user context. In one embodiment, a user context is provided to one or more electronic devices to control one or more of mode activation and notification on one or more electronic devices.

      In one embodiment, the organized information is continuously provided and includes life event information collected via the timeline. The life event information may be stored in one or more of a cloud-based system, a network system, and one or more electronic devices. In one embodiment, the one or more electronic devices comprise mobile electronic devices, and the mobile devices constitute one or more of a mobile phone, a wearable computing device, a tablet device, and a mobile computing device.

      43 is a high-level block diagram illustrating an information processing system including a computing system 500 that executes one or more embodiments. The system 500 includes one or more processors 511 (e.g., ASIC, CPU, etc.) and includes an electronic display device 512 (for displaying graphics, text, and other data), a main memory 513 (E.g., a random access memory (RAM), cache devices, etc.), a storage device 514 (e.g., a hard disk drive), a removable storage device 515 (E.g., a keyboard, a touch screen, a keypad, a positioning tool), and / or a computer readable medium on which data is stored A communication interface 517 (e.g., a modem, a wireless transceiver (such as Wi-Fi, a cellular phone), a network interface (such as an Ethernet card), a communication port, or a PCMCIA slot and card) .

      The communication interface 517 enables software and data transfer between the computer system and external devices via the Internet 550, the mobile electronic device 551, the server 552, the network 553, and the like. The system 500 further includes a communication infrastructure 518 (e.g., a communication bus crossbar or network) to which the aforementioned devices / modules 511 are connected via 517.

      The information transmitted via the communication interface 517 may be transmitted over a communication link that may be implemented using a line or cable carrying signals, an optical fiber, a telephone line, a cellular link, a radio frequency (RF) link, and / Electromagnetic, electro-optical or other signals that may be communicated by the communication interface 517. [0050]

      In performing one or more embodiments within a mobile wireless device (e.g., a mobile phone, a smart phone, a tablet, a mobile computing device, a wearable device, etc.), the system 500 may include an image, such as a camera 128 And an audio capture device 519 such as a capture device 520 and a microphone 122 (FIG. 2). The system 500 includes an MMS module 521, an SMS module 522, an email module 523, a social network interface (SNI) module 524, an audio / video (AV) player 525, a web browser 526, An image capture module 527, and the like.

      In one embodiment, the system 500 includes a life data module 300 that is capable of executing a timeline system 300 that is processed similar to that described with respect to components (Figure 2) of the block diagram 100 (Figure 3) (530). In one embodiment, the life data module 530 is configured to store the lifetime of the system 300 (FIG. 3), 400 (FIG. 4), 1400 (FIG. 14), 1800 (FIG. 18), 3200 (FIG. 41), and flow diagrams 1500 (FIG. 15, 1600, 2500, 2900, 2900, 3300, And 3600 (FIG. 36)). In one embodiment, the life data module 530 may be executed with executable code that resides in the memory of the system 500 with the operating system 529. [ In another embodiment, the life data module 530 may be provided in hardware, firmware, and the like.

      As is known to those of ordinary skill in the art, the above-described exemplary architectures described above in accordance with the above described architectures include program instructions for executing by a processor, software modules, microcode, computer program product on a computer readable medium, May be implemented in a variety of ways such as analog / logic circuits, specific application integrated circuits, firmware, consumer electronics, AV devices, wired / wireless transmitters, wired / wireless receivers, networks, multimedia devices, and the like. Further, embodiments of the above-described architectures may take the form of entirely hardware embodiments, entirely in the form of software embodiments, or in the form of embodiments including both hardware and software components.

      One or more embodiments have been described with reference to block diagrams, mechanisms (systems), and computer program products of the flowchart illustrations and / or methods in accordance with one or more embodiments. Each block, such as descriptions / diagrams or combinations thereof, may be executed by computer program instructions. When computer program instructions are provided to a processor, such instructions that are executed through the processor produce ways to execute the functions / operations disclosed in the flowchart and / or block diagram. Each block in the flowchart / block diagrams may represent hardware and / or software modules or logic executed in one or more embodiments. In alternative implementations, well-known functions in blocks may simultaneously deviate from well-known sequences in the drawings and the like.

      The terms "computer program medium", "computer usable medium", "computer readable medium", and "computer program product" It is used to mean the same medium. Computer program products are methods for providing a computer system with software. Computer readable media allow a computer system to read data, descriptions, messages or message packets and other computer readable information from a computer readable medium. Computer readable media can include, for example, non-volatile memory such as floppy disks, ROM, flash memory, disk drive memory, CD-ROM and other permanent storage. For example, it is useful for moving information, such as data and computer instructions, between computer systems. The computer program instructions may be stored in a computer-readable medium that can direct a computer, other programmable data processing instrument, or other device functioning in a particular manner, for example, instructions stored in the computer- And / or instructions that perform the functions / operations specified in the block diagram block or blocks.

      Computer program instructions that represent block diagrams and / or flowcharts may be loaded into a computer, programmable data processing apparatus, or processing devices to cause a series of operations to generate a computer-implemented process. Computer programs (i.e., computer control logic) are stored in main memory and / or auxiliary memory. The computer programs may also be received via a communication interface. As discussed herein, when these computer programs are executed, they enable the computer system to perform the features of the embodiments. In particular, when computer programs are executed, the computer programs enable a processor and / or a multi-core processor to perform the features of the computer system. These computer programs represent control devices of a computer system. The computer program product includes storing executable instructions by a computer system for performing the method of one or more embodiments and a type of storage medium readable by a computer system.

Although the embodiments have been described with reference to the specific embodiments set forth above, other descriptions are possible. Therefore, the meaning and scope of the appended claims should not be limited to the detailed description of the preferred descriptions contained herein.

Claims (33)

Collecting information including service activity data and sensor data from one or more electronic devices;
Organizing the collected information based on a time associated with the collected information; And
Providing one or more of the electronic devices with at least one of content information and service information for a potential interest based on at least one of a user context and a user activity,
Wherein the providing step divides one or more of the content information and the service information into one of a past category, a current category, and a future category,
Wherein the organized information is presented in a specific time order on a graphical timeline along a first axis of the screen of the one or more electronic devices, wherein the potential interest includes financial, health, and moving And displaying the selected potential interest information corresponding to the graphical timeline when one of the financial, health, and moving potential interests is selected, arranged along a second axis of the screen of the one or more electronic devices Features,
User data contextualization and presentation methods. The method according to claim 1,
≪ / RTI > further comprising filtering the organized information based on the one or more selected filters. 3. The method of claim 2,
Wherein the user context is determined based on at least one of location information, motion information, and user activity. delete The method of claim 3,
Providing one or more of the content and services for potential interests comprises providing one or more of the electronic devices with one or more of alerts, suggestions, events, and communications. 6. The method of claim 5,
Wherein the content information and the service information are user subscribable for use in the one or more electronic devices. delete The method according to claim 1,
Wherein the service activity data, the sensor data, and the content are captured with an event flagged based on a user action. The method according to claim 1,
Wherein the service activity data and the sensor data from the one or more electronic devices are provided to at least one of a cloud based system and a network system to determine the user context,
Wherein the user context is provided to the one or more electronic devices to control one or more of mode activation and notification on the one or more electronic devices. The method according to claim 1,
Wherein the organized information is continuously provided and includes life event information collected over a timeline,
Wherein the life event information is stored in one or more of a cloud based system, a network system, and the one or more electronic devices. The method according to claim 1,
Wherein the one or more electronic devices comprise mobile electronic devices, and wherein the mobile electronic devices include at least one of a mobile phone, a wearable computing device, a tablet device, and a mobile computing device. An activity module configured to collect information including service activity data and sensor data;
An organization module configured to organize the collected information based on the time associated with the collected information; And
An information analyzer module configured to provide at least one of content information and service information for a potential interest to one or more electronic devices based on at least one of a user context and a user activity,
Wherein the information analyzer module divides at least one of the content information and the service information into one of a past category, a current category, and a future category,
Presenting the organized information along a first axis in a specific time sequence on a graphical timeline and arranging and presenting the latent interests that include financial, health, and movement corresponding to each of the graphical timelines along a second axis, And a screen for displaying the selected potential interest information corresponding to the graphical timeline when one of the potential interests of finance, health, and movement is selected. 13. The method of claim 12,
Wherein the organizational module comprises providing filtering on the organized information based on one or more selected filters. 14. The method of claim 13,
Wherein the user context is determined by the information analyzer module based on at least one of location information, movement information and user activity. 15. The method of claim 14,
Wherein at least one of the content information and service information for a potential interest includes at least one of alerts, suggestions, events, and communication. 16. The method of claim 15,
Wherein the content information and the service information allow a user to subscript for use in one or more electronic devices. 13. The method of claim 12,
Wherein the one or more electronic devices comprise a plurality of haptic components for providing a haptic signal. 13. The method of claim 12,
Wherein in response to receiving a recognized user action at the one or more electronic devices, the service activity data, the sensor data, and the content are captured as flagged events. 13. The method of claim 12,
Wherein the service activity data and the sensor data from one or more electronic devices are provided to the information analyzer module running on at least one of a cloud based system and a network system to determine the user context,
Wherein the user context is provided to the one or more electronic devices for controlling one or more of mode activation and notification on the one or more electronic devices. 13. The method of claim 12,
Wherein the organized information is presented continuously and the organized information includes life event information collected via a timeline,
Wherein the life event information is stored in at least one of the cloud-based system, the network system, and the one or more electronic devices. 13. The method of claim 12,
Wherein the one or more electronic devices comprises mobile electronic devices, wherein the mobile electronic devices include one or more of a mobile phone, a wearable computing device, a tablet device, and a mobile computing device. Collecting information including service activity data and sensor data from one or more electronic devices;
Organizing the collected information based on a time associated with the collected information; And
Providing one or more of the electronic devices with one or more of content information and service information for a potential interest based on at least one of a user context and a user activity,
Wherein the providing step divides one or more of the content information and the service information into one of a past category, a current category, and a future category,
Wherein the organized information is presented in a specific time order on a graphical timeline along a first axis of the screen of the one or more electronic devices, wherein the potential interest includes financial, health, and moving And displaying potential interest information corresponding to the graphical timeline when one of the financial, health, and moving potential interests is selected, the graphical timeline being arranged along a second axis of the screen of the one or more electronic devices,
A computer-readable non-transitory recording medium on which a program for execution on a computer is recorded. 23. The method of claim 22,
Wherein the recording medium further comprises filtering the organized information based on one or more selected filters,
Wherein the user context is determined based on at least one of location information, movement information and user activity. 24. The method of claim 23,
Wherein the recording medium provides one or more of the alerts, suggestions, events, and communication to one or more electronic devices to provide one or more of the content information and service information for the potential interest. Recording medium. 25. The method of claim 24,
Wherein the recording medium is capable of subscribing to the content information and the service information for use in the one or more electronic devices,
Wherein the service activity data, the sensor data and the content are captured as an event flagged based on a user action. 23. The method of claim 22,
Wherein the recording medium provides the service activity data and the sensor data from one or more electronic devices to one or more of a cloud based system and a network system to determine the user context,
Wherein the user context is provided to the one or more electronic devices for controlling one or more of mode activation and notification on one or more electronic devices. 23. The method of claim 22,
Wherein the recording medium includes life event information in which the organized information is continuously provided and the organized information is collected through a time line,
Wherein the life event information is stored in at least one of the cloud-based system, the network system, and the one or more electronic devices. 23. The method of claim 22,
The recording medium being characterized in that the one or more electronic devices comprise mobile electronic devices, and wherein the mobile electronic devices comprise at least one of a mobile phone, a wearable computing device, a tablet device and a mobile computing device. In a graphical user interface (GUI) displayed on a display of an electronic device,
One or more timeline events associated with information including service activity data and sensor data collected from at least one electronic device; And
And at least one of selectable service categories and content information for a user's potential interest based on at least one of user activity and user context associated with the one or more timeline events,
Wherein one or more of the content information and the service categories are provided divided into one category of a past category, a current category, and a future category,
Wherein the one or more timeline events are presented in a specific time order along a first axis on the display, the potential interest includes financial, health, and movement corresponding to each of the timeline events and along a second axis on the display And wherein the selected potential interest information is displayed corresponding to the timeline events when one of the potential interests of the financial, health, and movement is selected. 30. The method of claim 29,
One or more icons may be selected to display one or more categories associated with the one or more timeline events,
Wherein one or more of the proposed content information and service information that the user is interested in is provided on the GUI. A display architecture of an electronic device,
A timeline comprising a plurality of content elements and one or more content elements for potential user interest,
Wherein the plurality of time-based elements comprise at least one of event information, communication information, and context alert information, the plurality of time-based elements being displayed in a specific time order,
Wherein the plurality of time-based components are extensible to provide extended information based on receiving a recognized user action,
Wherein one or more content components for the plurality of content components and potential user interests are provided divided into one of a past category, a current category, and a future category,
Wherein the plurality of time-based components are displayed along a first axis of the display architecture, the potential user concern includes financial, health, and movement corresponding to each of the plurality of time-based components, Characterized in that the selected potential interest information corresponding to the plurality of time-based elements is displayed when a potential interest of one of the financial, health and movement is selected, . In a wearable electronic device,
A processor;
A memory coupled to the processor;
Curved display; And
Determining contextual information and using context information determined based on sensor information and additional information received from one or more of the service activity data and additional sensor data from the paired host electronic device to provide content information and services And one or more sensors for providing sensor data to an analysis module that provides one or more of the information to a timeline module of the wearable electronic device,
Wherein at least one of the content information and the service information is provided divided into a category of a past category, a current category, and a future category, the time line module organizing content related to the timeline interface of the curved display,
Wherein the organized content is presented in a specific time order on a graphical timeline along a first axis of the curved display, wherein the potential interest includes financial, health, and movement corresponding to each of the graphical timelines, Wherein the selected potential interest information corresponding to the graphical timeline is displayed on the curved display when one of the potential interests of finance, health, and movement is selected. ≪ RTI ID = 0.0 > Device. Collecting information including service activity data and sensor data from one or more electronic devices;
Organizing the information based on a time associated with the collected information; And
Providing one or more of the electronic devices with at least one of content information and service information for a potential interest based on at least one of a user context and a user activity,
Wherein at least one of the content information and the service information is displayed for a predetermined duration,
Wherein the organized information is presented in a specific time order on a graphical timeline along a first axis of a screen of one or more electronic devices, wherein the potential interest includes financial, health, and movement corresponding to each of the graphical timelines Wherein the selected potential interest information corresponding to the graphical timeline is displayed when a potential interest of one of the financial, health and travel is selected,
User data contextualization and presentation methods.

Images