JP2013542470A - Method and system for controlling the user interface of a device using human exhalation - Google Patents

Abstract

  Certain aspects of a method and system for controlling the user interface of a device using human exhalation can include a device with an implantable micro electro mechanical system (MEMS) sensing and processing module. The MEMS sensing and processing module can detect movement caused by the exhalation of human breath by the user. One or more control signals may be generated in response to detecting motion caused by the exhalation of human breath. The generated control signals can control the user interface of the device and can allow navigation and / or component selection within the user interface.

Description

Incorporation of related applications by cross-reference / reference
[0001] This application is a continuation of US Patent Application No. 12 / 056,164, filed March 26, 2008, current US Patent No. 7,739,061, US Patent Application No. 12 / This is a continuation-in-part of 813,292.

  [0002] This application refers to US Provisional Patent Application No. 61 / 241,376 filed on September 11, 2009, and US Provisional Patent Application No. 61 / 242,201 filed on September 14, 2009, the priority of which is hereby incorporated by reference. Claims the right and claims that right.

[0003] This application also includes
US patent application Ser. No. 12 / 055,999, filed Mar. 26, 2008,
US patent application Ser. No. 12 / 056,203, filed Mar. 26, 2008,
US patent application Ser. No. 12 / 056,171 filed Mar. 26, 2008,
Reference is also made to US patent application Ser. No. 12 / 056,061 filed Mar. 26, 2008 and US Patent Application No. 12 / 056,187 filed Mar. 26, 2008.

[0004] Each of the above referenced applications is incorporated herein by reference in its entirety.
Research and development funded by the federal government
[0005] N / A Microfish / Copyright Reference
[0006] N / A
[0007] Certain embodiments of the invention relate to the control of a computer or electronic system. More particularly, certain embodiments of the invention relate to a method and system for controlling a user interface of a device using human exhalation.

  [0008] Mobile communications have changed the way people communicate, and mobile phones have changed from luxury goods to indispensable elements of everyday life. The use of mobile phones is required today by social conditions rather than being hampered by location or technology.

  [0009] Voice connections meet the basic needs of communication, and mobile voice connections continue to penetrate deeper into everyday life structures, but mobile access to services over the Internet is the next in the mobile communications revolution. It is a step. Currently, most mobile devices are equipped with a user interface that allows users to access services provided over the Internet. For example, some mobile devices may have a browser, and software and / or hardware buttons may be provided to allow navigation and / or control of the user interface. Some mobile devices, such as smartphones, have a touch screen feature that allows the user to navigate or control the user interface via touching with one hand while the device is held with the other hand Equipped with.

  [0010] Further limitations and shortcomings of conventional and traditional approaches will become apparent to those skilled in the art through comparison of such systems according to the present invention, as described in the remainder of this application, with reference to the drawings. It will be.

  [0011] As described more fully in the claims, substantially using human breath, as illustrated and / or described in connection with at least one of the drawings. A system and / or method for controlling a user interface of a device.

  [0012] Various advantages, aspects, and novel features of the present invention, as well as details of illustrated embodiments thereof, will be more fully understood from the following description and drawings.

[0013] FIG. 2 is a block diagram of an exemplary system for controlling user interfaces of multiple devices using human exhalation, according to one embodiment of the invention. [0014] FIG. 4 is a block diagram of an exemplary sensing module for detecting human exhalation, according to one embodiment of the invention. [0015] FIG. 4 is a block diagram of another embodiment of an exemplary system for controlling a user interface of a device using human exhalation according to an embodiment of the present invention. [0016] FIG. 4 is a block diagram of an exemplary processor interacting with a device being controlled, according to one embodiment of the invention. [0017] FIG. 2 is a block diagram of an exemplary system for sideloading information between two or more devices, according to one embodiment of the invention. [0018] FIG. 4 illustrates an exemplary MEMS sensing and processing module embedded in a device, according to one embodiment of the invention. [0019] FIG. 6 illustrates an exemplary MEMS sensing and processing module located on a stand-alone device that is communicatively coupled to a device via a USB interface, according to one embodiment of the invention. [0020] FIG. 6 illustrates an exemplary MEMS sensing and processing module located on a stylus, according to one embodiment of the invention. [0021] FIG. 5 illustrates an exemplary MEMS sensing and processing module located on a military headset, according to one embodiment of the invention. [0022] FIG. 6 illustrates an exemplary MEMS sensing and processing module located on a headrest of a seating device, according to one embodiment of the invention. [0023] FIG. 6 illustrates an exemplary MEMS sensing and processing module located inside an automobile, according to one embodiment of the invention. [0024] FIG. 5 illustrates an exemplary MEMS sensing and processing module located on removable eyewear, according to one embodiment of the invention. [0025] FIG. 6 illustrates an exemplary MEMS sensing and processing module located on a neckset, according to one embodiment of the invention. [0026] FIG. 6 illustrates an exemplary MEMS sensing and processing module located on a clip, according to one embodiment of the invention. [0027] FIG. 6 illustrates an exemplary MEMS sensing and processing module embedded in a fabric, according to one embodiment of the invention. [0028] FIG. 6 illustrates an example electronic device that may be controlled via a partition user interface according to one embodiment of the invention. [0029] FIG. 4 illustrates several example configurations of a partition user interface, according to one embodiment of the invention. [0030] FIG. 6 illustrates several exemplary fixed areas of a partition user interface, according to one embodiment of the invention. [0031] FIG. 6 illustrates several example content areas of a partition user interface, according to one embodiment of the invention. [0032] FIG. 6 illustrates interacting with a compartment user interface of an electronic device via respiratory and haptic input, according to one embodiment of the present invention. [0033] FIG. 6 illustrates an example partition user interface that can provide an indication of a category and / or icon sequence when scrolling, according to an embodiment of the invention. [0034] FIG. 6 illustrates interacting with an example compartment user interface via respiratory and haptic input, according to one embodiment of the present invention. [0035] FIG. 6 illustrates another exemplary partition user interface that can provide an indication of a category and / or icon sequence when scrolling, according to an embodiment of the invention. [0036] FIG. 6 illustrates launching an application via a user interface utilizing respiration and haptic input, according to one embodiment of the invention. [0037] FIG. 6 illustrates an exemplary interaction with an application executing on an electronic device, according to one embodiment of the invention. [0038] FIG. 6 illustrates an exemplary interaction with an application executing on an electronic device, according to one embodiment of the invention. [0039] FIG. 6 is a block diagram of an exemplary user interface interacting with a MEMS sensing and processing module and a host system, according to one embodiment of the invention. [0040] FIG. 6 is a flow chart illustrating exemplary steps for processing signals to control a device using human exhalation. [0041] FIG. 6 is a flow chart illustrating exemplary steps for controlling the user interface of a device using human exhalation according to one embodiment of the invention. [0042] FIG. 6 is a flowchart illustrating exemplary steps for performing sideloading of information, in accordance with one embodiment of the present invention.

[0043] Certain aspects of the present invention may be found in methods and systems for controlling the user interface of a device using human exhalation. Exemplary aspects of the invention can comprise a device comprising an implantable micro electro mechanical system (MEMS) sensing and processing module. The MEMS sensing and processing module can detect movement caused by the exhalation of human breath by the user. One or more control signals may be generated in response to detecting motion caused by the exhalation of human breath. The generated control signals may be utilized to control the user interface of the device and may allow navigation and / or component selection within the user interface. The generated control signal or signals are sent to the device being controlled by an external memory interface, a general purpose asynchronous transceiver (UART) interface, an extended serial peripheral interface (eSPI), a general purpose input / output (GPIO) interface, Pulse code modulation (PCM) and / or inter-IC sound (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface, Bluetooth interface, ZigBee interface, IrDA interface, and / or The wireless USB (W-USB) interface can communicate via one or more. Human exhalation may occur in open space, and detection of movement caused by the exhalation may occur without using a channel. One exemplary embodiment of a user interface is a graphical user interface (GUI).

  [0044] FIG. 1A is a block diagram of an exemplary system for controlling user interfaces of multiple devices using human exhalation, according to one embodiment of the invention. Referring to FIG. 1A, a user 102 and a multimedia device 106a, cell phone / smartphone / data phone 106b, personal computer (PC), laptop or notebook computer 106c, display device 106d, and / or television (TV) ) / Game console / other platforms 106e, etc. Multimedia device 106a, mobile phone / smartphone / dataphone 106b, personal computer (PC), laptop or notebook computer 106c, display device 106d, and / or television (TV) / game console / other platform 106e, etc. Each of the plurality of controlled devices can include an embedded micro electro mechanical system (MEMS) sensing and processing module 104.

  [0045] The multimedia device 106a can include a user interface 107a, and the mobile phone / smartphone / dataphone 106b can include a user interface 107b, and can be a personal computer (PC), laptop or notebook computer 106c. May include a user interface 107c. In addition, the display device 106d can include a user interface 107d, and the television (TV) / game console / other platform 106e can include a user interface 107e. Each of the plurality of controlled devices, for example, for loading information via sideloading or loading via a peer-to-peer connection and / or via a network connection and by wired and / or wireless communication In addition, it can be connected to a plurality of other devices 108 in a wired or wireless manner. Other exemplary devices 108 may include game consoles, immersive or 3D reality devices, and / or telematic devices. A telematic device refers to a device comprising integrated computing, wireless communications and / or a global navigation satellite system device, allowing transmission, reception and / or storage of information over a network. The user interface may be one or more inputs, eg, a fluid input such as air, a tactile input such as a button press, an audio input such as a voice command, and / or detected by an accelerometer and / or gyroscope Etc., movement of the electronic device 202 may allow interaction with the device being controlled.

  [0046] The MEMS sensing and processing module 104 may comprise suitable logic, circuitry, interfaces and / or code that may be operable to detect movement caused by the exhalation of human breath by the user 102. it can. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The MEMS detection and processing module 104 may be operable to detect kinetic energy generated by the exhalation of human breath and to generate one or more control signals accordingly. Or it can comprise a plurality of segments or members. The generated one or more control signals are the user interface 107a of the multimedia device 106a, the user interface 107b of the mobile phone / smartphone / dataphone 106b, the user interface 107c of the PC, laptop or notebook computer 106c, display device Control one or more user interfaces of multiple devices, such as user interface 107d of 106d, user interface 107e of TV / game console / other platform 106e, and mobile multimedia player and / or remote controller user interface May be operable to One exemplary embodiment of a user interface is a graphical user interface (GUI). Any information and / or data presented on the display, including programs and / or applications, can be part of the user interface. US patent application Ser. No. 12 / 055,999, filed Mar. 26, 2008, discloses an exemplary MEMS sensing and processing module and is incorporated herein by reference in its entirety.

  [0047] According to one embodiment of the present invention, detection of motion caused by the exhalation of human exhalation may occur without using a channel. Detection of movement caused by the exhalation of human exhalation can be responsive to the expulsion of human exhalation into an open space, which is then detected.

  [0048] According to another embodiment of the present invention, the MEMS sensing and processing module 104 may communicate the handheld device, eg, multimedia device 106a, mobile phone / smartphone, via the generated one or more control signals. / Navigating in one user interface of more of multiple devices, such as data phone 106b, PC, laptop or notebook computer 106c, display device 106d, and / or TV / game console / other platform 106e May be operable to The MEMS sensing and processing module 104 may be operable to select one or more components within the user interface of the plurality of devices via the generated one or more control signals. The generated one or more control signals may comprise one or more wired and / or wireless signals.

  [0049] According to another embodiment of the present invention, a handheld device, such as a multimedia device 106a, and / or a cell phone / smartphone / data phone 106b, and / or a PC, laptop or notebook computer 106c Etc., one or more of the plurality of devices may be operable to receive one or more inputs defining another user interface from another device 108. Other devices 108 may be one or more of a PC, laptop or notebook computer 106c, and / or handheld device, eg, multimedia device 106a and / or cell phone / smartphone / dataphone 106b. There may be. In this regard, data may be transferred from other devices 108 to the cell phone / smart phone / data phone 106b, which may be transferred to the cell phone / smart phone / data via a service provider, such as a cell phone or PCS service provider. It may be associated or mapped to media content that may be remotely accessed by phone 106b. The transferred data associated or mapped to the media content may be utilized to customize the user interface 107b of the mobile phone / smartphone / dataphone 106b. In this regard, the media content associated with one or more received inputs can be an integral part of the user interface of the device being controlled. The association and / or mapping may be done either on the other device 108 and / or one mobile phone / smartphone / dataphone 106b. If the association and / or mapping is performed on the other device 108, the associated and / or mapped data may be transferred from the other device 108 to the mobile phone / smartphone / dataphone 106b.

  [0050] In an exemplary embodiment of the invention, the icons transferred from the other device 108 to the mobile phone / smartphone / dataphone 106b are transmitted to the mobile phone / smartphone / data via the service provider of the mobile phone / smartphone 106b. It may be associated or mapped to media content, such as RSS feeds, markup languages such as HTML and XML, which may be remotely accessed by the phone 106b. Thus, when the user 102 blows on the MEMS sensing and processing module 104, the control signal generated by the MEMS sensing and processing module 104 can navigate to and select that icon. After the icon is selected, the RSS feed or markup language can be accessed via the mobile phone / smartphone / dataphone 106b service provider, and the corresponding RSS feed or markup language content can be accessed by the user interface 107b. Can be displayed above. US patent application Ser. No. 12 / 056,187, filed Mar. 26, 2008, discloses an exemplary method and system for customizing the user interface of a device and is hereby incorporated by reference in its entirety.

  [0051] In operation, the user 102 can exhale into an open space, and the exhaled breath or air is one or more sensors, sensing members, and / or sensing segments within the MEMS sensing and processing module 104. Etc., which may be sensed by one or more detection devices or detectors. The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. The one or more electrical, optical and / or magnetic signals are one or more detection devices or detectors in the MEMS sensing and processing module 104 in response to detecting motion caused by the exhalation of human breath. Can be generated. The processor firmware in the MEMS sensing and processing module 104 receives received electrical, optical and / or magnetic signals from one or more detection device (s) or detector (s) that utilize various algorithms. And may be operable to generate one or more control signals to the device being controlled, eg, multimedia device 106a. The generated one or more control signals may be communicated to the controlled device, eg, multimedia device 106a, via wired and / or wireless signals. The processor in the device being controlled includes a user interface 107a of the multimedia device 106a, a user interface 107b of a mobile phone / smartphone / dataphone 106b, a user interface 107c of a personal computer (PC), laptop or notebook computer 106c, To control the user interface of the device being controlled, such as the user interface 107d of the display device 106d, the user interface 107e of the TV / game console / other platform 106e, and the user interface of the mobile multimedia player and / or remote controller In addition, the communicated control signal can be used.

  [0052] FIG. 1B is a block diagram of an exemplary detection device or detector for detecting human expiration, according to one embodiment of the invention. Referring to FIG. 1B, a user 102 and a detection module 110 are illustrated. The detection module 110 can include a sensor control chip 109 and a plurality of sensors, for example, 111a, 111b, 111c, and 111d. Nevertheless, the present invention may not be so limited, and without limiting the scope of the present invention, the sensing module 110 may have more or fewer sensors or sensing members than shown in FIG. A segment can be provided. Thus, any number of detectors and sources may be utilized according to the desired size, sensitivity and desired resolution. Similarly, source and detector types may include other sensing mechanisms other than visible light. For example, piezoelectric, ultrasonic, Hall effect, electrostatic, and / or permanent or electromagnet sensors can be activated by a distorted MEMS member to generate a signal that is communicated to the sensor control chip 109.

  [0053] The sensing module 110 may be, for example, an electrochemical sensor or any other type of breath analysis sensor. The plurality of sensors or sensing members or segments 111a-d may be an integral part of one or more MEMS devices that may allow detection of air flow at various velocities from the user 102 exhalation. Good. The plurality of sensors or sensing members or segments 111a-d may be operable to detect kinetic energy and / or movement caused by the exhalation of human exhalation by the user 102. The sensor control chip 109 is operable to generate electrical, optical and / or magnetic signals that can be communicated to the processor in response to detection of kinetic energy and / or movement caused by human exhalation. sell.

  [0054] FIG. 1C is a block diagram of another embodiment of an exemplary system for controlling the user interface of a device using human exhalation, according to one embodiment of the present invention. Referring to FIG. 1C, the user 102 and multimedia device 106a, cell phone / smartphone / data phone 106b, PC, laptop or notebook computer 106c, display device 106d, and / or TV / game console / other A device 106 being controlled, such as platform 106e, is illustrated. The controlled device 106 may be wired and / or wirelessly connected to a plurality of other devices 108 for side loading of information. The device 106 being controlled can include a MEMS sensing and processing module 104 and a user interface 107.

  [0055] The MEMS detection and processing module 104 may comprise a detection module 110, a processing module 112, and a passive device 113. Passive devices 113, which may include resistors, capacitors and / or inductors, may be embedded within the substrate material of the MEMS process sensing and processing module 104. The processing module 112 can comprise, for example, an ASIC. The sensing module 110, commonly referred to as a sensing device or detector, may be operable to detect kinetic energy and / or movement caused by the exhalation of human breath by the user 102. Sensors, sensing members and / or sensing segments. The sensing module 110 is operable to generate electrical, optical and / or magnetic signals that can be communicated to the processing module 112 in response to detection of kinetic energy and / or movement caused by the exhalation of human breath. It is possible.

  [0056] The processing module 112 is operable to receive the generated electrical signal from the sensing module 110 and to generate one or more control signals to the device 106 being controlled. Appropriate logic, circuitry, interfaces and / or code may be provided. In this regard, the processing module 112 may comprise one or more analog-to-digital converters that may be operable to convert the sensed signal into one or more digital signals. Alternatively, a plurality of digital signals can be utilized to generate one or more control signals. The generated one or more control signals may be operable to control the user interface 107 of the device 106 being controlled.

  [0057] Accordingly, the generated signal or signals from the MEMS sensing and processing module 104 may be utilized to control the user interface 107. In an exemplary embodiment of the invention, the one or more signals generated by MEMS sensing and processing module 104 are controlled so that items in user interface 107 are selectable and / or operable. It may be operable to control the pointer on 106. In an exemplary embodiment of the invention, the device being controlled may be operable to receive one or more inputs from other devices 108, where the one or more inputs are It can be used to customize or define the interface 107. Other devices 108 may be one or more of a PC, laptop or notebook computer 106c, and / or handheld device, eg, multimedia device 106a and / or cell phone / smartphone / dataphone 106b. There may be. In this regard, other devices 108 may be similar to or different from the type of device 106 being controlled. In some embodiments of the present invention, processors in other devices 108 may be operable to associate or map data to media content that is remotely accessible by the controlled device 106. In other embodiments of the invention, the processor in the controlled device 106 may be operable to associate or map data to media content that is remotely accessible by the controlled device 106. US patent application Ser. No. 12 / 056,187, filed Mar. 26, 2008, discloses an exemplary method and system for customizing the user interface of a device and is hereby incorporated by reference in its entirety.

  [0058] FIG. 1D is a block diagram of an exemplary processor interacting with a controlled device, according to one embodiment of the invention. Referring to FIG. 1D, control such as multimedia device 106a, mobile phone / smartphone / dataphone 106b, PC, laptop or notebook computer 106c, display device 106d, and / or TV / game console / other platform 106e Device 106 is shown. The device 106 being controlled can comprise a processing module 112, a communication module 120, a processor 122, a memory 123, firmware 124, a display 126, and a user interface 128. The processing module 112 may be an ASIC and may include one or more analog-to-digital converters (ADC) 114, processor firmware 116, and a communication module 118. The device 106 being controlled may perform information loading, for example, via side loading or loading via a peer-to-peer connection and / or via a network connection and via wired and / or wireless communication. It can be connected to multiple other devices 108 in a wired and / or wireless manner.

  [0059] The processing module 112 may comprise suitable logic, circuitry, interfaces, and / or code that may be operable to receive digital and / or analog sensing signals from the sensing module 110. The ADC 114 may be operable to receive the generated analog sensing signal from the sensing module 110 and to convert the received signal to a digital signal, suitable logic, circuitry, interface and / or code. Can be provided.

  [0060] The processor firmware 116 utilizes a plurality of algorithms to receive a digital signal from the ADC 114 and / or a digital sensing signal from the sensing module 110 to generate one or more control signals. Appropriate logic, circuitry and / or code may be provided that may be operable to process. For example, the processor firmware 116 may be operable to read, store, calibrate, filter, modelize, calculate and / or compare the output of the sensing module 110. The processor firmware 116 may also be operable to incorporate artificial intelligence (AI) algorithms to adapt to a particular user's 102 breathing pattern. The processor firmware 116 may be operable to generate one or more control signals to the device 106 being controlled based on the processing of the received digital signals. The generated control signal or signals are operable to control the user interface of the controlled device 106, such as, for example, scrolling, zooming and / or 3-D navigation within the controlled device 106. It can be.

[0061] The communication module 118 comprises appropriate logic, circuitry, interfaces and / or code that may be operable to receive the generated one or more control signals and communicate to the communication module 120. be able to. Communication modules 118 and 120 may support multiple interfaces. For example, the communication modules 118 and 120 may include external memory interfaces, general purpose asynchronous transceiver (UART) interfaces, extended serial peripheral interfaces (eSPI), general purpose input / output (GPIO) interfaces, pulse code modulation (PCM) and / or ICs. Inter-sound (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface, Bluetooth interface, ZigBee interface, IrDA interface, and / or wireless USB (W-USB) interface Can be supported.

  [0062] The communication module 120 may be operable to receive communicated control signals via wired and / or wireless signals. The processor 122 may utilize appropriate logic, circuitry, interfaces and / or code that may be operable to control the user interface 128 and / or the display 126 utilizing the received one or more control signals. Can be provided. The memory can comprise suitable logic, circuitry, interfaces and / or code that can be operable to store data on the device 106 being controlled. Firmware 124 may comprise a plurality of drivers and an operating system (OS) library to convert received control signals into functional commands. Firmware 124 converts received control signals into compatible data, such as user customization features, applets and / or plug-ins, to map local functions and to control user interface 128. May be operable.

  [0063] The device 106 being controlled may be operable to receive one or more inputs defining a user interface 128 from another device 108. Other devices 108 can include a user interface 129 and a processor 125. Other devices 108 may be one or more of a PC, laptop or notebook computer 106c, and / or handheld device, eg, multimedia device 106a and / or cell phone / smartphone / dataphone 106b. There may be. In this regard, data may be transferred from another device 108 to a controlled device, such as a mobile phone / smartphone / dataphone 106b, which is routed through a service provider, such as a mobile or PCS service provider. And may be associated or mapped to media content that may be remotely accessed by the mobile phone / smartphone / dataphone 106b. The transferred data associated or mapped to the media content can be utilized to customize the user interface 128 of the device being controlled, such as cell phone / smart phone / data phone 106b. In this regard, media content associated with one or more received inputs can be an integral part of the user interface 128 of the device 106 being controlled.

  [0064] In some embodiments of the invention, the processor 125 in the other device 108 is operable to associate or map data to media content that is remotely accessible by the device 106 being controlled. It is possible. In other embodiments of the invention, the processor 122 in the controlled device 106 may be operable to associate or map data to media content that is remotely accessible by the controlled device 106. .

  [0065] FIG. IE is a block diagram of an exemplary system for sideloading information between two or more devices, according to one embodiment of the invention. Referring to FIG. 1E, a carrier network 124, a plurality of mobile phones 130a, 130b, 130c and 130d, such as a plurality of controlled devices 106, a PC, laptop or notebook computer 132 connected to a network 134 such as the Internet. Is illustrated. Network 134 may be coupled to web server 136, wireless carrier portal 138, web portal 140 and / or database 142. Each of the plurality of controlled devices 106 may have a user interface. For example, the mobile phone 130a can have a user interface 131a, the mobile phone 130b can have a user interface 131b, the mobile phone 130c can have a user interface 131c, and the mobile phone 130d can be a user An interface 131d can be included. The PC, laptop or notebook computer 132 can have a user interface 133.

  [0066] The carrier network 124 may be a radio access carrier network. Exemplary carrier networks may include 2G, 2.5G, 3G, 4G, IEEE 802.11, IEEE 802.16, and / or suitable networks capable of handling voice, video and / or data communications . A plurality of controlled devices 106 may be wirelessly connected to the carrier network 124. One of the devices being controlled, such as mobile phone 130a, may be connected to a plurality of mobile phones 130b, 130c and 130d via, for example, a peer-to-peer (P2P) network. A device being controlled, such as mobile phone 130a, may be communicatively coupled to a PC, laptop or notebook computer 132 via a wired or wireless network. For example, the mobile phone 130a can be communicatively coupled to a PC, laptop or notebook computer 132 via an infrared (IR) link, an optical link, a USB link, a wireless USB, a Bluetooth link and / or a ZigBee link. Nevertheless, the invention may not be so limited and other wired and / or wireless links may be utilized without limiting the scope of the invention. A PC, laptop or notebook computer 132 may be communicatively coupled to a network 134, eg, an Internet network 134, via a wired or wireless network. A plurality of controlled devices, such as a plurality of mobile phones 130a, 130b, 130c and 130d, may be wirelessly connected to the Internet network 134.

  [0067] The web server 136 is configured to receive HTTP and / or FTP requests from, for example, a client or web browser installed on a PC, laptop, or notebook computer 132 via the Internet network 134, and Appropriate logic, circuitry and / or code may be provided that may be operable to generate an HTTP response with optional data content such as, for example, HTML documents and linked objects.

  [0068] The wireless carrier portal 138 is suitable logic that may be operable to serve as a point of access to information on the Internet network 134 via a mobile device, such as a mobile phone 130a. And / or a code. The wireless carrier portal 138 may be, for example, a website that may be operable to provide a single function via, for example, a mobile web page.

  [0069] The web portal 140 may include suitable logic and / or code that may be operable to serve as a point of access to information on the Internet 134. Web portal 140 may be a site that may be operable to provide a single function via, for example, a web page or site. The web portal 140 can present information from a variety of sources, such as email, news, stock prices, infotainment, and various other feature articles in a unified manner. Database 142 may comprise suitable logic, circuitry, and / or code that may be operable, for example, to store a structured collection of records or data. Database 142 may be operable to organize the storage of data utilizing software.

  [0070] According to one embodiment of the present invention, the device being controlled, such as mobile phone 130a, may define one or more user interfaces 128 from another device, such as a PC, laptop or notebook computer 132. It may be operable to receive multiple inputs. The one or more processors 122 in the controlled device 106 may be an integral part of the user interface 128 of the device being controlled, such as the mobile phone 130a, with the content associated with the one or more received inputs. Can be operable to customize the user interface 128 of a device being controlled, such as mobile phone 130a. Mobile phone 130a may be operable to access content directly from a PC, laptop or notebook computer 132 rather than from the carrier network 124. This method of uploading and / or downloading customized information directly from a PC, laptop or notebook computer 132 rather than from the carrier network 124 may be referred to as side loading.

  [0071] According to one embodiment of the invention, the user interface 128 may be created, modified and / or organized by the user 102. In this regard, the user 102 may select, select, create, arrange, manipulate, and / or select content to be utilized for the user interface 128 and / or one or more content components. Can be organized. For example, the user 102 can organize content components on the screen and can select content such as personal photos for backgrounds and / or icon images. In addition, the user 102 can create and / or modify the manner in which the content component is activated or presented to the user 102. For example, the user 102 can create, import and / or edit icons and / or backgrounds for the user interface 128. Thus, the user 102 can associate and / or map an icon to a function so that the user 102 can enable or activate the function via the icon. Exemplary icons can enable features such as hyperlinks, bookmarks, programs / applications, shortcuts, widgets, RSS or markup language feeds or information, and / or favorite buddies.

  In addition, user 102 can organize and / or arrange content components within user interface 128. For example, icons can be organized into groups by category. A group of icons, such as content components, may be referred to as affinity banks, for example. In some embodiments of the invention, the processor 125 in the other device 108 may be operable to associate or map data to media content that is remotely accessible by the device 106 being controlled. In other embodiments of the invention, the processor 122 in the controlled device 106 may be operable to associate or map data to media content that is remotely accessible by the controlled device 106. . For example, the processor 122 can enable the user 102 to enable or activate functionality via an icon and can organize and / or arrange content components within the user interface 128. , May be operable to associate and / or map icons to functions.

  [0073] Creation, modification, and / or organization of user interface 128 and / or content components may be performed on a controlled device, such as mobile phone 130a, and / or a PC, laptop, or notebook. It may be performed on another device, such as computer 132. In this regard, user screens and / or audio that can be created, modified, and / or organized on another device, such as a PC, laptop, or notebook computer 132, are sideloaded to a controlled device, such as mobile phone 130a. Can be done. In addition, the sideloaded user interface 128 can be modified and / or organized on a controlled device, such as the mobile phone 130a. For example, the user interface 128 may be sideloaded from a PC, laptop or notebook computer 132 to the mobile phone 130a and can be customized on the mobile phone 130a. One or more tools may allow creation, modification, and / or organization of user interface 128 and / or audio or visual content components.

  [0074] FIG. 2A is a diagram illustrating an exemplary MEMS sensing and processing module embedded in a device, according to one embodiment of the invention. Referring to FIG. 2A, a user 102 and a device being controlled, such as a cell phone / smartphone / data phone 106b, is illustrated. The cell phone / smart phone / data phone 106b may include a user interface 107b and an embedded MEMS detection and processing module 104. In one embodiment of the invention, the user 102 may be allowed to exhale into open space and onto the MEMS sensing and processing module 104.

  [0075] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The MEMS detection and processing module 104 may be operable to detect kinetic energy generated by the exhalation of human breath and to generate one or more control signals accordingly. Or it can comprise a plurality of segments or members. The generated one or more control signals may be operable to control the user interface 107b of the mobile phone / smartphone / dataphone 106b. According to another embodiment of the present invention, the MEMS sensing and processing module 104 may be embedded in an interactive kiosk or panel, eg, an ATM machine. The user 102 is enabled to blow a blow of air with the MEMS detection and processing module 104 embedded in the interactive kiosk, for example, to access and / or interact with the interactive kiosk user interface. Also good.

  [0076] FIG. 2B is a diagram illustrating an exemplary MEMS sensing and processing module located on a stand-alone device that is communicatively coupled to the device via a USB interface, according to one embodiment of the invention. Referring to FIG. 2B, a stand-alone device 262 and another device such as a PC, laptop or notebook computer 132 are illustrated. Stand-alone device 262 may be placed on any suitable surface, for example on a table or desktop 263. Stand-alone device 262 may include a support structure 264 that is pliable. The support structure 264 can comprise, for example, a MEMS sensing and processing module 104 located at one end. Nevertheless, the present invention may not be so limited, and the MEMS sensing and processing module 104 may be located elsewhere on the stand-alone device 262, for example within the base of the stand-alone device 262. Nevertheless, the present invention may not be limited in this regard, and the location of the MEMS sensing and processing module 104 in or on the stand-alone device 262 may vary accordingly. The MEMS detection and processing module 104 can be communicatively coupled to a PC, laptop, or notebook computer 132 via, for example, a USB interface 265.

  [0077] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the discharge of fluid, such as air, from human exhalation, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The MEMS detection and processing module 104 may be operable to detect kinetic energy generated by the exhalation of human breath and to generate one or more control signals accordingly. Or it can comprise a plurality of segments or members. The generated one or more control signals can be communicated to another controlled device, such as a PC, laptop or notebook computer 132. The generated one or more control signals may be operable to control the user interface 133 of the PC, laptop or notebook computer 132.

  [0078] FIG. 2C is a diagram illustrating an exemplary MEMS sensing and processing module located on a stylus, according to one embodiment of the invention. Referring to FIG. 2C, a user 102 and a device being controlled, such as a mobile phone / smartphone / dataphone 106b, is illustrated. The cell phone / smart phone / data phone 106b may include a user interface 107b and a stylus 202. The stylus 202 may be retractable, foldable, rotatable about one or more axes, and / or bendable, and may fit within the body of the cell phone / smartphone / dataphone 106b. . Nevertheless, without limiting the scope of the present invention, the stylus 202 may be a foldable device that can be clipped to the body of the cell phone / smartphone / dataphone 106b.

  [0079] The stylus 202 may comprise, for example, a MEMS sensing and processing module 104 located at one end. In one embodiment of the present invention, the user 102 may be allowed to retract the stylus 202 to exhale into the open space and onto the MEMS sensing and processing module 104.

  [0080] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The MEMS detection and processing module 104 may be operable to detect kinetic energy generated by the exhalation of human breath and to generate one or more control signals accordingly. Or it can comprise a plurality of segments or members. The generated one or more control signals may be operable to control the user interface 107b of the mobile phone / smartphone / dataphone 106b.

  [0081] FIG. 2D is a diagram illustrating an exemplary MEMS sensing and processing module located on a military headset, according to one embodiment of the invention. Referring to FIG. 2D, the user 102 is illustrated. The user 102 can wear a removable helmet 208. The removable helmet 208 can include removable eyewear 204, a removable microphone 206, and a removable headset 210. The removable headset 210 can comprise, for example, a MEMS sensing and processing module 104 located at one end.

  [0082] The removable eyewear 204 may comprise, for example, night vision and / or infrared vision capabilities. The removable microphone 206 can be utilized, for example, to communicate with other users. In one embodiment of the present invention, the user 102 may be allowed to exhale into an open space and the MEMS sensing and processing module 104 is operable to detect or detect exhalation. sell. Exhaling may result from the user's 102 nostrils and / or mouth.

  [0083] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The generated control signal or signals are transmitted to the user interface 107a of the multimedia device 106a, the user interface 107b of the mobile phone / smartphone / dataphone 106b, the user interface of the personal computer (PC), laptop or notebook computer 106c. 107c and / or may be operable to control the user interface of the device being controlled, such as user interface 107d of display device 106d.

  [0084] FIG. 2E is a diagram illustrating an exemplary MEMS sensing and processing module located on a headrest of a seating device, according to one embodiment of the invention. Referring to FIG. 2E, the seat device 220 is illustrated. The seat device 220 can include a headrest 222 and a backrest 226. The headrest 222 can include a removable headset 224. User 102 may be allowed to sit on seat device 220.

  [0085] The removable headset 224 may comprise, for example, the MEMS sensing and processing module 104 located at one end. In one embodiment of the invention, the user 102 may be allowed to exhale into open space and onto the MEMS sensing and processing module 104. In one embodiment, the seat device 220 can be located, for example, inside a passenger car or any other automobile or vehicle. Nevertheless, the present invention may not be so limited, and the MEMS sensing and processing module 104 may be located elsewhere without limiting the scope of the present invention.

  [0086] The MEMS detection and processing module 104 may be operable to detect movement caused by the expulsion of human exhalation by the user 102 seated in the seating device 220. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The generated control signal or signals are transmitted to the user interface 107a of the multimedia device 106a, the user interface 107b of the mobile phone / smartphone / dataphone 106b, the user interface of a personal computer (PC), laptop or notebook computer 106c. 107c, user interface 107d of display device 106d, and / or a user interface of a multimedia player, such as an audio and / or video player, may be operable to control the user interface of the device being controlled.

  [0087] FIG. 2F is a diagram illustrating an exemplary MEMS sensing and processing module located inside an automobile, according to one embodiment of the invention. Referring to FIG. 2F, an automobile 230 is illustrated. The automobile 230 can include a visor 232 and a handle 234.

  [0088] In one embodiment of the invention, the visor 232 may comprise a support structure 233 that is easy to bend. The support structure 233 can comprise, for example, a MEMS sensing and processing module 104 located at one end. In another embodiment of the invention, the handle 234 can include a support structure 235 that is pliable. The support structure 235 can comprise, for example, a MEMS sensing and processing module 104 located at one end. Nevertheless, the present invention may not be so limited, and the MEMS sensing and processing module 104 may be located elsewhere in the automobile 230 without limiting the scope of the present invention.

  [0089] For example, without limitation, the user 102 may be seated in a seat behind the handle 234, and the processing module 104 is mounted on the handle 234. User 102 may be seated in a seat behind handle 234. User 102 may be allowed to exhale into open space and onto MEMS sensing and processing module 104. The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human exhalation, the MEMS sensing and processing module 104 performs a user interface 107a of the multimedia device 106a, a user interface 107b of the mobile phone / smartphone / dataphone 106b, a personal computer ( PC), laptop or notebook computer 106c user interface 107c, display device 106d user interface 107d, and / or multimedia and other devices such as audio and / or video players or navigation (eg, GPS) devices One or more controls to control the user interface of the device being controlled, such as the user interface It may be operable to generate an issue.

  [0090] FIG. 2G is a diagram illustrating an exemplary MEMS sensing and processing module located on removable eyewear, according to one embodiment of the invention. Referring to FIG. 2G, the user 102 is illustrated. The user 102 can wear, for example, removable goggles or any other type of eyewear 240. The removable eyewear 240 can comprise a removable headset 242. The removable headset 242 may be pliable and / or distortable. The removable headset 242 can comprise, for example, the MEMS sensing and processing module 104 located at one end. In one embodiment of the invention, the user 102 may be allowed to exhale into open space and onto the MEMS sensing and processing module 104.

  [0091] The MEMS detection and processing module 104 may be operable to detect movement caused by the expulsion of human breath by a user 102 seated in the seating device 102. In response to detecting motion caused by the exhalation of human exhalation, the MEMS sensing and processing module 104 performs a user interface 107a of the multimedia device 106a, a user interface 107b of the mobile phone / smartphone / dataphone 106b, a personal computer ( PC), user interface 107c of a laptop or notebook computer 106c, user interface 107d of a display device 106d, and / or a user interface of a multimedia player such as an audio and / or video player, etc. It may be operable to generate one or more control signals to control the interface.

  [0092] FIG. 2H is a diagram illustrating an exemplary MEMS sensing and processing module located on a neckset, according to one embodiment of the invention. Referring to FIG. 2H, a removable neck set 250 is illustrated. The removable neck set 250 can include a flexible printed circuit board (PCB) 254 and processing and / or communication circuitry 252. The flexible PCB 254 can include, for example, a MEMS sensing and processing module 104 located at one end.

  [0093] The processing and / or communication circuit 252 includes a battery, a voltage regulator, one or more switches, one or more light emitting diodes (LEDs), a liquid crystal display (LCD), a resistor, a capacitor. , Other passive devices such as inductors, communication chips capable of handling one or more wireless communication protocols, such as Bluetooth, and / or one or more wired interfaces. In an exemplary embodiment of the invention, processing and / or communication circuit 252 may be packaged in a PCB. Nevertheless, the invention may not be so limited, and the processing and / or communication circuit 252 may comprise other components and circuits without limiting the scope of the invention.

  [0094] In one embodiment of the present invention, the user 102 may be allowed to wear the neck set 250 around his neck and exhale into an open space, and the MEMS detection and processing module 104. May be operable to detect or detect exhalation. Exhaling may result from the user's 102 nostrils and / or mouth.

  [0095] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, MEMS sensing and processing module 104 generates one or more control signals to processing and / or communication circuit 252 via flexible PCB 254. May be operable. The processing and / or communication circuit 252 processes the generated control signal or signals and is a multimedia device 106a, mobile phone / smartphone / dataphone 106b, personal computer (PC), laptop or notebook Computer 106c and / or display device 106d may be operable to communicate with a controlled device. One or more processors in the device being controlled utilize the communicated control signals to utilize the user interface 107a of the multimedia device 106a, the user interface 107b of the mobile phone / smartphone / dataphone 106b, the personal computer ( PC), a user interface 107c of a laptop or notebook computer 106c, and / or a user interface 107d of a display device 106d, may be operable to control the user interface of the device being controlled.

  [0096] FIG. 2I is a diagram illustrating an exemplary MEMS sensing and processing module located on a clip, according to one embodiment of the present invention. Referring to FIG. 2I, user 102 and clip 272 are illustrated. The clip 272 can be placed on any suitable part of the garment, such as a shirt collar, coat wrap, or pocket. The clip 272 can include a support structure 274 that is easy to bend, for example. Although a clip 272 is illustrated, other suitable attachment structures may be utilized to attach the support structure 274. The support structure 274 can comprise the MEMS sensing and processing module 104, the back of which can be located, for example, at one end of the support structure 274 or anywhere above it. In other exemplary embodiments of the present invention, support structure 274 may not be utilized and MEMS sensing and processing module 104 may be attached to clip 272 or other suitable attachment structure.

  [0097] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The MEMS detection and processing module 104 may be operable to detect kinetic energy generated by the exhalation of human breath and to generate one or more control signals accordingly. Or it can comprise a plurality of segments or members. The generated one or more control signals may be capable of controlling the user interface 107b of the mobile phone / smartphone / dataphone 106b.

  [0098] FIG. 2J is a diagram illustrating an exemplary MEMS sensing and processing module embedded in a fabric, according to one embodiment of the invention. Referring to FIG. 2J, user 102 and fabric 276 are illustrated. The fabric 276 may be any suitable piece of clothing, such as a shirt collar, coat wrap, or pocket. The fabric 276 can include an embedded MEMS sensing and processing module 104, the back of which can be located anywhere within or on the fabric 276 274, for example. Nevertheless, the present invention may not be so limited, and without limiting the scope of the present invention, the MEMS sensing and processing module 104 is located elsewhere on the user's 102 outerwear or underwear. sell. In this regard, the MEMS sensing and processing module 104 can be made to be flexible so that the MEMS sensing and processing module 104 can be placed or interwoven within the fabric 276.

  [0099] The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The MEMS detection and processing module 104 may be operable to detect kinetic energy generated by the exhalation of human breath and to generate one or more control signals accordingly. Or it can comprise a plurality of segments or members. The generated one or more control signals may be capable of controlling the user interface 107b of the mobile phone / smartphone / dataphone 106b.

  [00100] FIG. 3A is a diagram illustrating an exemplary electronic device that may be controlled via a partition user interface, according to one embodiment of the invention. With reference to FIG. 3A, an electronic device 302 comprising a touch screen display 304 is illustrated. In this regard, a touch screen display is utilized for illustration, but without departing from the scope of the present invention, the electronic device 302 can be a non-touch screen display, as well as a trackball, one or more multifunction buttons. And / or one or more input devices, such as a keyboard.

  [00101] The electronic device 302 can include a user interface, such as a graphical user interface (GUI), that allows the user to navigate through and launch various applications and / or functions on the electronic device 302. Can be made possible. In this regard, the user interface may be electronic, such as breathing input such as exhaling, tactile input such as button presses, audio actions such as voice commands, and / or those detected by accelerometers and / or gyroscopes. It may be possible to interact with an electronic device via movement of the device 302. In this regard, the user interface can allow interaction with the electronic device 302 via any combination of one or more of the input methods. Further, the user interface may be operable to detect errors and / or failures of one or more input methods and default to one or more other input methods. Thus, interacting with the user interface may not be significantly affected by the failure and / or lack of a particular input method.

  [00102] The user interface of the electronic device 302 displays information about the status and / or function of the electronic device 302 and / or information generated by one or more applications on the electronic device 302 and / or Content can be displayed. In various embodiments of the present invention, when the electronic device 302 is turned on, a home screen of the user interface may be displayed or presented. In various embodiments of the present invention, the electronic device 302 is a mobile phone, smartphone, wireless phone, notebook computer, personal media player, personal digital assistant, multimedia device, handheld device and / or multifunction mobile device, One or more may be provided.

  [00103] The user interface includes a fixed area 310 comprising one or more fixed zones 311, a control area 314 comprising one or more control zones 315, and a content area 318 comprising one or more content zones 319. Of these, it may be partitioned into one or more. In this regard, each of the regions 310, 314, and 318, when present, may be any size and / or shape and may be in any location (s) of the display 304. Also, the presence, size, shape, and location (s) of regions 310, 314, and 320 can be configured (ie, personalized or customized) by the user of electronic device 302. For example, the electronic device 302 is a user interface customization application that can be executed by a user to configure areas of the user interface based on preferences, such as whether the user is right-handed or left-handed. Can be provided. In this regard, exemplary user interface configurations 306a, 306b, and 306c are illustrated in FIG. 3B.

  [00104] The fixed area 310 may display information independent of the state and / or activity in the control area 314. Exemplary information that may be displayed in the fixed area 310 includes date, time, weather, calendar appointments, RSS (or XML or other markup language) feeds, recent email contacts, and / or recent Phone contacts. However, the above is merely an example of information that can be displayed in the fixed area 310, and the present invention may not be so limited. In addition, the size, shape, and / or location of the anchoring region 310 may vary depending on what functions and / or applications are being executed on the electronic device 302. Further, the type and / or amount of information displayed in the fixed area 310 can be customized by the user of the electronic device 302. In this regard, FIG. 3C illustrates a number of exemplary fixation regions 310a, 310b, and 310c.

  [00105] The control area 314 allows the electronic device 302 to be controlled such that desired information can be displayed on the electronic device 302 and / or desired applications and / or functions can be activated. be able to. In this regard, breathing and / or tactile input can be utilized to scroll, select, manipulate, or otherwise interact with the object, such as text, images, links and / or icons of the user interface. In this regard, additional details of interacting with user interface objects using respiration and haptic input are described below with respect to FIG. 3E. In addition, the type and / or amount of information displayed in the control area 314 can be customized by the user of the electronic device 302. Further, the size, shape and / or location of the control region 314 may vary depending on what functions and / or applications are running on the electronic device 302.

  [00106] The content area 318 may display information that may depend on the state and / or activity in the control area 314. For example, the information in the content area 318 may be determined by an active icon in the control area. In this regard, the active icon has been navigated there (via exhalation and / or tactile input), but a “click” (eg, a tap on the touch screen, a button press, or a blow of air). It can be an icon that has not been selected via. In one exemplary embodiment of the invention, the active icon may be linked to a website and the content area 318 can display an RSS feed from that website. In another exemplary embodiment of the present invention, the active icon may be a shortcut for launching an email client and the content area 318 may display one or more recent email messages. it can. In this regard, exemplary information displayed in the content area 318 may include RSS or XML feeds, images, calendars, recent calls, recent text, and / or recent email. However, the above is merely an example, and the present invention is not so limited. In addition, the information displayed in content area 318 may be customizable by the user of electronic device 302. Further, the size, shape and / or location of the content area 318 may vary depending on what functions and / or applications are running on the electronic device 302. In this regard, FIG. 3D illustrates a small number of example content regions 318a, 318b, and 318c.

  [00107] In various embodiments of the invention, display 304 may be a touch screen, as opposed to fixed area 310 and / or content area 318, which may have a limited response to haptic input. In addition, the control region 314 can respond to various ranges of haptic input. For example, the control area 314 can respond to tactile movements, the number of touches, and / or the duration of the touch, while the fixed area 310 and the content area 318 respond to multiple touches (eg, a double tap). can do. In this way, limiting the amount of display 304 that can be allocated to control area 314 reduces the amount of area that the user needs to be reachable to navigate and select icons, and thus electronic device 302. Can be easily operated with one hand. In addition, limiting the tactile responsiveness of the fixed region 310 and the content region 318 can reduce inadvertent actions and / or selections (ie, inadvertent “clicks”).

  [00108] Information in fixed area 310 and / or content area 318 may be displayed in the form of one or more objects, such as images, text, links, and / or icons. In various embodiments of the present invention, objects in the fixed area 310 and / or the content area 318 may be selectable via tactile and / or respiratory input. However, in order to prevent inadvertent clicks, the response of the fixed area 310 and / or the content area 318 may be limited as described above. In addition, in various embodiments of the present invention, objects in content area 318 may be scrolled to control area 314 so that they can be selected. For example, breathing input can be used to scroll an object from content area 318 to control area 314 so that the object can be selected via tactile input to control area 310.

  [00109] Thus, the partition user interface of the electronic device 302 can be described as a universal content access manager (UCAM), which can provide advantages over traditional graphical user interfaces. One advantage is that UCAM configurability (ie, customization or personalization) greatly increases the usefulness and / or ease of use of electronic device 302 over similar devices having a traditional graphical user interface. Can be. In this regard, the objects within each compartment can be sequenced, juxtaposed, superimposed, overlayed, or otherwise as the user can quickly access the desired information, applications and / or functions. Positioning and / or knitting can be performed. Another advantage is that the UCAM can be partitioned into one or more regions, greatly increasing the usability and / or ease of use of the electronic device 302 over similar devices having a traditional graphical user interface. It can be that In this regard, a portion of each region may be configured to respond to or not respond to various input types and may be active (eg, updated in real time) with respect to information and / or objects displayed therein. May be configured to be passive) (eg, displayed statically until changed by the user). Another advantage of UCAM may be compatibility with various platforms. In this regard, the user can load the UCAM onto multiple his / her electronic devices so that the user can interact with all of the user's electronic devices in the same way.

  [00110] FIG. 3B is a diagram illustrating several example configurations of a partition user interface, according to one embodiment of the invention. With reference to FIG. 3B, exemplary user interface configurations 306 a, 306 b, and 306 c are illustrated, each of a fixed area 310 comprising one or more fixed zones 311, a control area 314 comprising one or more control zones 315. And a content area 318 comprising one or more content zones 319. In this regard, the size, shape and / or location of fixed area 310, control area 314 and content area 318 may be based on user preferences and / or based on functions and / or applications executing on electronic device 302. Can be configured.

  [00111] FIG. 3C is a diagram illustrating several exemplary fixed areas of a partition user interface, according to one embodiment of the present invention. Referring to FIG. 3C, fixed regions 310a, 310b and 310c are illustrated. In this regard, each of the fixed areas 310 a can comprise one or more objects 312. In addition, the portion of the fixed area 310 allocated to each object 312 can be configured to have any shape and / or size. Exemplary objects 312 include text, images, links, and / or dates, times, weather information, calendar appointments, RSS or XML feeds, recent email contacts, and / or recent phone contacts. An icon can be provided.

  [00112] FIG. 3D is a diagram illustrating several exemplary content areas of a partition user interface, according to one embodiment of the present invention. In this regard, in various embodiments of the present invention, the user may configure the attributes of the content area 318, such as the number of objects displayed, the size of objects displayed, and the order of objects displayed. it can. For example, the content area 318 may be customized to have different attributes for each icon, each group of icons, and / or each user. In this manner, exemplary content regions 318a, 318b, and 318c are drawn to provide an indication of the type of information that can be displayed in content region 318 and the customizability of content region 318.

[00113] The content area 318 can correspond to an active icon, which can be, for example, a folder or website with digital photos. Therefore, the objects 320 ₁ ,. . . , 320 _4, it is possible to correspond to the latest of the four photos that have been uploaded to a folder or Web site. The content area 318b can correspond to an active icon, which can be, for example, a link to a social networking website. Therefore, the objects 322 ₁ ,. . . 322 _N may correspond to the latest “N” events that occurred on one or more profiles on the social networking site. In another example, content area 318b can correspond to, for example, an active icon that can launch an email client, such as objects 322 ₁ ,. . . 322 _N may correspond to the latest “N” emails sent or received. The content area 318c can correspond to, for example, an active icon, which can be a shortcut for starting a web browser. Thus, objects 324 ₁ and 324 ₂ can be favorites and / or links to recently visited web pages. In another embodiment, the MEMS detection and processing module 104 is operable to modify the interaction with the user interface of the device 106 being controlled by activating one or more portions of the control area 314. It is possible. For example, the user 102 may be allowed to control the speed of background auto-scrolling in order to scroll through mobile game options or menus on the device 106 being controlled.

  [00114] FIG. 3E illustrates interacting with a compartment user interface of an electronic device via respiratory and haptic input, according to one embodiment of the invention. Referring to FIG. 3E, a control area 314a that can include an active icon area 328 is illustrated. The control region 314a depicted in FIG. 3E is an exemplary configuration of the control region 314, and the present invention is not limited to the depicted embodiment. In various embodiments of the present invention, icons may be represented in a variety of ways, may comprise visual information, such as images and / or text, and / or audio, such as tones, songs and / or speech. Information can be provided.

  [00115] The active icon area 328 determines, at least in part, the information displayed in the content area 318 and how the electronic device 302 can respond to tactile and / or respiratory inputs. Can do. In this regard, the content area 318 can display information corresponding to icons within the active icon area 328 as described with respect to FIGS. 3A and 3D. Further, when a “click” (eg, touch screen tap, button press, or air blow) is performed, an application or function associated with the icon in the active icon area 328 may be activated.

  [00116] In various embodiments of the present invention, icons can be grouped by category, and each category can comprise one or more icons. In this regard, the number of categories and / or the number of icons in each category can be configured by the user. In this regard, in the exemplary embodiment depicted in FIG. 3E, categories 00, 10,. . . , 90, and each category has icons 1, 2,. . . , 9 icons. Exemplary categories may include telephone and messaging, news, multimedia, music, photos and videos. In addition, the information and / or objects displayed in the fixed zone 310 can be determined based on which category is active.

  [00117] In various embodiments of the present invention, each icon is descriptive text, image (s) and user configurable to indicate which functions and / or applications can be associated with the icon. Audio can be provided.

  [00118] In various embodiments of the present invention, a user configurable background image on display 304 can be associated with each category and can indicate which category is currently selected. To place the desired icon in the active icon area 328, the user scrolls between categories using tactile and / or respiratory input and scrolls between icons using respiratory and / or tactile input. Can do. In this regard, the speed, direction and / or duration of scrolling can be determined based on the type, duration, strength, direction and / or number of tactile and / or respiratory inputs.

  [00119] In an exemplary embodiment of the invention, a user can scroll between categories using tactile input and scroll between icons using breathing input. For example, the user can shift categories 00, 10,... By shifting the position of their thumb within the control area 314 or by rotating the trackball. . . , 90 and the user can scroll the icons in the active category 326 by exhaling. The user can scroll through the categories until the background image displayed on the electronic device 302 corresponds to the desired category. The user may scroll the icons until the icons in the active icon area 328 correspond to the desired function and / or application and / or result in the desired information and / or objects in the content area 318. it can.

  [00120] FIG. 3F illustrates an example partition user interface that can provide an indication of a category and / or icon sequence when scrolling. In this regard, the icons and / or categories can be scrolled in a linear fashion with the first (eg, leftmost or top) and last (eg, rightmost or bottom) icons and / or categories. Alternatively, icons and / or categories can be accessed by scrolling in either direction, regardless of which icon and / or category is active at the beginning of the scroll. You can scroll in a recursive format. Regardless of the manner in which icons and / or categories scroll, it may be desirable to provide the user with an indication of the next and / or previous icons and / or categories in a scrolling sequence. Accordingly, various aspects of the present invention may allow the next and / or previous icon and / or category indications to be displayed in the fixed area 310, the control area 314, and / or the content area 318. . In this regard, this instruction allows the user to determine in which direction the icons and / or categories should be scrolled in order to reach the desired icon and / or category in the fastest and / or most efficient manner. Can be determined. For example, the fixed area 310, the control area 314, and / or the portion (s) of the content area 318 may include the next icon (s) and / or one category / multiple category translucent image (s) in the scroll sequence. Yes). In this regard, translucency is sufficiently opaque for the user to identify the next and / or previous icon (s) and / or one category / multiple categories, and within the fixed area 310 and the content area. The information in 318 can be made sufficiently transparent so as not to be greatly obstructed.

  [00121] In the exemplary embodiment of the invention depicted in FIG. 3F, icons and categories may be scrolled in the form of "windmills" or "slot machines". In this regard, the translucent images of the previous icon 330b, the previous icon 330a, the current icon 330, the first icon 330c, and the second icon 330d of the previous category 340 are displayed. Can be overlaid on the user interface. Similarly, the translucent images of the previous icon 332b, the previous icon 332a, the current icon 332, the first icon 332c, and the second icon 332d of the active category 326 are displayed on the user interface. Can be overlaid. Similarly, the translucent images of the second previous icon 334b, the previous icon 334a, the current icon 334, the first icon 334c, and the second icon 334d of the first category 342 are: It can be overlaid on the user interface.

  [00122] FIG. 3G illustrates interacting with an exemplary compartment user interface via respiratory and haptic input, according to one embodiment of the invention. Referring to FIG. 3G, it is illustrated that a user interacts with an example compartment user interface via haptic and respiratory inputs.

  [00123] Area 340 of the user interface may be a control area and may display elements that can be moved by breathing input and selected by a thumb tap in area 340. The arrow 344 in FIG. 3G is utilized to illustrate that the category of icons can be scrolled through a thumb shift (ie, a slight drag of the thumb) within the region 340.

  [00124] The information and / or objects displayed in region 342 may be a superimposed transparency sheet that allows the user to see a preview of the next and previous icons. Information and / or objects displayed in region 342 may be fixed or may be changed and / or updated. Some objects displayed in region 342 may be selectable via a thumb tap.

  [00125] Thus, by combining breathing and tactile inputs and well-defined responses to those inputs, a partitioned user interface comprising regions 340 and 342 is less ambiguous compared to conventional user interfaces. Provided solutions. In this regard, the partition user interface can allow configurable (ie, customized or personalized) and predictable control of electronic devices and multi-layer and / or multi-dimensional displays of content.

  [00126] FIG. 3H illustrates another example partition user interface that may provide an indication of a category and / or icon sequence when scrolling. In the exemplary embodiment of the invention depicted in FIG. 3H, icons and categories may be scrolled in a “flipbook” format. In this regard, the translucent images of the second previous icon 332b, the previous icon 332a, the current icon 332, the first icon 332c, and the second icon 332d of the active category 326 are displayed in the user interface. Can be overlaid on top.

  [00127] FIG. 4A illustrates launching an application via a user interface utilizing haptic and respiratory inputs according to one embodiment of the invention. Referring to FIG. 4A, exemplary screenshots 410a, 410b, 410c, and 410d are illustrated that depict an exemplary sequence of operations for navigating to a desired icon and launching an application associated with that icon.

[00128] This sequence of operations may begin with the electronic device 302 as depicted by the screen shot 410a. In this regard, in the screen shot 410a, the icon 402 can be present in the active icon area 328 of the control area 314a. Accordingly, the background image of the diagonal stripe can correspond to the category to which the icon 402 can belong. In addition, the objects 402 ₁ ,. . . , 402 ₄ may correspond to the icon 402.

[00129] Subsequently, the user can scroll through the sequence of categories via a haptic action, such as a thumb shift or a trackball rotation. In this regard, the user can search for a category associated with the dot background image. Thus, when the user reaches the dot background image, the device may be in the state depicted in screenshot 410b. In screenshot 410b, icon 404 can be present in active icon area 328 of control area 314b, and objects 404 ₁ ,. . . 404 _N can correspond to the icon 404.

[00130] Subsequently, the user can scroll through categories of icons having a background image of dots via a breathing input, such as exhaling one or more times. In this regard, the user can scroll through the sequence of icons until the device is in the state depicted in screenshot 410c. In the screenshot 410c, an icon 406 corresponding to the desired function or application may be present in the active icon area 328 of the control area 314c. Therefore, the objects 406 ₁ ,. . . , 406 _N can correspond to the icon 406.

  [00131] Thus, in screenshot 410c, the user may have reached icon 406, which is his desired icon, and icon 406 is displayed via tactile input, such as a touch screen tap or button press. By selecting, a desired application and / or function can be activated. In the exemplary embodiment of the present invention depicted in FIGS. 4A, 410d, a web page can be associated with icon 406, and selecting icon 406 can launch a web browser, depicted in screenshot 410d. As such, the web page can be displayed full screen.

  [00132] FIG. 4B illustrates an exemplary interaction with an application executing on an electronic device, according to one embodiment of the invention. Referring to FIG. 4B, aspects of the present invention may allow zooming in (enlargement) and / or zooming out (reduction) via a combination of breathing and haptic input. For example, a web browser running on the electronic device 302 may be displaying a full web page 422 and the user may wish to zoom in on a portion 424 of the web page. Thus, the user can use the haptic input to control the zoom reference point (s) and the breathing input to control the zoom direction and / or amount. For example, the user can touch a reference point on the touch screen and can zoom in or out based on that reference point by exhaling. In this regard, the direction and / or amount of zoom can be controlled, for example, by exhaling intensity, duration, direction and / or number of times.

  [00133] FIG. 4C illustrates interaction with an application executing on an electronic device, according to one embodiment of the invention. Referring to FIG. 4C, aspects of the present invention can enable scrolling through a combination of breathing and haptic input. Exemplary applications may include web browsers, media players, still cameras, video cameras, and file system browsers. For example, the web browser may be displaying a portion of the web page 424 and the user may wish to scroll to another portion 428 of the web page. Therefore, the user can perform coarse scrolling using the respiratory input, and can perform fine scrolling using the tactile input. For example, the user exhales in the direction he wants to scroll to reach part 426 of the web page, and then fine tune the position of the web page via the rotation of the trackball to Part 428 may be displayed.

  [00134] FIG. 5 is a block diagram of an exemplary user interface interacting with a MEMS sensing and processing module and a host system, according to one embodiment of the invention. Referring to FIG. 5, the device 106 being controlled is illustrated. The device 106 being controlled comprises a communication module 502, a user interface 504 and a host interface 506, a plurality of drivers and / or libraries 506, 518, 520 and 522, and a plurality of applets 508, 510, 512 and 514. Can do. The user interface 504 can be, for example, a graphical user interface (GUI).

[00135] The communication module 502 receives one or more signals from the MEMS sensing and processing module 104 operable to serve as an interface, such as a driver and / or a human interface device (HID). Appropriate logic, circuitry, interfaces and / or code may be provided that may be operable. For example, if the received signal does not conform to a supported HID profile, i.e., the signal is not a supported HID class, the received signal is processed for processing on the controlled device 106. It can be passed to the driver, such as a custom air exhaust driver or air detection driver. The received signal can be processed in the device 106 being controlled using that driver. One or more signals may be generated in response to detecting air movement caused by the exhalation of human exhalation by the user 102. The communication module 502 may be operable to receive one or more signals from the MEMS sensing and processing module 104 via wired and / or wireless signals. The communication module 502 can support multiple drivers, interfaces and / or HID profiles. For example, the communication module 120 may include an external memory interface, a universal asynchronous transceiver (UART) interface, an extended serial peripheral interface (eSPI), a general purpose input / output (GPIO) interface, pulse code modulation (PCM) and / or inter-IC sound. (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface and / or HID profile, Bluetooth interface and / or HID profile, ZigBee interface and / or HID profile, IrDA interface and / Or HID profile and / or wireless USB (W-USB) interface and / or HID profile Can support files.

  [00136] The user 102 may be enabled to interface with the GUI 504 of the device 106 being controlled via one or more received signals. The received one or more signals may be sent to one or more drivers, universal serial bus (USB) HID class, and / or wireless protocol HID class, for example, wireless USB HID class and / or ZigBee HID class. Can be compliant. Nevertheless, the invention may not be so limited, and one or more drivers and / or other wireless protocol HID classes may be utilized without limiting the scope of the invention. Currently, Bluetooth uses the USB HID class. Further, if the received signal does not conform to a supported HID profile, i.e., the signal is not a supported HID class, the received signal is processed for processing on the controlled device 106. It can be passed to the driver, such as a custom air exhaust driver or air detection driver. The received signal can be processed in the device 106 being controlled using that driver.

  [00137] The communication module 502 may be operable to format the received one or more signals into an HID profile. The HID profile may comprise one or more drivers and / or libraries 516-522 that may allow to interface with the GUI 504 of the device 106 being controlled. One or more drivers and / or libraries 516-522 allow one or more of the controlled device 106 to initiate, establish, and / or terminate communication with the MEMS sensing and processing module 104. can do. The HID profile may define protocols, procedures and / or usage scenarios for using HID, such as MEMS detection and processing module 104, via wired and / or wireless links, such as Bluetooth. The device 106 being controlled can host a wireless protocol stack, such as a Bluetooth stack, which can use a service discovery protocol (SDP) to discover HIDs, such as the MEMS detection and processing module 104.

  [00138] One user interface of more of the devices 106 being controlled can also be activated and modified based on the functional mode activated by a button or key. Functional modes may include firmware embedded in the MEMS detection and processing module 104 and a portion of one or more applications, drivers, and / or libraries installed on the device 106 being controlled. . The functional mode may be activated from the MEMS sensing and processing module 104 and / or via one or more stimuli on the device 106 being controlled. The one or more stimuli may include a blow of air, touch, audio, vision, gestures and / or other stimuli.

  [00139] According to one embodiment of the present invention, the device 106 being controlled is from the HID, such as the MEMS detection and processing module 104, to the class driver and / or library 516-522 before it is activated. It may be operable to receive device information, such as a descriptor. Drivers and / or libraries 516-522 may be operable to determine device characteristics utilizing descriptors to allow control on the device 106 being controlled.

  [00140] For example, the library variable # 1 516 detects the direction of the exhalation of human exhalation on the HID, such as the MEMS sensing and processing module 104, and the received signal accordingly to the user One or more components of interface 504 may be operable to convert to direction signals that control. The library, momentum # 1 518, detects a blow of air exhaled by the user 102 and uses the corresponding received signal from the MEMS sensing and processing module 104 accordingly to utilize the user interface. 504 may be operable to scroll through one or more menus and slow down after a certain period of time. For example, the library momentum # 1 518 may be a combination of the number of iterations of blowing human exhalation within a certain time period, or a combination of directions of blowing human exhalation within a certain period of time, It may be operable to detect a quick blow-off left-right-left sequence and to generate a control signal for activation and / or switching through the user interface of the device 106 being controlled. For example, by blowing a blow of air over the MEMS sensing and processing module 104, the scroll direction and speed can be determined based on the air flow across the surface of the MEMS sensing and processing module 104 from the lower left to the upper right. . The MEMS sensing and processing module 104 can generate a control signal that can result in a corresponding two-axis scrolling of the user interface, where the speed of the scroll is determined by the air flow on the MEMS sensing and processing module 104 or the air flow. It can be determined based on the duration of sensing the strength of the flow pressure. In another embodiment, one or more fixed air blows to the MEMS sensing and processing module 104 within a period after scroll interaction allows zooming in to an area visible as a result of scrolling. Can be processed as a zoom function mode. The user 102 may be allowed to exit the zoom function mode by blowing another air to the MEMS detection and processing module 104 and returning to the scroll function mode.

  [00141] The library Boolean # 1 520 is operable to select one or more menus and / or icons in the user interface 504 utilizing signals received from the MEMS sensing and processing module 104. It can be. Library Boolean # 2 522 is also operable to utilize the signals received from MEMS sensing and processing module 104 to select one or more menus and / or icons in user interface 504. sell. Library Boolean # 2 522 may also be operable to determine a functional mode based on a sequence of human exhalation breaths received within a particular time period. For example, several blows of human breath received within a period of time can switch the function mode from the scroll function mode to the expanded function mode in the user interface 504. Nevertheless, the invention may not be so limited, and other drivers and / or libraries may be utilized without limiting the scope of the invention.

  [00142] The controlled device 106 may be operable to interface with a detection device, such as the MEMS sensing and processing module 104, utilizing one or more applets 508-514. Applets 508-514 may comprise software components, code and / or programs that may be operable to execute in the context of another program, such as, for example, a web browser. For example, the applet UI skin # 1 508 can include software components, code and / or programs that can act as a windmill, where multiple icons circulate in the background of the user interface 504. can do. The user 102 may be prompted to select one or more icons from the background of the user interface 504 of the device 106 being controlled. UI skin # 2 510, which is an applet, allows one or more icons on user interface 504 to be shattered, for example, when user 102 blows air over icons displayed on GUI 504. Software components, code and / or programs can be provided. In another embodiment, one of the applets can comprise software components, code and / or programs that can be switched between one or more components of the user interface 504, for example upon activation.

  [00143] In another embodiment, one of the applets can comprise software components, code and / or programs that can serve as a 3-D flipbook, in which the 3-D flipbook: User 102 may be allowed to blow a book on GUI 504 to turn one or more pages in the book. The applet Faves # 1 512 morphs two or more photos of a user or friend on the GUI 504 into a single photo when the user 102 blows air on two or more photos of the user or friend on the GUI 504 Software components, code and / or programs can be provided that can be made possible. The scroll function 514, which is an applet, can comprise software components, code and / or programs that can allow scrolling through multiple menus, pages and / or icons on the GUI 504. The GUI 504 of the device 106 being controlled may be operable to interface with the MEMS sensing and processing module 104 based on one or more outputs generated by the applets 508-514.

  [00144] The host computer interface (HCI) 506 comprises an interface to a display, other hardware and / or processor in the device 106 being controlled, eg, for controller management, link establishment and / or maintenance. be able to. The HCI transport layer may be operable to deliver HCI commands to other hardware within the device 106 being controlled.

  [00145] According to one embodiment of the present invention, the MEMS detection and processing module 104 can be utilized to enable multiple functional modes on the device 106 being controlled. For example, the MEMS detection and processing module 104 allows a scrolling function mode to allow document scrolling in multi-dimensional axes, and allows zooming in and out of the document, or enabling directional magnification and text In an exemplary embodiment, a zoom function mode for moving the cursor within the body of the text, and a point and click function mode, where the click blows air one or more times over the MEMS detection and processing module 104 Possible functional modes and dragging the item on the user interface 128 by stopping the pointer over the item, dragging the item, and then blowing again on the MEMS sensing and processing module 104 Drop the item by For enabling it may be operable to enable a drag-and-drop function mode.

  [00146] For example, the MEMS detection and processing module 104 moves characters and / or avatars in games and other multimedia applications in one or more dimensions, and auto-scrolling speeds in games and other applications. Move the background by controlling or moving other superimposed elements, scroll through the list of options without interrupting gameplay, and weapons while shooting in the game And / or existing controllers that can be customized in-game based on mapping specific controls to be manipulated by exhalation to allow more simultaneous input May be operable to do that in addition to . For example, the MEMS sensing and processing module 104 is GUI based, such as enabling multimodal input with a keyboard, mouse or any other input device, enabling multimodal input with a touch screen, virtual element operation, etc. May be operable to allow multimodal input with voice and / or speech for interaction and / or enable multimodal input with gesture and / or motion tracking.

  [00147] According to another embodiment of the present invention, the MEMS detection and processing module 104 may set audio levels or volumes in audio-video devices, skip tracks forward or backward, and / or telephone voice. Allows control of non-GUI variable functions, such as skipping mail messages, browsing icons, applications, windows or widgets during data entry and / or as illustrated in FIG. 2F It may be operable to interact with a navigation system or an in-car dashboard and entertainment system, with both hands on the steering wheel.

  [00148] According to another embodiment of the present invention, the MEMS detection and processing module 104 can be used for real-world interaction, such as blowing fire, candles, windmills, soap bubbles, or drifting dust in virtual reality environments and games. It may be operable to allow imitation and modify audio and / or video parameters, such as filters, pitch or source, during music playback. For example, the MEMS detection and processing module 104 may be used to scroll or zoom content on a head mounted display such as that disclosed in FIG. 2D, interact remotely with a large display or video projector, and / or Function modes such as controlling toys or electronic devices by adjusting direction and / or speed, setting parameters such as line width while drawing or providing other input in graphics design editing applications By enabling, it may be operable to allow hands-free operation of wearable devices in companies, law enforcement, home security, medical emergencies, and military operations.

  [00149] Nevertheless, the present invention may not be so limited, and the MEMS sensing and processing module may be utilized in other applications without limiting the scope of the present invention.

  [00150] According to one embodiment of the invention, a human or user 102 that interfaces with the GUI 504 may be ignorant of any particular operating system (OS) platform on the device 106 being controlled. For example, the controlled device 106 may be any one or more of a Windows OS, a Symbian OS, an Android OS, a Palm OS, or other operating system on a mobile phone, such as an iPhone or Blackberry phone. May be running. Nevertheless, the present invention may not be so limited, and other operating systems may be utilized without limiting the scope of the present invention.

  [00151] FIG. 6 is a flowchart illustrating exemplary steps for processing signals to control a device using human exhalation. With reference to FIG. 6, an exemplary step may begin at step 602. At step 604, one or more signals may be received from a detection device operable to act as a human interface device (HID), such as the MEMS sensing and processing module 104. The detection device can comprise a micro electro mechanical system (MEMS) detector that can be embedded in the controlled device. One or more signals may be generated in response to detecting air movement caused by the exhalation of human breath. At step 606, the controlled device 106 may be operable to format the received signal or signals into an HID profile. The HID profile may comprise one or more drivers and / or libraries 516-522 that may allow to interface with the GUI 504 of the device 106 being controlled. At step 608, one or more drivers and / or libraries 516-522 may select one or more of the initiation, establishment and / or termination of communication with the MEMS sensing and processing module 104 by the controlled device 106. Multiple can be allowed. At step 610, one or more applets 508-514 in the device 106 being controlled may be operable to interface with a detection device, such as the MEMS sensing and processing module 104. At step 612, the user 102 utilizes one or more applets 508-514 to communicate with the graphical user interface (GUI) 128 of the device 106 being controlled via one or more received signals. It may be possible to interface. Control then passes to end step 514.

  [00152] FIG. 7A is a flowchart illustrating exemplary steps for controlling the user interface of a device using human exhalation, according to one embodiment of the present invention. With reference to FIG. 7A, an exemplary step may begin at step 702. At step 704, the sensing module 110 in the MEMS sensing and processing module 104 is operable to detect movements or changes in composition, such as, for example, ambient air composition caused by the exhalation of human breath by the user 102. It can be. At step 306, the sensing module 110 may be operable to generate one or more electrical, light and / or magnetic signals in response to detecting motion caused by the exhalation of human breath. At step 708, processor firmware 116 may be operable to process electrical, magnetic and / or optical signals received from sensing module 110 utilizing various algorithms. The processor firmware 116 may also be operable to incorporate artificial intelligence (AI) algorithms to adapt to a particular user's 102 breathing pattern.

  [00153] At step 710, the processor firmware 116 generates one or more control signals to the device 106 being controlled based on the processing of the electrical, optical and / or magnetic signals received from the sensing module 110. May be operable to In step 712, the generated one or more control signals are sent to the user interface 107a of the multimedia device 106a, the user interface 107b of the mobile phone / smartphone / dataphone 106b, a personal computer (PC), a laptop or notebook computer. 106c user interface 107c, display device 106d user interface 107d, TV / game console / user interface 107e user interface 107e, and mobile multimedia player and / or remote controller user interface 106 The user interface 128 may be operable to control the user interface 128. Control then passes to end step 714.

  [00154] FIG. 7B is a flowchart illustrating exemplary steps for performing sideloading of information according to one embodiment of the invention. With reference to FIG. 7B, an exemplary step may begin at step 752. At step 754, a controlled device 106, such as mobile phone 130a, may be operable to receive data and / or media content from another device 108, such as a PC, laptop or notebook computer 132. . At step 756, the controlled device 106, such as the mobile phone 130a, may be operable to retrieve data and / or media content from a network, such as the Internet 134. For example, retrieved data and / or media content may include RSS feeds, URLs, and / or multimedia content.

  [00155] At step 758, the laptop, PC, and / or notebook 132 may associate and / or map the received data and / or media content and the retrieved data and / or media content. Whether it can be determined. If the association or mapping is done on the laptop, PC and / or notebook 132, control passes to step 760. At step 760, one or more processors in the laptop, PC, and / or notebook 132 operate to associate and / or map received and retrieved data and / or media content to an icon or group. It may be possible. For example, a laptop, PC and / or notebook 132 operates to associate and / or map an icon to a function so that the user 102 can activate or activate the function via the icon. It may be possible. Exemplary icons can enable features such as hyperlinks, bookmarks, shortcuts, widgets, RSS feeds and / or favorite buddies. At step 762, the laptop, PC and / or notebook 132 may be operable to communicate the associated icon or group to the device 106 being controlled, such as the mobile phone 130a. Control then passes to step 766.

  [00156] If the association or mapping is not performed on the laptop, PC, and / or notebook 132, control passes to step 764. At step 764, one or more processors in controlled device 106, such as mobile phone 130a, associate and / or map the received and retrieved data and / or media content to an icon or group. It may be operable. For example, the mobile phone 130a may be operable to associate and / or map an icon to a function so that the user 102 can enable or activate the function via the icon.

  [00157] At step 766, the controlled device 106, such as the mobile phone 130a, receives the content associated with the received data and / or media content, the user interface 131a of the controlled device, such as the mobile phone 130a. May be operable to customize the associated icon or group. User interface 131a may be modified and / or organized by user 102. In this regard, the user 102 can select, create, arrange, and / or organize content to be utilized for the user interface 131a and / or one or more content components. For example, the user 102 can organize content components on the screen and can select content, such as personal photos, for the background and / or icon images. In addition, the user 102 can create and / or modify the manner in which the content component is activated or presented to the user 102. For example, the user 102 can create, import and / or edit icons and / or backgrounds for the user interface 128. Control then passes to end step 768.

  [00158] According to one embodiment of the present invention, a method and system for controlling the user interface of a device using human exhalation is provided for implantable micro electro mechanical system (MEMS) sensing and processing. A device 106 (FIG. 1A) can be provided, including a module 104 (FIG. 1A). The MEMS sensing and processing module 104 may be operable to detect movement caused by the exhalation of human breath by the user 102. In response to detecting motion caused by the exhalation of human breath, the MEMS sensing and processing module 104 may be operable to generate one or more control signals. The generated control signal or signals may include multimedia device 106a, mobile phone / smartphone / dataphone 106b, PC, laptop or notebook computer 106c, display device 106d, TV / game console / other platform 106e, It can be utilized to control the user interface 128 of the device 106, such as a mobile multimedia player and / or a remote controller.

  [00159] In an exemplary embodiment of the invention, detection of motion caused by the expulsion of human exhalation may occur without using a channel. The detection of movement caused by the exhalation of human exhalation is that the human exhalation is exhaled into open space and onto one or more detectors in the MEMS sensing and processing module 104 that allows detection. Can respond.

[00160] According to another embodiment of the present invention, the MEMS detection and processing module 104 can be used to generate one user of more of the devices 106 being controlled via the generated one or more control signals. It may be operable to navigate within the interface. The MEMS sensing and processing module 104 may be operable to select one or more components within the user interface 128 of the device 106 being controlled via the generated one or more control signals. . The generated control signal or signals are sent to the device being controlled by an external memory interface, a general purpose asynchronous transceiver (UART) interface, an extended serial peripheral interface (eSPI), a general purpose input / output (GPIO) interface, Pulse code modulation (PCM) and / or inter-IC sound (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface, Bluetooth interface, ZigBee interface, IrDA interface, and / or The wireless USB (W-USB) interface can communicate via one or more.

  [00161] The MEMS detection and processing module 104 may be operable to allow one or more of the initiation, establishment and / or termination of communications by the device 106. The MEMS sensing and processing module 104 is detected by a human expiratory or fluid expelling such as air, a tactile input such as a button press, an audio input such as a voice command, and / or an accelerometer and / or a gyroscope Based on one or more of the movements of the device 106 being controlled, such as those, may be operable to allow interaction within the user interface 128 of the device 106 being controlled.

  [00162] In accordance with another embodiment of the present invention, the MEMS sensing and processing module 104 may include one or more analogs and / or within one user interface 128 of more of the devices 106 being controlled. It may be operable to generate control signals to control digital functions. The MEMS sensing and processing module 104 may be operable to detect omnidirectional air blows at ultra-low pressures. The MEMS detection and processing module 104 enables intuitive functional modes, such as scrolling, panning, zooming and / or clicking functional modes, prevents unintentional content selection, and / or blocks content. It may be operable to do what is minimized.

  [00163] One or more of a plurality of devices, such as a handheld device, eg, a multimedia device 106a and / or a cell phone / smartphone / dataphone 106b and / or a PC, laptop or notebook computer 106c May be operable to receive one or more inputs defining a user interface 128 from another device 108. Other devices 108 may be one or more of a PC, laptop or notebook computer 106c, and / or handheld device, eg, multimedia device 106a and / or cell phone / smartphone / dataphone 106b. There may be. In this regard, data may be transferred from other devices 108 to the cell phone / smart phone / data phone 106b, which may be transferred to the cell phone / smart phone / data via a service provider, such as a cell phone or PCS service provider. It may be associated or mapped to media content that may be remotely accessed by phone 106b. The transferred data associated or mapped to the media content can be utilized to customize the mobile phone / smartphone / dataphone 106b user interface. In this regard, media content associated with one or more received inputs can be an integral part of the user interface 128 of the device 106 being controlled.

  [00164] The present invention is not limited to exhalation. Thus, in various exemplary embodiments of the invention, the MEMS may be operable to detect the discharge of any type of fluid, such as air, and the source of fluid may be an animal, machine and / or It may be a device.

  [00165] Other embodiments of the present invention have at least one code portion executable by a machine and / or computer, thereby allowing the machine and / or computer to use a device for human device breathing. Non-transitory computer readable and / or storage media and / or non-transitory storing machine code and / or computer programs that cause the steps as described herein to control the interface Machine-readable media and / or storage media can be provided.

  [00166] Thus, aspects of the invention may be implemented in hardware, software, firmware, or a combination thereof. The invention can be implemented in a centralized form in at least one computer system or in a distributed form in which different elements are spread across several interconnected computer systems. Any type of computer system or other apparatus adapted to carry out the methods described herein is suitable. A typical combination of hardware, software and firmware is a general purpose computer system having a computer program that, when loaded and executed, controls the computer system to perform the methods described herein. Also good.

  [00167] One embodiment of the present invention is integrated on a single chip as a board level product, as a single chip application specific integrated circuit (ASIC), or as a separate component with the rest of the system. Can be implemented at various levels. The degree of system integration is determined primarily by considering speed and cost. Due to the high performance of modern processors, a commercially available processor can be used, which can be implemented outside the ASIC implementation of the system. Alternatively, if the processor is usable as an ASIC core or logic block, a commercially available processor can be implemented as part of an ASIC device, with various functions implemented as firmware.

  [00168] The present invention is also embeddable in a computer program product, which has all the features that enable the implementation of the methods described herein when loaded into a computer system. These methods can be performed. The computer program in this context can be, for example, directly on a system with information processing functions, or a) conversion to another language, code or notation, b) reproduction in different material forms, After either or both can mean any representation in any language, code or notation of a set of instructions intended to perform a particular function. However, other meanings of computer programs within the understanding of those skilled in the art are also contemplated by the present invention.

  [00169] Although the present invention has been described with reference to certain embodiments, those skilled in the art will recognize that various modifications can be made and equivalents can be substituted without departing from the scope of the invention. Will be understood. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the scope of the invention. Accordingly, the invention is not limited to the specific embodiments disclosed, and the invention is intended to include all embodiments included within the scope of the appended claims.

Claims (30)

A method for interaction,
In a device comprising an embedded micro electro mechanical system (MEMS) sensing and processing module,
Detecting movement caused by human breath exhalation (8 expulsion) by the MEMS sensing and processing module;
Generating one or more control signals to control a user interface of the device in response to the detection.   The method of claim 1, wherein the device comprises one or more of a personal computer (PC), laptop, notebook computer, television (TV), game console, display device and / or handheld device.   The method of claim 2, wherein the handheld device comprises one or more of a mobile phone, a mobile multimedia player, a navigation device and / or a remote controller.   The method of claim 1, wherein the step of detecting the movement caused by the exhalation of the human exhalation occurs without using a channel.   The step of detecting the movement caused by the exhalation of the human exhalation includes one or more of the MEMS sensing and processing modules in the MEMS sensing and processing module that allow the human exhalation to and into the open space. The method of claim 1, wherein the method is responsive to being exhaled onto the detector. The generated control signal or signals to the device as an external memory interface, general purpose asynchronous transceiver (UART) interface, extended serial peripheral interface (eSPI), general purpose input / output (GPIO) interface, pulse code modulation (PCM) and / or inter-IC sound (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface, Bluetooth interface, ZigBee interface, IrDA interface, and / or wireless USB The method of claim 1, comprising communicating via one or more of (W-USB) interfaces.   The method of claim 1, comprising navigating within the user interface via the generated one or more control signals.   The method of claim 1, comprising selecting one or more components within the user interface via the generated one or more control signals.   The method of claim 1, comprising enabling one or more of initiation, establishment and / or termination of communication by the device.   Enabling interaction within the user interface of the device based on one or more of the exhalation, tactile input, audio input, and / or movement of the device of the human breath. The method of claim 1. A non-transitory machine-readable storage medium storing a computer program having at least one code part for interaction, wherein the at least one code part comprises:
In a device comprising an embedded micro electro mechanical system (MEMS) sensing and processing module,
Detecting, by the MEMS sensing and processing module, movement caused by exhalation of human breath;
In response to the detection, the method comprising: generating one or more control signals to control a user interface of the device; A temporary machine-readable storage medium.   The non-transitory of claim 11, wherein the device comprises one or more of a personal computer (PC), laptop, notebook computer, television (TV), game console, display device and / or handheld device. Machine-readable storage medium.   The non-transitory machine-readable storage medium of claim 12, wherein the handheld device comprises one or more of a mobile phone, a mobile multimedia player, a navigation device, and / or a remote controller.   The non-transitory machine-readable storage medium of claim 11, wherein the step of detecting the movement caused by the exhalation of the human exhalation occurs without using a channel.   The step of detecting the movement caused by the exhalation of the human exhalation includes one or more of the MEMS sensing and processing modules in the MEMS sensing and processing module that allow the human exhalation to and into the open space. The non-transitory machine-readable storage medium of claim 11, responsive to being exhaled onto the detector. The at least one code unit sends the generated one or more control signals to the device to an external memory interface, a general purpose asynchronous transceiver (UART) interface, an extended serial peripheral interface (eSPI), a general purpose input / output (GPIO) interface, pulse code modulation (PCM) and / or inter-IC sound (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface, Bluetooth interface, ZigBee interface, IrDA The non-transitory machine-readable storage medium of claim 11, comprising code for communicating via one or more of an interface and / or a wireless USB (W-USB) interface.   The non-transitory machine-readable storage of claim 11, wherein the at least one code portion comprises code for navigating within the user interface via the generated one or more control signals. Medium.   12. The non-code of claim 11, wherein the at least one code portion comprises code for selecting one or more components within the user interface via the generated one or more control signals. A temporary machine-readable storage medium.   The non-transitory machine readable storage of claim 11, wherein the at least one code portion comprises code for enabling one or more of initiation, establishment and / or termination of communication by the device. Medium.   The at least one code portion interacts within the user interface of the device based on one or more of the exhalation of the human breath, haptic input, audio input, and / or movement of the device. The non-transitory machine-readable storage medium of claim 11, comprising code for enabling A system for interaction,
One or more processors and / or circuits for use in a device comprising an embedded micro electro mechanical system (MEMS) sensing and processing module, said one or more processors and / or circuits Is
One or more control signals are detected by the MEMS sensing and processing module to detect movement caused by the exhalation of human breath and to control the user interface of the device in response to the detection. A system that is operable to do generating.   The system of claim 21, wherein the device comprises one or more of a personal computer (PC), a laptop, a notebook computer, a television (TV), a game console, a display device and / or a handheld device.   23. The system of claim 22, wherein the handheld device comprises one or more of a mobile phone, a mobile multimedia player, a navigation device and / or a remote controller.   The system of claim 21, wherein the detection of the movement caused by the exhalation of the human exhalation occurs without using a channel.   The detection of the movement caused by the exhalation of the human exhalation is one or more detectors in the MEMS sensing and processing module that allow the human exhalation to go into the open space and the detection. The system of claim 21 responsive to being exhaled up. The one or more processors and / or circuits send the generated one or more control signals to the device to an external memory interface, a universal asynchronous transceiver (UART) interface, an extended serial peripheral interface (eSPI). ), General purpose input / output (GPIO) interface, pulse code modulation (PCM) and / or inter-IC sound (I ² S) interface, inter-integrated circuit (I ² C) bus interface, universal serial bus (USB) interface, Bluetooth interface 24. The system of claim 21, wherein the system is operable to communicate via one or more of: a ZigBee interface, an IrDA interface, and / or a wireless USB (W-USB) interface. .   The system of claim 21, wherein the one or more processors and / or circuits are operable to navigate within the user interface via the generated one or more control signals. .   The one or more processors and / or circuits are operable to select one or more components within the user interface via the generated one or more control signals. Item 22. The system according to Item 21.   The system of claim 21, wherein the one or more processors and / or circuits are operable to allow one or more of initiation, establishment and / or termination of communication by the device. .   The one or more processors and / or circuits may be configured such that the device's one or more of the exhalation of the human breath, tactile input, audio input, and / or movement of the device, The system of claim 21, wherein the system is operable to allow interaction within a user interface.

Images