US20220335792A1

US20220335792A1 - Methods and apparatus regarding electronic eyewear for a dwelling of a user

Info

Publication number: US20220335792A1
Application number: US17/856,954
Authority: US
Inventors: Thomas A. Howell; David Chao; C. Douglass Thomas; Peter P. Tong
Original assignee: Ingeniospec LLC
Current assignee: IpVenture Inc
Priority date: 2018-04-12
Filing date: 2022-07-02
Publication date: 2022-10-20
Also published as: US11721183B2; US20200364992A1; US20190318589A1; US10777048B2

Abstract

Novel methods and apparatus regarding electronic eyewear are disclosed. Different embodiments can be applicable for seniors. One embodiment includes an eyewear that can charge when placed in an eyewear case. A user can access a live operator by tapping a button at the eyewear. The live operator can connect the user to a person. The eyewear can detect falls. In another embodiment, the eyewear with a digital assistant can be voice activated, and can provide assistance via the digital assistant. In yet another embodiment, the eyewear includes a position identifying system to identify a position of the eyewear. In one embodiment, the eyewear could wirelessly interact with a device coupled to an apparatus in the vicinity of the eyewear.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 16/987,781, filed on Aug. 7, 2020, entitled “Methods and Apparatus Regarding Electronic Eyewear Applicable for Seniors,” which is hereby incorporated herein by reference, which in turn is a continuation of U.S. patent application Ser. No. 16/382,036, filed on Apr. 11, 2019, now U.S. Pat. No. 10,777,048, and entitled “Methods and Apparatus Regarding Electronic Eyewear Applicable for Seniors,” which is hereby incorporated herein by reference and which in turn claims benefit of (a) U.S. Provisional Patent Application No. 62/656,621, filed on Apr. 12, 2018, entitled “Electronic Eyewear for Seniors,” which is hereby incorporated herein by reference; (b) U.S. Provisional Patent Application No. 62/668,762, filed on May 8, 2018, entitled “Methods and Apparatus regarding Electronic Eyewear Applicable for Seniors,” which is hereby incorporated herein by reference; (c) U.S. Provisional Patent Application No. 62/681,292, filed on Jun. 6, 2018, entitled “Methods and Apparatus regarding Electronic Eyewear Applicable for Seniors,” which is hereby incorporated herein by reference; (d) U.S. Provisional Patent Application No. 62/686,174, filed Jun. 18, 2018, entitled “Methods and Apparatus Regarding Electronic Eyewear Applicable for Seniors,” which is hereby incorporated herein by reference; and (e) U.S. Provisional Patent Application No. 62/718,597, filed on Aug. 14, 2018, entitled “Methods and Apparatus Regarding Electronic Eyewear Applicable for Seniors,” which is hereby incorporated herein by reference.

BACKGROUND OF THE INVENTION

As the elderly population continues to grow, technologies for aging will become crucial for ensuring seniors' well-being. Existing devices and services attempt to address some of these needs, but can be unwieldy and conspicuous. It is desirable to have methods and apparatuses combining ease-of-use with compact electronics in items many seniors already use every day that could address the needs of seniors, particularly those who live by themselves.

SUMMARY OF THE INVENTION

Different embodiments focus on using electronic eyewear to address the needs of seniors, particularly those who live by themselves and may need extra help, such as to get around or keep in touch with loved ones.
As the elderly population continues to grow, technologies for aging are becoming more crucial for ensuring seniors' well-being. A number of embodiments combine ease-of-use with compact electronics in an item many seniors already use every day: eyewear.
Different embodiments can benefit elderly users who want to remain independent. The different embodiments empower and guide the elderly through routine obstacles, from maintaining a schedule to addressing critical needs in the event of an emergency.
One embodiment prioritizes a simple learning curve: a waterproof eyewear that can charge automatically when placed in an appropriate eyewear case, and can allow a user to access different features of the eyewear through the eyewear's one-button interface. For example, one tap of the button can summon a digital assistant-available in the user's preferred language—that can place calls to contacts and respond to questions like “Major news today?”; “Have I walked enough today?”; and “How do I get home from here?”. If the digital assistant can't help, it can connect the user to a live professional operator. The user can also connect to the operator directly at any time, simply by, for example, tapping the button more than once. The operator can offer more personalized assistance, including responding to more complicated requests, or connecting the user to the right person for the situation, such as a loved one or emergency services.
Another embodiment includes features for the user who can benefit from additional assistance and monitoring. The eyewear can detect falls, and can continually measure the user's key vital signs. In the event of a problem, the eyewear can send a caregiver to check on the user. In different embodiments, the eyewear can also provide convenient reminders for eating, taking medications, drinking water, and sleeping.
One embodiment not only can detect falls but can connect the eyewear to a live operator. The eyewear can be charged easily via inductive charging by placing the eyewear in an appropriate eyewear case. In another embodiment, the eyewear can also be voice activated.
One embodiment includes a position identifying system to help guide the user, such as when the user is lost. Many seniors want to keep independent, and sometimes, want to go for a walk. They may forget how to get home. In one embodiment, by pushing a button on the eyewear, a user could be linked to a person who could guide the user home. Or in another embodiment, instead of a person, a digital assistant in the eyewear could help.
In one embodiment, the eyewear could wirelessly interact with a device in an apparatus in its vicinity. For example, the apparatus could be a stove. When the user turns on a burner at the stove, the device could send a signal to the eyewear. And a digital assistant in the eyewear could alert the user if the eyewear has not received from the device another wireless signal regarding the burner being turned off after a preset amount of time.
Different embodiments offer an easy, familiar way for seniors to maintain their independence, facilitating fast and automatic communication, and providing guidance. The different embodiments would enable seniors to retire their worries, not their lifestyles, and offer loved ones the sound tranquility of relief.
Other aspects and advantages of the present invention will become apparent from the following detailed description, which, when taken in conjunction with the accompanying drawings, illustrates by way of example the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an embodiment of an eyewear in an eyewear casing that could help charge the eyewear.

FIG. 2 shows an embodiment of an eyewear in a plate with a slot that could help charge the eyewear.

FIG. 3 shows an embodiment of an eyewear sitting in a structure that could help charge the eyewear.

FIG. 4 shows an embodiment of an oscillator circuit for charging.

FIG. 5 shows an embodiment of a receiving charging circuit to charge a battery.

FIG. 6 shows an embodiment for RF charging in an eyewear.

FIG. 7 shows an embodiment of an eyewear that a senior can use.

Same numerals in FIGS. 1-7 are assigned to similar elements in all the figures. Embodiments of the invention are discussed below with reference to FIGS. 1-7. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes as the invention extends beyond these limited embodiments.

DETAILED DESCRIPTION OF THE INVENTION

Different embodiments of electronic eyewear for everyday use are disclosed herein and may be used individually or in any combination. They could be used, for example, by seniors.
One embodiment is equipped with an easily accessible button located, for example, at a side of the eyewear. Pressing the button once enables the user to speak to a digital assistant in her preferred language. The assistant can respond to a number of preset requests, including “How can I get to the library?”; “Louder!”; “Call Katherine for me.”; and “Tell me the major news today.”
If the user becomes lost, she can ask the digital assistant, “How do I get home?” In response, the assistant can provide real-time step-by-step directions to guide the user home using the eyewear's built-in GPS or other location guidance system.
In one embodiment, for more complex inquiries, the user can simply ask the digital assistant to connect her to a live professional operator. Or, by pressing the button more than once, the user can connect directly to the operator at any time, without going through the digital assistant. The operator can, if necessary, soothe the user, provide more personal help, and answer more complicated questions. The operator can also connect the user to family and friends, caregivers, or emergency services, as the particular situation requires.
In one embodiment, when the eyewear needs to be recharged, the user can place the eyewear in its case, which functions as an automatic charging station without any extra effort from the user. The next day, the glasses can be fully charged for use. It can be that easy!
One embodiment can detect falls. For example, if the user falls while walking and is then practically motionless, the eyewear could ask if the user is okay or directly alert a caregiver. It also can measure the user's other vital signs, such as body temperature, heart rate, and blood oxygen level. The eyewear can automatically connect to emergency services if monitored data surpasses predetermined safety thresholds. The eyewear can be waterproof, so it can continue to function even when it gets wet, like on a rainy day.
In one embodiment, the eyewear can add features for seniors who would benefit from additional assistance and monitoring. The eyewear can provide convenient reminders for eating, taking medications, drinking water, and sleeping. Motion sensors can be built into the eyewear to detect whether the user has remained stationary for too long. A digital assistant in the eyewear can encourage the sedentary user to stand, walk a few steps, or stroll outside. In one embodiment, the eyewear can light up the user's path to facilitate night activity.
In one embodiment, the eyewear can include a being-worn sensor, which could sense if the eyewear is worn. The eyewear can be automatically activated or turned on when the eyewear is worn. For example, when a user gets up in the morning and puts on the eyewear, the eyewear could be turned on automatically. Different embodiments of a being-worn sensor in an eyewear are described in U.S. Pat. No. 8,434,863, entitled, “Eyeglasses with Printed Circuit Board,” which is incorporated herein by reference.
In one embodiment, the eyewear could be automatically charged if placed in a charging station. For example, the eyewear could come with a case, which could be connected to a wall power outlet (directly or via a wire/cable). If the eyewear is placed in the case, the eyewear can then be charged. To illustrate, in one embodiment, a user could take off a pair of eyeglasses with folding arms. With the arms folded, the glasses could be turned off. The user could put the glasses into its case, where the eyeglasses could be charged. One charging mechanism can be via a connector at the eyewear being connected to a connector at a case (e.g., an electrical connector, such as a USB connector, accessible inside the case). Different embodiments regarding a charging station for an eyewear are disclosed in “Eyewear Housing for Charging Embedded Battery in Eyewear Frame,” with application serial number, 15/409,723, which is incorporated herein by reference.
Another charging mechanism can be via inductive coupling. For example, there could be receiving charging coils of wires in the eyewear, such as inside the frame of the eyewear. To illustrate, the receiving charging coils could be at a lens holder and/or at one or more arms of a pair of eyeglasses. At corresponding position(s) inside a case where the eyewear could be housed, there could also be transmitting charging coils of wires. Via inductive coupling, a rechargeable battery in the eyewear can be charged via coupling between the receiving charging coils in the frame of the eyewear and the transmitting charging coils in the case. One such structure 100 is shown, for example, in FIG. 1, where there could be receiving charging coils at least around the lens holders of the eyewear. When the eyewear is placed in the case, for example, with the case closed, the eyewear could be charged automatically by transmitting charging coils in the case, such as at the oval-shaped area 102 at a side wall of the case as shown in FIG. 1, which could be arranged to be in close proximity to at least one of the lens holders 104 when the eyeglasses are placed in the case and the case is closed. In one embodiment, the eyewear and the case can be designed so that the eyewear could go into the case in only one way, such as when one wants to close the case. In one embodiment, there could be a switch at the case, such as close to the hinge of the case, so that when the case is closed, the switch could be activated to start the charging process.
FIG. 2 shows an embodiment of a plate 200 with a slot 202 to help charge an eyewear. In this example, transmitting charging coils can be in a wall area 204 at one end of the plate next to the slot 202. When the lens holders of an eyewear are in the slot, as shown, for example, in FIG. 2, the lens holders could be in close proximity to the transmitting charging coils in the wall area 204. Then via receiving charging coils, for example, in the lens holders, the battery within the eyewear could be charged.
In yet another embodiment, the eyewear could be placed on or in a structure, with areas in close proximity to parts of the eyewear, such as in close proximity to at least one of its arms, to charge the eyewear. One such structure 300 is shown, for example, in FIG. 3. There could be transmitting charging coils at a back panel 302 of the structure 300 to charge the battery within the eyewear, which could have receiving charging coils at least in one of the arms 304 of the eyewear. To illustrate, when the user retires at night and puts the eyewear on the structure 300 shown in FIG. 3, the eyewear's battery can be automatically recharged by transmitting charging coils in the back panel 302 sending charging signals to receiving charging coils in at least one arm 304 of the eyewear.
FIG. 4 shows an example of an oscillator circuit 400, serving as at least a portion of transmitting charging circuit for charging. The circuit 400 in FIG. 4 is sometimes known as a Colpitts oscillator. The circuit 400 includes an oscillator (such as the C1, C2 and L structure) generating AC signals at the base of a transistor (such as 2N2222). The output of the oscillator circuit, Vout, could be used to drive a set of transmitting charging coils, such as 10 turns of coils with a diameter of 1.5”, connected between Vout and ground. With appropriate components, the oscillation could be at, for example, 140 KHz. One could also place an amplifier at Vout to increase its power to drive the transmitting charging coils. The oscillator circuit 400 shown in FIG. 4 is just an example. Other types of oscillator circuits can be used.
FIG. 5 shows an example of a receiving charging circuit 500 to charge one or more batteries, such as in an eyewear. The circuit 500 could include a set of receiving charging coils, such as 10 turns of coils or wires with a diameter of 1.5”. The receiving charging coils could wirelessly receive charging signals from transmitting charging coils. AC signals at the outputs from the receiving charging coils, V1, could be connected to a resonator, such as L1 and C1, tuned at the charging frequency, such as 140 KHz. Then the AC signals at V1 can be converted to DC signals at V2, which can be stepped up by a voltage multiplier to raise the DC voltage to, for example, 5V at V3. The 5V DC signals can run a battery charger to charge a set of batteries at VBat in an eyewear.
Yet another charging mechanism can be via RF charging. FIG. 6 shows an example of such an implementation 600 in an eyewear. In the example, an antenna could run along an arm of the eyewear. The antenna can be tuned to the charging frequency, such as 2.4 Ghz, to wirelessly capture RF signals from a RF transmitting circuit. The captured RF signals could be received by a RF receiving charging chip to charge one or more batteries in the eyewear. The RF transmitting circuit with a corresponding charging antenna could be at a charging structure, such as at the back panel 302 of the structure 300 shown in FIG. 3, to generate the RF signals to charge the one or more batteries in the eyewear.
In one embodiment, when a battery within the eyewear is being charged, other than the charging electronics, other electronics in the eyewear could be off; and when the eyewear is not being charged, electronics in the eyewear could be on. This could be done, for example, by measuring a voltage at the charging electronics. A high voltage value could indicate the operation of charging being on, which could cause the other electronics in the eyewear to be turned off; and a voltage value below a threshold could indicate not charging, which could cause electronics in the eyewear to be on.
In one embodiment, the battery in the eyewear can be sufficient to operate the electronics in the eyewear for at least 24 hours when fully charged.
In one embodiment, the eyewear can notify the user when the battery level is low via an indicator, such as a visual indicator, like a LCD display or one or more LEDs. In another example, the eyewear could orally tell the user, via a speaker at the eyewear, that the battery level is low and should be recharged. To illustrate, when the battery level has dropped to 25% of its fully-charged value, the speaker could alert the user.
In one embodiment, the eyewear could monitor the heart rate of its user via a heart-beat sensor. Different embodiments of a heart beat in an eyewear are described in U.S. Pat. No. 7,677,723, entitled, “Eyeglasses with a Heart Rate Monitor,” which is incorporated herein by reference.
In one embodiment, the eyewear could include a pulse oximeter. At least one of the nose pads of the eyewear could have the pulse oximeter. In another embodiment, instead of at the nose pads, a pulse oximeter can be clipped onto a portion of the ear of the user, such as the user's earlobe, or can be attached to other areas of the skin of the user with capillaries. Based on the measurements of a pulse oximeter, a controller in the eyewear could determine the % of oxygen in the user's blood.
In one embodiment, the eyewear can include a temperature sensor. Different embodiments of a temperature sensor in an eyewear are described in U.S. Pat. No. 7,380,936, entitled, “Eyeglasses with a Clock or other Electrical Component,” which is incorporated herein by reference.
In one embodiment, the eyewear can include a motion sensor, such as a pedometer. Different embodiments of a motion sensor in an eyewear are described in U.S. Pat. No. 7,255,437, entitled, “Eyeglasses with Activity Monitoring,” which is incorporated herein by reference.
In one embodiment, the eyewear can include a blood pressure sensor. In one embodiment, a blood pressure sensor can be based on optical Ballistocardiography and pulse oximetry. Electronics for both mechanisms could be at the nose pads of the eyewear, or they could be pressing onto parts of the skin of the user with capillaries. To illustrate, optical Ballistocardiography can measure physical displacements of a section of the user's skin as a function of the user's pulse. The optical device can be made as a mat of optic fibers with some emitting light, and some sensing with photo transistors. The user's pulses can vibrate the skin, which could press onto the mat (such as the mat at a nose pad). The vibrating skin could alternately compress and relax the mat of fibers, leading to the light received by the photo transistors to be modulated as a function of the movement of the mat. With that, optical Ballistocardiography could produce a first pulse signal waveform. The pulse oximeter could produce a second pulse signal waveform. Depending on the time-lag between the two waveforms, a controller in the eyewear could calculate the user's blood pressure.
In one embodiment, the eyewear can include a blood glucose sensor. The blood glucose sensor could be at a nose pad of the eyewear.
In one embodiment, to save battery, one or more of the sensors do not sense the user continuously, but at regular intervals, such as every 10 minutes. Different sensors could sense at different intervals. For example, in one embodiment, heart rate could be monitored more frequently than temperature.
In one embodiment, the eyewear could alert a user to drink water, eat, exercise, weigh himself, sleep, and/or involve in other activities. This could be based on time. In one embodiment, typical times can be used, such as alerting the user to have dinner at 6 pm.
In one embodiment, the eyewear could be programmed to be tailored to the user, for example, supporting a set of predetermined routine. To illustrate, the eyewear could be programmed to alert the user to take medication (such as asking the user, “Taken the medication yet?”), follow a treatment plan, and/or sleep at specific times. As to taking medication, the eyewear could alert the user how and when to take medication, and what medication to take. The eyewear could alert the user to walk a little more, if the user has not walked much that day (as shown, for example, by a pedometer in the eyewear). Or, at certain time each day, the eyewear could tell the user major news of the day. These could be major news identified by Facebook, and wirelessly downloaded to the eyewear.
In one embodiment, the eyewear can be wirelessly coupled to a calendar to remind the user of appointments, such as “time to play bridge in the lunch room.” The calendar could be downloaded to a memory device in the eyewear, such as when the user has a new calendared event for the calendar. In another example, a new calendared event could be entered, such as wirelessly, into the calendar in the eyewear. In yet another example, a new calendared event could be entered wiredly via a connector, such as a USB connector, into the calendar in the eyewear.
In one embodiment, based on motion monitored by a pedometer in an eyewear, the eyewear could alert the user, such as by 5 pm, to walk another certain number of steps before the end of the day.
In one embodiment, the eyewear could alert a user by sound, such as via an ear bud coupled to the eyewear. The alert could be based on an audio tone, such as beeps. In another embodiment, the alerts can be via human voices, and could be in a language preferred by the user.
In one embodiment, the eyewear could send emergency alerts to others. For example, the eyewear could send emergency signals wirelessly to a near-by device, such as the user's cell phone, which could automatically make a cellular call to an interested party. In another embodiment, the eyewear could send emergency signals directly to the interested party.
In one embodiment, the eyewear could periodically, such as every 12 hours, wirelessly send its monitored data to an interested party, who could review the data, to determine if there are issues of concern. Or a system could analyze the data. Such analysis could be performed automatically, and the interested party could be alerted if there is an issue. The wireless transmission could be performed via another device in the vicinity of the user. The eyewear could send the monitored data to the another device, such as wirelessly to the user's cell phone, or wiredly to a computer that is plugged to a wall power outlet. In another embodiment, at least a portion, or a significant portion, or all of the analysis can be performed by a processor in the eyewear.
In one embodiment, a user could activate an emergency call. To prevent false alarm, one embodiment can require the user to press and hold a switch or button at the eyewear and let go. Then an interested party would call the user, and ask if the user needs assistance. In another embodiment, the user needs to press and hold the switch twice to initiate a distress call.
In one embodiment, the eyewear could have just a switch for a user to activate, and the switch can be a switch to at least activate an emergency call. For example, the user could activate the switch by pushing it. The switch can be made conspicuously. In addition to the switch, the eyewear could also include a microphone to receive the user's voice input. In another embodiment, the eyewear could include at least a speaker, which could be at an ear bud, and the eyewear could also include a user-controlled volume-changing switch to change the volume of the speaker. The simplicity of having fewer switches at the eyewear for a user to activate or push could help make the eyewear easier for a senior to use.
In one embodiment, the eyewear includes voice recognition software or firmware embedded therein. The software could have a dictionary to recognize words and sentences commonly used in specific areas, such as in time of emergency. For example, when the user wears the eyewear, the eyewear can passively listen. The eyewear could be programmed to be activated by a specific word. When that word is captured by the eyewear, the eyewear could try to recognize subsequent voice inputs based on its voice recognition capabilities.
In one embodiment, the eyewear could send out emergency call if, for example, abnormal signals or signals below certain preset thresholds have been tracked by the eyewear. To illustrate, if a pulse oximeter indicates that the oxygen level of the user is below 93%, an emergency call could be sent out to an interested party, with the reasons of the call also included. Or a caregiver could be automatically sent to check on the user.
In one embodiment, the eyewear also can include a cellular phone that could at least receive calls.
In one embodiment, the eyewear can include a connector, which could be used to download monitored information from the eyewear. Different embodiments of a connector coupled to an eyewear are described in U.S. Pat. No. 7,500,747, entitled, “Eyeglasses with Electrical Components,” which is incorporated herein by reference.
In one embodiment, instead of downloading via a connector, monitored information or data could be accessed wirelessly. In another embodiment, the wireless access can be via a short-range wireless network, such as Bluetooth. The information in the eyewear could be password protected.
In one embodiment, the eyewear can include a light, such as a LED, which can turn on if the eyewear is worn at specific time frames, such as in the middle of the night. The time period to turn on the light could be programmed. Turning on the light can also depend on the user putting on the eyewear, which could, for example, activate the eyewear. In other words, in this embodiment, the light could turn on if the eyewear goes from the off state to the on state during the specific time frames. In one embodiment, the light can automatically turn off after a predetermined period of time, such as 3 minutes. To turn it back on, in one embodiment, one could take off the eyewear and put it back on again. In another embodiment, the user could activate or turn on the light manually via, for example, a switch at the eyewear. In yet another embodiment, the user could turn on the light via voice, with the eyewear having voice recognition electronics.
In one embodiment, the eyewear can include hearing enhancement abilities. Different embodiments of hearing enhancement abilities in an eyewear are described in U.S. Pat. No. 7,760,898, entitled, “Eyeglasses with Hearing Enhanced and Other Audio Signal-generating Capabilities,” which is incorporated herein by reference.
In one embodiment, the eyewear can include a position identifying system, such as a GPS device, or other position identifying system, such as a system using wifi and/or cellular networks via, for example, triangulation. In one embodiment, a combination of more than one position identifying systems could be used to identify position, and the combination could become the position identifying system. In one embodiment, the position identifying system can be normally off and activated remotely to track the position of the eyewear. For example, the user could send out an emergency call to an interested party, who could activate the position identifying system. This could be used to track the user, such as when the user is lost, and provide instructions orally to guide the user home. In one embodiment, the user could activate the position identifying system, and the system could guide the user to a preset place, such as back home via, for example, voice.
In one embodiment, the eyewear can include a lanyard, which could be permanently attached to the eyewear. The lanyard could be attached to a battery pack to provide power (or additional power) to electronics in the eyewear. Different embodiments of electronics tethered to an eyewear are described in U.S. Pat. No. 7,192,136, entitled, “Tethered Electrical Components for Eyeglasses,” which is incorporated herein by reference.
In one embodiment, the eyewear could detect if the user has fallen. For example, the eyewear could include an accelerometer. As one example, if the eyewear (a) detects an accelerating and then a stop, (b) with the being-worn sensor indicating eyewear being worn, and (c) with little or no subsequent motion, the eyewear could assume the user has fallen, and send an emergency signal to an interested party. In another embodiment, the eyewear also could include a up/down sensor (or level sensor) showing the orientation of the eyewear. If the eyewear detects the above, with the eyewear orientation still showing it being in at least a substantially up orientation at the stop, the eyewear could send out an emergency signal asking an interested party to contact the user.
Different embodiments could be used to determine, for example, if a user wearing the eyewear is in danger, such as has fainted. One approach could be based on changes in the patterns of the monitored measurements, as determined by a controller in the eyewear. For example, the eyewear can be aware of the average heart beat or heart beat pattern of the user. A dangerous condition could be significant deviation from the average. Another approach could be based on measurements and/or changes in the patterns of measurements from different sensors in combination. For example, the eyewear has detected that the user might have fallen, and a microphone in the eyewear detects no sound from the user in a subsequent duration of time.
In one embodiment, in response to detecting danger, an interested party could try to contact the user. For example, the interested party could call the user and ask the user to, for example, push a button on the eyewear, such as pushing the button multiple times if the user is in danger.
In one embodiment, the eyewear can include an imaging system, such as a camera. The imaging system could be used, for example, to read barcodes off products, such as medicine bottles. And a controller in the eyewear could identify the product based on the barcodes read. In one embodiment, the imaging system could be a 3D imaging system, and the controller could, for example, identify the product based on its 3D image. Different embodiments of eyewear with cameras are described in U.S. Pat. No. 7,806,525, entitled, “Eyeglasses having a camera,” which is incorporated herein by reference.
In one embodiment, as shown, for example, in FIG. 7, the eyewear 700 includes (a) a button 702, which could be at an extended endpiece 704 of the eyewear 700, such as at its top surface (Different embodiments of eyewear with extended endpieces are described in U.S. Pat. No. 8,109,629, entitled, “Eyewear Supporting Electrical Components and Apparatus Therefor,” which is incorporated herein by reference); (b) at least a microphone, which could be at an extended endpiece of the eyewear; (c) at least a speaker 706, which could be at an extended endpiece of the eyewear, (d) at least a battery, which could be at an extended endpiece, or could be at an arm, of the eyewear; (e) a battery indicator; (f) voice recognition capability, with a digital assistant, to respond to a limited range of requests in voice; (g) a phone directory; and (h) cellular connection capabilities. The eyewear could also include a local wireless system that could track locations (such as based on WiFi signals), or a navigation satellite system that could track locations (such as a GPS system); and a pedometer. The eyewear could further include inductive charging ability, such as, for example, with wires at lens holders or arms of the eyewear. The eyewear, including the button, the microphone, and the speaker, could be waterproofed.
For the above embodiment, in operation, for example, such as when the button is pushed once, the digital assistant could respond to a limited range of requests from the user, such as: “What is the time?”; “Major news today?”; “Have I walked enough today?”; “Call Andy for me.”; “Louder.”; “Softer.”; “What is the time?”; “Help!”; “Everything is fine.”; and “How do I get home from here?” The digital assistant could be trained to respond accordingly. For example, if the question is “How do I get home?”, the digital assistant could respond, for example, by providing turn-by-turn directions via the built-in position identifying system to the user. The interaction with the digital assistant could be in a language preferred by the user.
In one embodiment, the user could use the eyewear to listen to audio books. For example, one or more digital books could be stored in or downloaded into the eyewear. The embodiment could include a display, such as a LCD display, to show the books and allow the user to scroll down the list of books to select the one the user wants. The books could be categorized. The user could select a category to have books under the category listed. Selection could be done via a button or a switch at the eyewear. In one embodiment, selection could be done via voice. In the embodiment with a digital assistant, the user could ask the digital assistant to start playing a digital book by describing to the digital assistant the book, such as telling the assistant the title of the book. In another embodiment, if the eyewear does not have the book, the digital assistant could find out how much it would cost to get the book, such as from Amazon, and ask the user if the user wants to acquire the book. If the user wants to, the digital assistant could download the book, such as from Amazon, based on, for example, the user's charge card information previously stored in the eyewear.
In one embodiment, the eyewear could include noise cancellation circuits, such as based on multiple microphones. For example, there could be a first microphone in an area on a top portion of the eyewear, such as a top portion of a lens holder or a top portion of an arm of the eyewear, for capturing sound from the environment. And there could be a second microphone in an area on a bottom portion of the eyewear, such as a bottom portion of a lens holder or a bottom portion of an arm of the eyewear, for capturing the user's voice. The audio signals from the second microphone capturing the user's voice could be adjusted based on the audio signals from the first microphone for noise cancellation, via techniques known to those skilled in the art.
In one embodiment, the eyewear could also identify where the user parked via another device in the car of the user, turning light on or off when asked, if the eyewear includes a light output, which could be pointing forward at an extended endpiece; and asking the user some brain exercise questions.
In one embodiment, if the digital assistant can't help, it could connect the user to a live professional operator (e.g. via cellular connection). The user can also connect to the operator directly at any time by quickly tapping the button, such as two or more times within a second. The operator can offer more personalized assistance, including responding to more difficult requests, or connecting the user to the right person for the situation, such as a close relative, as needed. When the battery is low, the battery indicator could provide an indication to the user, and the indication could be oral via the speaker.
In one embodiment, the eyewear could detect if the user has fallen. For example, the eyewear could include an accelerometer, which could operate as a motion sensor, such as a pedometer. As one example, if the eyewear detects an accelerating and then a stop, with little or no subsequent motion, the digital assistant could orally ask the user whether everything is fine. If the user doesn't respond, the eyewear could assume the user has fallen, and send an emergency signal to an interested party. In another embodiment, the eyewear also could include a up/down sensor (or level sensor) showing the orientation of the eyewear. If the eyewear detects the above regarding a fall, (a) with the eyewear orientation still showing it being in at least a substantially up orientation at the stop, and/or (b) with the eyewear orientation showing it being not in a substantially up orientation after the stop for a duration of time, the digital assistant could orally ask the user to respond. If the user doesn't respond, the eyewear could assume the user has fallen, and send an emergency signal asking for a caregiver to check on the user. In one embodiment, the caregiver, in addition to using the position identifying system in the eyewear, could also activate the eyewear wirelessly to give out a beeping sound to help the caregiver locate the user.
In one embodiment, the digital assistant can engage in simple dialogues with the user. This could be initiated by the user, such as by the user pushing the button and starting to talk to the digital assistant. For example, the user could say, “I don't feel good.” The assistant could respond open-endedly, such as, “Why don't you feel good?” Through the dialogue, the assistant could spot patterns in the user's spoken language. (I am here)
For example, once a day, or several times a day, the assistant could ask the user how the user is feeling, and could offer choices for the user to select to respond, such as lonely, sad, or depressed. Then the digital assistant could recognize words and phrases in subsequent dialogues with the user that could be contextually likely to be associated with the selected choice. In one embodiment, the assistant could provide some simple counseling based on standard cognitive behavioral therapy. In another embodiment, if the assistant decides that the user is depressed, the assistant could suggest activities the user likes that could enhance the user's mood or distract the user. For example, the assistant could play some music the user likes.
In one embodiment, the digital assistant could be trained to analyze (a) syntactic patterns in the user's words, such as the frequency of nouns used versus adjectives; (b) classes of words the user uses, such as whether they are more related to perception or action; (c) how often the user talks to the digital assistant; (d) whether the user changes topics more abruptly than the user's average responses; and (e) acoustic features, such as changes in volume, pitch, and frequency of pauses by the user.
In one embodiment, the digital assistant could identify dangerous signs. For example, by analyzing dialogue from the user for an extended period of time, the assistant could identify a drastic change in the user's verbal communication. Note that the analysis could be done by a remote device, based on the user's daily communication data transmitted by the eyewear. Or, in another embodiment, the analysis, or a portion of the analysis, could be done by the eyewear. The analysis could be based on machine learning. In view of the dangerous signs identified, in one embodiment, a professional operator or a close relative of the user could be alerted, and this could be done, for example, by the digital assistant.
One embodiment for voice recognition could use processors or servers remotely accessed via the Internet (such as in the cloud), such as GPU, to train the voice recognition models, which can be a machine learning-driven task. And the embodiment could use a lower power processor, a client processor, or an edge processor in the eyewear to run inferences. In one embodiment, with the models trained, the client processor could run voice recognition without the need to wirelessly call the servers. This could lead to consuming less power, reducing latency, and giving a better user experience. In another embodiment, the eyewear could directly interact with the remote servers without using a client processor.
In one embodiment, for the client processor, the eyewear could use a lighter-version TPU (Tensor Processing Unit), a custom chip optimized for inference in edge devices. Instead of using TensorFlow, a Google machine learning development platform, other embodiments could use development platforms, such as Caffe2 or PyTorch.
Instead of voice recognition, different embodiments could train and/or use machine learning models in the remote servers for other applications, such as image recognition.
In one embodiment, the eyewear could include mental exercises to sharpen mental skills of the user, such as word games, testing recall, and doing math mentally. In another embodiment, the eyewear could play games or other activities with the user to entertain the user. As an example, this could be administered to the user verbally by the eyewear.
In one embodiment, the eyewear could turn off automatically if there is no audio input, such as from the user, received for a preset amount of time, such as 2 minutes. In another embodiment, the user could ask the digital assistant to turn the eyewear off by, for example, saying, “Goodbye, glasses.” In one embodiment, the eyewear or the digital assistant could be programmed to be called a name, such as Joe. To turn the eyewear off, the user could say, “Goodbye, Joe.”
In one embodiment, instead of or in addition to a button, the eyewear could be programmed to be activated by voice, such as when the user said a specific word or phrase. When that word/phrase by the user is captured by the eyewear, the eyewear could be activated. In one embodiment, the eyewear, with voice recognition capability, does not have the button and could be activated by voice. In one embodiment, when the eyewear is activated, a digital assistant in the eyewear could address the user.
In one embodiment, the eyewear could be wirelessly coupled to a medicine box. This could be done via short-range wireless communication system, such as Bluetooth. When the medicine box is opened, the box could send a signal to the eyewear. The signal could be an indication that the user has initiated an action to take medication. The signal could also initiate an audio script by the eyewear as to the medication to take and how much. In another example, based on time monitored by the eyewear, the audio script could indicate to the user that it is not time to take any medication yet.
In one embodiment, the eyewear could be coupled, such as wirelessly coupled, to a remote medical monitoring system. The system could keep track of measurements performed by the eyewear on a user, and perform analysis accordingly. Different embodiments of a medical monitoring system are described in U.S. Pat. No. 8,112,293, entitled, “Medical monitoring system,” which is incorporated herein by reference.
In one embodiment, the eyewear could wirelessly interact with one or more other types of electronic devices in its vicinity (or local e-devices). A local e-device could be at one's property indoor or outdoor. It typically includes computing and communication electronics that, for example, allow it to interact, pair, or be authorized to participate and communicate via a local network or short-range wireless network (e.g. a wifi network or a Bluetooth network) with at least the eyewear. In one embodiment, a local e-device can be an IOT device. In another embodiment, a local e-device can be connected to an electrical power outlet at a wall. In yet another embodiment, a local e-device could also interact wirelessly with one or more other local e-devices in its vicinity.
In one embodiment, a local e-device could be an indoor device, such as in a house of the user.
In one embodiment, an indoor device could be coupled to an apparatus or a physical object in an indoor environment, such as in the house of the user; and the apparatus includes at least a status that could be changed by the user. As the status changes, a signal could be sent to the indoor device, which could send a corresponding signal to the eyewear. The eyewear does not have to be fully turned on, but can be in a listen mode to receive the signal, and the eyewear could act according to the signal received.
For example, the apparatus could be a stove with, for example, a status indicating a burner being turned on or off; a refrigerator with, for example, a status indicating its door being opened or shut; a door (such as a garage door, a front door or a patio door) with, for example, a status indicating it being opened or closed; a faucet with, for example, a status indicating it being turned on or off; and a shower with, for example, a status indicating it being turned on or off.
In one embodiment, as the user changes a status of an apparatus, the corresponding indoor device could send a signal to the eyewear of the user (which could be worn by the user), alerting the eyewear of the change. In another embodiment, if the status is changed in a way different from its norm, an alert signal could be sent to the eyewear. Depending on the alert signal received, the eyewear could react accordingly.
For example, when the user turns on a burner at a stove, an indoor device of the stove could send a wireless signal regarding the burner being turned on to the eyewear of the user. In one approach, a digital assistant at the eyewear could alert the user if the eyewear has not received from the indoor device another wireless signal regarding the burner being turned off after a preset amount of time.
As another example, when the user opens a door, an indoor device at the door could send a wireless signal regarding the door being opened to the eyewear of the user. In one approach, an alert signal at the eyewear could notify the user if the eyewear does not receive from the indoor device another wireless signal regarding the door being closed, such as after a preset amount of time.
In one embodiment, an indoor device can be used to open the door of a user's house. For example, the corresponding system could include at least two local e-devices, such as a microphone/speaker device at the door and a lock-control device controlling the lock of the door. When a visitor, for example, rings the door bell, the user could turn on the eyewear and ask the digital assistant to ask the visitor questions, such as the visitor's identity and the purpose of the visit, via the speaker at the door. The visitor's response can be captured by the microphone at the door, and presented to the user via the eyewear. If the user wants to open the door, the user could instruct the digital assistant to remotely open the door via the lock-control device. The digital assistant could re-confirm with the user before remotely activating the lock-control device to open the door. In one embodiment, the door bell also could be a local e-device. When the visitor rings the door bell, the eyewear could be notified, which, in turn, could notify the user accordingly.
In another embodiment, the corresponding system to open the door of a user's house could include a video camera to capture video/images of the visitor and transmit such information to the user via, for example, a display. In one embodiment, the display could be at the eyewear. In another embodiment, the display could be a monitor external to the eyewear. The eyewear could alert the user to watch the captured video/images.
In one embodiment, an indoor device can be used to lock a door, and the lock could be controlled electrically. In one embodiment, the device could be electrically coupled to the lock; and in another embodiment, the device includes the lock. A user could control to lock the door via the user's eyewear, which can be wirelessly coupled to the indoor device.
In one embodiment, an indoor device could be used to adjust one or more thermostats in the house. In one embodiment, an indoor device could be used to turn on/off or dim one or more lights in the house. In one embodiment, an indoor device could be used to control one or more other appliances in the house. In one embodiment, the adjustment/control could be through a digital assistant in the eyewear.
One embodiment previously described includes a medicine box sending signals to the eyewear. In one embodiment, a medicine holder could include an indoor device. When the medicine holder is opened, the indoor device could send a message to the eyewear. If the user opens the medicine holder again the same day, based on another message from the indoor device, the eyewear could alert the user that the holder has been opened before that day.
As yet another example, an indoor device could monitor one or more statuses of a car, such as the location of the car. After the car has been parked for a preset amount of time, such as 15 minutes, inside a garage of the house of the user, if a digital assistant in the eyewear has not received a signal from the indoor device that the car has been turned off, the digital assistant could alert the user.
In one embodiment, a signal from the eyewear to the user, such as an alert signal, could be a beeping sound. In another example, such a signal could be optical, such as via light from a LED at the eyewear. In yet another example, such a signal could be a voice message from a digital assistant from the eyewear.
In one embodiment, an indoor device of an apparatus could restrict the user from using the apparatus unless the user could satisfy certain predetermined criteria. For example, the apparatus could be a gun or a chain saw, which could be stored in a container. When the user wants to open the container or when the user wants to operate the apparatus, the user could, for example, be asked a question, such as the address of the house with the apparatus. Whether the user could answer the question correctly determines if the user could use the apparatus. In one embodiment, the indoor device could have the apparatus send a signal to the eyewear, requesting the eyewear to ask the question and to monitor the answer. Or, the indoor device could have the apparatus ask the question and monitor the answer, and send information regarding the interrogation, including the monitoring, to the eyewear.
In one embodiment, an alert or a signal from the eyewear regarding the status of a corresponding apparatus could be deactivated. For example, a digital assistant could alert the user after a burner at a stove has been turned on for 5 minutes. To illustrate, it turns out the user is making a stew, and the user could tell the digital assistant not to send any more alert regarding this episode. Or, the user could ask the digital assistant to send a reminder after a certain amount of time, such as 30 minutes. In another embodiment, the digital assistant could be configured to override the user's request of not sending any more alert if after a unreasonable or predetermined amount of time, the eyewear still has not received any signal regarding the burner being turned off.
In one embodiment, a local e-device could be an outdoor device in the vicinity of the eyewear.
In one embodiment, an outdoor device could be coupled to an apparatus or a physical object out in the open; and the apparatus includes at least a status that could be changed by the user. As the status changes, a signal could be sent to or received by the outdoor device, which could wirelessly send a corresponding signal to the eyewear.
For example, the apparatus could be a car door with at least a status indicating the door being opened or closed. When the user opens the car door, an outdoor device at the door could send a wireless signal regarding the door being opened to the eyewear of the user. In one approach, a signal at the eyewear could notify the user if the eyewear has not received from the outdoor device another wireless signal regarding the door being closed, such as after a preset amount of time.
In one embodiment, an outdoor device could be in a car, and could be electrically coupled to batteries in the car and a position identifying system in the car that could track locations (such as via a navigation satellite system). An eyewear could be wirelessly coupled to the outdoor device via, for example, a wifi network or a cellular network. To illustrate, the car is in an outdoor parking lot, and a user could not find it. The user could ask a digital assistant in her eyewear for guidance. The digital assistant could send a wireless signal to the outdoor device. The outdoor device would determine its position (or the position of the car) via the position identifying system in the car. Then the outdoor device could wirelessly message the position to the digital assistant. The digital assistant could determine the position of the eyewear via a position identifying system in the eyewear. Based on the positions of the outdoor device and the eyewear, the digital assistant could guide the user back to the car.
In one embodiment, a local e-device could couple to an object, and operate both indoor and outdoor, in the vicinity of the eyewear.
A number of examples have been described regarding local e-devices. In one embodiment, at least each local e-device that is configured to interact with the eyewear has an unique identifier relative to other local e-devices that are configured to interact with the eyewear. This could serve as a way to distinguish signals from one local e-device relative to signals from another local e-device, received by the eyewear.
In one embodiment, the eyewear could keep track of its interactions with the one or more local e-devices in its vicinity. The eyewear could analyze the interactions; or information regarding the interactions could be sent, such as wirelessly, to another computer to be analyzed. In one embodiment, based on the analysis, norms could be established regarding typical, normal, or regular behavior of the user toward different physical objects indoor or outside in the open.
In one embodiment, if a user behavior to a physical object indoor or outside in the open is significantly different from the corresponding norm, a digital assistant in the eyewear could ask the user whether there is any concern. Based on the user's response, the digital assistant could notify different persons of interest, and could present to them what has happened. In one embodiment, significantly different from the norm could be more than one and a half standard deviations from the norm or mean. In another embodiment, significantly different from the norm could be two or more than two standard deviations from the norm or mean.
In one embodiment, the eyewear includes an imaging system, which could include one or more cameras, such as optical cameras and/or infrared cameras. The imaging system could be used to gather images, which could be videos, of the environment of the eyewear. The images could be analyzed to help the user. The analysis could be done at the eyewear. In another embodiment, the eyewear could be wirelessly coupled to a computing device, and the eyewear could send the images to the computing device to perform the analysis. The computing device could be coupled to the eyewear via a local network (e.g. a local area network, or a wifi network). After the analysis, the computing device could send the results back to the eyewear.
For example, the user is standing in front of an oven and does not know how to operate the different buttons on the oven. The user could ask the digital assistant in the eyewear, and the digital assistant could have the imaging system took images and then have them analyzed to help the user. If the user is not satisfied with the help, the user could ask the digital assistant to have the images sent to a live person and connect the user to the live person to help the user. In one embodiment, the imaging system could keep taking images, which could be videos, of the oven as the user is talking to the live person, who could help the user to operate the different buttons on the oven.
In another embodiment, when the user is standing in front of a stove and tries to turn on the stove, an indoor device at the stove could, for example, ask the user a question before the stove can be turned on. In yet another embodiment, the indoor device could send a signal to the eyewear, which could be in a listening mode. The signal could activate the eyewear, which could analyze the signal, and ask the user a question. For example, the question could be, “Why do you want to use the stove?” The user could respond, “I want to heat up some soup.” The response can be analyzed, based on, for example, natural-language processing techniques, to determine if it is an allowed activity in the context of operating a stove. The analysis could use different pre-configured rules. For example, a rule could be time based, such as certain activities being allowed at a certain time, but not allowed at a different time.
In one embodiment, if the response is deemed appropriate, then the requested action would be permitted. For example, the user could turn on the stove if the response is deemed appropriate. However, if the response is deemed not appropriate, then the user would not be able to turn on the stove. The eyewear, the indoor device at the stove, or the stove, could inform the user, for example, that it is not a good time to use the stove, such as to heat some soup.
In another embodiment, the imaging system could include additional sensors to assist the user. For example, the imaging system could include a flame detector to detect flames, such as flames of a stove; and the eyewear can interact with the user to insure the stove is used appropriately. To illustrate, one rule could be that two burners should not be turned on and left on for a preset amount of time. Based on such a rule, if the imaging system determines that two burners have been turned on for more than the preset amount of time, such as 10 seconds, the eyewear could react, such as alert the user, accordingly. For example, the eyewear could interact with a corresponding indoor device at the stove, which could react accordingly, such as turning down the flames at both burners or turning both burners off.
In yet another embodiment, the eyewear can also take and analyze images to see if the user is actually using the stove appropriately, such as to heat soup. Based on the analysis, the eyewear could act accordingly. For example, the eyewear could send signals to an indoor device at the stove to have the stove operated accordingly.
In one embodiment, instead of performing the analysis, the eyewear could communicate with a supervising person to determine whether the user is allowed to perform certain activity. For example, when the user wants to turn on a stove, a corresponding indoor device could alert the eyewear, which could take images of the user intending to turn on a stove, and send the images to the person. Or, the eyewear could send a text message to the person asking whether it is ok for the user to use the stove, such as to heat soup. Based on the response from the supervisor, the eyewear could send a corresponding signal to the indoor device to act accordingly. For example, if the corresponding signal is no, the indoor device could forbid the stove to be turned on.
A number of embodiments have been described regarding an eyewear interacting with the one or more local e-devices in its vicinity. In one embodiment, instead of an eyewear, the embodiment could include a wearable device interacting with the one or more local e-devices in its vicinity. The wearable device could be in the format of a watch, a wristband, a piece of jewelry, a chest strap, a piece of clothing, or other types of devices wearable on the body of the user.
Different embodiments regarding an eyewear have been described. The eyewear could be, for example, a pair of sunglasses, auxiliary frames, fit-over glasses, prescription glasses, safety glasses, swim masks, and goggles (such as ski goggles). In one embodiment, the eyewear could be incorporated in a helmet or other type of headgear.
Different embodiments of electronics in an eyewear are described in (a) U.S. Pat. No. 9,488,520, entitled, “Eyewear With Radiation Detection,” which is incorporated herein by reference; (b) U.S. Pat. No. 8,905,542, entitled, “Eyewear Supporting Bone Conducting Speaker,” which is incorporated herein by reference; and (c) U.S. Pat. No. 8,337,013, entitled, “Eyeglasses with RFID Tags or with a Strap,” which is incorporated herein by reference.
The various embodiments, implementations and features of the invention noted above can be combined in various ways or used separately. Those skilled in the art will understand from the description that the invention can be equally applied to or used in other various different settings with respect to various combinations, embodiments, implementations or features provided in the description herein.
Numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will become obvious to those skilled in the art that the invention may be practiced without these specific details. The description and representation herein are the common meanings used by those experienced or skilled in the art to most effectively convey the substance of their work to others skilled in the art. In other instances, well-known methods, procedures, components, and circuitry have not been described in detail to avoid unnecessarily obscuring aspects of the present invention.
Also, in this specification, reference to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the order of blocks in process flowcharts or diagrams representing one or more embodiments of the invention do not inherently indicate any particular order nor imply any limitations in the invention.
Other embodiments of the invention will be apparent to those skilled in the art from a consideration of this specification or practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

Claims

What is claimed is:

1. An eyewear frame for a user comprising:

a first wireless communication component configured to:

couple to a second wireless communication component in an electronic device associated with an apparatus of a dwelling of the user; and

receive a message from the electronic device via the second wireless communication component;

an output device configured to present the message;

a microphone configured to receive a voice input from the user, at least in view of the message being presented;

a memory storing at least voice-recognition software code configured to analyze the voice input; and

a controller configured to:

generate an output based on the analyzed voice input; and

transmit the output via the first wireless communication component to the apparatus to change a physical status of the apparatus.

2. An eyewear frame as recited in claim 2, wherein the output is configured for turning the apparatus off.

3. An eyewear frame as recited in claim 3, wherein the apparatus includes a stove.

4. An eyewear frame for a user as recited in claim 1 comprising at least two microphones and at least an electronic component configured for noise cancellation connected to the at least two microphones.

5. An eyewear frame for a user as recited in claim 1, wherein the first wireless communication component is configured to receive at least an indication to be presented to the user regarding the physical status of the apparatus being changed.