GB2565534A - Input method and apparatus - Google Patents

Input method and apparatus Download PDF

Info

Publication number
GB2565534A
GB2565534A GB1712760.6A GB201712760A GB2565534A GB 2565534 A GB2565534 A GB 2565534A GB 201712760 A GB201712760 A GB 201712760A GB 2565534 A GB2565534 A GB 2565534A
Authority
GB
United Kingdom
Prior art keywords
user
digit
optical sensor
input device
wearable electronic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB1712760.6A
Other versions
GB201712760D0 (en
Inventor
Breugelmans Mark
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Interactive Entertainment Inc
Original Assignee
Sony Interactive Entertainment Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Interactive Entertainment Inc filed Critical Sony Interactive Entertainment Inc
Priority to GB1712760.6A priority Critical patent/GB2565534A/en
Publication of GB201712760D0 publication Critical patent/GB201712760D0/en
Publication of GB2565534A publication Critical patent/GB2565534A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304Detection arrangements using opto-electronic means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304Detection arrangements using opto-electronic means
    • G06F3/0317Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/033Indexing scheme relating to G06F3/033
    • G06F2203/0331Finger worn pointing device

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)

Abstract

A wearable electronic device comprises at least a first input device (310) configured to be positioned on a portion of a user's digit (350) and to detect a presence of a surface within a threshold distance, the first input device comprises a light source (311) configured to emit light, an optical sensor (312) configured to detect a presence of a surface within a threshold distance of the optical sensor based on reflection of the emitted light, to determine a direction and a magnitude of a movement of the optical sensor relative to the surface based on changes in the reflected light, and to generate corresponding direction and magnitude information, wherein the optical sensor is positioned proximate to a distal portion of the user's digit, and a transmitter configured to transmit data indicative of the information generated by the optical sensor. The input device may include a sleeve that covers a portion of an extremity of a user’s finger such that a portion of the extremity is not covered (figure 4) allowing the use of a keyboard or the like.

Description

INPUT METHOD AND APPARATUS
The present invention relates to computer input devices and more particularly to a wearable computer input device.
Even with the recent advances in computer performance and computer input techniques (e.g., wireless computer mice, touchscreen devices, handheld controllers), a conventional computer mouse comprising a light source and an optical sensor remains the most popular input device for the industry.
While the control afforded by the conventional computer mouse has made it a popular input means for both general purpose computing and electronic gaming, the conventional computer mouse is nevertheless restricted to use with approximately horizontal surfaces over which the underside of the mouse can reliably slide and upon which the mouse can rest when not in use. This means that operation of the conventional computer mouse has typically been limited to use with flat work surfaces where the user is seated.
Means and techniques to enhance the functionality of computer input devices comprising one or more optical sensors are therefore advantageous.
The present invention seeks to improve the functionality of computer input devices comprising one or more optical sensors.
In a first aspect, there is provided a wearable electronic device for detecting a presence of an external surface within a threshold distance of a user’s digit in accordance with claim 1.
In another aspect, there is provided an information processing device configured to receive data from a wearable electronic device in accordance with claim 13.
In another aspect, there is provided a method of detecting a presence of an external surface within a threshold distance of a user’s digit in accordance with claim 14.
Further respective aspects and features of the invention are defined in the appended claims.
Embodiments of the present invention will now be described by way of example with reference to the accompanying drawings, in which:
Figure 1A illustrates an isometric view of a wearable electronic device comprising a first input device which comprises a light source, an optical sensor positioned proximate to a distal portion of a user’s digit and a transmitter, in accordance with an embodiment of the present invention.
Figure IB illustrates an isometric view of a light source and an optical sensor of a first input device of a wearable electronic device, in accordance with an embodiment of the present invention.
Figure 2 is a schematic diagram of apparatus comprising an information processing device, in accordance with an embodiment of the present invention.
Figure 3A illustrates an isometric view of a wearable electronic device with an optical sensor positioned on a mounting surface of a cap that surrounds an extremity of a user’s digit, in accordance with an embodiment of the present invention.
Figure 3B illustrates an isometric cross-sectional view of a wearable electronic device with an optical sensor positioned on a mounting surface of a cap that surrounds an extremity of a user’s digit.
Figure 4A illustrates an isometric view of a wearable electronic device with an optical sensor positioned on a mounting surface of a sleeve that covers a portion of an extremity of a user’s digit, in accordance with an embodiment of the present invention.
Figure 4B illustrates an isometric view of a wearable electronic device with an optical sensor positioned on a mounting surface of a cylinder that covers a portion of a distal portion of a user’s digit and does not cover an extremity of the user’s digit, in accordance with an embodiment of the present invention.
Figure 4C illustrates an isometric view of a wearable electronic device with an optical sensor positioned on a mounting surface connected to a partial ring that grips around a portion of a circumference of a distal portion of a user’s digit, in accordance with an embodiment of the present invention.
Figure 5A illustrates an isometric view of a wearable electronic device with a tactile switch that detects whether a surface is in contact with the wearable electronic device, in accordance with an embodiment of the present invention.
Figure 5B illustrates an isometric view of a wearable electronic device with a first tactile switch and a second tactile switch positioned on respective portions of a mounting surface proximate to an extremity of a user’s digit, in accordance with an embodiment of the present invention.
Figure 6A illustrates an isometric view of a wearable electronic device with a tactile switch positioned on a mounting surface of a sleeve which is contactable by another digit of a user’s hand.
Figure 6B illustrates an isometric view of a wearable electronic device with a tactile switch positioned on a mounting surface of a cap which is contactable by another digit of a user’s hand.
Figure 6C illustrates an isometric view of a wearable electronic device with a plurality of tactile switches positioned on a mounting surface which is contactable by another digit of a user’s hand, in accordance with an embodiment of the present invention.
Figure 7 illustrates an electric motor of a first input device for generating a force to provide haptic feedback to a user’s digit when a surface is in contact with the first input device.
Figure 8 is a functional block diagram of hardware to detect an orientation of an input device of a wearable electronic device, in accordance with an embodiment of the present invention.
Figure 9 illustrates an isometric view of a first input device 610 of a wearable electronic device comprising a light-emitting-diode (LED) for detection by a camera, in accordance with an embodiment of the present invention.
Figure 10 is a functional block diagram of hardware for wireless communication of data indicative of information generated by an optical sensor of a wearable electronic device, in accordance with an embodiment of the present invention.
Figure 11 is a flow diagram of a method of detecting a presence of an external surface within a threshold distance of a user’s digit and determining a relative movement of the external surface, in accordance with an embodiment of the present invention.
For clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. Further, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding or analogous elements.
Methods and systems are disclosed for improving the functionality of computer input devices comprising one or more optical sensors. In the following description, a number of specific details are presented in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to a person skilled in the art that these specific details need not be employed to practice the present invention. Conversely, specific details known to the person skilled in the art are omitted for the purposes of clarity where appropriate.
The conventional computer mouse includes a light source for emitting visible or infrared light, and an optical sensor capable of detecting the emitted light that is reflected off a surface proximate to the computer mouse. The optical sensor measures the properties of the reflected light and changes in the properties of the reflected light, which can occur due to movement of the computer mouse relative to the surface, can be detected by the optical sensor. A user can move the computer mouse relative to the surface, and changes in the light sensed by the optical sensor can be measured to determine a direction and a magnitude of the movement of the computer mouse relative to the external surface. This means that the direction and the magnitude associated with the movement of the computer mouse relative to the surface can be determined and corresponding information is generated that is transmitted (by a wired or wireless link) to an information processing device, so that movement of the computer mouse by the user can be translated to user input for an application executing on the information processing device.
During normal use, the conventional computer mouse is held by the user’s hand so that the user’s hand movements are used to control the movement of the computer mouse. When not held, the computer mouse is typically left to reside upon a substantially horizontal work surface so that the user is able to locate the computer mouse and resume use when required. Consequently, the conventional computer mouse has been typically restricted to use with conventional desktop computers and the user’s hand must locate the computer mouse when use is resumed.
In an embodiment of the present invention, a wearable electronic device, as described later herein, may be implemented by a wearable electronic device comprising at least a first input device configured to be positioned on a portion of a user’s digit and detect a presence of a surface within a threshold distance.
The wearable electronic device comprises at least the first input device, wherein the first input device comprises a light source configured to emit light.
The first input device comprises an optical sensor configured to detect a presence of a surface within a threshold distance of the optical sensor based on reflection of the emitted light, determine a direction and a magnitude of a movement of the optical sensor relative to the surface based on changes in the reflected light, and generate corresponding information, wherein the optical sensor is positioned proximate to a distal portion of the user’s digit.
The first input device comprises a transmitter configured to transmit data indicative of the information generated by the optical sensor.
Figure 1A illustrates an isometric view of a wearable electronic device 100 comprising a first input device 110 which comprises a light source 111, an optical sensor 112 positioned proximate to a distal portion of a user’s digit 150 and a transmitter (not shown), in accordance with an embodiment of the present invention. The light source 111 may comprise a lightemitting-diode (LED) positioned on a mounting surface 113 of the first input device 110, and the light source Illis thus capable of emitting either visible light or infrared light depending on the choice of LED. The light source 111 may comprise more than one LED and may be capable of emitting either visible light, or infrared light, or both infrared and visible light. The light source 111 optionally comprises a prism system so that light emitted from the LED enters the prism system and is refracted in a direction away from the mounting surface 113. The light source 111 is configured to emit light in a direction away from the mounting surface and the presence of a proximate surface will cause the emitted light to be reflected back towards the first input device 110. The light source 111 can be positioned either on a plane substantially the same as the mounting surface 113, as in Figure 1 A, or on a plane formed by a recess in the mounting surface 113, as shown in Figure IB described later herein. The light source 111 may be positioned within the recess in the mounting surface 113 and the recess can be shaped so that light emitted by the light source Illis emitted from the mounting surface in a direction determined by the shape of the recess. Alternatively or in addition, the prism system may be positioned within the recess in the mounting surface 113 so that light is refracted in a direction compatible with the shape of the recess.
The first input device 110 comprises an optical sensor 112 configured to detect the presence of the surface within the threshold distance of the optical sensor 112 based on reflection of the emitted light, determine a direction and a magnitude of a movement of the optical sensor 112 relative to the surface based on changes in the reflected light, and generate corresponding information, wherein the optical sensor is positioned proximate to a distal portion of the user’s digit 150. The optical sensor 112 typically comprises a complementary metal oxide semiconductor (CMOS) sensor having a two-dimensional array of pixels for sensing visible and infrared light, and may also comprise a lens for focussing the reflected light onto the CMOS sensor. The respective pixels of the CMOS sensor are sensitive to incident light and each pixel can be readout and digitised so that information can be generated indicating the changes in incident light as a function of time. Light that is incident upon the optical sensor 112 can be detected and an electrical signal is generated by each pixel that is dependent on the amount of light incident upon that pixel. The optical sensor may comprise a digital signal processor for analysing the information generated by the CMOS sensor and determining the direction and magnitude of the movement based on the changes in the information generated by the CMOS sensor.
The first input device 110 comprises a transmitter configured to transmit data indicative of the information generated by optical sensor 112. The transmitter is typically integrated within the first input device 110 and uses radio frequencies to transmit the data from the first input device 110 to an information processing unit. It will be apparent to those skilled in the art that data may be transmitted or communicated from the transmitter of the first input device 110 via wired or wireless communication means (such as Bluetooth® or other personal area network (PAN) communication means). For example, the transmitter may be configured to wirelessly transmit data using Bluetooth® RF technology operating in the 2.4 GHz range. The operation of the transmitter is discussed later herein.
For a surface (e.g., a wall, a table, a book or part of the user’s clothing) that is proximate to the first input device 110, light that is emitted from the light source 111 will scatter off the surface and reflect back towards the first input device 110. As the first input device 110 moves closer to the surface, the amount of light reflected back towards the first input device 110 will increase, and thus the amount of light sensed by the optical sensor 112 will increase, which causes the magnitude of the electrical signal generated by the optical sensor to increase. As such, the electrical signal generated by the optical sensor 112 is dependent on the distance between the first input device 110 and the surface, wherein a larger electrical signal indicates that the surface is closer to the first input device 110, and a smaller electrical signal indicates that the surface is further away from the first input device 110. The electrical signal generated by the optical sensor 112 can be compared with a first threshold to determine whether the surface is within a threshold distance of the optical sensor 112, and thus whether the surface is within a threshold distance of the first input device 110 positioned on a portion of a user’s digit 150. Hence more generally, the optical sensor 112 is configured to generate an electrical signal based on the amount of sensed light, and detect the presence of a surface within a threshold distance of the optical sensor 112 and the first input device 110 by comparing the generated electrical signal with a first threshold value. When the electrical signal generated by the optical sensor 112 is greater than the first threshold value, this indicates that the surface is within the threshold distance of the optical sensor 112 and the first input device 110. Alternatively, when the electrical signal generated by the optical sensor 112 is smaller than the first threshold value, this indicates that the surface is not within the threshold distance of the optical sensor 112 and the first input device 110. It will be appreciated that this is only one possible embodiment of this feature and that others are considered to be within the scope of the invention, such as for example the use of structured light (e.g. an illuminated pattern), where the apparent scale and/or distortion of the light (optionally in conjunction with brightness) may be used to determine the relative distance of the mouse to the surface. Alternatively or in addition, the light may be angled so that reflections from a surface will only impinge on the sensor when the mouse is within a predetermined distance from the surface. Indeed, conventional mice recess their sensor to provide such predetermined distance when the mouse is resting flat on a surface. In this case there is no threshold value as such, merely situations where the light is detected by the sensor to an extent that a measurement can be made, and situations where it does not work.
The readout rate and ADC clock frequency of the optical sensor 112 is such that signals from respective pixels can be readout and digitized, and successive images can be captured by the optical sensor 112 so that the optical sensor 112 can detect changes in the light incident upon the optical sensor 112 and generate corresponding information. For a surface detected to be within the threshold distance of the optical sensor 112 based on the amount of light detected by the optical sensor 112, a movement of the surface relative to the optical sensor 112 can cause changes in the properties of the light sensed by the optical sensor 112. Consequently, the movement of the surface relative to the optical sensor 112 can be determined based on changes in the light detected by the optical sensor 112. A direction and a magnitude of a displacement associated with the movement of the optical sensor 112 relative to the surface can be determined by detecting the changes in the light incident upon the optical sensor 112 due to changes in the light reflected from the surface. Hence more generally the optical sensor 112 is configured to detect a presence of a surface within a threshold or working distance of the optical sensor 112 and is configured to detect changes in the light reflected by the surface and generate information corresponding to the movement and displacement of the surface relative to the optical sensor 112.
Referring now again to Figure 1 A, in an embodiment of the present invention a wearable electronic device 100 comprising at least a first input device 110 configured to be positioned on a portion of a user’s digit 150 and detect a presence of a surface within a threshold or working distance is provided. The wearable electronic device 100 may comprise a plurality of input devices such that the first input device 110 is worn on a first digit 150, a second input device is worn on a second digit, and so-on.
The wearable electronic device 100 comprises at least the first input device 110, wherein the first input device 110 comprises a light source 111 configured to emit light. The light source 111 may for example be an LED capable of emitting either visible or infrared light. The light source 111 is configured to emit light in a direction away from the first input device 110, and the light source 111 may further comprise a prism system through which the emitted light passes and is refracted in a direction away from the first input device 110.
The first input device comprises an optical sensor 112 configured to detect a presence of a surface within a threshold distance of the optical sensor 112 based on reflection of the emitted light, and configured to determine a direction and a magnitude of a movement of the optical sensor 112 relative to the surface based on changes in the reflected light, and configured to generate corresponding information, wherein the optical sensor is positioned proximate to a distal portion of the user’s digit 150. The optical sensor 112 may for example comprise a CMOS sensor comprising a two-dimensional array of pixels.
The first input device 110 comprises a transmitter configured to transmit data indicative of the information generated by the optical sensor 112. The transmitter is integrated in the first input device 110 and uses radio frequencies to transmit the data from the first input device 110 to an information processing unit according to a low power wireless communication such as Bluetooth® or another personal area network (PAN) communication means.
With regards to the optical sensor, in an embodiment of the present invention, the optical sensor is configured to detect the presence of a surface within a threshold distance, determine whether the surface is in contact with the first input device, and generate information indicating whether the surface is in contact with the first input device. Figure IB illustrates an isometric view of a light source 111 and an optical sensor 112 of a first input device 110 of a wearable electronic device 110, in accordance with an embodiment of the present invention. The light source 111 and the optical sensor 112 may be positioned on the mounting surface 113 of the first input device 110 and on substantially the same plane as the mounting surface 113, as shown in Figure 1A, or may alternatively be positioned within the recesses formed on the mounting surface 113, as shown in Figure IB, and light can be emitted from the light source 111 in a direction away from the mounting surface 113.
It will be appreciated that as noted previously, references to a threshold distance or value herein can equally refer to detection by active measurement and comparison, or passive detection though design, e.g. as a function of successful and sufficient reflection from light source to sensor in order to work.
As noted previously, the optical sensor 112 may be configured to detect a presence of a surface within the threshold distance of the optical sensor 112 by detecting the amount of light emitted from the light source 111 that is reflected from the surface back towards the optical sensor 112 of the first input device 110. The optical sensor 112 detects the presence of the surface within the threshold distance by comparing a signal generated by the optical sensor 112 with a first threshold value. Based on the amount of light detected by the optical sensor 112, the optical sensor can firstly detect whether the surface is within the threshold distance using the first threshold value, and if so the optical sensor 112 can determine whether or not the surface is in contact with the first input device 110 using a second threshold value. The optical sensor 112 is configured to determine whether the surface is in contact with the first input device 110 by comparing the signal generated by the optical sensor 112 with the second threshold value. For example, if an electrical signal generated by the optical sensor 112 is greater than the first threshold value but less than the second threshold value, this indicates that the surface is within the threshold distance of the optical sensor 112 but the surface is not in contact with the first input device 110. When the electrical signal generated by the optical sensor 112 is greater than the first threshold value and approximately equal to the second threshold value, this indicates that the surface is in contact with the first input device 110 and the optical sensor 112 is configured to generate information indicating that the surface in contact with the first input device 110.
Hence more generally, the optical sensor 112 is configured to detect the presence of a surface within a threshold distance of the optical sensor 112 based on a first threshold value and if the surface is within the threshold distance, the optical sensor 112 is configured to determine whether the surface is in contact with the first input device 110 based on a second threshold value. The optical sensor 112 is configured to generate information indicating whether the surface is in contact with the first input device 110 in addition to the information indicating the direction and the magnitude of the movement of the surface relative to the optical sensor 112. Alternatively, the optical sensor may simply detect when the surface is sufficiently close for direction and the magnitude values to be obtained, and not make an explicit distinction between being close to and on the surface.
In an embodiment of the present invention, an information processing device (such as the Sony ® PlayStation 4 ® entertainment device) comprises a receiver adapted to receive from at least the first input device positioned on the portion of the user’s digit, for a surface within the threshold distance of the first input device, data indicative of the direction and the magnitude of the movement of the first input device relative to the surface, and a processor operable to update a state of an executing application responsive to the received data.
For the purposes of explanation only and as a non-limiting example, the present description will refer to the wearable electronic device operating in conjunction with a Sony ® PlayStation 4®.
Hence Figure 2 schematically illustrates the overall system architecture of a Sony ® PlayStation 4 ® entertainment device. A system unit 10 is provided, with various peripheral devices connectable to the system unit.
The system unit 10 comprises an accelerated processing unit (APU) 20 being a single chip that in turn comprises a central processing unit (CPU) 20A and a graphics processing unit (GPU) 20B. The APU 20 has access to a random access memory (RAM) unit 22.
The APU 20 communicates with a bus 40, optionally via an I/O bridge 24, which may be a discreet component or part of the APU 20.
Connected to the bus 40 are data storage components such as a hard disk drive 37, and a Blu-ray ® drive 36 operable to access data on compatible optical discs 36A. Additionally the RAM unit 22 may communicate with the bus 40.
Optionally also connected to the bus 40 is an auxiliary processor 38. The auxiliary processor 38 may be provided to run or support the operating system.
The system unit 10 communicates with peripheral devices as appropriate via an audio/visual input port 31, an Ethernet ® port 32, a Bluetooth ® wireless link 33, a Wi-Fi ® wireless link 34, or one or more universal serial bus (USB) ports 35. Audio and video may be output via an AV output 39, such as an HDMI port.
The peripheral devices may include a monoscopic or stereoscopic video camera 41 such as the PlayStation Eye ®; wand-style videogame controllers 42 such as the PlayStation Move ®;conventional handheld videogame controllers 43 such as the Dual Shock 4 ®;wearable devices such as the wearable electronic device 100 illustrated in Figure 1; portable entertainment devices 44 such as the PlayStation Portable ® and PlayStation Vita ®; a keyboard 45 and/or a mouse 46; a media controller 47, for example in the form of a remote control; and a headset 48. Other peripheral devices may similarly be considered such as a printer, or a 3D printer (not shown). Data from the wearable electronic device 100 comprising one or more input devices 110 can be received at the entertainment device 10 for example via the Bluetooth ®port (33).
The GPU 20B, optionally in conjunction with the CPU 20A, generates video images and audio for output via the AV output 39. Optionally the audio may be generated in conjunction with or instead by an audio processor (not shown).
The video and optionally the audio may be presented to a television 51. Where supported by the television, the video may be stereoscopic. The audio may be presented to a home cinema system 52 in one of a number of formats such as stereo, 5.1 surround sound or 7.1 surround sound. Video and audio may likewise be presented to a head mounted display unit 53 worn by a user 60.
The user may also interact with the system unit using a video camera 41 such as the PlayStation Eye ®. This may provide monoscopic or stereoscopic video images to the system unit 10 via for example AV input 31. Where these images capture some or all of the user, the user may enact gestures, facial expressions or speech as appropriate to interact with the currently presented user interface.
Alternatively or in addition, a peripheral designed to assist with camera-based user interaction, such as the PlayStation Move ® 42, may be provided. This peripheral has a wand form factor and an illuminated region that facilitates detection of the peripheral within a captured video image. Illuminated regions may similarly be provided on other peripherals, such as on the wearable electronic device 100, as described later herein. Both kinds of peripheral may comprise motion sensors to detect transverse movement along three axes and rotational movement around three axes, and wireless communication means (such as Bluetooth®) to convey movement data to the system unit. Optionally such peripherals can also receive control data from the system unit to enact functions such as a rumble effect, or to change the colour or brightness of the illuminated region, where these are supported by the peripheral.
The system unit may also communicate with a portable entertainment device 44. The portable entertainment device 44 will comprise its own set of control inputs and audio/visual outputs. Consequently, in a ‘remote play’ mode some or all of the portable entertainment device’s inputs may be relayed as inputs to the system unit 10, whilst video and/or audio outputs from the system unit 10 may be relayed to the portable entertainment device for use with its own audio/visual outputs. Communication may be wireless (e.g. via Bluetooth ® or Wi-Fi ®) or via a USB cable.
Other peripherals that may interact with the system unit 10, via either wired or wireless means, include a keyboard 45, a mouse 46, a media controller 47, and a headset 48. The headset may comprise one or two speakers, and optionally a microphone.
In operation, the entertainment device defaults to an operating system such as a variant of FreeBSD 9.0. The operating system may run on the CPU 20A, the auxiliary processor 38, or a mixture of the two. The operating system provides the user with a graphical user interface such as the PlayStation Dynamic Menu. The menu allows the user to access operating system features and to select games and optionally other content.
Referring now again to Figure 2, in an embodiment of the present invention, an information processing device 10 device (such as the Sony ® PlayStation 4 ® entertainment device) comprises a receiver (such as a Bluetooth ® wireless link 33) adapted to receive from at least the first input device 110, of the wearable electronic device 100, positioned on the portion of the user’s digit 150, for a surface within a threshold distance of the first input device 110, data indicative of the direction and the magnitude of the movement of the optical sensor 112 of the first input device 110 relative to the surface, and a processor (such as the CPU 20A or APU 20 operating under suitable software instruction) operable to update a state of an executing application responsive to the received data.
Hence more generally data indicative of the direction and the magnitude of the movement of the optical sensor 112 of the first input device 110 relative to the surface within the threshold distance may be received by the receiver 33 of the information processing device 10 and the processor (CPU 20A or APU 20) is operable to update the state of the executing application responsive to received data indicative of the information generated by the optical sensor 112. As such, the state of the executing application can be updated in response to the direction and the magnitude of the movement of the optical sensor 112 of the first input device 110, positioned on the portion of the user’s digit 150, relative to the surface. The state of the application being executed by the information processing device 10 can thus be updated responsive to the direction and the magnitude of the movement of the portion of the user’s digit 150 relative to the proximate surface.
In an embodiment of the present invention, the first input device comprises an optical sensor positioned on a mounting surface of a cap with a closed end configured to surround an extremity of the user’s digit. Figure 3A illustrates an isometric view of a wearable electronic device 300 comprising a first input device 310 with an optical sensor 312 positioned on a mountable surface 313 of a cap configured to surround an extremity of a user’s digit 350, in accordance with an embodiment of the present invention. The mountable surface 313 of the cap is configured to surround the extremity of the user’s digit 350 so that the first input device 310 may be worn on the user’s digit 350 like a thimble. The mountable surface 313 of the cap can extend around the full circumference of the user’s digit thus covering the extremity of the digit, including the portion of the extremity of the user’s digit that is typically used when typing on a keyboard or using a touch screen. Additionally, the mountable surface 313 can extend along the length of the user’s digit towards the base of the digit (towards user’s hand) so as to cover at least the extremity of the user’s digit.
Figure 3B illustrates an isometric cross-sectional view of the wearable electronic device 300 illustrated in Figure 3 A, in order to better show the position of the mountable surface 313 of the cap in relation to the extremity of the user’s digit. The optical sensor 312 is positioned on the mountable surface 313 of the cap and the presence of a surface within a threshold distance of the optical sensor 312, and thus the extremity of the user’s digit, can be detected based on the amount of light that is detected by the optical sensor. The direction and the magnitude of the movement of the surface relative to the extremity of the user’s digit can be detected based on changes in the light detected by the optical sensor 312, due to changes in the light reflected by the surface. It will be appreciated that the positions of the light source 311 and the optical sensor 312 on the mountable surface 313 of the cap may be different to the positions illustrated in Figure 3A and Figure 3B. For example, the light source 311 and optical sensor 312 may be positioned on the opposite side of the user’s digit to that shown in Figure 3B (above the nail of the digit), or the light source 311 and the optical sensor 312 may be positioned on a front portion of the mountable surface 313 at the limit of the extremity.
Hence the wearable electronic device 300 provides a means for detecting a presence of a surface within the threshold distance of a user’s digit 350, determining a direction and a magnitude of the movement of the surface relative to the user’s digit 350, generating information corresponding to the direction and magnitude of the movement, and transmitting data indicative of the information. The user wearing the electronic device 300 can move their digit 350 relative to the surface within the threshold distance and data indicative of the direction and the magnitude of the relative movement can be transmitted to an information processing device. The transmitted data can be received by an information processing device and a state of an executing application can be updated in response to the movement or gesture performed by the user’s digit relative to the surface.
For example, the user wearing the wearable electronic device 300 may move their digit in a first direction with a first magnitude relative to the surface (such as a table or wall) within the threshold distance. In response to the movement, the information processing device receives the corresponding data and is operable to update the state of the executing application so that a position of a cursor (pointer), on a display displaying the executing application, is updated by moving the cursor in a direction corresponding to the first direction with a magnitude based on the first magnitude.
When the optical sensor 312 of the wearable electronic device 300 detects the movement of the surface relative to the optical sensor 312, the optical sensor 312 is configured to determine at least a first direction and a first magnitude associated with the movement. The state of the executing application can be updated responsive to the received data so that the position of the cursor on the display is moved in a direction the same as the first direction with a second magnitude that may or may not be equal to the first magnitude depending on a scaling factor or mapping used. A one-to-one scaling factor may be used for which the first magnitude detected by the optical sensor is equal to the second magnitude with which the cursor is moved on the display. Alternatively, a scaling factor may be used such that the movement with the first magnitude causes the cursor to be moved with a second magnitude that is greater than the first magnitude (i.e., detecting a small movement of the optical sensor 312 relative to the surface may cause a much larger movement of the cursor on the display).
Alternatively or in addition, a range of scaling values or mappings may be chosen depending either on the user’s preference or the position of the light source 311 and the optical sensor 312 on the mountable surface 313. For example, the information generated by the optical sensor positioned as in Figure 3A may be interpreted, by the information processing device, differently to the information generated by the optical sensor positioned on the opposite side of the user’s digit (above the nail of the digit) so that the sensitivity varies depending on where the light source 311 and the optical sensor 312 are positioned on the mountable surface 313. Hence more generally a first scaling factor is associated with a first position of the optical sensor 312 on the mounting surface 313 and a second scaling factor is associated with a second position of the optical sensor 312 on the mounting surface 313. For the wearable electronic device 300 comprising two or more input devices, the first input device may have an optical sensor positioned on a first portion of the mounting surface 313 and the second input device may have the optical sensor positioned on the second portion of the mounting surface 313 and the scaling factor for the first input device may be different to the scaling factor for the second input device.
In an embodiment of the present invention, the first input device comprises an optical sensor positioned on a mounting surface of a sleeve configured to cover a portion of the extremity of the user’s digit. Figure 4A illustrates an isometric view of a wearable electronic device 400 with an optical sensor 412 positioned on a mounting surface 413 of a sleeve that covers a portion of an extremity of a user’s digit 450, in accordance with an embodiment of the present invention. The sleeve is configured to cover a portion of the distal portion of the user’s digit 450 and also cover a portion of the extremity of the user’s digit 450. In the illustration in Figure 4A, a portion of the extremity of the user’s digit 450 is not covered by the sleeve so that the user may still use the digit 450 to comfortably type on a keyboard or touchscreen or use some other controller device whilst wearing the first input device 410. As such, the user is able to use the wearable electronic device 400 whilst using other handheld devices (such as the wand-style videogame controllers 42 PlayStation Move ®, conventional handheld videogame controllers 43, the keyboard 45 and/or the mouse 46, for example). The user is able to slide the sleeve over their digit 450 like when pulling a glove over the respective digits of a hand.
Referring to Figure 4A, the optical sensor 412 is positioned on a portion of the mountable surface 413 that is proximate to the extremity of the user’s digit 450, and the user can bring this portion of the mountable surface 413 of the sleeve to within a threshold distance of a surface or into contact with the surface by moving the digit 450 upon which the wearable electronic device 400 is worn. The optical sensor 412 is positioned on the portion of the mountable surface 413 of the sleeve, and the sleeve is configured to cover a first portion of the extremity of the user’s digit 450 and expose a second portion of the extremity of the user’s digit 450, and the second portion of the extremity of the user’s digit 450 may be capable of directly contacting a surface without the optical sensor 412 detecting the presence of the surface within the threshold distance of the optical sensor 412. It will be appreciated that the optical sensor 412 can be appropriately positioned anywhere on the mounting surface 413 of the sleeve and Figure 4A illustrates a possible configuration. Alternatively, the optical sensor 412 may be positioned on a portion of the mountable surface 413 on the underside of the user’s digit similar to that shown in Figure 4B, as described later herein.
In an embodiment of the present invention, the first input device comprises an optical sensor positioned on a mounting surface of a cylinder configured to cover a portion of the distal portion of the user’s digit and not cover the extremity of the user’s digit. Figure 4B illustrates an isometric view of a wearable electronic device 400 with an optical sensor 412 positioned on a mounting surface 413 of a cylinder that covers a portion of a distal portion of a user’s digit 450 and does not cover an extremity of the user’s digit 450, in accordance with an embodiment of the present invention. The mountable surface 413 of the cylinder does not cover the extremity of the user’s digit 450, and as such the extremity of the user’s digit 450 may contact a surface, such as a touchscreen or the keyboard 45, without the optical sensor 412 detecting the presence of the surface within the threshold distance of the optical sensor 412. The mountable surface 413 of the cylinder and the position of the optical sensor 412 on the mountable surface 413 is configured so that the extremity of the user’s digit is capable of directly contacting a surface and the wearable electronic device 400 can be used in conjunction with other handheld devices (such as the wandstyle videogame controllers 42 PlayStation Move ®, conventional handheld videogame controllers 43, the keyboard 45 and/or the mouse 46, for example).
In an embodiment of the present invention, the first input device comprises an optical sensor positioned on a mounting surface connected to a whole or partial ring configured to grip around a portion of a circumference of the distal portion of the user’s digit. Figure 4C illustrates an isometric view of a wearable electronic device 400 with an optical sensor 412 positioned on a mounting surface 413 connected to a partial ring 420 that grips around a portion of a circumference of a distal portion of a user’s digit 450, in accordance with an embodiment of the present invention. The partial ring 420 is configured to grip the circumference of the distal portion of the user’s digit 450, and the mounting surface 413 extends in a direction from the partial ring 420 towards the extremity of the user’s digit 450. The optical sensor 412 is positioned on a portion of the mounting surface 413 and the mounting surface 413 is configured so that the optical sensor 412 can be positioned proximate to the extremity of the user’s digit 450. Hence it will be appreciated that the extremity of the user’s digit 450 may contact a surface without the optical sensor 412 detecting the presence of the surface within the threshold distance of the optical sensor 412 and the wearable electronic device can be used in conjunction with other handheld devices or devices which require contact with a portion of a user’s digit for input.
In an embodiment of the present invention, the first input device comprises at least one tactile switch configured to detect whether a surface is in contact with the first input device and generate information indicating whether the surface is in contact with the first input device depending on whether the tactile switch either in a closed state or an open state. Figure 5A illustrates an isometric view of a wearable electronic device 500 with a tactile switch 525 that detects whether a surface is in contact with the first input device 510 of the wearable electronic device 500, in accordance with an embodiment of the present invention. The tactile switch 525 is positioned on a portion of the mountable surface 513 of the first input device 510, and is either in a closed state or an open state (binary configuration) depending on whether or not a surface is in contact with the tactile switch 525. The tactile switch 525 is configured to generate information when the switch is in the closed state and may not generate information when the switch is in the open state. Alternatively, the tactile switch 525 may generate first information when in the closed state and second information when in the open state. Pressure applied to the tactile switch 525 by a contacting surface will close the tactile switch 525, causing a transition from the open state to the closed state such that the tactile switch 525 generates information indicating when the surface is in contact with the tactile switch 525 positioned on the mountable surface 513 of the first input device 510.
It will be appreciated that optionally activation of this tactile switch may be used to supply power to the input device, thereby activating it when the user brings it into contact with a surface. Alternatively, the input device may have a dormant mode where it only maintains wireless contact with the information processing device (for example via a low-power Bluetooth ® connection, which can otherwise take a short but inconvenient time to establish), and pressure on the tactile switch wakes the input device, causing activation of the light source, sensor and processing required to obtain the direction and magnitude data.
With regards to the at least one tactile switch 525, in an embodiment of the present invention, the first input device comprises a first tactile switch positioned on a first portion of the mounting surface proximate to the extremity of the user’s digit and a second tactile switch positioned on a second portion of the mounting surface proximate to the extremity of the user’s digit, wherein at least one of the first tactile switch and the second tactile switch is in the closed state when a portion of the first input device proximate to the distal portion of the user’s digit is in contact with the surface. Figure 5B illustrates an isometric view of a wearable electronic device with a first tactile switch 525 and a second tactile switch 530 positioned on respective portions of a mounting surface 513 proximate to the extremity of the user’s digit 550, in accordance with an embodiment of the present invention. It will be appreciated that the first tactile switch 525 and the second tactile switch 530, as described later herein, may be positioned on the mounting surface 513 of any of the cap, the sleeve, the cylinder or the mounting surface 513 connected to the partial ring, as illustrated in Figures 4A-4C.
The first tactile switch 525 may be positioned on the first portion of the mounting surface 513 and the second tactile switch may be positioned on the second portion of the mounting surface 513. The first tactile switch 525 and the second tactile switch 530 can be positioned so that either the first tactile switch 525 is in the closed state and the second tactile switch 530 is in the open state when the first portion of the mounting surface 513 is in contact with a surface, or the first tactile switch 525 is in the open state and the second tactile switch 530 is in the closed state when the second portion of the mounting surface 513 is in contact with a surface. For example, if the first input device is in contact with a surface, the user may roll their digit onto its left side in order to close the first tactile switch, and the user may roll their digit onto its right side in order to close the second tactile switch. The information generated by the first tactile switch 525 may be different to the information generated by the second tactile switch 530, such that the first tactile switch 525 is associated with a first user input and the second tactile switch 530 is associated with a second user input and either the first or the second user input can be generated depending on the orientation of the user’s digit with respect to the surface (for example, the first tactile switch 525 may be associated with a left click or shooting a gun in a game and the second tactile switch may be associated with a right click or jumping in a game).
Hence more generally, the first tactile switch is configured to generate a first information when in a closed state and the second tactile switch is configured to generate a second information when in a closed state, and one of the first information and the second information is generated depending on the orientation of the user’s digit when the first input device is in contact with a surface. As such, the first input device 510 comprising the first tactile switch 525 and the second tactile switch 530 is configured to generate information indicating whether a surface is in contact with the first input device and also generates information indicating the orientation of the user’s digit with respect to the surface.
Optionally, the first portion of the mounting surface 513 and the second portion of the mounting surface 513 may be positioned so that a portion of the first portion and a portion of the second portion can both simultaneously contact a surface and both the first tactile switch 525 and the second tactile switch 530 may simultaneously be in a closed state depending on the orientation of the user’s digit. For example, the first portion and the second portion may be positioned with a small distance separating them, such that a portion of both the first portion and the second portion contact the surface when the user’s digit brings the first input device into contact with the surface and applies a pressure to a region between the first portion and the second portion. The user may adjust the orientation of the digit with respect to surface in order to either close only the first tactile switch 525 or close only the second tactile switch 530. Consequently, the first information can be generated when the first tactile switch is in a closed state, the second information can be generated when the second tactile switch is in a closed state, and a third information can be generated, instead of or as well as one or both of the first and second information, when the first tactile switch and second tactile switch are simultaneously in a closed state, such that different information can be generated depending on the orientation of the user’s digit when in contact with a surface.
The first input device 510 may comprise a plurality of tactile switches positioned on respective portions of the mounting surface 513 proximate to an extremity of the user’s digit 550. For example, four respective tactile switches may be positioned symmetrically about the optical sensor 512 such that the each tactile switch can generate information when in the closed state and each tactile switch may be associated with a different user input.
In an embodiment of the present invention, the first input device comprises at least one tactile switch positioned on a portion of the mounting surface which is contactable by another digit of the user’s hand, the tactile switch configured to generate information indicating whether the tactile switch is either closed or open. Figure 6A illustrates an isometric view of a wearable electronic device 600 with a tactile switch 635 positioned on a mounting surface 613 of a sleeve which is contactable by another digit of a user’s hand, in accordance with an embodiment of the present invention. The tactile switch 635 illustrated in Figure 6A is positioned on the mounting surface 613 of a sleeve that is configured to cover a portion of the extremity of the user’s digit. It will be appreciated that the tactile switch 635 may be positioned on the mounting surface 613 of any of the cap, the sleeve, the cylinder or the mounting surface 613 connected to the partial ring. Figure 6B illustrates an isometric view of a wearable electronic device 600 with a tactile switch 635 positioned on a mounting surface 613 of a cap which is contactable by another digit of a user’s hand. The tactile switch 635 may be positioned on the mounting surface 613 proximate to the distal portion of the user’s digit 650 so that another digit of the user’s hand can contact the tactile switch 635 so that the tactile switch 635 is either in a closed state or an open state depending on whether the user’s other digit is in contact with the tactile switch 635. The tactile switch 635 is configured to generate information when in a closed state.
For example, the first input device 610 of the wearable electronic device 600 could be worn on the user’s index finger or little finger and the user’s thumb can be used to contact the tactile switch 635 in order to close the tactile switch 635. Alternatively or in addition, the first input device 610 of the wearable electronic device 600 could be worn on the user’s thumb and the other fingers of the user’s hand can be used to contact the tactile switch 635. Consequently, the first input device 610 comprises at least one tactile switch 635 contactable by another digit of the user’s hand and the first input device 610 is capable of generating information without the presence of a surface, and data indicative of the generated information can be transmitted by the transmitter. The user can close the tactile switch 635 using another digit in order to generate information for which data indicative of the information is transmitted to the information processing device 10, and the information processing device is operable to update the state of the executing application in response to the information generated by the closed tactile switch 635. Hence the wearable electronic device 610 comprising at least one tactile switch 635 positioned on a portion of the surface 613 which is contactable by another digit of the user’s hand is capable of generating information without the presence of a surface within a threshold distance, or the tactile switch 635 may generate information in addition to the information generated by the optical sensor when a surface within a threshold distance is detected.
The wearable electronic device 600 may comprise a plurality of input devices, with each input device 610 comprising at least one tactile switch 635, so that the first input device 610 is worn on a first digit, a second input device is worn on a second digit, a third input device is worn on a third digit, a fourth input device is worn on a fourth digit and a fifth input device is worn on a fifth digit, or any suitable combination thereof. For example, the wearable electronic device 600 comprising five input devices (one on each digit) will comprise at least five tactile switches, and the information generated by each tactile switch may correspond to a different user input. The user may use their thumb to close the tactile switch 635 on the first input device worn on the little finger to generate information, and different information may be generated when the user uses their thumb to close the tactile switch 635 on the second input device worn on the index finger.
Alternatively or in addition, when the extremity of one or more of the user’s digits is covered by the mountable surface 613 (e.g. a mounting surface of a cap with a closed end configured to surround an extremity of the user’s digit), the optical sensor positioned on the mounting surface can detect whether a surface is in contact with the input device comprising the optical sensor, and based on the time at which the optical sensor detects a contacting surface and the time at which the tactile switch 635 is closed, it is possible to identify which of the user’s digits is responsible for closing the tactile switch 635 on another digit.
Hence more generally, the first input device 610 comprising at least one tactile switch 635 positioned on a portion of the mountable surface 613 contactable by another digit of the user’s hand is configured to generate information when the tactile switch 635 is in a closed state and the wearable electronic device 600 is configured to determine whether the tactile switch 635 is closed by another digit wearing a second input device based on the information generated by an optical sensor of the second input device.
Hence in an embodiment of the present invention, the at least one tactile switch positioned on a portion of the mounting surface which is contactable by another digit of the user’s hand is configured to generate information indicating whether the tactile switch is either closed or open and the wearable electronic device is configured to detect whether another digit wearing a second input device contacted the tactile switch.
Figure 6C illustrates an isometric view of a wearable electronic device 610 with a plurality of tactile switches 635, 640 positioned on a mounting surface 613 which is contactable by another digit of a user’s hand, in accordance with an embodiment of the present invention. The respective tactile switches 635, 640 can be contacted by another digit of the user’s hand and the information generated when the first tactile switch 635 is closed is different to the information generated when the second tactile switch is closed 640. Data indicative of the generated information can be transmitted by the transmitter and the information processing device adapted to receive the data is operable to update the state of the executing application responsive to whether the first tactile switch 635 is closed or the second tactile switch 640 is closed.
In an embodiment of the present invention, the first input device comprises an electric motor configured to generate a force responsive to the detected presence of a surface in contact with the first input device, the force providing haptic feedback to the user’s digit when the surface is in contact with the first input device. Figure 7 illustrates an electric motor 720 of a first input device 710 for generating a force to provide haptic feedback to a user’s digit when a surface is in contact with the first input device 710. A small electric motor 720 may be positioned under the mounting surface 713 proximate to the distal portion of the user’s digit so as to provide haptic feedback to the user’s digit. Information generated either by the optical sensor or a tactile switch in a closed state may indicate the presence of a surface in contact with the first input device. The information generated by the optical sensor or closed tactile switch can be communicated to the electric motor 720 and the electric motor is configured to generate vibrations in response to the information depending on whether or not a surface is in contact with the first input device. The electric motor may be a coin vibration motor using an eccentric rotating mass (ERM). Rotation of the eccentric rotating mass creates an asymmetric force and when rotating at high speeds a vibration effect is achieved. The enclosed vibration mechanism and small size of the electric motor means that the first input device 710 may optionally comprise a plurality of electric motors 720 positioned under different portions of the mounting surface 713.
Figure 8 is a functional block diagram of hardware to detect an orientation of an input device of a wearable electronic device, in accordance with an embodiment of the present invention. With regards to the first input device, in embodiments of the present invention, the first input device comprises one or more sensors 810 for detecting an orientation of the first input device, the one or more sensors 810 comprising one or more from the list consisting of: an accelerometer 820, a gyroscope 830, and a magnetometer 840. Other sensors may similarly be considered. Generally, the sensor 810 may be any sensor known in the art that is capable of measuring relative changes in a magnetic field or measuring a magnitude and a direction of acceleration in its own rest frame and generating information corresponding to a detected change in the measured property. Data indicative of the information generated by the one or more sensors 810 can be transmitted by the transmitter, and the state of the executing application at the information processing device can be updated in response to changes in the orientation of the user’s digit wearing an input device. Hence more generally the orientation of the first input device and the user’s digit can be detected by the one or more sensors 810, data indicative of the detected orientation can be transmitted to the information processing device and the state of an executing application can be updated in response to changes in the orientation. The information processing device is configured to receive from at least the first input device, data indicative of the detected orientation and is operable to detect a gesture performed by the first input device based on either a change or a time dependent change in the orientation, and the processor of the information processing device is operable to update the state of the executing application responsive to the gesture performed by the first input device positioned on the user’s digit.
Alternatively or in addition, the respective digits of the user’s hand may each wear an input device, each input device comprising one or more sensors 810. Data indicative of the information generated by the one or more sensors 810 can be transmitted for each input device and the orientation of the respective digits with respect to each other can be determined by the information processing device. The information processing device may receive the data corresponding to the information generated for the respective digits and determine the orientation of each digit with respect to each other digit and update the state of the executing application in response to a relative orientation of the one or more digits. Hence more generally the information processing device is adapted to receive first orientation information from a first input device and second orientation information from a second input device and the processor of the information processing device is operable to determine a relative digit orientation and update the state of the executing application in response to the relative digit orientation. For example, when the user wears a wearable electronic device comprising two or more input devices the processor may determine that the user’s hand performs a thumbs up or thumbs down gesture based on the relative orientation of the respective digits.
In an embodiment of the present invention, the first input device comprises at least a first light-emitting-diode for detection by a remote camera such as camera 41. Figure 9 illustrates an isometric view of a first input device 910 of a wearable electronic device 900 comprising a lightemitting-diode (LED) 960 for detection by a camera, in accordance with an embodiment of the present invention. The LED 960 is positioned on the mountable surface 913 of the first input device 910 and is capable of emitting either visible or infrared light depending on the choice of LED. The LED 960 may be positioned on the mountable surface 913 proximate to the top side (same side of the digit as the nail of the digit) of the digit 950 to reduce the likelihood of the LED being obscured from the view of the camera by the user’s hand or digit 950. The light emitted by the LED can be detected by the camera and the position of the LED and thus the position of the first input device 910 and user’s digit 950 can be detected and tracked in successive images captured by the camera. For example, the wearable electronic device 900 may comprise a plurality of respective input devices worn on respective digits of the same hand or different hands, and each input device may have an LED that emits a different colour of visible light, or the LEDs may have different rates of flashing. Alternatively, the LEDs may be IR (infrared) LEDs with different rates of flashing. Hence more generally, the first input device comprises a first LED configured to emit light with a first wavelength and the second input device comprises a second LED configured to emit light with a second wavelength and the position of the first LED and the second LED can be respectively tracked by the camera, and a state of an executing application can be updated responsive to the tracked positions.
With regards to the transmitter, in an embodiment of the present invention, the transmitter is configured to transmit the data indicative of the information generated by the optical sensor according to a wireless communication comprising one or more from the list consisting of: Bluetooth®; infrared wireless; and ultra wideband. Figure 10 is a functional block diagram of hardware 1000 for wireless communication of data indicative of information generated by an optical sensor of a wearable electronic device, in accordance with an embodiment of the present invention. The transmitter may comprise hardware configured to transmit data according to a wireless communication comprising one or more from the list consisting of: Bluetooth® 1010; infrared wireless 1020; and ultra wideband 1030. Other personal area network (PAN) wireless technologies may similarly be considered such as ZigBee or Body area network.
The Bluetooth® 1010 wireless communication uses a low power technology standard that is capable of transmitting data over distances of approximately 10 metres using shortwavelength radio waves from 2.4 GHz to 2.485 GHz. Alternatively or in addition, the transmitter may wirelessly communicate data using infrared wireless 1020 signals to establish a wireless communication link with another device in line-of-sight. Alternatively or in addition, the transmitter may use ultra wideband 1030 wireless communication, which uses direct sequence ultra wide band and multiband orthogonal frequency division multiplexing to transmit information at time intervals.
It will be appreciated that the transmitter is configured to transmit the data indicative of the information generated by the optical sensor as well the information generated by any of the tactile switches 525, 530, 635, 640, and the one or more sensors for detecting orientation 810.
Finally it will be appreciated that whilst the above description discloses that the input device generates for transmission data such as direction and magnitude data, or first, second and/or third information, or position / orientation data. Optionally for any one or more of the generated data items disclosed herein, raw signal data or partially processed data may be transmitted instead, for additional processing by the information processing unit 10 (e.g. the PlayStation 4 ®). This may consequently reduce the cost, size and/or weight of the input device, and/or increase battery life and/or enable use of a brighter light source for a similar battery life, for example.
It will be apparent to those skilled in the art that various combinations of the embodiments of the present invention as described and claimed herein are considered within the scope of the present invention.
In an embodiment of the present invention, a method of detecting a presence of a surface within a threshold distance of a user’s digit and determining a relative movement of the surface is provided. Figure 11 is a flow diagram illustrating a method of detecting a presence of an external surface within a threshold distance of a user’s digit, in accordance with an embodiment of the present invention comprising:
a first step SI 100 of emitting light from a light source;
a second step SI 110 of detecting a presence of a surface within a threshold distance based on light sensed by an optical sensor positioned on a surface proximate to a distal portion of the user’s digit (where, as noted previously herein, the detection step may be active in terms of detecting the distance with respect to a threshold, and/or may be passive, in terms of having light emitter and sensor positioned so as to work when within a threshold distance of the surface);
a third step SI 120 of determining a direction and a magnitude of a movement of the optical sensor relative to the surface based on changes in the sensed light;
a fourth step SI 130 of generating information corresponding to the determined direction and magnitude of the relative movement; and a fifth step SI 140 of transmitting data indicative of the information corresponding to the determined direction and magnitude of the relative movement.
It will be apparent to a person skilled in the art that variations in the above method corresponding to operation of the various embodiments of the system as described and claimed herein are considered within the scope of the present invention.
It will be appreciated that the above methods may be carried out on conventional hardware (such as that described previously herein) suitably adapted as applicable by software instruction or by the inclusion or substitution of dedicated hardware.
Thus the required adaptation to existing parts of a conventional equivalent device may be implemented in the form of a computer program product comprising processor implementable instructions stored on a non-transitory machine-readable medium such as a floppy disk, optical disk, hard disk, PROM, RAM, flash memory or any combination of these or other storage media, or realised in hardware as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device. Separately, such a computer program may be transmitted via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these or other networks.

Claims (15)

1. A wearable electronic device, comprising:
at least a first input device configured to be positioned on a portion of a user’s digit and to detect a presence of a surface within a threshold distance, comprising:
a light source configured to emit light;
an optical sensor configured to detect a presence of a surface within a threshold distance of the optical sensor based on reflection of the emitted light, to determine a direction and a magnitude of a movement of the optical sensor relative to the surface based on changes in the reflected light, and to generate corresponding direction and magnitude information, wherein the optical sensor is positioned proximate to a distal portion of the user’s digit; and a transmitter configured to transmit data indicative of the information generated by the optical sensor.
2. A wearable electronic device according to claim 1, wherein the optical sensor is positioned on one or more from the list consisting of:
i) a mounting surface of a cap with a closed end configured to surround an extremity of the user’s digit;
ii) a mounting surface of a sleeve configured to cover a portion of the extremity of the user’s digit;
iii) a mounting surface of a cylinder configured to cover a portion of the distal portion of the user’s digit and not cover the extremity of the user’s digit; and iv) a mounting surface connected to a ring configured to grip around a portion of a circumference of the distal portion of the user’s digit.
3. A wearable electronic device according to claim 1 or claim 2, wherein the optical sensor is a CMOS sensor configured to detect either visible light or infrared light.
4. A wearable electronic device according to any one of the preceding claims, wherein the optical sensor is configured to detect the presence of a surface within a threshold distance, determine whether the surface is in contact with the first input device, and generate information indicating whether the surface is in contact with the first input device.
5. A wearable electronic device according to any one of the preceding claims, wherein the first input device comprises at least one tactile switch configured to detect whether a surface is in contact with the first input device and generate information indicating whether the surface is in contact with the first input device depending on whether the tactile switch is either in a closed state or an open state.
6. A wearable electronic device according to claim 5, comprising:
a first tactile switch positioned on a first portion of a mounting surface proximate to the extremity of the user’s digit; and a second tactile switch positioned on a second portion of the mounting surface proximate to the extremity of the user’s digit, wherein one of the first tactile switch and the second tactile switch is in the closed state when a portion of the first input device proximate to the distal portion of the user’s digit is in contact with the surface.
7. A wearable electronic device according to any one of the preceding claims, wherein the first input device comprises at least one tactile switch positioned on a portion of a mounting surface which is contactable by another digit of the user’s hand, the tactile switch configured to generate information indicating whether the tactile switch is either closed or open.
8. A wearable electronic device according to claim 7, wherein the at least one tactile switch is configured to generate information indicating whether the tactile switch is either closed or open and the wearable electronic device is configured to detect whether another digit wearing a second input device contacted the tactile switch.
9. A wearable electronic device according to any one of claims 4 to 8, wherein the first input device comprises an electric motor configured to generate a force responsive to the detected presence of a surface in contact with the first input device, the force providing haptic feedback to the user’s digit when the surface is in contact with the first input device.
10. A wearable electronic device according to any one of the preceding claims, wherein the first input device comprises one or more from the list consisting of:
i) an accelerometer;
ii) a gyroscope; and iii) a magnetometer, configured to detect an orientation of the first input device.
11. A wearable electronic device according to any one of the preceding claims, wherein the wearable electronic device comprises at least a first light-emitting-diode for detection by a remote camera.
12. A wearable electronic device according to any one of the preceding claims, wherein the transmitter is configured to transmit the data indicative of the information generated by the optical sensor according to a wireless communication comprising one or more from the list consisting of:
i) Bluetooth;
ii) infrared wireless;
iii) ultra wideband; and iv) induction wireless.
13. An information processing device, comprising:
a receiver adapted to receive from at least a first input device positioned on a portion of a user’s digit, for a surface within a threshold distance of the first input device, data indicative of a direction and a magnitude of a movement of an optical sensor of the first input device relative to the surface; and a processor operable to update a state of an executing application responsive to the received data.
14. A method for a wearable electronic device of detecting a presence of a surface within a threshold distance of a user’s digit and determining a relative movement of the surface, comprising:
emitting light from a light source of the wearable electronic device;
detecting a presence of a surface within a threshold distance of an optical sensor of the wearable electronic device based on light sensed by the optical sensor, positioned on a surface proximate to a distal portion of the user’s digit;
determining a direction and a magnitude of a movement of the optical sensor relative to the surface based on changes in the sensed light;
generating information corresponding to the determined direction and magnitude of the relative movement; and transmitting data indicative of the information corresponding to the determined direction and magnitude of the relative movement.
15. A computer readable medium having computer executable instructions adapted to cause a 5 computer system to perform the method of claim 14.
GB1712760.6A 2017-08-09 2017-08-09 Input method and apparatus Withdrawn GB2565534A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB1712760.6A GB2565534A (en) 2017-08-09 2017-08-09 Input method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB1712760.6A GB2565534A (en) 2017-08-09 2017-08-09 Input method and apparatus

Publications (2)

Publication Number Publication Date
GB201712760D0 GB201712760D0 (en) 2017-09-20
GB2565534A true GB2565534A (en) 2019-02-20

Family

ID=59894749

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1712760.6A Withdrawn GB2565534A (en) 2017-08-09 2017-08-09 Input method and apparatus

Country Status (1)

Country Link
GB (1) GB2565534A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006031743A1 (en) * 2006-07-10 2008-01-24 Kastriot Merlaku Computer input device, particularly computer mouse, has optical or infrared-detection or sensing-system, which takes-over mouse-scroll-function, and infrared-detection or sensing-system is coupled with scroll-function
KR20130081543A (en) * 2012-01-09 2013-07-17 유범수 Memo input device, memo display method using the same in terminal and memo system
US20140098067A1 (en) * 2012-10-02 2014-04-10 Autodesk, Inc. Always-available input through finger instrumentation
US20170205879A1 (en) * 2016-01-18 2017-07-20 Anoop Molly JOSEPH Multipurpose computer mouse

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006031743A1 (en) * 2006-07-10 2008-01-24 Kastriot Merlaku Computer input device, particularly computer mouse, has optical or infrared-detection or sensing-system, which takes-over mouse-scroll-function, and infrared-detection or sensing-system is coupled with scroll-function
KR20130081543A (en) * 2012-01-09 2013-07-17 유범수 Memo input device, memo display method using the same in terminal and memo system
US20140098067A1 (en) * 2012-10-02 2014-04-10 Autodesk, Inc. Always-available input through finger instrumentation
US20170205879A1 (en) * 2016-01-18 2017-07-20 Anoop Molly JOSEPH Multipurpose computer mouse

Also Published As

Publication number Publication date
GB201712760D0 (en) 2017-09-20

Similar Documents

Publication Publication Date Title
US20240171478A1 (en) Electronic Device with Intuitive Control Interface
US11231786B1 (en) Methods and apparatus for using the human body as an input device
EP2733574B1 (en) Controlling a graphical user interface
JP6669069B2 (en) Detection device, detection method, control device, and control method
US9729608B2 (en) Information processing device, table, display control method, program, portable terminal, and information processing system
JP5969626B2 (en) System and method for enhanced gesture-based dialogue
KR101524575B1 (en) Wearable device
US11229836B2 (en) Input method and apparatus
US20230004231A1 (en) Sensors for Electronic Finger Devices
US20060125789A1 (en) Contactless input device
TWI596378B (en) Portable virtual reality system
US20170087455A1 (en) Filtering controller input mode
US20110095983A1 (en) Optical input device and image system
CN104808938B (en) Bimodulus optical navigator and its mode switching method
KR20210006278A (en) Electronic device for recognizing user input using sensor and method thereof
US10276036B2 (en) Electronic system and control method for the same
US9310903B2 (en) Displacement detection device with no hovering function and computer system including the same
TW202119175A (en) Human computer interaction system and human computer interaction method
WO2018058673A1 (en) 3d display method and user terminal
US9652052B2 (en) Optical mini-mouse
GB2565534A (en) Input method and apparatus
KR20240022503A (en) Wearable ring device and user interface processing
US20110069007A1 (en) Pointing device
US20100207885A1 (en) Optical input device and operating method thereof, and image system
KR20200002413A (en) Band-type optical wireless mouse

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)