TWI516989B - Three-dimensional user interface device - Google Patents

Description

Three-dimensional user interface device

The present invention relates to a three-dimensional user interface device.

Users interact with consumer electronic devices (CEDs) in a variety of different ways. A user can use a telecontrol device to interact with a CED that is not connected to the CED but is in communication with the CED. For example, the remote control device can be a remote control, a wireless mouse, a wireless keyboard, or a game controller. The user can interact with a device that is part of the CED or a device that is connected to the CED (eg, a keyboard, a touchpad). The CED can include a touch sensitive display such that the user can interact with the CED by touching the display. Many user interfaces are two-dimensional, which limits the types of interactions that can be made with the CED.

The invention provides an apparatus comprising: a first a body supporting a display and a first array of proximity sensors; and a second body extending from an edge of the first body, wherein the second body supports a second array of proximity sensors The proximity sensor of the first array and the proximity sensor of the second array are used to define a three-dimensional (3D) user interface for the device.

100‧‧‧ Laptop

110‧‧‧ cover

150‧‧‧Base

120‧‧‧ display

160‧‧‧ keyboard

170‧‧‧ Trackpad

200‧‧‧ Laptop

210‧‧‧ proximity sensor

220‧‧‧ display

230‧‧‧User Interface (Keyboard)

340‧‧ ‧ coverage area

410‧‧‧Defined area

410A‧‧‧Defined area

410B‧‧‧Defined area

410C‧‧‧Defined area

410D‧‧‧Defined area

500‧‧‧ Laptop

510‧‧‧ display

520‧‧‧ proximity sensor

530‧‧‧Logical Analyzer

The features and advantages of the various embodiments will be apparent from the following detailed description, wherein: FIGS. 1A-C illustrate various views of an exemplary laptop; FIG. 2 illustrates an exemplary lap in accordance with an embodiment. a computer for use in a cover (display) and a proximity sensor in the base (user interface); FIG. 3 illustrates an application for the laptop generated by the proximity sensors in accordance with an embodiment An exemplary footprint of a computer; FIG. 4 illustrates the generation of an exemplary defined area (eg, a box) within a coverage area in accordance with an embodiment; and FIG. 5 illustrates an embodiment for providing a An exemplary system diagram for a 3D user interface for a laptop.

1A-C illustrate various views of an exemplary laptop 100. The computer 100 includes an upper frame (lid) 110 and a lower frame (base) 150 that pass through a hinge or the like (they are not marked Code) and pivotally connected to each other. The computer 100 can be changed between an open configuration in which the cover 110 extends from the base 150 in an upward direction and a closed configuration in which the cover 110 lies flat on the base 150. The cover 110 can include a display 120 at which content can be viewed by a user. The base 150 can include one or more user interfaces that interact with the computer 100. The user interfaces can include, for example, a keyboard 160 and a touch pad 170. When the computer 100 is operational, it may be in the open configuration (see Figures 1A and 1C) and when the computer is turned off and/or shipped, it may be in the closed configuration (see Figure 1B). ).

When the computer 100 is operational, the user interacts with the computer 100 through the user interfaces. The keyboard 160 allows the user to enter data and/or select certain parameters, for example, using a button. The touchpad 170 allows a user to scroll around the display 120, for example by moving their finger on the touchpad (eg, the detected finger motion can be mapped to screen motion). Move to view and/or select certain content. Whether using keyboard 160, trackpad 170, or other user interface device (eg, a mouse), the interaction is limited to two-dimensional (2D) interaction (the user interaction is limited to the plane of display 120) . For example, the user can move left, right, up, down, or a combination of these directions (eg, diagonally).

The display 120 is configured to provide an additional user interface option similar to a touch screen display for use with a touch screen display on a tablet. However, this device still only provides a 2D user interface (a device that the user can only interact with the display in the plane of the display).

The proximity sensor can detect if an object is at a specific distance from it. For example, a proximity sensor used on a computer can detect if a person is within a normal operating distance (eg, 3 inches) of it. However, the proximity sensor cannot determine the exact distance between the human and the computer (eg, 1 inch to 3 inches). The proximity sensors can be, for example, inductive sensors, capacitive sensors, magnetic sensors, photoelectric sensors, other forms of sensors, or some combination thereof. The photoelectric sensor can include a light source (eg, infrared light) and a receiver to determine whether the light is reflected back.

A proximity sensor can be used in the display 120 to detect the user's use without touching the display 120 or using the user interface device (eg, keyboard 160, trackpad 170). The position and/or movement of a particular portion of the user, such as a hand or finger, relative to the display 120. However, in order to select certain content or take certain actions, the user may need to touch the display 120 and/or use the user interface devices (eg, keyboard 160, trackpad 170). That is, interfacing with this device is still limited to 2D interaction.

2 illustrates an exemplary laptop 200 that uses proximity sensor 210 within a cover (display) 220 and a base (user interface) 230. The proximity sensors 210 can be, for example, inductive sensors, capacitive sensors, magnetic sensors, photoelectric sensors, other forms of sensors, or some combination thereof. The proximity sensors 210 are illustrated as being viewable for ease of illustration. But the proximity sensors 210 can be The user does not see and/or the user does not notice. The proximity sensors 210 do not affect the content presented on the display 220 and do not affect the user using the user interface (keyboard) 330.

The proximity sensors 210 can be organized into an array comprising a sensor column and a sensor row 210 extending across the length and height of the display 220 and the length and depth of the user interface (keyboard) 230 . The rows of sensors 210 on the display 220 can be aligned with the rows of sensors 210 on the keyboard 230. Utilizing the plane of the two separate sensors 210 allows the computer 200 to detect not only the user (or a particular portion of the user, such as a hand or finger) or the position and/or movement of the device relative to the display 220, It is also capable of detecting the distance from the display.

FIG. 3 illustrates an exemplary footprint 340 for the laptop 200. The proximity sensors 210 on the display 220 are capable of detecting objects that are substantially equal or slightly larger than the distance that the keyboard 230 extends from the display 220. The proximity sensors 210 on the keyboard 230 are capable of detecting objects that are at a distance equal to or slightly greater than the distance that the display 220 extends from the keyboard 230. The footprint 340 can be an area where the coverage of the sensor 210 on the display 220 overlaps with the coverage of the sensor 210 on the keyboard 230. That is, the footprint 340 can extend upwardly as high as the display 220, extending outwardly as far as the keyboard 230, and encompassing the length of the display 220 and keyboard 230.

FIG. 4 illustrates the generation of an exemplary defined area (eg, box) 410 within the footprint 340. Such for a display (not shown) A proximity sensor (not shown) can be oriented on the x-axis (length of the display) and the z-axis (height of the display). The proximity sensors for a keyboard (not shown) can be oriented on the x-axis (the length of the keyboard) and the y-axis (the depth of the keyboard). The sensors on the x-axis for the display and keyboard can be aligned with one another. Each defined area may be a surrounding area centered at a point of intersection of a proximity sensor of each axis. The size of the region can be determined based on the number of such sensors and the proximity of the sensors to each other. As shown, each axis has three defined regions (e.g., x = 1-3, y = 1-3, and z = 1-3) that show three sensors associated with each axis. Thus, the display and the keyboard each can have nine (3 x 3) sensors and their associated. There are a total of 27 (3 × 3 × 3) defined areas.

For example, a defined region 410A can be defined to be located at x=1 (the first aligned row), y=2 (the second column on the keyboard), and z=2 (the second column on the display) The intersection of the proximity sensor is centered around the area. A defined region 410B may include a proximity sensor positioned at x=3 (the third aligned row), y=1 (the first column on the keyboard), and z=3 (the third column on the display) . A defined area 410C may include proximity sensors positioned at x=2 (second aligned line), y=2 (second column on the keyboard), and z=3 (third column on the display) . A defined region 410D may include a proximity sensor positioned at x=3 (the third aligned row), y=3 (the third column on the keyboard), and z=1 (the first column on the display) .

If there are three proximitys associated with the defined area The detector indicates that an object (eg, a finger, a stylus) is present, and the computer can determine that an object is within the defined area. The defined regions may be associated with content that appears on a corresponding area of the display. According to an embodiment, proximity sensors on the x-axis and z-axis can be used to define an area on the display. For example, an icon for a particular program can be associated with a defined area. If a finger, hand or other device is determined to be within the defined area, the image can be illuminated and if the user wants to select the image, he can implement a display toward the display Move to select the image.

According to an embodiment, the proximity sensors on the three axes can be used to assign items on the display to the defined regions. For example, if the display presents a 3D desktop, then many items on the desktop can be associated with the different defined areas. For example, an item in the upper right corner of the display can be associated with a proximity sensor on row x=3 and column z=3. An item closest to the user in the upper right corner of the 3D desktop (eg, the first image of three overlapping images) may be associated with column y=3, an item that is intermediate from the user (eg, The second image of the three overlapping images can be associated with column y=2, and an item that is furthest away from the user (eg, the third image of three overlapping images) can be Column y=1 is associated. The image may be illuminated if a finger, hand or other device is determined to be present in the defined area. If the user wants to select the image, he can implement a defined motion (e.g., a gesture toward the display) to select the image.

In addition to selecting images from the desktop, the proximity sensors can be used to track motion associated with the display, similar to that used for touch screens and/or touchpads, but without the need for actual and Device interaction. For example, if a user wants to flip a page on a book on the display he is watching, he can wave his finger from right to left to advance one page or swipe his finger from left to right to go back one page. The action taken in accordance with the movement of the user's finger (hand, device or the like) depends on the mode of operation of the computer and any application that can operate on the computer.

Hardware and/or software processing can be used to analyze data from the proximity sensors to determine the position and motion of the device (e.g., fingers, hands, sticks) relative to the display. The proximity sensors can provide data to a processor for analyzing the data for detecting and/or identifying motion and/or objects within the coverage area 340. As mentioned above, the action taken based on the detection/identification depends on the current state of operation of the computer.

According to an embodiment, the proximity sensor can be used to make the computer function as a 3D scanner. Processing may be performed in accordance with the defined regions that are determined to contain the object to obtain a concept of the size and shape of the object. It should be noted that the resulting 3D image will be limited to the surface of the object facing the display and the keyboard.

According to an embodiment, the generation of a 3D image can be used for security measures of authentication and/or access. For example, when a user attempts to log into the computer, the computer can utilize the proximity sensors to, for example, generate a 3D image of the user's hand or face. The The generated 3D image can be compared to a 3D image that has been authenticated to determine if the access should be granted. For example, a particular movement of a user's finger through the coverage area may also be used as a safeguard for authenticity identification and/or access. For example, to identify a user, the motion of the detected user's finger can be compared to the stored motion. The actual action may be, for example, a swipe of a finger from the upper left front portion to the lower right rear portion of the coverage area.

It will be appreciated by those skilled in the art that the more proximity sensors are used, the finer the granularity of the 3D user interface. Thus, at least a certain number of sensors may be required for each region in the use of 3D scanning and/or user-identified proximity sensors.

FIG. 5 illustrates an exemplary system diagram for a laptop 500 that provides a 3D user interface. The computer 500 can include a display 510, a plurality of proximity sensors 520, and a logic 530. The display 510 is used to present information. The proximity sensors 520 are used to detect the presence of objects associated with it (eg, the user's finger, the user's hand, the digital pen). The logic analyzer 530 is for processing inputs from the proximity sensors. The logic analyzer 530 can be a hardware and/or software logic analyzer. The logic analyzer 530 can be one or more processors that are used within the computer 500.

The logic analyzer 530 can be configured to define a plurality of coverage areas corresponding to intersections of the proximity sensors. The logic analyzer 530 can be configured to indicate the presence of an object within a coverage area in response to detection of a proximity sensor associated with the coverage area. The logic analyzer 530 It can be constructed to allow a user to identify an item on the display 510 for selection by placing an object within a corresponding coverage area. The logic analyzer 530 can be constructed to highlight the identified item for selection. The logic analyzer 530 can be configured to select the item to echo the motion of the object toward the display. The logic analyzer 530 can be configured to identify an action to be taken on the display in response to detection of motion of an object through one or more coverage areas in the coverage areas. The logic analyzer 530 can be constructed to generate a 3D image of an object to detect the detection of the object within one or more coverage areas. The logic analyzer 530 can be configured to perform authenticity identification using the 3D image.

The 3D user interface has been described with particular reference to a laptop, but is not limited thereto. Conversely, the 3D interface can be used with any device that includes a display and a keyboard extending from the display, such as a desktop computer, some tablets with a mating keyboard (for example, sold by Microsoft®). Surface ^TM ), some wireless phones, and/or some personal digital assistants (PDAs) and the like. Furthermore, the second surface is not limited to the keyboard. Conversely, the other surface can be any type of user interface or device that extends from a display. For example, the second surface can be another display or can be a cover for the device that extends outwardly when the cover is opened. For example, a 3D interface can be provided to a tablet computer by utilizing a cover for the device. The proximity sensors within the cover may need to provide information to the computer, so some type of communication interface may be required.

Although the disclosure has been made with reference to a particular embodiment It is to be understood that the disclosure is not limited to the embodiments, and various changes and modifications may be made to the embodiments without departing from the scope. "One embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described herein is included in at least one embodiment. Therefore, the terms "a" or "an" or "an"

Various embodiments are intended to provide the most general protection of the spirit and scope of the appended claims.

200‧‧‧ Laptop

210‧‧‧ proximity sensor

220‧‧‧ display

230‧‧‧User Interface (Keyboard)

Claims (29)

An apparatus comprising: a first body supporting a display and a first array of proximity sensors; a second body extending from an edge of the first body, wherein the second body supports a a proximity sensor of the second array, wherein the proximity sensor of the first array and the proximity sensor of the second array are used to define a three-dimensional (3D) user interface for the device; and logic An analyzer, at least a portion of which is a hardware, wherein the logic analyzer is configured to define a 3D region between the first body and the second body for the 3D user interface as at least one corresponding to the The 3D coverage area of the intersection of the proximity sensor. The device of claim 1, wherein the logic analyzer is configured to indicate that an object is present in a 3D coverage area of the at least one 3D coverage area in response to a proximity sensor associated with the 3D coverage area Detection. The device of claim 2, wherein the logic analyzer is configured to allow a user to identify an item on the display for selection by placing an object within a corresponding 3D coverage area . The device of claim 3, wherein the object is a user's finger. The device of claim 3, wherein the object is a user's hand. For example, the device of claim 3, wherein the object is a Indicator device. A device as claimed in claim 3, wherein the logic analyzer is constructed to highlight the designated item for selection. The device of claim 7, wherein the logic analyzer is configured to select the item to reflect the movement of the object toward the display. The device of claim 2, wherein the logic analyzer is configured to identify an action to be taken on the display in response to movement of an object through one or more 3D coverage areas of the at least one 3D coverage area Detection. The device of claim 1, wherein the proximity sensor of the first array comprises a plurality of sensors configured in columns and rows; the proximity sensor of the second array comprises a plurality of configured columns And a line of sensors. The device of claim 10, wherein the proximity sensor arrays in the first array are aligned relative to the proximity sensor columns in the second array. The device of claim 10, wherein the proximity sensor rows in the first array are aligned relative to the proximity sensor rows in the second array. The device of claim 1, wherein the second body is a user interface device. Such as the device of claim 13 of the patent scope, wherein the user The interface is a keyboard. The apparatus of claim 1, wherein the second body extends from a lower edge of the first body. The device of claim 1, wherein the second body extends from a side edge of the first body. A device as claimed in claim 2, wherein the logic analyzer is constructed to generate a 3D image of an object to reflect the detection of the object within one or more 3D coverage areas. The device of claim 17, wherein the logic analyzer is configured to perform authenticity identification using the 3D image. The device of claim 17, wherein the object is a user's hand. The device of claim 17, wherein the object is a user's head. An apparatus comprising: a display, wherein the display is for supporting a first array of proximity sensors; a user interface device extending from an edge of the display, wherein the user interface device is a proximity sensor for supporting a second array; and a logic analyzer, at least a portion of which is a hardware, configured to at least partially approximate the proximity of the proximity sensor and the second array from the first array The input of the detector detects the three-dimensional (3D) user interaction with the display, Wherein the logic analyzer is configured to define a 3D area between the display and the user interface device as at least one 3D coverage area corresponding to an intersection of the proximity sensors and detect at least 3D user interaction within a 3D coverage area. The device of claim 21, wherein the logic analyzer is configured to indicate the presence of an object within a 3D coverage area of the at least one 3D coverage area in response to the proximity sense associated with the 3D coverage area Detector detection. The device of claim 21, wherein the logic analyzer is configured to identify a purpose selection on the display in response to an object being placed in a corresponding 3D coverage area. The device of claim 23, wherein the object comprises at least one selected from the group consisting of a user's finger, a user's hand, and a list of pointing devices. The apparatus of claim 23, wherein the logic analyzer is constructed to highlight the designated item for selection. The device of claim 22, wherein the logic analyzer is configured to identify the selection of the item to reflect the movement of the object toward the display. The device of claim 21, wherein the logic analyzer is configured to identify an action to be taken on the display in response to movement of an object through one or more 3D coverage areas of the at least one 3D coverage area Detection. Such as the device of claim 21, wherein the logical division The analyzer is constructed to generate a 3D image of an object to reflect the detection of the object within one or more 3D coverage areas. The device of claim 28, wherein the logic analyzer is configured to utilize the 3D image to identify the authenticity of a user.