JP6161078B2 - Detection of user input at the edge of the display area - Google Patents

Description

  The present disclosure relates to computing devices.

  Mobile (mobile) computing devices are being developed to increase the functionality that is made available to users in the mobile environment. For example, a user can interact with a cell phone, tablet computer, or other mobile computing device to check email, surf the web, write text, or interact with an application . Traditional mobile computing devices often employ displays with touch screen capabilities to allow users to enter various data or requests into the computing device. However, with such traditional mobile computing devices, it is difficult to recognize certain user inputs, which can give the user a frustrating and difficult to use experience.

  Disclosed is a technique (display area edge user input sensing technique) for detecting user input at a display area edge.

  In one or more implementations, input data relating to user input is received. This input data includes both data relating to at least part of user input within the active display area of the device and data relating to at least part of user input within an area outside the active display area of the device. This user input is determined based on the received input data.

  In one or more implementations, the computing device includes a housing configured in a hand-held form factor and a display device supported by the housing. The display device includes an active display area and one or more sensors, the one or more sensors based at least in part on the proximity of the object to the active display area and outside the active display area. Is arranged to detect user input based at least in part on the proximity of the object to the region.

  This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed matter, but is also intended to be used as an aid in determining the scope of the claimed matter. Not a thing.

The detailed description is described with reference to the accompanying drawings. In the drawings, the leftmost digit of a reference number identifies the figure in which the reference number first appears. The use of the same reference symbols in the description and in the several different examples in the drawings indicates the same or the same item. An entity represented in the figure may refer to one or more entities, and thus may be referred to interchangeably in the description as singular or plural.
FIG. 6 illustrates an environment in one implementation that is operable to use the techniques described herein. FIG. 6 illustrates an environment in another implementation that is operable to use the techniques described herein. FIG. 3 is a diagram depicting one implementation of the input device of FIG. 2 to show the flexible hinge in more detail. It is the figure which drew one example of mounting which shows the perspective view of the connection part of FIG. 3 containing the protrusion for mechanical coupling | bonding and a some communication contact. It is a figure which shows an example of the display apparatus which mounted the display area user input sensing technique. It is a figure which shows sectional drawing of an example of the display apparatus which mounted the display area user input sensing technique. It is a figure which shows sectional drawing of the other example of the display apparatus which mounted the display area user input sensing technique. FIG. 6 illustrates a system in an implementation that is operable to use the techniques described herein. It is a figure which shows the example of a display apparatus of FIG. 5 with an example of user input. It is a figure which shows the example of a display apparatus of FIG. 5 with another example of user input. It is a figure which shows the example of a display apparatus of FIG. 5 with another example of user input. It is a figure which shows the example of a display apparatus of FIG. 5 with another example of user input. 6 is a flowchart illustrating an example process for using the techniques described herein in accordance with one or more embodiments. Of a system that includes various components of an example of a device that can be implemented as any type of computing device described with reference to FIGS. 1-13 to use the embodiments of the technology described herein. It is a figure which shows an example.

Outline A technique (display area edge user input sensing technique) for detecting user input at a display area edge is described. One or more sensors are arranged to detect user input in the active display area and to detect user input in an extended area outside the active display area. Data relating to user input, such as gestures, may include data from user input detected either in the active display area or outside thereof. Thus, user input can begin and / or end outside the active display area.

  In the following description, first, an example environment in which the technique described herein can be used will be described. Thereafter, an example of a procedure that can be executed not only in the environment example but also in other environments will be described. Therefore, the execution of the procedure example is not limited to the environment example, and the environment example is not limited to the execution of the procedure example.

Example Environment and Procedure FIG. 1 illustrates an environment 100 in one implementation that is operable to use the techniques described herein. The example environment 100 includes a computing device 102 according to an example. The computing device 102 can be variously configured. For example, the computing device 102 may be configured for mobile, such as a mobile phone and a tablet computer. However, the techniques described herein are applicable to many types of devices other than mobile devices, and can be used with any of a variety of different devices that use input sensors above or in the display area. May be. For example, the computing device 102 may be a desktop computer, a storefront kiosk terminal, an interactive display or monitor (eg, in a hospital, airport, mall, etc.). The computing device 102 can range from a full resource device with sufficient memory and processor resources to limited memory and / or processing resources to low resource devices. The computing device 102 may also relate to software that causes the computing device 102 to perform one or more processes.

  The computing device 102 is illustrated as including an input / output module 108 by way of example. The input / output module 108 represents functions related to input processing and output rendering of the computing device 102. A variety of different inputs can be processed by the input / output module 108, for example, relating to functions corresponding to keys of an input device coupled to the computing device 102 or keys of a virtual keyboard displayed by the display device 110. An input or a gesture recognized through the touch screen function of the display device 110 and an input for executing a process corresponding to the gesture may be processed. Thus, because the display device has the ability to receive user input via any of a variety of input sensing technologies, the display device 110 is also referred to as an interactive display device. The input / output module 108 may support a variety of different input technologies by recognizing and utilizing the distinction between multiple types of inputs, including key presses and gestures.

  FIG. 2 illustrates an environment 200 in another implementation that is operable to use the techniques described herein. The exemplary environment 200 includes an example computing device 202 that is physically and communicatively coupled to an input device 204 via a flexible hinge 206. The computing device 202 can be variously configured, similar to the computing device 102 of FIG. The computing device 202 may also relate to software that causes the computing device 202 to perform one or more processes.

  The computing device 202 is illustrated as including an input / output module 208 by way of example. The input / output module 208 represents functions related to input processing and output rendering of the computing device 202. A variety of different inputs can be processed by the input / output module 208, such as inputs relating to functions corresponding to keys on the input device 204 or virtual keyboard displayed by the display device 210, A gesture recognized through the touch screen function, and an input for executing a process corresponding to the gesture may be processed. Thus, because the display device has the ability to receive user input via any of a variety of input sensing technologies, the display device 210 is also referred to as an interactive display device. The input / output module 208 may support a variety of different input technologies by recognizing and utilizing the distinction between multiple types of inputs, including key presses and gestures.

  In the illustrated example, the input device 204 is configured as a keyboard having a QWERTY key, but other keys are also contemplated. Other non-conventional configurations are also contemplated, such as configurations that mimic game controllers and musical instruments, for example. Thus, the input device 204 and the keys incorporated therein may assume a variety of different configurations that support a variety of different functions.

  As previously noted, the input device 204 is physically and communicatively coupled to the computing device 202, in this example through the use of a flexible hinge 206. Flexible hinge 206 is a flexure of the material that forms the hinge, as opposed to mechanical rotation (where such embodiments are also contemplated) where the rotational motion supported by the hinge is supported by the pin. It is flexible in that it is achieved by (for example, bending). This flexible rotation also supports movement in one direction (eg, the vertical direction in the figure), but other movements, such as the lateral movement of the input device 204 relative to the computing device 202, for example. Can be configured to limit. This can be used to assist in consistent alignment of the input device 204 with respect to the computing device 202, such as, for example, to align sensors used to change power states, application states, and the like.

  The flexible hinge 206 can be formed using, for example, one or more layers of fabric, and is conductively formed as a flexible wiring that communicatively couples the input device 204 to the computing device 202 and vice versa. May contain body. This communication may include, for example, communicating the result of a key press to the computing device 202, receiving power from the computing device 202, performing authentication, providing auxiliary power to the computing device 202, etc. Can be used for. The flexible hinge 206 can be variously configured and further description thereof will be given in conjunction with subsequent figures.

  FIG. 3 shows an implementation 300 of the input device 204 of FIG. 2 to show the flexible hinge 206 in more detail. In this example, the input device connection 302 is shown configured to provide a communication and physical connection between the input device 204 and the computing device 202. In this example, the connection 302 has a height and cross section configured to be received in a groove in the housing of the computing device 202, but this configuration departs from its spirit and scope. It may be reversed.

  The connecting portion 302 is flexibly connected to a part of the input device 204 including a key through the use of the flexible hinge 206. Therefore, when the connection unit 302 is physically connected to the computing device, the combination of the connection unit 302 and the flexible hinge 206 is an operation of the input device 204 with respect to the computing device 202, which is similar to the back sticking (hinge) of a book. To help.

  For example, rotational movement can be assisted by the flexible hinge 206 so that the input device 204 can be placed against the display device 210 of the computing device 202 and thereby act as a cover. The input device 204 is also placed against the back of the computing device 202 (eg, against the rear housing of the computing device 202 located on the opposite side of the computing device 202 from the display device 210). It may be rotated as follows.

  Naturally, various other positions are also supported. For example, the computing device 202 and the input device 204 can be arranged such that both are placed flat against the surface shown in FIG. In other examples, the input device 204 is placed flat against the surface so that typing can be supported, and the computing device 202 uses, for example, a stand placed on the back of the computing device 202, etc. Thus, the display device 210 is arranged at an angle that allows viewing. Other examples are also contemplated, such as a tripod arrangement, a meeting arrangement, and a presentation arrangement.

  Connection 302 is shown in this example as including magnetic coupling devices 304, 306, mechanical coupling protrusions 308, 310, and a plurality of communication contacts 312. The magnetic coupling devices 304, 306 are configured to magnetically couple with a complementary magnetic coupling device of the computing device 202 through the use of one or more magnets. Thus, the input device 204 can be physically attached to the computing device 202 using magnetic force.

  The connection 302 also includes mechanical coupling protrusions 308 and 310 for forming a mechanical physical connection between the input device 204 and the computing device 202. The mechanical coupling protrusions 308, 310 are shown in more detail in the following figure.

  FIG. 4 shows a mounting example 400 showing a perspective view of the connecting portion 302 of FIG. 3 including the mechanical coupling protrusions 308, 310 and a plurality of communication contacts 312. As shown, the mechanical coupling protrusions 308, 310 are configured to extend away from the surface of the connection 302 and are vertical in this example, but other angles are contemplated.

  The mechanical coupling protrusions 308, 310 are configured to be received in complementary cavities (holes) in the grooves of the computing device 202. When so received, the mechanical coupling protrusions 308, 310 are mechanical between these devices when a force is applied that is not aligned with an axis defined to correspond to the height of the protrusions and the depth of the cavity. Promote social ties.

  For example, when a force is applied that coincides with the previously described longitudinal axis according to the height of the protrusion and the depth of the cavity, the user overcomes the force provided by the magnet alone and separates the input device 204 from the computing device 202. . However, at other angles, the mechanical coupling protrusions 308, 310 are mechanically coupled within the cavity so that the force to withstand removal of the input device 204 from the computing device 202 is the magnetic force of the magnetic coupling devices 304, 306. In addition to being created. Thus, the mechanical coupling protrusions 308, 310 can bias the removal of the input device 204 from the computing device 202 to mimic a page from the book and attempt to isolate the device. Other attempts may be limited.

  Connection 302 is also shown as including a plurality of communication contacts 312. The plurality of communication contacts 312 are configured to contact corresponding communication contacts of the computing device 202 to form a communication coupling between the devices. Communication contact 312 may be configured in a variety of ways, for example, by forming with a plurality of spring-loaded pins configured to provide a consistent communication contact between input device 204 and computing device 202. obtain. Thus, the communication contacts can be configured to remain intact during small movements such as device collisions. Various other examples are also contemplated, including placing pins on the computing device 202 and placing contacts on the input device 204.

  Display area edge user input sensing technology uses one or more sensors located in the extended sensor area to detect user input outside the active display area. One or more sensors are also arranged to detect user input within the active display area. The extended sensor area is in the vicinity of the active display area (for example, within 5 millimeters) and is typically adjacent to the active display area.

  FIG. 5 shows an example of a display device 500 that implements the display area user input sensing technology. Display device 500 is an interactive display device that includes an active display area 502 in which various data and information can be displayed by a computing device. Display area 502 is referred to as the active display area because displayed data and information can be changed over time, possibly in response to user input received by the computing device. The display device 500 also includes an extended sensor region 504 illustrated in a parallel line pattern surrounding and adjacent to the active display region 502. For example, when an object such as a finger, stylus, and pen of a user's hand is touching and / or close to the surface of the active display area 502 and / or the extended sensor area 504 Can receive. The extended sensor area 504 helps to detect user input along the edge of the active display area 502. The edge of the active display area 502 means the outer edge of the active display area 502 that is the part of the active display area 502 that is closest to the extended sensor area 504.

  The extended sensor area 504 can, for example, extend 2 millimeters outside the active display area 502, although other extensions are contemplated. The extended sensor area 504 may extend the same amount outside the active display area 502 around the entire active display area 502, or may instead extend by a different amount. For example, the extended sensor area 504 may extend outside the active display area 502 by 2 millimeters in the vertical direction and 4 millimeters in the horizontal direction. The extended sensor area 504 may also be varied with different types of devices and may be customized for a particular type of device. For example, an interactive device that can receive input from a greater distance (eg, an interactive display and a storefront kiosk terminal that can detect input even as far as 10 centimeters) is a device that receives input from a closer interaction ( For example, it may have an extended sensor area that extends beyond the display area farther (eg, 10-15 centimeters rather than 2-4 millimeters) than a tablet that detects touch.

  A display device that implements display area edge user input sensing technology can use various active display technologies. These active display technologies include, for example, flexible display technology, e-book reader display technology, liquid crystal display (LCD) technology, light emitting diode (LED) display technology, organic light emitting diode (OLED) display technology, and plasma display technology. obtain. Although examples of display technologies are described herein, other display technologies are also contemplated.

  A display device that implements display area edge user input sensing technology can use a variety of different input sensing technologies. These input sensing techniques may include capacitive (capacitive) and / or resistive (resistive) systems that detect touch. These input sensing techniques may also include inductive systems that sense pen (or other object) input. These input sensing techniques also reflect or disrupt light from objects touching (or near) the surface of a display device, such as sensor-in-pixel (SIP) systems, infrared systems, and optical imaging systems. A light-based system for detecting Other types of input sensing technologies can also be used, such as surface acoustic wave systems, acoustic pulse recognition systems, and distributed signal systems. While example input sensing techniques are described herein, other input sensing techniques are also contemplated. Also, these input sensing techniques may be combined together to implement other tactile inputs, such as a piezoelectric sensor with an extended capacitive sensor, for example.

  Depending on the input sensing technology used in the display device, user input may occur when an object (eg, a user's hand finger, stylus, and pen) touches the surface of the display area and / or the surface of the display device. Can be received when in proximity. This proximity can be, for example, 5 millimeters, but various proximities are contemplated and can vary depending on the manner in which the display is implemented. The proximity of an object to the display device refers to the distance of the object from the display device along a direction perpendicular to the plane of the display device.

  FIG. 6 shows a cross-sectional view of a display device 600 according to an example in which the display area user input sensing technology is implemented. Display device 600 includes an active display layer 602 and an input sensing layer 604 disposed thereon. Although these layers 602 and 604 are illustrated as individual layers, each of the layers 602 and 604 may be composed of a plurality of layers. The input sensing layer 604 and the active display layer 602 can be implemented using a variety of different technologies, as described above. Note that although not shown in FIG. 6, any number of additional layers may be included in display device 600. For example, a further protective layer made of glass or plastic can be disposed on the input sensing layer 604.

  A user's finger 606 (or other object) in contact with or close to the input sensing layer 604 is detected by the input sensing layer 604. The position at which the user's finger 606 (or other object) is detected by the layer 604 is provided by the layer 604 as the detected object position and is used to identify user input, as described in more detail below.

  The input sensing layer 604 includes a plurality of sensors and extends to the extended sensor regions 608 and 610 outside the active display region 602. The number of sensors and how they are arranged can vary based on the implementation and the input sensing technology used for the input sensing layer 604. Input sensing layer 604 includes a portion 612 and portions 614 and 616.

  Within the portion 612, one or more sensors may be disposed on the input sensing layer 604 above the active display layer 602. These sensors located above layer 602 detect that a user's finger 606 (or other object) is in contact with or close to layer 604 above the active display layer 602, and thus Also referred to as detecting user input within and / or on the active display area and being located within the active display area.

  One or more sensors may also be disposed in the input sensing layer 604 above the extended sensor regions 608 and 610, respectively, within the portions 614 and 616, respectively. The extended sensor areas 608, 610 are not on the active display layer 602, as illustrated in FIG. These sensors located above the extended sensor areas 608, 610 detect that the user's finger 606 (or other object) is in contact with or close to the layer 604 above the extended sensor areas 608, 610. And therefore also referred to as sensing user input in and / or on the extended sensor area 608,610. Since the extended sensor areas 608, 610 are not on the active display layer 602, these sensors located above the extended sensor areas 608, 610 are also sensitive to user input in areas outside the active display area. , And also as being located in an area outside the active display area.

  In other examples, the sensor may be disposed in the input sensing layer 604 in other ways, for example, along the outer edge (periphery) of the input sensing layer 604 or at a corner of the input sensing layer 604. Such sensors can still sense user input within and / or above the active display area and user input within areas outside the active display area.

  FIG. 7 shows a cross-sectional view of a display device 700 according to another example in which the display area user input sensing technology is implemented. The display device 700 includes an active display layer 702 and an input sensing layer 704 disposed thereon. The input sensing layer 704 and the active display layer 702 can be implemented using a variety of different technologies, as described above. These layers 702 and 704 are disposed between the lower panel layer 706 and the upper panel layer 708. The panel layers 706 and 708 can be made of various materials such as glass and plastic. Although the layers 702, 704, 706, and 708 are illustrated as separate layers, each of the layers 702, 704, 706, and 708 may be composed of a plurality of layers. These layers may also be flexible layers and may be applied to three-dimensional (3D) interactive devices.

  Optionally, additional support material 714, 716 is included between the panel layers 706 and 708 as shown in the parallel line pattern in FIG. Support material 714, 716 provides additional support for regions between panel layers where layers 702 and 704 do not extend. The support material 714, 716 can be a variety of materials such as glass, plastic, and bonding adhesives.

  A user's finger 606 (or other object) in contact with or close to the input sensing layer 704 is detected by the input sensing layer 704. The position at which the user's finger 606 (or other object) is detected by layer 704 is provided by layer 704 as the detected object position and is used to identify user input, as will be described in more detail below.

  The input sensing layer 704 includes a plurality of sensors and extends to the extended sensor regions 710 and 712 outside the active display region 702. However, the input sensing layer 704 need not extend as far as the panel layers 706, 708 as shown. The number of sensors included in the input sensing layer 704 and how the sensors are arranged can vary based on the implementation and the input sensing technology used for the input sensing layer 704. Input sensing layer 704 includes a portion 718 and portions 720 and 722.

  Within the portion 718, one or more sensors are disposed on the input sensing layer 704 above the active display layer 702. These sensors, located above layer 702, detect that a user's finger 606 (or other object) is in contact with or in proximity to panel layer 708 above active display layer 702, and therefore Also referred to as detecting user input in and / or on the active display area and being located within the active display area.

  One or more sensors are also disposed in the input sensing layer 704 above the extended sensor regions 710 and 712 within the portions 720 and 722, respectively. The extended sensor areas 710, 712 are not on the active display layer 702, as illustrated in FIG. These sensors located above the extended sensor areas 710, 712 detect that the user's finger 606 (or other object) is in contact with or close to the panel layer 708 above the extended sensor areas 710, 712. Therefore, it is also referred to as sensing user input in and / or on the extended sensor area 710, 712. Since the extended sensor areas 710, 712 are not on the active display layer 702, these sensors located above the extended sensor areas 710, 712 also detect user input in areas outside the active display area. , And also as being located in an area outside the active display area.

  In other examples, the sensors may be disposed in the input sensing layer 704 in other ways, for example, along the outer edge (periphery) of the input sensing layer 704 or at a corner of the input sensing layer 704. Such sensors can still sense user input within and / or above the active display area and user input within areas outside the active display area.

  Although the input sensing layer of FIGS. 6 and 7 is shown as being disposed above the active display layer, other configurations are also contemplated. For example, the input sensing layer may be inside or below the active display layer. The input sensing layer may also have multiple configurations. The input sensing layer may be on both sides of the plastic and / or glass substrate, or on the same side of the plastic layer, glass layer and / or other optically transparent layers.

  FIG. 8 illustrates a system 800 in one implementation operable to use the techniques described herein. System 800 includes an input data collection module 802 and an input handler module 804. System 800 may be implemented, for example, in computing device 102 of FIG. 1 or computing device 202 of FIG. Note that although modules 802 and 804 are shown in system 800, one or more additional modules may be included in system 800. Further, the function of the module 802 and / or the function of the module 804 can be separated into a plurality of modules.

  The input data collection module 802 receives the detected object position indication 806. These detected object position indications 806 indicate the position of an object (for example, a user's finger or pen) detected by the input sensing layer of the display device. If desired, timing information associated with a position detected by the input sensing layer can also be included as part of the detected object position indication 806. This timing information indicates when a particular position has been detected and can take a variety of forms. For example, this timing information may be for an unchanging time frame or clock, or may be the amount of time since the preceding position was detected. In other examples, the timing information may be generated by the input data collection module 802 based on the timing of reception of the sensed object position indication 806.

  Input data collection module 802 generates input data 808 using sensed object position indication 806. Input data 808 describes the position and movement of user input. Input data 808 may be sensed object position indication 806 and any relevant timing information regarding those positions received and / or generated by module 802.

  Further, the user input can have an associated lifetime, which starts when an object is detected touching (or close to) the surface so that the object is no longer a display device. Represents the time period that ends when it is no longer detected as being in contact with (or in close proximity to) the surface. This associated lifetime may be identified by the input data collection module 802 and included as part of the input data 808.

  User input can also have an associated speed, which represents the speed at which the object being sensed is moving. This speed is a specific distance divided by a specific amount of time, such as inches per second or millimeters per millisecond. This associated speed may be specified by the input data collection module 802 and included as part of the input data 808, or used in other ways (eg, as described in more detail below, input data To determine when to provide 808 to the input handler module 804).

  The input data collection module 802 provides input data 808 to the input handler module 804, which determines what the input is. User input can take various forms, such as gestures or mouse movements. A gesture represents an action or path taken by an object (eg, a user's finger) to activate one or more functions of the computing device. For example, the gesture may be to slide the user's finger in a specific direction, or to draw a specific character or symbol (eg, a circle, the letter “Z”, etc.) with the user's finger. Gestures may also include multi-touch input in which multiple objects (eg, multiple fingers of a user) take a specific action or path to activate one or more functions of the computing device. Mouse movement represents an action or path taken by an object (eg, a user's finger) to move something (eg, a cursor or pointer, an object being dragged and dropped, etc.) on the display device. While gestures and mouse movement are described herein, various other types of user input are also contemplated.

  The input handler module 804 may determine what the user input is based on the input data 808 using any of a variety of public and / or proprietary technologies. For example, the input handler module 804 can determine that the user input is a specific gesture, a specific mouse movement, or the like. The input handler 804 may also analyze input characteristics (eg, input magnitude and / or input speed) to set up a display or other output for a customized user experience. For example, small inputs with small fingers can be processed to adjust fonts, colors, applications, etc. suitable for children.

  The input handler module 804 may also take various actions based on the determined user input. For example, the input handler module 804 may provide one or more other modules of the computing device that indicate the determined user input to perform the requested function or operation. As another example, the input handler module 804 itself may perform the requested function or operation.

  Input data collection module 802 may provide input data 808 to input handler module 804 at various times. For example, input data collection module 802 may provide input data 808 to input handler module 804 when input data 808 is generated. As another example, the input data collection module 802 is detected after a user input is completed (eg, the lifetime for that user input has passed and the object is no longer in contact with (or in proximity to) the surface of the display device. The input data 808 may be provided to the input handler module 804 after it has run out.

  In other examples, the input data collection module 802 may retain the input data 808 regarding a user input without providing the input data 808 to the input handler module 804 until a certain event occurs. Good. A variety of different events can cause module 802 to provide input data 808 to module 804. One event that may cause module 802 to provide input data 808 to module 804 is that the user input pointed to by the position of the object is in the active display area. Thus, in response to the user input being within the active display area, module 802 provides input data 808 to module 804.

  Another event that may cause module 802 to provide input data 808 to module 804 is that the user input is outside the active display, but in the future (eg, during the relevant lifetime of the user input) It is expected to be inside. User input can be predicted to be in the active display area in the future based on various rules or criteria, eg, based on the speed of the user input and / or the orientation of the user input. For example, when the user input is outside the active display area and the direction of the user input faces the active display area, the user input is predicted to be in the active display area in the future. As another example, when the user input is outside the active display area, the direction of the user input is facing the active display area, and the speed of the user input is greater than the threshold amount, the user input will be active in the future. Expected to be in region. This threshold amount may be, for example, 4 inches per second, although other threshold amounts are contemplated. Thus, in response to the user input expected to be in the active display area in the future, module 802 provides input data 808 to module 804.

  FIG. 9 shows the example display device 500 of FIG. 5 along with an example of user input. As described above, the display device 500 includes the active display region 502 surrounded by the extended sensor region 504 (illustrated using a parallel line pattern). User input is received via the user's finger 606.

  The user input in FIG. 9 is shown as a right-to-left operation, which starts in the extended sensor area 504 and moves into the active display area 502. The end position of the user's finger is shown using a dashed hand. Detection of user input begins in the extended sensor area 504 before the user's finger 606 moves into the active display area 502. The user input indicated by the movement of the user's finger 606 in FIG. 9 can be identified more quickly than when the extended sensor area 504 is not included in the display device 500. The user input can be identified more quickly because in the absence of the extended sensor area 504, the position of the user's finger 606 does not begin to be detected until the user's finger 606 reaches the edge of the active display area 502. is there.

  The user input in FIG. 9 is shown as starting in the extended sensor area. It should be noted, however, that user input may begin outside both the active display area 502 and the extended sensor area 504 (eg, along the edge of the display device 500). User input can still be identified more quickly than if extended sensor region 504 was not included in display device 500. This is because when the user's finger 606 reaches the extended sensor area 504 (rather than waiting for the user's finger 606 to reach the active display area 502), motion will begin to be detected.

  FIG. 10 shows the example display device 500 of FIG. 5 along with another example of user input. As described above, the display device 500 includes the active display region 502 surrounded by the extended sensor region 504 (illustrated using a parallel line pattern). User input is received via the user's finger 606.

  The user input in FIG. 10 is shown as a left-to-right operation, which starts in the active display area 502 and ends in the extended sensor area 504. The end position of the user's finger is shown using a dashed hand. In another example, the end position of the operation may be outside both the active display area 502 and the extended sensor area 504 (for example, along the edge of the display device 500). Detection of user input begins in the active display area before the user's finger 606 moves into the extended sensor area 504. By identifying the user input by ending the movement of the user's finger 606 within the extended sensor area 504 (or allowing the user's finger movement past the extended sensor area 504), The position of the user input at can be used. For example, the input handler module 804 of FIG. 8 causes the user input to cross the display device as opposed to the input intended by the user to remain on a particular icon or object displayed near the edge of the display area. It can be determined to be a left-to-right gesture or swipe.

  FIG. 11 shows the example display device 500 of FIG. 5 along with another example of user input. As described above, the display device 500 includes the active display region 502 surrounded by the extended sensor region 504 (illustrated using a parallel line pattern). User input is received via the user's finger 606.

  The user input in FIG. 11 is shown as a right-to-left and top-to-bottom operation with a “<” shape. The user input in FIG. 11 starts and ends in the active display area 502, but passes through the extended sensor area 504. The end position of the user's finger is shown using a dashed hand. Sensing user input within the extended sensor area 504 allows the user input shown in FIG. 11 to be entered along the edge of the active display area 502. Even if the user input passes outside the edge of the active display area 502, the user input is detected in the extended sensor area 504.

  FIG. 12 shows the example display device 500 of FIG. 5 along with another example of user input. As described above, the display device 500 includes the active display region 502 surrounded by the extended sensor region 504 (illustrated using a parallel line pattern). User input is received via the user's finger 606.

  The user input in FIG. 12 is shown as a left-to-right operation, which starts and ends in the extended sensor area 504 without moving into the active display area 502. The end position of the user's finger is shown using a dashed hand. Detection of user input begins within the extended sensor area 504. However, since the user's finger 606 is not moved into the active display area 502 and the moving direction of the user's finger 606 is not directed toward the active display area 502, the input data relating to this user input is the input of FIG. There is no need to be provided to the handler module 804. Thus, when the user input remains in the extended sensor area 504, no action based on the user input need be taken.

  FIG. 13 is a flowchart illustrating an example process 1300 using the techniques described herein in accordance with one or more embodiments. Process 1300 may be performed by a computing device, such as computing device 102 of FIG. 1 or computing device 202 of FIG. 2, and may be implemented in software, firmware, hardware, or a combination thereof. . Process 1300 is shown as a set of steps and is not limited to performing the steps of the various steps in the order shown. Process 1300 is an example process using the techniques described herein, and further description of using the techniques described herein is included herein with reference to the various figures.

  In process 1300, input data is received (step 1302). The input data includes data relating to at least part of user input within the active display area of the device and data relating to at least part of user input within an area outside the active display area of the device, as described above. .

  Based on the input data, user input is determined (step 1304). Any of a variety of published and / or proprietary technologies can be used to determine what the user input is, as described above.

  The action indicated by the user input is performed (step 1306). This action may be the execution of various functions or operations, as described above.

Example System and Device FIG. 14 is an example of a system that includes an example computer device 1402 that represents one or more computing systems and / or devices that may implement the various techniques described herein (generally denoted by reference numerals). 1400). The computing device 1402 can be configured assuming a mobile configuration using, for example, a housing sized to be held and carried by one or both hands of the user, examples of which are Mobile phones, mobile gaming and music devices, and tablet computers, although other examples and configurations are also contemplated.

  The computing device 1402 according to the illustrated example includes a processing system 1404, one or more computer-readable media 1406, and one or more I / O interfaces 1408 that are communicatively coupled to each other. Although not shown, computing device 1402 may further include a system bus or other data command transmission system that couples the various components together. The system bus uses any of a variety of bus architectures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or any of various bus structures such as a processor or local bus. One or a combination can be included. Various other examples are also contemplated, such as control and data lines.

  The processing system 1404 represents a function for executing one or more processes using hardware. Accordingly, the processing system 1404 is illustrated as including hardware elements 1410 that may be configured as processors, functional blocks, and the like. This may include hardware implementation as an application specific integrated circuit or other logic device formed using one or more semiconductors. The hardware element 1410 is not limited by the material from which it is formed or the processing mechanism employed therein. For example, the processor may be comprised of semiconductors and / or transistors (eg, electronic integrated circuits (ICs)). In such a case, the processor executable instructions may be electronically executable instructions.

  Computer readable storage media 1406 is illustrated as including memory / storage 1412. Memory / storage 1412 represents the memory / storage capacity associated with one or more computer-readable media. Memory / storage component 1412 may include volatile media (eg, random access memory (RAM), etc.) and / or non-volatile media (eg, read only memory (ROM), flash memory, optical disk, magnetic disk, etc.). . The memory / storage component 1412 may include fixed media (eg, RAM, ROM, fixed hard drive, etc.), and removable media (eg, flash memory, removable hard drive, optical disc, etc.). The computer readable medium 1406 may be configured in various other ways as further described below.

  The input / output interface 1408 allows the user to enter commands and information into the computing device 1402 using various input / output devices and presents information to the user and / or other components or devices. Represents a feature that allows to be done. Examples of input devices include a keyboard, a cursor control device (eg, a mouse), a microphone, a scanner, a touch function (eg, a capacitive sensor or other sensor configured to detect a physical touch), and a camera (eg, , Visible wavelengths or non-visible wavelengths such as infrared frequencies may be used to recognize movement as a gesture without touch). Examples of output devices include display devices (eg, monitors or projectors), speakers, printers, network cards, tactile response devices, and the like. Thus, the computing device 1402 can be configured in a variety of ways to support user interaction.

  Computing device 1402 is further illustrated as including one or more modules 1418 that can be configured to support a variety of functions. These one or more modules 1418 are configured to generate input data based on, for example, indication of the sensed object position, determine what the user input is based on the input data, and the like. Can be done. Module 1418 may include, for example, input data collection module 802 and / or input handler module 804 of FIG.

  Here, various techniques may be described in the general context of software, hardware elements, or program modules. In general, such modules include routines, programs, objects, elements, components, data structures, etc. that perform particular tasks or use particular abstract data types. The terms “module”, “function”, and “component” generally refer to software, firmware, hardware, or a combination thereof. The mechanism of the technology described here is platform independent, that is, these technologies can be implemented on a variety of commercial computing platforms with a variety of processors.

  Implementations of the described modules and techniques may be stored on or transmitted between some form of computer readable media. Computer readable media can include a variety of media that can be accessed by computing device 1402. By way of example, and not limitation, computer readable media may include “computer readable storage media” and “computer readable signal media”.

  “Computer-readable storage media” may refer to media and / or devices that enable permanent and / or non-transitory storage of information, not just signal transmission, carrier waves, or signals themselves. Thus, a computer readable storage medium represents something that is not a signal carrier medium. Computer-readable storage media are suitable, for example, for storing volatile and non-volatile removable and non-removable media and / or information such as computer-readable instructions, data structures, program modules, logic elements / circuits or other data Hardware such as a storage device implemented by a method or technique. Examples of computer readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, hard disk, magnetic cassette , Magnetic tape, magnetic disk storage or other magnetic storage device, or other storage device, tangible medium or article of manufacture suitable for storing the desired information and accessible by computer.

  “Computer readable signal medium” may refer to a signal carrier medium configured to transmit instructions to the hardware of computing device 1402, for example, over a network. A signal medium typically may embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave, data signal, or other transport mechanism. The signal medium also includes any information delivery medium. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

  As previously noted, the hardware element 1410 and the computer readable medium 1406 are used in some embodiments to perform at least some aspects of the techniques described herein, eg, executing one or more instructions. Represents a module, programmable logic device, and / or fixed logic device implemented in a hardware form that can be implemented. Hardware includes integrated circuits or on-chip system components, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other It may include a silicon implementation or other hardware. In this situation, the hardware stores instructions for execution, such as a treatment device that performs the program tasks defined by the instructions and / or logic embodied by the hardware, and the computer-readable storage medium described above, for example. It can operate as hardware used for

  Combinations of the above can also be used to implement various techniques described herein. Thus, software, hardware, or executable modules may be on one form of computer-readable storage media and / or as one or more instructions and / or logic embodied by one or more hardware elements 1410. Can be implemented. The computing device 1402 may be configured to implement specific instructions and / or functions corresponding to software and / or hardware modules. Accordingly, the implementation of modules executable as software by computing device 1402 may be achieved at least partially in hardware, such as by use of computer-readable storage media and / or hardware elements 1410 of processing system 1404, for example. . The instructions and / or functions may be performed / operated by one or more articles of manufacture (eg, one or more computing devices 1402 and / or processing system 1404) to implement the techniques, modules, and examples described herein. It can be possible.

Summary Although implementations have been described in terms specific to structural mechanisms and / or method actions, as will be appreciated, the matters defined in the appended claims are not necessarily limited to the specific mechanisms or actions described. Is not to be done. Rather, these specific features and acts are disclosed as example forms of implementing the claimed features.

Claims (6)

Receiving input data relating to a user input, the input data representing a position of an object in both an active display area of the interactive display device and an area outside the active display area of the interactive display device; The active display area of the interactive display device described has a touch screen display on which data or information is displayed, and the area outside the active display area is outside the active display area where no display pixels are present Extending, receiving, and
Determining the user input based on the input data, the user input having a gesture indicating a function to be activated by a computing device including the interactive display device, the gesture being determined by the object; A path to be taken, said path being at least partly within said area outside said active display area, said input data being
A plurality of positions of the object in the active display area prior to the path passing through the area outside the active display area;
A plurality of positions of the object in the area outside the active display area when the path passes through the area outside the active display area;
Describing and determining a plurality of positions of the object within the active display area following the path passing through the area outside the active display area;
Having a method.   The method of claim 1, wherein the receiving comprises receiving the input data in response to the user input being within the active display area. A computing device comprising:
A housing with a hand-held form factor;
An active display area that is a display device supported by the housing and is a touch screen display on which data or information is displayed, and an outside of the active display area that extends outside the active display area where no display layer is present A display device having a region of
The display device has one or more sensors arranged to detect the position of an object in both the active display region and the region outside the active display region, the computing device Further comprises an input handler configured to determine a gesture based on the sensed position, the gesture indicating a function of the computing device to be performed and a path taken by the object. The path is at least partially within the area outside the active display area, and the sensed position is prior to the path passing through the area outside the active display area, a plurality of positions of the object along the path of the active display area, wherein the path of the outside of the active display area When passing through the pass, and a plurality of positions of the object along the path of the outside of the area of the active display area, the route continues to pass through the region outside the active display area, the A plurality of positions of the object along the path in an active display area ,
Computing device.   The computing device of claim 3, wherein at least one of the one or more sensors is disposed within an extended sensor area surrounding the active display area.   4. The computing device of claim 3, wherein the one or more sensors are included in an input sensor layer of the display device that extends to the outside of the display layer of the display device.   4. The computing device of claim 3, wherein the area outside the active display area has an extended sensor area adjacent to and surrounding the active display area.

Images