WO2018028003A1 - Procédé et système d'entrée de type en matrice destinés à être utilisés dans un espace tridimensionnel - Google Patents

Procédé et système d'entrée de type en matrice destinés à être utilisés dans un espace tridimensionnel Download PDF

Info

Publication number
WO2018028003A1
WO2018028003A1 PCT/CN2016/095986 CN2016095986W WO2018028003A1 WO 2018028003 A1 WO2018028003 A1 WO 2018028003A1 CN 2016095986 W CN2016095986 W CN 2016095986W WO 2018028003 A1 WO2018028003 A1 WO 2018028003A1
Authority
WO
WIPO (PCT)
Prior art keywords
input
user
dimensional
dimensional space
array
Prior art date
Application number
PCT/CN2016/095986
Other languages
English (en)
Chinese (zh)
Inventor
李小波
朱郁丛
杨小冬
Original Assignee
北京英梅吉科技有限公司
李小波
朱郁丛
杨小冬
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京英梅吉科技有限公司, 李小波, 朱郁丛, 杨小冬 filed Critical 北京英梅吉科技有限公司
Publication of WO2018028003A1 publication Critical patent/WO2018028003A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233—Character input methods
    • G06F3/0236—Character input methods using selection techniques to select from displayed items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Definitions

  • the present invention relates to the field of virtual three-dimensional space processing technologies, and in particular, to an array input method and system for use in a three-dimensional space.
  • the input mode is mostly in a planar or two-dimensional manner.
  • these operations are feasible for light operations or non-large input operations, but when users need to input a long list of passwords, texts, etc., It seems very inefficient, and it can't make full use of the depth information in 3D space, and the security of information is not improved.
  • the object of the present invention is to provide an array input method and system for use in a three-dimensional space, which can realize the distribution of numbers, letters or symbols in three-dimensional space on different depth layers, thereby greatly reducing each layer.
  • the number of elements solves the problem of inconvenient input in three-dimensional space, making the input of the user in three-dimensional space more organized and efficient.
  • an embodiment of an aspect of the present invention provides an array input method for use in a three-dimensional space, comprising the following steps:
  • Step S1 Receive an input request of a user, where the input request includes: a type of an element to be input;
  • Step S2 in response to the input request, and setting an alternate input element to display a layout in a three-dimensional space according to the type of the element to be input, wherein the display layout includes a number of layers of the candidate input element in three-dimensional space And arranging, each of the candidate input elements having unique three-dimensional coordinates in the three-dimensional space;
  • Step S3 displaying the candidate input elements in a three-dimensional array in a three-dimensional space in front of the user;
  • Step S4 using a three-dimensional spatial positioning technology to detect a user's input means medium, and real-time positioning the user with the input hand A moving position of the segment in the three-dimensional array, detecting three-dimensional coordinates at each position, and then using an alternate input element corresponding to each of the three-dimensional coordinates as an input element of the user until the user's confirmation completion operation is detected;
  • step S5 the input element of the user is transmitted to a subsequent operating system to complete an input operation.
  • the type of the input element includes at least one or a combination of the following: numbers, letters, symbols, and operation control keys.
  • step S2 setting the candidate input element to display the layout in the three-dimensional space according to the type of the element to be input includes:
  • the types of the input elements include: numbers, letters, symbols, and operation control keys, the operation control keys, numbers, letters, and symbols are sequentially arranged in a manner that is close to and far from the user, each type
  • the input elements are arranged in one or more layers;
  • the type of the input element includes: a number and an operation control key, and the user requests to input a password, sequentially arranging the operation control key and the number in a manner that is close to and far from the user, wherein the The operation control keys are distributed in the same layer, the numbers are arranged in multiple layers, and the same numbers are repeatedly arranged in different layers;
  • the type of the input element includes: a letter and an operation control key
  • the operation control key and the letter are sequentially arranged in a manner that is close to and far from the user, wherein the operation control keys are distributed on the same layer.
  • the letters are arranged in a plurality of layers, and the same letters are repeatedly arranged in different layers.
  • the selection of the display layout of the candidate input element in three-dimensional space includes the following two methods:
  • the three-dimensional spatial positioning technology includes at least one of the following: a time-of-flight TOF positioning technology, a binocular camera positioning technology, an ultrasonic positioning technology, an optical positioning technology, a kinect sensing positioning technology, and a dynamic controller Ps.
  • Move positioning technology lighthouse positioning technology, laser radar positioning technology, ultra-wideband UWB positioning technology, synchronous positioning and mapping SLAM technology, Virtuix omni positioning technology, structured light positioning technology.
  • the user input means medium includes: a hand input, an input by an external control device, and input by using a keyboard and a mouse.
  • the three-dimensional spatial positioning technology is used to acquire the coordinates of the x-y-axis plane of the current input position of the user in the three-dimensional array parallel to the user's perspective;
  • the three-dimensional spatial positioning technology is used to detect the movement of the user in the three-dimensional array by input means, and acquire coordinates on the z-axis depth information of the user perpendicular to the user's perspective, thereby obtaining the user's The three-dimensional coordinates of the current input position;
  • the candidate input element corresponding to the three-dimensional coordinates is used as an input element of the user;
  • each input element is located, the input element is highlighted to prompt the user, and after receiving the confirmation operation of the user, the input element is transmitted to the subsequent operating system.
  • an array input system for use in a three-dimensional space, comprising: a communication module, configured to receive an input request of a user, wherein the input request includes: a type of an element to be input; three-dimensional array generation And a display module, the three-dimensional array generation and display module is connected to the communication module, configured to respond to the input request, and set an alternate input element to display a layout in a three-dimensional space according to the type of the element to be input.
  • the alternative input elements are displayed in a three-dimensional array in a three-dimensional array in front of the user, wherein the display layout comprises a number of layers and an arrangement of the candidate input elements in three-dimensional space, each of the candidate input elements
  • the three-dimensional space has a unique three-dimensional coordinate
  • the three-dimensional spatial positioning module is connected to the three-dimensional array generation and display module, and is configured to detect a user input medium by using a three-dimensional spatial positioning technology, and real-time Positioning the user to detect the position of movement in the three-dimensional array by the input means, detecting each position 3D coordinates, the alternate input element corresponding to each of the three-dimensional coordinates is used as an input element of the user until the confirmation of the user is completed
  • the communication module is further configured to transmit the input element of the user to a subsequent operating system , complete the input operation.
  • the type of the input element includes at least one or a combination of the following: numbers, letters, symbols, and operation control keys,
  • the three-dimensional array generation and display module sets a display layout of the candidate input elements in three-dimensional space, including the following forms:
  • the types of the input elements include: numbers, letters, symbols, and operation control keys, the operation control keys, numbers, letters, and symbols are sequentially arranged in a manner that is close to and far from the user, each type
  • the input elements are arranged in one or more layers;
  • the type of the input element includes: a number and an operation control key and the user requests to input a password
  • the operation control key and the number are sequentially arranged in a manner that is close to and far from the user, wherein the operation control
  • the keys are distributed in the same layer, the numbers are arranged in multiple layers, and the same numbers are repeatedly arranged in different layers;
  • the type of the input element includes: a letter and an operation control key
  • the operation control key and the letter are sequentially arranged in a manner that is close to and far from the user, wherein the operation control keys are distributed on the same layer.
  • the letters are arranged in a plurality of layers, and the same letters are repeatedly arranged in different layers.
  • the selection of the display layout of the candidate input element in three-dimensional space includes the following two methods:
  • the three-dimensional spatial positioning technology adopted by the three-dimensional spatial positioning module includes at least one of the following: a time-of-flight TOF positioning technology, a binocular camera positioning technology, an ultrasonic positioning technology, an optical positioning technology, a kinect sensing positioning technology, and a dynamic controller Ps move. Positioning technology, lighthouse positioning technology, laser radar positioning technology, ultra-wideband UWB positioning technology, synchronous positioning and mapping SLAM technology, Virtuix omni positioning technology, structured light positioning technology.
  • the three-dimensional spatial positioning module acquires the coordinates of the xy-axis plane of the current input position of the user in the three-dimensional array parallel to the user's perspective, and then uses the three-dimensional spatial positioning technology to detect The user acquires coordinates on the z-axis depth information of the user perpendicular to the user's perspective by the movement of the input means in the three-dimensional array, thereby obtaining the three-dimensional coordinates of the current input position of the user; Corresponding relationship of the candidate input elements corresponding to each three-dimensional coordinate, the candidate input element corresponding to the three-dimensional coordinates is used as an input element of the user,
  • the three-dimensional spatial positioning module locates an input element, highlights the input element to prompt the user, and after receiving the confirmation operation of the user, transmits the input element to the subsequent operating system.
  • the input means medium of the user includes: hand input, input by using an external control device, input by using a keyboard and a mouse;
  • the array input system for use in a three-dimensional space is mounted on a virtual reality VR device or an augmented reality AR and a mixed reality MR device, and communicates with the virtual reality VR device or an operating system within the AR and MR devices.
  • the array input method and system for three-dimensional space fully utilizes the depth dimension information of the (virtual) three-dimensional space, and the elements such as numbers, letters or symbols having a large number of originals are input according to convenience.
  • the rules are distributed on different depth layers, which greatly reduces the number of elements in each layer, solves the problem of inconvenient input in three-dimensional space, and makes the input of users in three-dimensional space more organized and efficient.
  • the user's input elements can be accurately located. Displaying the numerical arrangement through the three-dimensional array can greatly improve the security of the user input information with respect to the two-dimensional planar display form.
  • FIG. 1 is a flow chart of an array input method for use in a three-dimensional space, in accordance with one embodiment of the present invention
  • FIG. 2 is a flow chart of an array input method for use in three-dimensional space, in accordance with another embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a three-dimensional array according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a three-layer numeric keypad array in accordance with an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a three-layer English alphabet keyboard array according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a three-dimensional digital array for selecting a first layer of "5" elements in accordance with an embodiment of the present invention
  • FIG. 7 is a schematic diagram of a three-dimensional digital array point-selecting a second layer "0" element according to an embodiment of the invention.
  • Figure 8 is a block diagram of an array input system for use in a three dimensional space in accordance with an embodiment of the present invention.
  • the terms “installation”, “connected”, “connected”, “fixed” and the like shall be understood broadly, and may be either a fixed connection or a detachable connection, unless explicitly stated and defined otherwise. , or connected integrally; may be mechanical connection or electrical connection; may be directly connected, or may be indirectly connected through an intermediate medium, and may be internal communication between the two elements.
  • installation shall be understood broadly, and may be either a fixed connection or a detachable connection, unless explicitly stated and defined otherwise.
  • , or connected integrally may be mechanical connection or electrical connection; may be directly connected, or may be indirectly connected through an intermediate medium, and may be internal communication between the two elements.
  • the specific meanings of the above terms in the present invention can be understood on a case-by-case basis.
  • the present invention provides an array input method and system for use in a (virtual) three-dimensional space, by using depth information of a three-dimensional space to display alternate input elements in a three-dimensional array form in a three-dimensional space in front of the user.
  • Forms are relatively flat and two-dimensional in the way of elements, which can make full use of depth information and improve the security of information input by users.
  • the method for inputting an array in a three-dimensional space includes the following steps:
  • step S1 an input request of the user is received.
  • the input mode of the device is triggered to send an input request.
  • the input request includes: a type of the element to be input.
  • the user may specify in the input request that the elements to be input include: a combination of letters and numbers or a combination of letters, numbers, and symbols.
  • the device may provide a selection prompt box for displaying the element to be input, and the user selects according to the need, and sends the input request after confirmation.
  • Step S2 in response to the input request, and setting the candidate input element to display the layout in three-dimensional space according to the type of the element to be input.
  • the response to the input request is initiated, and the type of the element to be input recorded in the input request is acquired.
  • the candidate input elements are displayed in a three-dimensional space according to the type of the element to be input.
  • one of the following means of implementing the above display layout is as follows:
  • An optical light guide lens is placed a few centimeters in front of the eye, and a three-dimensional effect is achieved by striking a corresponding three-dimensional image on the lens.
  • the means for realizing the three-dimensional display layout is not limited to the above two types, and the user mode can also be adopted.
  • the purpose of three-dimensional display of input elements can be achieved, and will not be described here.
  • the display layout includes the number of layers and the arrangement of the candidate input elements in the three-dimensional space, and the three-dimensional display means is used to arrange the candidate input elements in front of the user in a three-dimensional array.
  • each candidate input element has unique three-dimensional coordinates in three-dimensional space for subsequent positioning and matching in three-dimensional space.
  • numbers, letters, and symbols are arranged in order from the user to the near and far, wherein the number of letters can be arranged in multiple layers.
  • FIG. 3 is only an example purpose, and the order, layer number and position of different types of elements can be set by the system and the user according to needs and user habits.
  • the type of input element includes at least one or a combination of the following: numbers, letters, symbols, and operational control keys.
  • the operation control keys are, for example, a backspace key, a newline key, and a confirmation key. It should be noted that the type of the input element is not limited to the above, and may also include its user type, and the type of the input element may be added and deleted as needed.
  • setting the candidate input element to display the layout in three-dimensional space according to the type of the element to be input includes:
  • each type of The input elements include: numbers, letters, symbols, and operation control keys, the operation control keys, numbers, letters, and symbols are sequentially arranged in a manner that is close to and far from the user, each type of The input elements are arranged in one or more layers.
  • the type of the input element includes: the number and the operation control key, the operation control key and the numerical layer are arranged in order according to the distance from the user, wherein the operation control keys are distributed on the same layer, and the number can be The distribution is arranged in one or more layers, and the same numbers are repeatedly arranged in different layers.
  • the order in which the operation controls and numbers are arranged between the different layers is set by the system or user. For example, the operation control key is on the first layer, the number is on the next layer; or the number is on the front, and the operation control key is on the last layer, which can be set according to the user's habits.
  • the numbers are arranged in multiple layers, which can improve the security of the input information.
  • 10 numbers of 0-9 are displayed in multiple layers, and the same numbers are repeatedly arranged in a plurality of layers, and different layers are distinguished.
  • the user can switch between multiple layers of numbers so that the surrounding person cannot infer the password entered by the user from the relative displacement of the hand.
  • the user's safety factor is 1/10 ⁇ 6.
  • input is performed in a three-dimensional array, and since the different layers are distinguished, the same 6-digit password has a security level of 1/30 ⁇ 6. It can be seen that the safety factor is increased by 729 times compared to the two-dimensional input. When the number of input passwords is more or the number of layers is more, the safety factor can be increased more.
  • the digital single layer can be arranged, referring to the first layer of FIG.
  • the user can specify whether he or she is a password input in the input request, or the system automatically infers that the user performs password input according to the function currently used by the user, thereby configuring the corresponding digital display layout.
  • the operation control keys and letters are arranged in layers according to the distance from the user in the distance, wherein the operation control keys are distributed in the same layer, and the letter distribution In a multi-layer arrangement, the same letters are repeatedly arranged in different layers.
  • the order in which the operation control keys and letters are arranged between different layers is set by the system or the user. For example, referring to FIG. 5, the operation control key is on the first layer, the letters are on the next layer; or the letter is on the front, and the operation control key is on the last layer, which can be set according to the user's habits.
  • 26 English letters can be arranged in two layers, each layer is arranged with 13 letters, and the specific arrangement of each letter in each layer is arbitrarily set.
  • 26 English letters can also be arranged in 3 layers, with 8 letter elements arranged in the first two layers and 10 letter elements arranged in the third layer.
  • the three-dimensional array is not limited to the number of layers and the number of elements per layer, the actual form depends on the specific input requirements.
  • the selection of the display layout of the candidate input elements in three dimensions includes the following two methods:
  • the candidate input elements display the layout in a three-dimensional space
  • the layering rules are determined according to the user's input content and the convenience of the operation, and the layered layer can be selected according to one's preference and each layer is to be input.
  • the arrangement of the content is determined according to the user's input content and the convenience of the operation, and the layered layer can be selected according to one's preference and each layer is to be input. The arrangement of the content.
  • step S3 the candidate input elements are displayed in a three-dimensional space in front of the user in a three-dimensional array.
  • a three-dimensional array in a (virtual) three-dimensional space is generated at a suitable location in the user's field of view, the three-dimensional array including all of the alternate input elements for viewing and selection by the user.
  • Step S4 using three-dimensional spatial positioning technology to detect the user's input means medium, and real-time positioning the user's moving position in the three-dimensional array by the input means, detecting the three-dimensional coordinates at each position, and then selecting each three-dimensional coordinate correspondingly
  • the input element is used as the user's input element until the user's confirmation is detected to complete the operation.
  • the numbers, letters or symbols to be input are arranged in a three-dimensional space as required, and displayed in a three-dimensional space in front of the user.
  • the user's input means medium is detected by a three-dimensional spatial positioning technique.
  • the user's input means medium includes: hand input, input by an external control device, input using a keyboard and mouse. That is, the user can move in the three-dimensional array space by means of a hand, a control device, a keyboard, a mouse, etc., and complete the selection confirmation operation.
  • the user can use one-hand or single-control device input means to select elements in the three-dimensional array with two- or two-control devices.
  • the three-dimensional spatial positioning technology is used to spatially locate the moving position of the user in the three-dimensional space, and the user is positioned in real time to move the position of the input means in the three-dimensional array, and the three-dimensional coordinates at each position are detected.
  • each candidate input element has a unique three-dimensional coordinate in three-dimensional space. Comparing the detected three-dimensional coordinates of the moving position with the three-dimensional coordinates of the candidate input elements, determining the candidate input elements selected by the user, and then using the alternate input elements corresponding to each three-dimensional coordinate as the input elements of the user until The user's confirmation completion operation was detected.
  • the three-dimensional spatial positioning technology includes at least one of the following: a time-of-flight TOF positioning technology, a binocular camera positioning technology, an ultrasonic positioning technology, an optical positioning technology, a kinect sensing positioning technology, and a dynamic controller Ps move positioning.
  • a time-of-flight TOF positioning technology a binocular camera positioning technology
  • an ultrasonic positioning technology an optical positioning technology
  • a kinect sensing positioning technology a dynamic controller Ps move positioning.
  • Technology lighthouse positioning technology, laser radar positioning technology, ultra-wideband UWB positioning technology, synchronous positioning and mapping SLAM technology, Virtuix omni positioning technology, structured light positioning technology.
  • the three-dimensional spatial positioning technology used in the present invention is not limited to the above examples, and may be a user-type positioning technology, as long as the positioning in the three-dimensional space can be implemented, and details are not described herein again.
  • step S5 the input element of the user is transmitted to the subsequent operating system to complete the input operation.
  • the flight time TOF positioning technology is taken as an example to illustrate the positioning process:
  • FIG. 2 is a flow chart of an array input method for use in three-dimensional space in accordance with another embodiment of the present invention.
  • step S201 the number of layers and the arrangement of the elements to be input are determined as required.
  • the type of input element is specified by the user according to the purpose of his input, that is, input of numbers, letters or symbols, or numbers, letters, symbols, and the like.
  • a three-dimensional array in (virtual) three-dimensional space ie an array containing z-axis depth information, is generated at a suitable location in the user's field of view, according to explicit input requirements.
  • the z-axis depth information is in one-to-one correspondence with the depth information of the corresponding operation means.
  • Step S202 displaying a corresponding three-dimensional array.
  • Step S203 positioning the input means medium using a three-dimensional spatial positioning technique.
  • the three-dimensional spatial positioning technology is used to acquire the input means medium of the user, for example, by means of a hand, a control device, a keyboard and a mouse.
  • Step S204 using the input means to move the selection in the three-dimensional array.
  • the three-dimensional spatial positioning technology is used to acquire the position information of the user in the three-dimensional array by using the input means, and the position information is mapped to the corresponding array element position in the generated three-dimensional space array, and the corresponding position is given. Feedback.
  • the coordinates of the x-y axis plane of the user's current input position in the three-dimensional array parallel to the user's perspective are obtained by using the three-dimensional spatial positioning technology.
  • the three-dimensional spatial positioning technology is used to detect the movement of the user in the three-dimensional array by the input means, and the coordinates on the z-axis depth information perpendicular to the user's perspective are obtained, thereby obtaining the three-dimensional coordinates of the current input position of the user.
  • the position change information is mapped in real time to the three-dimensional coordinates of the candidate input elements in the three-dimensional array with respect to the relative movement of the input means such as the hand or the control device.
  • the three-dimensional coordinates are corresponding to the corresponding relationship of the candidate input elements corresponding to each three-dimensional coordinate of the configuration.
  • the TOF positioning technology is used as a spatial positioning scheme.
  • the gesture input is used as a case input means, and the operation process of the present invention is described by inputting a 0-9 password as a specific case requirement.
  • a three-dimensional array including digital and operation control keys is generated in a three-dimensional space, and the number and order of the layers can be determined according to the user, for example, set to three layers.
  • a three-dimensional array with depth information is then generated and presented in the (virtual) three-dimensional space in the user's field of view.
  • the specific generation method is: each array element (that is, an array block with a single number or operation control key) has a unique three-dimensional coordinate, including: two-dimensional coordinate information and depth information of the z-axis.
  • the array shape of the second layer and the third layer will perform some deformation and perspective in the two-dimensional space according to the principle of perspective, so that the visual effect also has a three-dimensional array feeling.
  • the z-axis depth information is in one-to-one correspondence with the depth information of the subsequent input means.
  • the user uses a gesture as an input means.
  • the medium of the input means is located by the TOF positioning technique, that is, the position of the specific hand.
  • the TOF continuously transmits a light pulse to the medium of the input means, that is, the user's hand through a unique camera, and then receives the light pulse returned from the hand with the sensor, and obtains the depth information of the hand by detecting the round-trip time of the light pulse.
  • the TOF camera can simultaneously obtain the depth information of the entire image in the field of view, that is, the z-axis depth information of the hand.
  • the TOF camera is used as a reference, and the relative position of the TOF camera and the generated three-dimensional array and the TOF camera is determined to be unchanged.
  • a three-dimensional coordinate system with reference to the TOF camera and the three-dimensional array is formed, and the X, Y-axis and Z-axis coordinates of the fixed three-dimensional array are pre-defined.
  • the depth information of the hand and the x and y axis coordinates in this coordinate system are given by the TOF camera.
  • the hand moves within this coordinate system relative to the invariant three-dimensional array, so that the relative movement coordinates of the hand in three-dimensional space can be calculated.
  • the z-axis depth information and the depth information of the three-dimensional array are one-to-one correspondence. Therefore, by acquiring the relative movement of the hand on the z-axis, it can also be reflected on the z-axis of the invariant three-dimensional array depth information. Based on this, it is possible to realize the calibration and acquisition of the medium of the input means by using the TOF positioning technology: the three-dimensional coordinates of the hand.
  • the user moving the hand can move the selection in the established three-layer digital array.
  • the initial position of the hand is initialized and corresponds to the initial position in the generated three-dimensional array.
  • V1 the three-dimensional coordinates of the number "5" of the first layer of the generated three-layer digital array are determined by the judgment.
  • V0 it is preliminarily judged that the input element currently selected by the user is 5.
  • the "5" of the first layer will be highlighted to prompt the user for the correspondence between the position of the hand and the three-dimensional array.
  • the user After seeing the highlighted prompt, if the user confirms that it is correct, it is completed in the three-dimensional array space by moving his own hand. Choose Confirm. For example, if the user wants to select the "0" element of the second layer, the hand needs to be moved down. At this point, the TOF camera will monitor the position of the hand and give feedback in real time. During the movement, it will pass the “8” of the first layer, and it will also highlight it, indicating that it has been selected. As the hand continues to move down, the button will change from highlight to normal when the response range of the first layer "8" is removed; the hand continues to move down, and the "0" element of the first layer will Highlight response.
  • the hand will stop moving downward and move toward the extension of the z-axis.
  • the TOF camera will capture the three-dimensional information of the hand, and the depth information of the hand is detected to increase, when compared with the depth information of the three-dimensional array element.
  • the relative displacement is generated, after the hand moves to the three-dimensional coordinate response range of the "0" element of the second layer, the "0" element of the second layer becomes highlighted, and thus the moving point selection operation is completed.
  • the triggering method of the specific input means may be arbitrary and is not limited.
  • the index finger is selected by hand as a trigger method.
  • the "0" of the second layer is input to the corresponding register.
  • step S205 the confirmation operation is completed according to the standard of the input means.
  • Step S206 obtaining corresponding input requirements and transmitting to the operating system.
  • the input element is highlighted to prompt the user, and after receiving the user's confirmation operation, the input element is transmitted to the subsequent operating system.
  • an embodiment of the present invention further provides an array input system for use in a three-dimensional space, comprising: a communication module 1, a three-dimensional array generation and display module 2, and a three-dimensional spatial positioning module 3.
  • the communication module 1 is configured to receive an input request of a user.
  • the input mode of the device is triggered and an input request is sent to the communication module 1 of the input system of the present invention.
  • the input request includes: a type of the element to be input.
  • the user may specify in the input request that the elements to be input include: a combination of letters and numbers or a combination of letters, numbers, and symbols.
  • the device may provide a selection prompt box for displaying the element to be input, and the user selects according to the need, and sends the input request after confirmation.
  • the three-dimensional array generation and display module 2 is connected to the communication module 1 for responding to the input request, and setting the candidate input element to display the layout in three-dimensional space according to the type of the element to be input, and the candidate input element is in front of the user in a three-dimensional array form.
  • the three-dimensional space is displayed.
  • one of the following means of implementing the above display layout is as follows:
  • An optical light guide lens is placed a few centimeters in front of the eye, and a three-dimensional effect is achieved by striking a corresponding three-dimensional image on the lens.
  • the means for realizing the three-dimensional display layout is not limited to the above two types, and the user mode can also be adopted.
  • the purpose of three-dimensional display of input elements can be achieved, and will not be described here.
  • the display layout includes the number of layers and the arrangement of the candidate input elements in the three-dimensional space, and the three-dimensional display means is used to arrange the candidate input elements in front of the user in a three-dimensional array.
  • each candidate input element has unique three-dimensional coordinates in three-dimensional space for subsequent positioning and matching in three-dimensional space.
  • the type of input element includes at least one or a combination of the following: numbers, letters, symbols, and operational control keys.
  • the operation control keys are, for example, a backspace key, a newline key, and a confirmation key. It should be noted that the type of the input element is not limited to the above, and may also include its user type, and the type of the input element may be added and deleted as needed.
  • the three-dimensional array generation and display module 2 sets the display layout of the candidate input elements in the three-dimensional space according to the type of the element to be input, including:
  • each type of The input elements include: numbers, letters, symbols, and operation control keys, the operation control keys, numbers, letters, and symbols are sequentially arranged in a manner that is close to and far from the user, each type of The input elements are arranged in one or more layers.
  • the operation control keys and the numbers are sequentially arranged in a manner that is close to and far from the user, wherein the operation control keys are distributed in
  • numbers can be arranged in one or more layers, and the same numbers are repeatedly arranged in different layers.
  • the order in which the operation controls and numbers are arranged between the different layers is set by the system or user. For example, the operation control key is on the first layer, the number is on the next layer; or the number is on the front, and the operation control key is on the last layer, which can be set according to the user's habits.
  • the operation control keys and letters are arranged in layers according to the distance from the user in the distance, wherein the operation control keys are distributed in the same layer, and the letter distribution In a multi-layer arrangement, the same letters are repeatedly arranged in different layers.
  • the order in which the operation control keys and letters are arranged between different layers is set by the system or the user.
  • the selection of the display layout of the candidate input elements in three dimensions includes the following two methods:
  • the candidate input elements display the layout in a three-dimensional space
  • the layering rules are determined according to the user's input content and the convenience of the operation, and the layered layer can be selected according to one's preference and each layer is to be input.
  • the arrangement of the content is determined according to the user's input content and the convenience of the operation, and the layered layer can be selected according to one's preference and each layer is to be input. The arrangement of the content.
  • the three-dimensional array generation and display module 2 further displays the alternate input elements in a three-dimensional array in front of the user in a three-dimensional space.
  • the three-dimensional array generation and display module 2 is generated at a suitable position of the user's visual field of view.
  • a three-dimensional array in (virtual) three-dimensional space that includes all of the alternate input elements for viewing and selection by the user.
  • the three-dimensional spatial positioning module 3 is connected to the three-dimensional array generating and displaying module 2, and is configured to detect the input means medium of the user by using the three-dimensional spatial positioning technology, and locate the moving position of the user in the three-dimensional array by the input means in real time, and detect each position.
  • the three-dimensional coordinates, the alternate input element corresponding to each three-dimensional coordinate is taken as the input element of the user until the confirmation of the user is completed.
  • the communication module 1 transmits the input element of the user to the subsequent operating system to complete the input operation.
  • the three-dimensional array generation and display module 2 displays the numbers, letters or symbols to be input in the three-dimensional space as required, and displays them in a three-dimensional space in front of the user.
  • the user's input means medium is detected by the three-dimensional spatial positioning module 3.
  • the user's input means medium includes: hand input, input by an external control device, input using a keyboard and mouse. That is, the user can move in the three-dimensional array space by means of a hand, a control device, a keyboard, a mouse, etc., and complete the selection confirmation operation.
  • the user can use one-hand or single-control device input means to select elements in the three-dimensional array with two- or two-control devices.
  • the three-dimensional spatial positioning module 3 uses the three-dimensional spatial positioning technology to spatially locate the moving position of the user in the three-dimensional space, and locates the moving position of the user in the three-dimensional array by the input means in real time, and detects the three-dimensional coordinates at each position. According to each candidate input element, there is a unique three-dimensional coordinate in three-dimensional space. Comparing the detected three-dimensional coordinates of the moving position with the three-dimensional coordinates of the candidate input elements, determining the candidate input elements selected by the user, and then using the alternate input elements corresponding to each three-dimensional coordinate as the input elements of the user until The user's confirmation completion operation was detected.
  • the three-dimensional spatial positioning technology adopted by the three-dimensional spatial positioning module 3 includes at least one of the following: a time-of-flight TOF positioning technology, a binocular camera positioning technology, an ultrasonic positioning technology, an optical positioning technology, and a kinect sensing positioning technology.
  • Dynamic controller Ps move positioning technology, lighthouse Light house positioning technology, laser radar positioning technology, ultra-wideband UWB positioning technology, synchronous positioning and mapping SLAM technology, Virtuix omni positioning technology, structured light positioning technology.
  • the three-dimensional spatial positioning technology used in the present invention is not limited to the above examples, and may be a user-type positioning technology, as long as the positioning in the three-dimensional space can be implemented, and details are not described herein again.
  • the array input system for three-dimensional space provided by the present invention is installed on a virtual reality VR device or an augmented reality AR and a mixed reality MR device, and a virtual reality VR device or an AR and MR device.
  • the internal operating system communicates.
  • the device to which the present invention is applicable is not limited to the above examples, and the three-dimensional array arrangement input provided by the present invention can be applied as long as there is a requirement for three-dimensional display and input, and details are not described herein again.
  • the array input method and system for three-dimensional space of the present invention fully utilizes the depth dimension information of the (virtual) three-dimensional space, and distributes elements such as numbers, letters or symbols that are originally large in number according to the rules of input convenience. Different depth layers, thus greatly reducing the number of elements per layer, solving the problem of inconvenient input in three-dimensional space, making the input of the user in three-dimensional space more organized and efficient. Moreover, by using the three-dimensional spatial positioning technology to accurately locate the user's input, the user's input elements can be accurately located. Displaying the numerical arrangement through the three-dimensional array can greatly improve the security of the user input information with respect to the two-dimensional planar display form.

Abstract

La présente invention concerne un procédé et un système d'entrée de type en matrice destinés à être utilisés dans un espace tridimensionnel. Le procédé comprend les étapes consistant à : recevoir une demande d'entrée provenant d'un utilisateur ; répondre à la demande d'entrée et configurer une disposition d'affichage d'un élément d'entrée optionnel dans l'espace tridimensionnel en fonction du type d'un élément devant être entré ; afficher l'élément d'entrée optionnel dans l'espace tridimensionnel devant l'utilisateur sous la forme d'une matrice tridimensionnelle ; utiliser une technique de positionnement d'espace tridimensionnel pour détecter un support de moyen d'entrée de l'utilisateur, positionner une position mobile de l'utilisateur dans la matrice tridimensionnelle en utilisant ledit moyen d'entrée en temps réel, détecter une coordonnée tridimensionnelle au niveau de chaque position et, par conséquent, considérer l'élément d'entrée optionnel correspondant à chaque coordonnée tridimensionnelle comme un élément d'entrée de l'utilisateur jusqu'à une détection indiquant que l'utilisateur confirme qu'une opération est achevée ; et transmettre l'élément d'entrée de l'utilisateur à un système fonctionnel ultérieur, ce qui achève l'opération d'entrée. La présente invention distribue des nombres, des lettres ou des symboles, etc., au niveau de différentes couches de profondeur dans l'espace tridimensionnel, ce qui règle le problème selon lequel une entrée dans un espace tridimensionnel est difficile et permet à un utilisateur d'entrer dans un espace tridimensionnel d'une manière plus organisée et plus efficace.
PCT/CN2016/095986 2016-08-10 2016-08-19 Procédé et système d'entrée de type en matrice destinés à être utilisés dans un espace tridimensionnel WO2018028003A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610652247.0 2016-08-10
CN201610652247.0A CN106293127B (zh) 2016-08-10 2016-08-10 用于三维空间中的阵列式输入方法及系统

Publications (1)

Publication Number Publication Date
WO2018028003A1 true WO2018028003A1 (fr) 2018-02-15

Family

ID=57668149

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/095986 WO2018028003A1 (fr) 2016-08-10 2016-08-19 Procédé et système d'entrée de type en matrice destinés à être utilisés dans un espace tridimensionnel

Country Status (2)

Country Link
CN (1) CN106293127B (fr)
WO (1) WO2018028003A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106873796B (zh) * 2017-01-13 2019-11-26 河源市汉祖文化科技有限公司 一种基于三笔数码输入法的手势输入方法及系统
CN108958573B (zh) * 2017-05-26 2021-06-29 创新先进技术有限公司 基于虚拟现实场景的身份认证方法及装置
CN107329566B (zh) * 2017-06-23 2020-03-24 联想(北京)有限公司 虚拟对象处理方法及其系统、虚拟现实设备
CN107688388B (zh) * 2017-08-20 2020-08-28 平安科技（深圳）有限公司 密码输入的控制设备、方法及计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102446064A (zh) * 2010-10-04 2012-05-09 Lg电子株式会社 移动终端及其控制方法
US20120162213A1 (en) * 2010-12-24 2012-06-28 Samsung Electronics Co., Ltd. Three dimensional (3d) display terminal apparatus and operating method thereof
CN103430138A (zh) * 2011-01-20 2013-12-04 黑莓有限公司 与用户接口的不同输入组件相关联的图标的三维多深度呈现
CN105808015A (zh) * 2016-05-27 2016-07-27 京东方科技集团股份有限公司 防窥用户交互装置及方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1042250A (zh) * 1989-10-20 1990-05-16 胡书彬 汉字层次三位编码方法及其键盘
CN1050432C (zh) * 1995-12-25 2000-03-15 中国中文信息学会 全拼双拼规范码汉字输入方法
US7151530B2 (en) * 2002-08-20 2006-12-19 Canesta, Inc. System and method for determining an input selected by a user through a virtual interface
US9207859B2 (en) * 2010-09-14 2015-12-08 Lg Electronics Inc. Method and mobile terminal for displaying fixed objects independent of shifting background images on a touchscreen
CN105224069B (zh) * 2014-07-03 2019-03-19 王登高 一种增强现实虚拟键盘输入方法及使用该方法的装置
CN105808086B (zh) * 2014-12-30 2019-04-19 Tcl集团股份有限公司 虚拟键盘的输入方法及装置
CN105511618A (zh) * 2015-12-08 2016-04-20 北京小鸟看看科技有限公司 三维输入装置、头戴式装置及三维输入方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102446064A (zh) * 2010-10-04 2012-05-09 Lg电子株式会社 移动终端及其控制方法
US20120162213A1 (en) * 2010-12-24 2012-06-28 Samsung Electronics Co., Ltd. Three dimensional (3d) display terminal apparatus and operating method thereof
CN103430138A (zh) * 2011-01-20 2013-12-04 黑莓有限公司 与用户接口的不同输入组件相关联的图标的三维多深度呈现
CN105808015A (zh) * 2016-05-27 2016-07-27 京东方科技集团股份有限公司 防窥用户交互装置及方法

Also Published As

Publication number Publication date
CN106293127A (zh) 2017-01-04
CN106293127B (zh) 2019-03-19

Similar Documents

Publication Publication Date Title
WO2018028003A1 (fr) Procédé et système d'entrée de type en matrice destinés à être utilisés dans un espace tridimensionnel
CN105100390B (zh) 一种控制移动终端的方法
CN102841710B (zh) 感测方法与装置
KR101591579B1 (ko) 증강 현실 시스템들에서 실세계 표면들에의 가상 이미지들의 앵커링
US10235121B2 (en) Wirelessly communicating configuration data for interactive display devices
EP2965299B1 (fr) Modification d'une fonctionnalité suivant la distance entre des dispositifs
EP2945407A1 (fr) Appariement déclenché d'image
JP6253127B2 (ja) 情報提供装置
US20210056370A1 (en) Optical tag based information apparatus interaction method and system
KR101282361B1 (ko) 3차원 입력 인터페이스 제공 장치 및 방법
CN111309183A (zh) 触控显示系统及其控制方法
US10594993B2 (en) Image projections
US20120291093A1 (en) Communication device, communication method, and program
KR102077665B1 (ko) 혼합 현실에서의 가상 모바일 단말 구현 시스템 및 이의 제어 방법
KR101632220B1 (ko) 이동 단말기, 이동 단말기의 제어방법, 그리고, 제어시스템과 그 제어방법
KR20150008688A (ko) 디스플레이장치 및 그 제어방법
US20190027021A1 (en) Mobile device and method for controlling mobile device
US20190096130A1 (en) Virtual mobile terminal implementing system in mixed reality and control method thereof
CN117941323A (zh) 用于管理虚拟工作空间的方法、设备和介质
US20180267757A1 (en) System and method for interacting with media displays
EP2829949B1 (fr) Système et procédé permettant de manipuler un objet dans un environnement de bureau en trois dimensions
KR101091984B1 (ko) 정보제어장치를 이용한 데이터 전송시스템 및 방법, 그 단말기 및 정보제어장치
WO2013156987A1 (fr) Système d'interface de proximité et procédé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16912458

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16912458

Country of ref document: EP

Kind code of ref document: A1