CN110442245A - Display methods, device, terminal device and storage medium based on physical keyboard - Google Patents

Display methods, device, terminal device and storage medium based on physical keyboard Download PDF

Info

Publication number
CN110442245A
CN110442245A CN201910684470.7A CN201910684470A CN110442245A CN 110442245 A CN110442245 A CN 110442245A CN 201910684470 A CN201910684470 A CN 201910684470A CN 110442245 A CN110442245 A CN 110442245A
Authority
CN
China
Prior art keywords
key
physical keyboard
virtual
terminal device
control function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910684470.7A
Other languages
Chinese (zh)
Inventor
卢智雄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Virtual Reality Technology Co Ltd
Original Assignee
Guangdong Virtual Reality Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Virtual Reality Technology Co Ltd filed Critical Guangdong Virtual Reality Technology Co Ltd
Priority to CN201910684470.7A priority Critical patent/CN110442245A/en
Publication of CN110442245A publication Critical patent/CN110442245A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/0202—Constructional details or processes of manufacture of the input device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233—Character input methods

Abstract

The embodiment of the present application discloses a kind of display methods based on physical keyboard, device, terminal device and storage medium, the display methods based on physical keyboard includes: the first relative tertiary location information obtained between terminal device and physical keyboard, and the physical keyboard includes multiple keys；Determine one or more keys in the physical keyboard with control function；According to control function corresponding to determining each key, virtual instruction content corresponding with each key is obtained respectively；According to the first relative tertiary location information, the second relative tertiary location information between the terminal device and each key is obtained respectively；Based on each corresponding second relative tertiary location information of key and virtual instruction content, generates and distinguish matched virtual instruction with each key determined, the display position virtually indicated is corresponding with matched key.This method can be convenient user and be controlled by physical keyboard, realize the control function of user demand.

Description

Display methods, device, terminal device and storage medium based on physical keyboard
Technical field
This application involves field of display technology, more particularly, to a kind of display methods based on physical keyboard, device, Terminal device and storage medium.
Background technique
Currently, physical keyboard is commonly used for inputting information and instruction.Physical keyboard in use, user generally according to object The content printed on the keycap of keyboard is managed to be inputted.But for user when using keyboard input information and instruction, there are unclear Or the case where forgetting keypress function, without convenient for the user to operate.
Summary of the invention
The embodiment of the present application proposes a kind of display methods based on physical keyboard, device, terminal device and storage medium, User can be facilitated to control by physical keyboard.
In a first aspect, the embodiment of the present application provides a kind of display methods based on physical keyboard, which comprises obtain The first relative tertiary location information between terminal device and physical keyboard is taken, the physical keyboard includes multiple keys；It determines With one or more keys of control function in the physical keyboard；According to control function corresponding to determining each key Can, virtual instruction content corresponding with each key is obtained respectively；According to the first relative tertiary location information, respectively Obtain the second relative tertiary location information between the terminal device and each key；It is corresponding based on each key The second relative tertiary location information and virtual instruction content, generate and distinguish matched virtual instruction with each key determined, The display position virtually indicated is corresponding with matched key.
Second aspect, the embodiment of the present application provide a kind of display device based on physical keyboard, and described device includes: One position acquisition module, key determining module, content obtain module, the second position obtains module and content generating module, In, the first position obtains module and is used to obtain the first relative tertiary location information between terminal device and physical keyboard, The physical keyboard includes multiple keys；The key determining module is used to determine in the physical keyboard with control function One or more keys；The content obtains module and is used for the control function according to corresponding to determining each key, obtains respectively Take virtual instruction content corresponding with each key；The second position obtains module for relatively empty according to described first Between location information, obtain the second relative tertiary location information between the terminal device and each key respectively；It is described Content generating module is used to generate based on each corresponding second relative tertiary location information of key and virtual instruction content Matched virtual instruction is distinguished with determining each key, and the display position virtually indicated is corresponding with matched key.
The third aspect, the embodiment of the present application provide a kind of terminal device, comprising: one or more processors；Memory； One or more application program, wherein one or more of application programs be stored in the memory and be configured as by One or more of processors execute, one or more of programs be configured to carry out that above-mentioned first aspect provides based on The display methods of physical keyboard.
Fourth aspect, the embodiment of the present application provide a kind of storage medium, deposit in the computer-readable storage medium Program code is contained, said program code can be called by processor executes the showing based on physical keyboard that above-mentioned first aspect provides Show method.
Scheme provided by the present application is believed by the first relative tertiary location obtained between terminal device and physical keyboard Breath, physical keyboard includes multiple keys, one or more keys in physical keyboard with control function is determined, according to determining Control function corresponding to each key obtains virtual instruction content corresponding with each key respectively, relatively empty according to first Between location information, obtain the second relative tertiary location information between terminal device and each key respectively, be based on each key Corresponding second relative tertiary location information and virtual instruction content, generate and distinguish matched virtual finger with each key determined Show, the display position virtually indicated is corresponding with matched key, so that user can be according to virtual instruction, to control function Key operated, realize to the control function of virtual content, user facilitated to interact with virtual content.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, the drawings in the following description are only some examples of the present application, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.
Fig. 1 shows a kind of schematic diagram of application scenarios suitable for the embodiment of the present application.
Fig. 2 shows the display methods flow charts based on physical keyboard according to the application one embodiment.
Fig. 3 shows a kind of display effect schematic diagram provided according to the application one embodiment.
Fig. 4 shows the display methods flow chart based on physical keyboard according to another embodiment of the application.
Fig. 5 shows the display methods flow chart based on physical keyboard according to another embodiment of the application.
Fig. 6 shows the display methods flow chart based on physical keyboard according to the application further embodiment.
Fig. 7 shows a kind of display effect schematic diagram provided according to the application further embodiment.
Fig. 8 shows another display effect schematic diagram provided according to the application further embodiment.
Fig. 9 shows the display methods flow chart based on physical keyboard according to the application a still further embodiment.
Figure 10 shows a kind of display effect schematic diagram provided according to the application a still further embodiment.
Figure 11 shows the structural block diagram according to the display device provided by the embodiments of the present application based on physical keyboard.
Figure 12 is the embodiment of the present application for executing the display methods based on physical keyboard according to the embodiment of the present application Terminal device block diagram.
Figure 13 is the embodiment of the present application for saving or carrying realization according to the embodiment of the present application based on secondary or physical bond The storage unit of the program code of the display methods of disk.
Specific embodiment
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described.
The application scenarios of the display methods of virtual content provided by the embodiments of the present application are introduced below.
Referring to Figure 1, the augmented reality system 10 provided in one embodiment is provided.The augmented reality system 10 packet It includes: terminal device 100 and physical keyboard 200, wherein terminal device 100 is connect with physical keyboard 200.
In some embodiments, terminal device 100 can be head-wearing display device, be also possible to the shifting such as mobile phone, plate Dynamic equipment.When terminal device 100 is head-wearing display device, head-wearing display device can be integral type head-wearing display device, can also To be the head-wearing display device being connect with external electronic equipment.Terminal device 100 is also possible to wear with circumscribed/access type aobvious The intelligent terminals such as the mobile phone of showing device connection, i.e. terminal device 100 can be used as the processing and storage equipment of head-wearing display device, insert Enter or access circumscribed head-wearing display device, virtual content is shown in head-wearing display device.
In some embodiments, it may be provided with marker 201 on physical keyboard.Wherein, marker 201 can be pasted attached Or be integrated on physical keyboard 200, it is not limited here.The quantity for being set to the marker 201 on physical keyboard 200 can be with Not as restriction, the quantity of marker 201 can be one or more.As an implementation, the quantity of marker 201 It is four corners region that 4,4 markers 201 can be respectively arranged at physical keyboard 200, passes through marker to be promoted The accuracy that 201 pairs of physical keyboards 200 and its key are positioned.
In some embodiments, terminal device 100 and physical keyboard 200 can pass through bluetooth, WiFi (Wireless- Fidelity, Wireless Fidelity), the communication modes connection such as ZigBee (purple peak technology), the wire communications such as data line can also be passed through Connection, it is not limited here.
When terminal device 100 and physical keyboard 200 are used collectively, marker 201 can be made to be located at terminal device 100 In visual range, so that terminal device 100 can collect the image comprising marker 201, to carry out to marker 201 Identification tracking, obtains spatial positional informations and the markers 201 such as position, the posture of 201 relative termination equipment 100 of marker The recognition results such as identity information, and then spatial positions such as position, posture for obtaining 200 relative termination equipment 100 of physical keyboard Information, realize to physical keyboard 200 and it includes each key location tracking.Terminal device 100 can according to secondary or physical bond Relative position and posture information between disk 200, show corresponding virtual content.
In some embodiments, marker 201 is the pattern with topological structure, and topological structure refers in marker Connected relation between sub- marker and characteristic point etc..In some embodiments, marker 201 can also be the mark of light-spot type Note, terminal device is by tracking with spatial positional informations such as relative position, postures luminous point.In a specific embodiment In, luminous point and Inertial Measurement Unit (Inertial measurement unit, IMU) can be set on physical keyboard 200, eventually End equipment can acquire the dot pattern picture on physical keyboard 200 by imaging sensor, and obtained and surveyed by Inertial Measurement Unit Data are measured, are the space that can determine between physical keyboard 200 and terminal device 100 according to the dot pattern picture and measurement data Location information realizes positioning and tracking to physical keyboard 200.Wherein, the luminous point being arranged on physical keyboard 200 can be visible The spot sequence that the quantity of luminous point perhaps infrared light spot luminous point can be one or be made of multiple luminous points.
It in some embodiments, may include multiple keys 202 on physical keyboard 200.Physical keyboard can according to Operation of the family to key 202 generates operation data, and operation data is sent to terminal device 100, and terminal device 100 is connecing After receiving operation data, corresponding operation can be executed, such as control etc. the virtual content of display.
In some embodiments, physical keyboard 200 can also include Trackpad, and Trackpad can be set in physical keyboard 200 surface, so that user is operated and controlled by Trackpad.
In some embodiments, physical keyboard 200 can also connect other external equipments, and behaviour can be generated in other peripheral hardwares Make data and terminal device is sent to by physical keyboard 200, realization is operated and controlled by other peripheral hardwares.Other peripheral hardwares It may include mouse, handle etc., it is not limited here.Wherein, physical keyboard 200 can be connected with other peripheral hardwares by wired mode It connects, for example, other peripheral hardwares access the Peripheral Interface of physical keyboard 200 by connecting line；Physical keyboard 200 and other peripheral hardwares can also It connects wirelessly, for example, passing through the connection of the communications such as bluetooth, ZigBee.
In some embodiments, physical keyboard 200 can be foldable cellular keyboard, and user is by folding physics Keyboard 200 reduces 200 area occupied of physical keyboard, so that user be facilitated to carry.
It should be noted that physical keyboard 200 is not limited to the form of physical keyboard, in some embodiments, object Reason keyboard 200 is also possible to dummy keyboard shown by the touch screen of mobile terminal (such as tablet computer, mobile phone etc.).
The display methods specifically based on physical keyboard is introduced below.
Referring to Fig. 2, the application one embodiment provides a kind of display methods based on physical keyboard, can be applied to The terminal device in augmented reality system is stated, this method may include:
Step S110: the first relative tertiary location information between terminal device and physical keyboard, physical keyboard packet are obtained Include multiple keys.
In some embodiments, physical keyboard can be used to implement the scene of control function, such as physical keyboard is used for It operates video games, be used to control video pictures, is used for the scenes such as input content.
When physical keyboard be used to realize control function, terminal device and physical keyboard may be at same site environment, And the marker on physical keyboard is in the image collecting device of terminal device within sweep of the eye, so that terminal device can be with Physical keyboard is tracked by identification tag.
In some embodiments, the first space between the available terminal device of terminal device and physical keyboard Location information is shown so that terminal device generates relevant virtual content according to the first relative tertiary location information.Specifically Ground, terminal device can identify the marker on physical keyboard, to obtain terminal device according to the recognition result of identification tag The first relative tertiary location information between physical keyboard.
The recognition result of terminal device identification tag includes at least the location information of marker relative termination equipment, posture The identity information of information etc. and marker, each marker can have different identity informations, so that terminal device can root The position being arranged on physical keyboard according to marker and above-mentioned recognition result obtain the between terminal device and physical keyboard One relative tertiary location information.The first relative tertiary location information between terminal device and physical keyboard may include: terminal Relative position information and posture information between equipment and physical keyboard etc., posture information can be physical keyboard relative termination Direction and rotation angle of equipment etc..
Certainly, the concrete mode of the first relative tertiary location information between terminal device and physical keyboard is obtained in this Shen It please can be not as restriction in embodiment.
Step S120: one or more keys in physical keyboard with control function are determined.
In some embodiments, one or more keys of physical keyboard can have control function, i.e., these keys can To be configured with corresponding control function, the corresponding relationship of key and its specific control function with control function be can store In terminal device.For example, at least partly key on physical keyboard can be configured as technical ability key, object in scene of game Product key, adjustment key of game parameter etc.；In another example in the scene of video or music, on physical keyboard extremely Small part key, which can be configured as, plays the functions such as control, pause control, volume control, playback progress control；Further for example, In In the scene of word content input, at least partly key on physical keyboard can be configured as the shortcut functions such as duplication, stickup And play the role of shortcut key.
In some embodiments, for physical keyboard when for realizing control function, terminal device can determine secondary or physical bond In all keys that disk includes with control function one or more keys, with control function key quantity not as It limits, it can be depending on actual scene and demand.Specifically, terminal device can be according to the key and control function being locally stored The corresponding relationship of energy obtains one or more keys on physical keyboard with control function.The corresponding relationship can be with physics The form of the configuration file of keyboard is stored in terminal device, and the type of configuration file can be the types such as table, does not limit herein It is fixed.Key and key with control function and the corresponding relationship of control function can be matched with hobby according to demand by user It sets, can also be preset by manufacturer, it is not limited here.
For example, the part key of physical keyboard, which has, plays control function in the scene of music control, key with The control function that the corresponding relationship of control function can play for the corresponding pause of key F5 is played, key F6 corresponds to next control Function processed, the corresponding upper one first control function of key F7, key F8 correspond to the function of speed broadcasting, and terminal device can be according to this Corresponding relationship, determining has the key for playing control function on physical keyboard be F5, F6, F7 and F8, the above music control Key and the corresponding relationship of control function are only for example in the scene of system, are not used to limit.
Step S130: it according to control function corresponding to determining each key, obtains respectively corresponding with each key Virtual instruction content.
The corresponding virtual instruction of each key in the available one or more keys with control function of terminal device Content, it is virtual to indicate that content generates virtual instruction corresponding with having the key of control function for terminal device, it is virtual to indicate It can serve to indicate that the control function of key.Wherein, virtually instruction content may include for generating the three-dimensional mould virtually indicated Type data, the three-dimensional modeling data may include for constructing the color of the corresponding model of threedimensional model, model vertices coordinate, mould Type outline data etc..As an implementation, virtually instruction content can be iconic content, and iconic content is specific for characterizing Control function；As another embodiment, virtually instruction content may be word content, and word content can be summary The descriptive matter in which there of control function, such as title etc.；As another embodiment, it is virtual indicate content may be icon with The content that text combines, specific virtual instruction content can be without limitation.
In some embodiments, terminal device obtains the corresponding virtual instruction content of key each with control function When, can the control function according to corresponding to determining each key, void corresponding to the determining control function with each key Quasi- instruction content.Wherein, virtual instruction content corresponding to different control functions can be different, virtual to indicate content and control function The corresponding relationship of energy can store in terminal device, and terminal device can be closed according to virtual instruction content is corresponding with control function Virtual instruction content corresponding to the control function of system, determination and each key, and the control function institute with each key is right The virtual instruction content answered, as the corresponding virtual instruction content of each key.
Step S140: according to the first relative tertiary location information, the between terminal device and each key is obtained respectively Two opposite spatial positional informations.
Terminal device can be according to the first relative tertiary location information between the terminal device and physical keyboard of acquisition, really The second relative tertiary location information between terminal device and each key with control function is determined, so as to the void being subsequently generated Quasi- instruction is corresponding with key.
In some embodiments, terminal device can be according to the position of each key in keys all on physical keyboard The first space position between (i.e. relative position of the key with respect to physical keyboard entirety) and terminal device and physical keyboard Confidence breath determines the second relative tertiary location information between terminal device and each key with control function.As one Kind of specific embodiment, terminal device can according to the positional relationship between key each on physical keyboard and marker, and The first relative tertiary location information between terminal device and physical keyboard determines terminal device and each has control function The second relative tertiary location information between key.
In some embodiments, when physical keyboard is dummy keyboard shown by the touch screen of mobile terminal, due to not Key in the dummy keyboard of same type or format is different in the position of touch-control screen display, therefore then terminal device can basis The type or format for the dummy keyboard that mobile terminal is shown determine position of each key of dummy keyboard on touch screen, For example, dummy keyboard can be 9 key types, it is also possible to 26 key types etc., further according to position of each key on touch screen, And the first relative tertiary location information, determine the second relative tertiary location information between terminal device and each key.
In the embodiment of the present application, terminal device obtains the corresponding virtual instruction content of each key, obtains with terminal device Take the sequencing of the second relative tertiary location information between each key can be not as restriction, for example, step S140 It can be executed before step S130, step S140 can also be executed after step s 130.
Step S150: based on the corresponding second relative tertiary location information of each key and virtual instruction content, generate with Determining each key distinguishes matched virtual instruction, and the display position virtually indicated is corresponding with matched key.
Terminal device is getting the second space position between terminal device and each key with control function After confidence breath, and virtual instruction content corresponding to each key with control function, then can be generated has with each The matched virtual instruction of the key of control function, and by generation it is virtual instruction show, so as to user watch with it is each The matched virtual instruction of key with control function.
In some embodiments, terminal device can be according to the second phase between each key with control function To spatial positional information, rendering position of the virtual instruction content in Virtual Space is obtained, generates simultaneously wash with watercolours further according to rendering position Three-dimensional virtual instruction is dyed, and shows the virtual instruction of rendering.The position of display virtually indicated is corresponding with virtually indicating Key is corresponding, which refers to that terminal device will virtually indicate that Overlapping display arrives by way of augmented reality Position in true environment, it is understood that virtually position of the instruction in true environment seen for user by terminal device It sets.
In some embodiments, terminal device can be opposite according to second between each key with control function Relative positional relationship between spatial positional information, and each key with control function and corresponding virtual instruction, obtains The spatial position coordinate of the virtual position for indicating to need Overlapping display of each of realistic space, and by each space bit of acquisition It sets coordinate and is converted to space coordinate in Virtual Space, so that rendering coordinate of each virtual instruction in Virtual Space is obtained, The rendering position each virtually indicated is obtained.Wherein, which can be used as the rendering coordinate virtually indicated, to realize At rendering position, rendering coordinate is referred to virtually indicating in Virtual Space with default origin (such as generation for virtual instruction rendering Boundary's coordinate origin, Softcam origin etc.) three dimensional space coordinate.As an implementation, each virtual instruction needs Relative positional relationship between the position of Overlapping display and each key with control function can be that virtual instruction needs folded The position of display is added to be at the position of the key with control function.
After the rendering coordinate that terminal device each virtually indicates in obtaining Virtual Space, terminal device can be according to every A each virtual instruction of virtual instruction content building, and virtually refer to according to each rendering coordinate rendering virtually indicated is each Show, wherein rendering virtually indicates the apex coordinate and color value etc. on each vertex in available virtual instruction.
In some embodiments, after terminal device generates each virtual instruction, all virtual instructions can be shown Show.Specifically, after terminal device constructs and renders three-dimensional virtual instruction, virtual screen can be converted by virtual instruction, Obtain the display data that virtually indicate, which may include showing in picture the rgb value of each pixel and corresponding Pixel coordinate etc., terminal device can generate display picture according to the display data, and display picture is passed through display screen or throwing It penetrates mould group to project on display eyeglass, to show virtual instruction, user passes through the display eyeglass of head-wearing display device, can be with See that virtual instruction Overlapping display at the key in real world on physical keyboard with control function, realizes augmented reality Display effect facilitates user according to the corresponding virtual control function for indicating to know each key of each key.
In some embodiments, when terminal device also shows virtual screen, then user can use physical keyboard It interacts with virtual screen, to facilitate user according to virtual instruction, the key with control function is operated, and it is real Now to the control of virtual screen.
For example, as shown in figure 3, the key A in physical keyboard 200 has the function of that magnified content, key D have in diminution The function of appearance, then virtual enlarging function icon 301 corresponding with key A and corresponding with key D can be generated in terminal device Virtual reduction capability icon 302, and by virtual enlarging function icon 301 and virtual reduction capability icon 302 into Row display, user can watch the virtual amplification at the position of key A by the display eyeglass of head-wearing display device Icon 301 and the virtual reduction capability icon 302 at the position of key B are put to recognize that key A has The function of big content, key D are had the function of diminution content, are amplified the control of content using physical keyboard 200 convenient for user System and the control for reducing content.
Display methods provided by the embodiments of the present application based on physical keyboard, terminal device can be according to control functions The corresponding specific control function of key, generate with the matched virtual instruction of key with control function, so that user can be with According to virtual instruction, the key with control function is operated, realizes the control function to virtual content, facilitate user with Virtual content interacts.
Referring to Fig. 4, the application can be applied to another embodiment provides for a kind of display methods based on physical keyboard Terminal device in above-mentioned display system, this method may include:
Step S210: the first relative tertiary location information between terminal device and physical keyboard, physical keyboard packet are obtained Include multiple keys.
In the embodiment of the present application, step S210 can be refering to the content of above-described embodiment, and details are not described herein.
Step S220: current usage scenario is determined.
In the embodiment of the present application, since different usage scenario demands uses the control of the key and key of physical keyboard Function processed is different, therefore for different usage scenarios, can configure the key with control function corresponding with usage scenario And the control function of each key with control function.
In some embodiments, the number of the required control function controlled using physical keyboard of different usage scenarios Amount can be different, therefore the key with control function of different number can be set in different usage scenarios, for example, in difference In the scene of game of game, the operation amount that each game needs user to carry out is different, therefore has control function on physical keyboard The number of keys of energy is different.Further, can according to the required number of keys with control function of each usage scenario, The key for configuring the number of keys on physical keyboard, as the key with control function corresponding with each usage scenario.
In some embodiments, it between the corresponding key with control function of different usage scenarios, can also deposit In one or more identical key.In addition, in some different specifically used scenes, these identical keys can also be with Function having the same, for example, the key with control function can be equal in music scene and video playing scene It is set as F5, F6, F7 and F8, and F5 is the control function that pause plays in music scene and video playing scene Can, F6 is the control function of next program in music scene and video playing scene, and F7 is in music scene And be the control function of a upper program in video playing scene, F8 in music scene and video playing scene For the control function that speed plays, the above key and control function are only for example.
Certainly, apply for the key and the corresponding control of each key in embodiment corresponding to usage scenario with control function Function processed can configure, it is not limited here according to actual usage scenario, user demand etc..
The available current usage scenario of terminal device, to determine in physical keyboard have according to current usage scenario There are the key of control function and the control function of each key.As an implementation, terminal device can be according to current The application of operation and determine current usage scenario, for example, when the currently running application of terminal device is video reproduction program, then It can determine that current usage scenario is video playing scene.As another embodiment, terminal device can be according to user To the selection operation of target usage scenario in multiple usage scenarios, and target usage scenario is determined as to current usage scenario. The mode that terminal device obtains current usage scenario can be not as restriction.
Step S230: one or more keys in physical keyboard with control function are determined according to usage scenario, and really Control function corresponding to fixed each key.
In some embodiments, the corresponding relationship of each usage scenario and the key with control function and each make It can be pre-configured with key each in scene and the corresponding relationship of specific control function.Terminal device can be according to usage scenario With the corresponding relationship of the key with control function, determine in physical keyboard with current usage scenario to it is corresponding control by Key, and according to the corresponding relationship of key and specific control function each in each usage scenario, determine current usage scenario In control function corresponding to each key.
As an implementation, can store in terminal device scene identity, the key with control function and The relationship of the control function three of each key corresponds to table, as shown in table 1,
Table 1
As another embodiment, the key configurations file that can store each usage scenario in terminal device, matches Set file may include for describe corresponding to usage scenario with control function key information and for describing it is each The information of the corresponding control function of key.Terminal device can be read corresponding with the usage scenario according to current usage scenario Configuration file, to obtain key corresponding with current usage scenario in physical keyboard according to configuration file, and each The corresponding control function of key.
Certainly, the embodiment party of the control function of key corresponding with current usage scenario and each key is specifically determined Formula can be not as restriction.
Step S240: it according to control function corresponding to determining each key, obtains respectively corresponding with each key Virtual instruction content.
Step S250: according to the first relative tertiary location information, the between terminal device and each key is obtained respectively Two opposite spatial positional informations.
Step S260: based on the corresponding second relative tertiary location information of each key and virtual instruction content, generate with Determining each key distinguishes matched virtual instruction, and the display position virtually indicated is corresponding with matched key.
In the embodiment of the present application, step S240 to step S260 can be refering to the content of previous embodiment, herein no longer It repeats.
Display methods provided by the embodiments of the present application based on physical keyboard determines there is control according to current usage scenario The key of function processed, and determine control function corresponding to each key, then generation and the matched virtual instruction of these keys, To which user can operate the key with control function, realize needed for current usage scenario according to virtual instruction The control function asked, facilitates user to control by physical keyboard.
Referring to Fig. 5, another embodiment of the application provides a kind of display methods based on physical keyboard, can be applied to Terminal device in above-mentioned display system, this method may include:
Step S310: the selection instruction for being used for selection key, and one based on selection instruction selection physical keyboard are obtained Or multiple keys.
In some embodiments, when carrying out control function using physical keyboard, there is control function on physical keyboard Key, can be by user setting.Terminal device can show that the virtual key for selection key selects picture, then root According to the selection instruction that user inputs, one or more keys of physical keyboard are selected, by the key configurations selected for control The key of function.The input of selection instruction can be inputted by physical keyboard, can also be inputted according to other interactive devices, such as Mouse etc..
By the above-mentioned means, user can be according to a Man's Demands and hobby, and being arranged, there is the key of control function to carry out Control operation, so as to meet the needs of different user and hobby.For example, user 1 likes using in identical scene of game W, A, S and D carry out direction controlling, and user 2 likes carrying out direction controlling with 1,2,3 and 5, so as to meet the happiness of different user It is good.
Step S320: obtaining setting instruction, and control function corresponding to each key is arranged based on setting instruction.
In some embodiments, terminal device is pressed in selection physical keyboard with one or more of control function After key, then it can receive the setting instruction being configured to the control function of each key, and instruct according to setting, it will be each The corresponding control function of key is set as the control function with setting instructions match.
Terminal device then may be used after to control function corresponding to each key with control function to be configured To store the corresponding relationship of each key with control function and control function, so that physical keyboard is for realizing this When a little control functions, terminal device can determine that the key with control function and each key are corresponding according to the corresponding relationship Control function.
It should be noted that in step S310 and in step S320, it can be with one pressing with control function of every selection Key, is then arranged the control function of the key, also can choose all keys with control function and then is respectively set every The corresponding control function of a key, it is not limited here.
Step S330: the first relative tertiary location information between terminal device and physical keyboard, physical keyboard packet are obtained Include multiple keys.
Step S340: one or more keys in physical keyboard with control function are determined.
Step S350: it according to control function corresponding to determining each key, obtains respectively corresponding with each key Virtual instruction content.
Step S360: according to the first relative tertiary location information, the between terminal device and each key is obtained respectively Two opposite spatial positional informations.
Step S370: based on the corresponding second relative tertiary location information of each key and virtual instruction content, generate with Determining each key distinguishes matched virtual instruction, and the display position virtually indicated is corresponding with matched key.
In the embodiment of the present application, step S330 to step S370 can be refering to the content of previous embodiment, herein no longer It repeats.
Display methods provided by the embodiments of the present application based on physical keyboard has the key institute of control function by determining Corresponding control function, then generation and the matched virtual instruction of these keys, so that user can be according to virtual instruction, to tool There is the key of control function to be operated, realizes to the control function of virtual content, user is facilitated to interact with virtual content. In addition, with control function key and key corresponding to specific control function can by user's self-setting, therefore Meet the demand and hobby of user.
Referring to Fig. 6, the application further embodiment provides a kind of display methods based on physical keyboard, can be applied to Terminal device in above-mentioned display system, this method may include:
Step S400: the first relative tertiary location information between terminal device and physical keyboard, physical keyboard packet are obtained Include multiple keys.
Step S410: one or more keys in physical keyboard with control function are determined.
Step S420: it according to control function corresponding to determining each key, obtains respectively corresponding with each key Virtual instruction content.
Step S430: according to the first relative tertiary location information, the between terminal device and each key is obtained respectively Two opposite spatial positional informations.
Step S440: based on the corresponding second relative tertiary location information of each key and virtual instruction content, generate with Determining each key distinguishes matched virtual instruction, and the display position virtually indicated is corresponding with matched key.
In the embodiment of the present application, step S410 to step S440 can be refering to the content of previous embodiment, herein no longer It repeats.
Step S450: the first content data of virtual screen to be shown are obtained.
In the embodiment of the present application, terminal device can also show virtual screen to be shown.Void to be shown Quasi- picture can be operating system or unlatching using corresponding virtual screen.For example, virtual screen to be shown can be void Quasi- game picture；In another example virtual screen to be shown can be virtual video picture, then for example；Virtual screen to be shown It can be virtual document picture, the above virtual screen is only for example.
In some embodiments, the first content data of the available virtual screen to be shown of terminal device, first Content-data generates the virtual screen for terminal device, and first content data may include three for generating the virtual screen The particular content of dimension module data, first content data can be refering to the description for virtually indicating content in previous embodiment, herein It repeats no more.
Step S460: the first relative positional relationship between the first setting regions and physical keyboard, the first setting area are obtained Domain is that virtual screen needs to be superimposed on the overlap-add region in reality scene, the first setting regions be in physical keyboard region with Outside.
In some embodiments, the available virtual screen of terminal device needs to be superimposed on the superposition position in reality scene The first relative positional relationship between physical keyboard is set, is shown to generate virtual screen.Wherein, virtual screen can be shown Show in a region, therefore the available virtual screen of terminal device needs to be superimposed on the overlap-add region (first in reality scene Setting regions) and physical keyboard between the first relativeness.
In some embodiments, the first setting regions may be at outside physical keyboard region, thus virtual screen After the display will not be Chong Die with physical keyboard region, facilitate user while watching interior on virtual screen and physical keyboard Hold, for example, virtual screen can suspend, Overlapping display is in the front etc. of terminal device.As a kind of specific embodiment, first Setting regions can be with other than physical keyboard region and corresponding with the wherein one side edge region of physical keyboard, wherein side Edge region refers to the periphery region of physical keyboard, for instance in other than physical keyboard region and and first side Edge region is adjacent.
In some embodiments, the first setting regions can also be preset with physical keyboard region at default angle The specific size of angle can not as restriction, for example, default angle can be 45 °~70 °, in another example, can for 65 °~ 90 ° etc..Virtual screen is watched to which virtual screen can be convenient user after the display, promotes the display effect of virtual screen.
Under embodiment of above, the virtual screen of display can be enable to be incorporated as the use of computer with physical keyboard, That is the display screen of virtual screen analogue computer, the keyboard of physical keyboard analogue computer, user only need to carry physical keyboard and The use of computer can be realized in terminal device.
Step S470: raw according to the first relative tertiary location information, the first relative positional relationship and first content data It is shown at virtual screen and by virtual screen.
In the embodiment of the present application, terminal device can be according to the first relative tertiary location information and the first relative position Relationship obtains the rendering position of virtual screen, then according to rendering position and first content data render virtual screen after, will Virtual screen is shown.Terminal device generates virtual screen and shows virtual screen, can be refering to previous embodiment Middle generation is virtual to be indicated and will virtually indicate the description content shown.
It should be noted that terminal device generates virtual instruction and will virtually indicate the process shown, set with terminal The sequencing of the standby process for generating virtual screen and showing virtual screen can be not as restriction.
Step S480: it when being triggered in one or more keys there are key, obtains corresponding to the key being triggered Display adjustment function.
In some embodiments, the corresponding control function of key in physical keyboard with control function can be display Adjust function.Wherein, display adjustment function can be to carry out display adjustment to virtual screen, for example, comparing virtual screen The control functions such as example amplification, scale smaller, position movement, it is not limited here.
In some embodiments, physical keyboard can detecte the key being triggered, and be generated according to the key being triggered Operation data is sent to terminal device by operation data.Terminal device can receive the operation data of physical keyboard generation, determine The key being triggered in physical keyboard.If the operation data that terminal device is sent according to physical keyboard, determine that there is control When being triggered in the key of function there are key, then the corresponding display adjustment function of the key being triggered can be determined, so as to right Virtual screen carries out corresponding display adjustment.
Step S490: the corresponding display of display adjustment function is carried out to virtual screen and is adjusted.
In some embodiments, terminal device display adjustment function corresponding to the key for determining to be triggered it Afterwards, then it can carry out adjusting that function is corresponding shows adjustment to virtual screen with the display.For example, the key being triggered is corresponding Display adjustment function is that when carrying out ratio enlargement to virtual screen, then can be triggered according to the key and once need to carry out ratio Virtual screen is carried out the ratio enlargement of the scale by the scale of amplification.
For example, physical keyboard can be used for the scene of documents editing, please referring also to Fig. 7 and Fig. 8, terminal device are shown Virtual screen can be virtual document picture 303, in this scenario, key A in physical keyboard 200, which has, draws document Face 303 carries out the function of ratio enlargement, and key D has the function that scale smaller is carried out to document picture 303, corresponding with key A Enlarging function icon display position it is corresponding with key A, the display position of reduction capability icon corresponding with key D and key D is corresponding, and when key D is triggered, then the scale of document picture 303 is reduced.
The display methods of dummy keyboard provided by the embodiments of the present application, terminal device can be according to pressing with control function The corresponding specific control function of key generates and has the matched virtual instruction of key of control function, can also generate to be shown Virtual screen and virtual screen is shown, user can show virtual screen to having according to virtual instruction The key of adjustment function is operated, and realizes that the display to virtual screen adjusts, and user is facilitated to draw by physical keyboard with virtual Face interacts.
Referring to Fig. 9, the application a still further embodiment provides a kind of display methods based on physical keyboard, can be applied to Terminal device in above-mentioned display system, this method may include:
Step S500: the first relative tertiary location information between terminal device and physical keyboard, physical keyboard packet are obtained Include multiple keys.
Step S510: one or more keys in physical keyboard with control function are determined.
Step S520: it according to control function corresponding to determining each key, obtains respectively corresponding with each key Virtual instruction content.
Step S530: according to the first relative tertiary location information, the between terminal device and each key is obtained respectively Two opposite spatial positional informations.
Step S540: based on the corresponding second relative tertiary location information of each key and virtual instruction content, generate with Determining each key distinguishes matched virtual instruction, and the display position virtually indicated is corresponding with matched key.
In the embodiment of the present application, step S510 to step S540 can be refering to the content of previous embodiment, herein no longer It repeats.
Step S550: the second content-data of virtual input frame to be shown is obtained.
In the embodiment of the present application, terminal device can also show that virtual input frame can be to virtual input frame For showing the Text Entry of the content of text such as input character, candidate word in text input scene.For example, virtual input frame can For candidate word corresponding to display input method；In another example virtual input frame is displayed for the text in chat interface The content of input area, the above virtual input frame are only for example.
In some embodiments, the second content-data of the available virtual input frame to be shown of terminal device, the Two content-datas generate the virtual input frame for terminal device, and the particular contents of the second content-data first content data can be with Refering to the description for virtually indicating content in previous embodiment, details are not described herein.
Step S560: the second relative positional relationship between the second setting regions and physical keyboard, the second setting area are obtained Domain is that virtual input frame needs to be superimposed on the overlap-add region in reality scene, and the second setting regions is located at the margin location of physical keyboard Set region.
In the embodiment of the present application, unlike the first setting regions in previous embodiment, the second setting regions is located at The marginal position region of physical keyboard, wherein the marginal position of physical keyboard is the periphery position of physical keyboard.Example Such as, the second setting regions can be the region of the top edge of fitting physical keyboard relative termination equipment, the top edge of physical keyboard It may refer to the long edge that key (is exited) close to Esc.
Step S570: raw according to the first relative tertiary location information, the second relative positional relationship and the second content-data It is shown at virtual input frame and by virtual input frame.
In the embodiment of the present application, step S570 can be refering to the description content of step S470 in previous embodiment, herein It repeats no more.
For example, as shown in Figure 10, terminal device can be generated virtual input frame 304 and show virtual input frame 304 Show, user can see virtual input frame 304 and physical keyboard 200 by the display eyeglass of head-wearing display device, observable Upper marginal position fitting, the visual effect of physical keyboard 200 is adsorbed on to the virtual input frame 304 of user, convenient for user with object Reason keyboard 200 carries out to see physical keyboard 200 and virtual input frame 304 simultaneously when text input, and user is facilitated to carry out text Word input.
It should be noted that terminal device generates virtual instruction and will virtually indicate the process shown, set with terminal The sequencing of the standby process for generating virtual input frame and showing virtual input frame can be not as restriction.
In some embodiments, control function corresponding to the key in physical keyboard with control function can be aobvious Show adjustment function, terminal device can carry out virtual input frame according to the key being triggered in the key with control function Corresponding display adjustment.
Step S580: it obtains in physical keyboard for inputting the operation data of the key of text.
In some embodiments, it in physical keyboard may include for inputting text key, for inputting text Specific key can be not as restriction.Terminal device can receive physical keyboard and detect the key being triggered, and the use generated In the operation data of the key of input text.
In some embodiments, can be identical as the key part with control function for inputting the key of text, When needing to carry out the identical key in this part and carrying out input text, the mode of this part key can be switched over, so that This part key is to input the mode of text from pattern switching for controlling the function.
Step S590: the virtual text information for showing in virtual input frame is generated according to operation data, and will be empty Quasi- text information is shown.
In the embodiment of the present application, terminal device can be obtained and be used according to the operation data of the key for inputting text The content of text of family input, and the virtual text information for showing in virtual input frame is generated according to content of text, it will be empty Quasi- text information is shown.
In some embodiments, terminal device generates virtual input frame and by virtual input frame in the embodiment of the present application Embodiment, the embodiment party that virtual screen can also be generated with terminal device in previous embodiment and show virtual screen Formula combines, i.e., terminal device had both shown the virtual screen in previous embodiment, also showed virtual input frame.For example, in aforementioned reality It applies terminal device in example and shows in the scene of virtual document content that terminal device can also also show virtual input frame, and virtual Input frame is displayed for the corresponding content of input method (such as candidate word), so that user can be simultaneously when inputting content of text See the key and the corresponding content of input method on physical keyboard, user is facilitated to carry out input text to virtual document content In.
In some embodiments, terminal device can also be generated for Overlapping display in the virtual decoration of physical keyboard Hold, and virtual decorative content is shown, to reach the decorative effect to physical keyboard, promotes the vision effect of physical keyboard Fruit.In a kind of specific embodiment, the flashing virtual content with different color is can be generated in terminal device, and flashing is virtual The display position of content can be corresponding with the position of the surrounding of physical keyboard, and flashing virtual content is shown, realizing will Virtual fluorescence content Overlapping display is in the surrounding of physical keyboard.The service condition different for physical keyboard, can show difference Virtual decorative content, it is stackable aobvious on each key for example, can show the flashing virtual content with different color when starting It is shown with halation result, when the button is pressed, the color etc. of halation result can be changed, but not limited to this.As another specific Embodiment, terminal device can be generated with the keycap content of each key (text, pattern for being printed on keycap etc.) institute it is right The virtual keycap content answered, the display position of virtual keycap content is corresponding with key, and virtual keycap content is shown, real Now by virtual keycap content Overlapping display in key.Certainly, both the above specific embodiment can also combine, and press to realize Key surrounding Overlapping display has fluorescence content, and Overlapping display and is reached to physical keyboard by virtual keycap content on key Decorative effect promotes the visual experience of user.
Display methods provided by the embodiments of the present application based on physical keyboard, terminal device can be according to control functions The corresponding specific control function of key, generate with control function the matched virtual instruction of key, facilitate user to understand The corresponding control function of key.In addition, terminal device can also generate virtual input frame to be shown and by virtual input frame into Row display, and virtual input frame is superimposed on the position in reality scene and is located in physical keyboard region and close to secondary or physical bond The marginal position region of disk, can see simultaneously convenient for user when carrying out text input with physical keyboard physical keyboard and Virtual input frame facilitates user and carries out text input.
Referring to Figure 11, it illustrates a kind of structural frames of the display device 400 based on physical keyboard provided by the present application Figure.The display device 400 based on physical keyboard is applied to the terminal device in above-mentioned display system.This is based on physical keyboard Display device 400 includes: that first position obtains module 410, key determining module 420, content acquisition module 430, the second position Obtain module 440 and content generating module 450.Wherein, first position obtains module 410 for obtaining terminal device and physics The first relative tertiary location information between keyboard, physical keyboard include multiple keys；Key determining module 420 is for determining object Manage one or more keys in keyboard with control function；Content obtains module 430 and is used for according to determining each key institute Corresponding control function obtains virtual instruction content corresponding with each key respectively；The second position obtains module 440 and is used for root According to the first relative tertiary location information, the second relative tertiary location information between terminal device and each key is obtained respectively； Content generating module 450 is used to generate based on the corresponding second relative tertiary location information of each key and virtual instruction content Matched virtual instruction is distinguished with determining each key, and the display position virtually indicated is corresponding with matched key.
In some embodiments, key determining module 420 may include scene determination unit and key acquiring unit. The usage scenario current in determination is applied alone in scene determination；Key acquiring unit is used to determine in physical keyboard have according to usage scenario There are one or more keys of control function, and determines control function corresponding to each key.
In some embodiments, should display device 400 based on physical keyboard can also include instruction acquisition module with And function setup module.Instruction acquisition module is used to obtain the first relative tertiary location between terminal device and physical keyboard Before information, the selection instruction for being used for selection key is obtained, and the one or more based on selection instruction selection physical keyboard is pressed Key；Control function corresponding to each key is arranged for obtaining setting instruction, based on setting instruction in function setup module.
In some embodiments, being somebody's turn to do the display device 400 based on physical keyboard can also include the first data acquisition mould Block, first position Relation acquisition module and picture generation module.First data acquisition module is used for；First position Relation acquisition Module is used for；Picture generation module is used for according in the first relative tertiary location information, the first relative positional relationship and first Hold data, generate virtual screen and shows virtual screen.
In this embodiment, should display device 400 based on physical keyboard can also include: detection trigger module and Picture adjusts module.Detection trigger module is used for when being triggered in one or more keys there are key, is obtained and is triggered Display corresponding to key adjusts function；Picture adjustment module is used to carry out virtual screen the corresponding display of display adjustment function Adjustment.
In some embodiments, being somebody's turn to do the display device 400 based on physical keyboard can also include: the second data acquisition mould Block, second position Relation acquisition module and input frame generation module.Second data acquisition module is for obtaining void to be shown Second content-data of quasi- input frame；Second position Relation acquisition module is for obtaining between the second setting regions and physical keyboard The second relative positional relationship, the second setting regions is that virtual input frame needs to be superimposed on the overlap-add region in reality scene, the Two setting regions are located at the marginal position region in physical keyboard region and close to physical keyboard；Input frame generates Module is used to be generated virtual defeated according to the first relative tertiary location information, the second relative positional relationship and the second content-data Enter frame and shows virtual input frame.
In this embodiment, being somebody's turn to do the display device 400 based on physical keyboard can also include: that operation data obtains module And input control module.Operation data obtains module and is used to obtain in physical keyboard for inputting the operand of the key of text According to；Input control module is used to generate the virtual text information for showing in virtual input frame according to operation data, and will Virtual text information is shown.
It is apparent to those skilled in the art that for convenience and simplicity of description, foregoing description device and The specific work process of module, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, the mutual coupling of module can be electrical property, mechanical or other The coupling of form.
It, can also be in addition, can integrate in a processing module in each functional module in each embodiment of the application It is that modules physically exist alone, can also be integrated in two or more modules in a module.Above-mentioned integrated mould Block both can take the form of hardware realization, can also be realized in the form of software function module.
To sum up, scheme provided by the present application, by obtaining the first space position between terminal device and physical keyboard Confidence breath, physical keyboard includes multiple keys, one or more keys in physical keyboard with control function is determined, according to true Control function corresponding to fixed each key obtains virtual instruction content corresponding with each key, according to the first phase respectively To spatial positional information, the second relative tertiary location information between terminal device and each key is obtained respectively, based on each The corresponding second relative tertiary location information of key and virtual instruction content, generate and distinguish matched void with each key determined Quasi- instruction, the display position virtually indicated is corresponding with matched key, so that user can be according to virtual instruction, to control The key of function is operated, and realizes to the control function of virtual content, user is facilitated to interact with virtual content.
Figure 12 is please referred to, it illustrates a kind of structural block diagrams of terminal device provided by the embodiments of the present application.The terminal is set Standby 100 can be smart phone, tablet computer, wear the terminal device for showing that equipment etc. can run application program.The application In terminal device 100 may include one or more such as lower component: processor 110, memory 120, image collecting device 130, one or more computer programs can be stored in memory 120, computer program can be configured to by one or more Processor 110 executes, for executing the method as described in preceding method embodiment.
Processor 110 may include one or more processing core.Processor 110 is whole using various interfaces and connection Various pieces in a terminal device 100, by run or execute the instruction being stored in memory 120, program, code set or Instruction set, and the data being stored in memory 120 are called, execute the various functions and processing data of terminal device 100.It can Selection of land, processor 110 can use Digital Signal Processing (Digital Signal Processing, DSP), field-programmable Gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA) at least one of example, in hardware realize.Processor 110 can integrating central processor (Central Processing Unit, CPU), in image processor (Graphics Processing Unit, GPU) and modem etc. One or more of combinations.Wherein, the main processing operation system of CPU, user interface and application program etc.；GPU is for being responsible for Show the rendering and drafting of content；Modem is for handling wireless communication.It is understood that above-mentioned modem It can not be integrated into processor 110, be realized separately through one piece of communication chip.
Memory 120 may include random access memory (Random Access Memory, RAM), also may include read-only Memory (Read-Only Memory).Memory 120 can be used for store instruction, program, code, code set or instruction set.It deposits Reservoir 120 may include storing program area and storage data area, wherein the finger that storing program area can store for realizing operating system Enable, for realizing at least one function instruction (such as touch function, sound-playing function, image player function etc.), be used for Realize the instruction etc. of following each embodiments of the method.Storage data area can also store the number that terminal 100 is created in use According to etc..
In the embodiment of the present application, image collecting device 130 is used to acquire the image of marker.Image collecting device 130 It can be infrared camera, be also possible to colour imagery shot, specific camera types are not intended as in the embodiment of the present application It limits.
In one embodiment, terminal device may also include following one or more components: display module, optics module, Communication module and power supply.Display module may include display control unit, after display control unit is for receiving processor rendering Virtual content display image, which is shown and is projected in optics module, allows users to pass through optical mode Group watches virtual content.Wherein, display module can be display screen or grenade instrumentation etc., for showing image.Optics module Off-axis optical system or waveguide optical system can be used, the display image that display module is shown can be thrown after optics module It is incident upon the eyes of user.The display image of display module projection can be seen by optics module by user.In some embodiments, User can also observe actual environment through optics module, experience virtual content and the superimposed visual effect of actual environment. Communication module can be the modules such as bluetooth, WiFi, ZigBee, and terminal device can be connect by communication module with inter device communication, To carry out the interaction of information and instruction.Power supply can be powered for entire terminal device, guarantee terminal device all parts It operates normally.
Figure 13 is please referred to, it illustrates a kind of structural frames of computer readable storage medium provided by the embodiments of the present application Figure.Program code is stored in the computer-readable medium 800, program code can be called by processor and execute above method implementation Method described in example.
Computer readable storage medium 800 can be such as flash memory, EEPROM (electrically erasable programmable read-only memory), The electronic memory of EPROM, hard disk or ROM etc.Optionally, computer readable storage medium 800 includes non-volatile meter Calculation machine readable medium (non-transitory computer-readable storage medium).Computer-readable storage Medium 800 has the memory space for the program code 810 for executing any method and step in the above method.These program codes can With from reading or be written in one or more computer program product in this one or more computer program product. Program code 810 can for example be compressed in a suitable form.
Finally, it should be noted that above embodiments are only to illustrate the technical solution of the application, rather than its limitations；Although The application is described in detail with reference to the foregoing embodiments, those skilled in the art are when understanding: it still can be with It modifies the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features；And These are modified or replaceed, do not drive corresponding technical solution essence be detached from each embodiment technical solution of the application spirit and Range.

Claims (10)

1. a kind of display methods based on physical keyboard, which is characterized in that the described method includes:
Obtain the first relative tertiary location information between terminal device and physical keyboard, the physical keyboard include it is multiple by Key；
Determine one or more keys in the physical keyboard with control function；
According to control function corresponding to determining each key, obtained in virtual instruction corresponding with each key respectively Hold；
According to the first relative tertiary location information, second between the terminal device and each key is obtained respectively Relative tertiary location information；
Based on the corresponding second relative tertiary location information of each key and virtual instruction content, generate with it is determining each Key distinguishes matched virtual instruction, and the display position virtually indicated is corresponding with matched key.
2. the method according to claim 1, wherein having control function in the determination physical keyboard One or more keys, comprising:
Determine current usage scenario；
One or more keys in the physical keyboard with control function are determined according to the usage scenario, and are determined each Control function corresponding to key.
3. the method according to claim 1, wherein the between the acquisition terminal device and physical keyboard Before one relative tertiary location information, the method also includes:
The selection instruction for being used for selection key is obtained, and the one or more based on selection instruction selection physical keyboard is pressed Key；
Setting instruction is obtained, control function corresponding to each key is arranged based on setting instruction.
4. method according to claim 1-3, which is characterized in that the method also includes:
Obtain the first content data of virtual screen to be shown；
The first relative positional relationship between the first setting regions and the physical keyboard is obtained, first setting regions is institute It states virtual screen and needs to be superimposed on overlap-add region in reality scene, first setting regions is in where the physical keyboard Other than region；
According to the first relative tertiary location information, the first relative positional relationship and the first content data, institute is generated It states virtual screen and shows the virtual screen.
5. according to the method described in claim 4, it is characterized in that, the method also includes:
When being triggered in one or more of keys there are key, the adjustment function of display corresponding to the key being triggered is obtained Energy；
The corresponding display adjustment of the display adjustment function is carried out to the virtual screen.
6. method according to claim 1-3, which is characterized in that the method also includes:
Obtain the second content-data of virtual input frame to be shown；
The second relative positional relationship between the second setting regions and the physical keyboard is obtained, second setting regions is institute It states virtual input frame and needs to be superimposed on overlap-add region in reality scene, second setting regions is located at the physical keyboard Marginal position region；
It is raw according to the first relative tertiary location information, second relative positional relationship and second content-data It is shown at the virtual input frame and by the virtual input frame.
7. according to the method described in claim 6, it is characterized in that, the method also includes:
It obtains in the physical keyboard for inputting the operation data of the key of text；
The virtual text information for showing in the virtual input frame is generated according to the operation data, and will be described virtual Text information is shown.
8. a kind of display device based on physical keyboard, which is characterized in that described device include: first position obtain module, by Key determining module, content obtain module, the second position obtains module and content generating module, wherein
The first position obtains module and is used to obtain the first relative tertiary location information between terminal device and physical keyboard, The physical keyboard includes multiple keys；
The key determining module is used to determine one or more keys in the physical keyboard with control function；
The content obtains module and is used for the control function according to corresponding to determining each key, obtains respectively and described each The corresponding virtual instruction content of key；
The second position obtains module and is used to obtain the terminal device respectively according to the first relative tertiary location information The second relative tertiary location information between each key；
The content generating module is used for based on each corresponding second relative tertiary location information of key and virtual instruction Content generates and distinguishes matched virtual instruction with each key determined, and the display position virtually indicated is pressed with matched Key is corresponding.
9. a kind of terminal device characterized by comprising
One or more processors；
Memory；
One or more application program, wherein one or more of application programs are stored in the memory and are configured To be executed by one or more of processors, one or more of programs are configured to carry out as claim 1-7 is any Method described in.
10. a kind of computer-readable storage medium, which is characterized in that be stored with journey in the computer-readable storage medium Sequence code, said program code can be called by processor and execute the method according to claim 1 to 7.
CN201910684470.7A 2019-07-26 2019-07-26 Display methods, device, terminal device and storage medium based on physical keyboard Pending CN110442245A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910684470.7A CN110442245A (en) 2019-07-26 2019-07-26 Display methods, device, terminal device and storage medium based on physical keyboard

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910684470.7A CN110442245A (en) 2019-07-26 2019-07-26 Display methods, device, terminal device and storage medium based on physical keyboard

Publications (1)

Publication Number Publication Date
CN110442245A true CN110442245A (en) 2019-11-12

Family

ID=68431814

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910684470.7A Pending CN110442245A (en) 2019-07-26 2019-07-26 Display methods, device, terminal device and storage medium based on physical keyboard

Country Status (1)

Country Link
CN (1) CN110442245A (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111158825A (en) * 2019-12-27 2020-05-15 联想(北京)有限公司 Control method and system
CN114706489A (en) * 2022-02-28 2022-07-05 北京所思信息科技有限责任公司 Virtual method, device, equipment and storage medium of input equipment
US20220253130A1 (en) * 2021-02-08 2022-08-11 Multinarity Ltd Keyboard sensor for augmenting smart glasses sensor
US11475650B2 (en) 2021-02-08 2022-10-18 Multinarity Ltd Environmentally adaptive extended reality display system
US11480791B2 (en) 2021-02-08 2022-10-25 Multinarity Ltd Virtual content sharing across smart glasses
CN115633119A (en) * 2022-09-24 2023-01-20 深圳市般若造物科技有限公司 Multifunctional keyboard control method, multifunctional keyboard, control device and storage medium
WO2023049248A1 (en) * 2021-09-24 2023-03-30 Apple Inc. Method and device for facilitating interactions with a peripheral device
US11748056B2 (en) 2021-07-28 2023-09-05 Sightful Computers Ltd Tying a virtual speaker to a physical space
US11846981B2 (en) 2022-01-25 2023-12-19 Sightful Computers Ltd Extracting video conference participants to extended reality environment
US11948263B1 (en) 2023-03-14 2024-04-02 Sightful Computers Ltd Recording the complete physical and extended reality environments of a user

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140035819A1 (en) * 2012-08-03 2014-02-06 Research In Motion Limited Method and Apparatus Pertaining to an Augmented-Reality Keyboard
CN106383652A (en) * 2016-08-31 2017-02-08 北京极维客科技有限公司 Virtual input method and system apparatus

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140035819A1 (en) * 2012-08-03 2014-02-06 Research In Motion Limited Method and Apparatus Pertaining to an Augmented-Reality Keyboard
CN106383652A (en) * 2016-08-31 2017-02-08 北京极维客科技有限公司 Virtual input method and system apparatus

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111158825B (en) * 2019-12-27 2022-08-19 联想(北京)有限公司 Control method and system
CN111158825A (en) * 2019-12-27 2020-05-15 联想(北京)有限公司 Control method and system
US11609607B2 (en) 2021-02-08 2023-03-21 Multinarity Ltd Evolving docking based on detected keyboard positions
US11514656B2 (en) 2021-02-08 2022-11-29 Multinarity Ltd Dual mode control of virtual objects in 3D space
US11620799B2 (en) 2021-02-08 2023-04-04 Multinarity Ltd Gesture interaction with invisible virtual objects
US11480791B2 (en) 2021-02-08 2022-10-25 Multinarity Ltd Virtual content sharing across smart glasses
US11481963B2 (en) 2021-02-08 2022-10-25 Multinarity Ltd Virtual display changes based on positions of viewers
US11496571B2 (en) 2021-02-08 2022-11-08 Multinarity Ltd Systems and methods for moving content between virtual and physical displays
US11516297B2 (en) 2021-02-08 2022-11-29 Multinarity Ltd Location-based virtual content placement restrictions
US11927986B2 (en) 2021-02-08 2024-03-12 Sightful Computers Ltd. Integrated computational interface device with holder for wearable extended reality appliance
US11924283B2 (en) 2021-02-08 2024-03-05 Multinarity Ltd Moving content between virtual and physical displays
US11561579B2 (en) 2021-02-08 2023-01-24 Multinarity Ltd Integrated computational interface device with holder for wearable extended reality appliance
US11567535B2 (en) 2021-02-08 2023-01-31 Multinarity Ltd Temperature-controlled wearable extended reality appliance
US11574451B2 (en) 2021-02-08 2023-02-07 Multinarity Ltd Controlling 3D positions in relation to multiple virtual planes
US11882189B2 (en) 2021-02-08 2024-01-23 Sightful Computers Ltd Color-sensitive virtual markings of objects
US11582312B2 (en) 2021-02-08 2023-02-14 Multinarity Ltd Color-sensitive virtual markings of objects
US11580711B2 (en) 2021-02-08 2023-02-14 Multinarity Ltd Systems and methods for controlling virtual scene perspective via physical touch input
US11588897B2 (en) 2021-02-08 2023-02-21 Multinarity Ltd Simulating user interactions over shared content
US11592872B2 (en) 2021-02-08 2023-02-28 Multinarity Ltd Systems and methods for configuring displays based on paired keyboard
US11592871B2 (en) 2021-02-08 2023-02-28 Multinarity Ltd Systems and methods for extending working display beyond screen edges
US11601580B2 (en) 2021-02-08 2023-03-07 Multinarity Ltd Keyboard cover with integrated camera
US11599148B2 (en) 2021-02-08 2023-03-07 Multinarity Ltd Keyboard with touch sensors dedicated for virtual keys
US11574452B2 (en) 2021-02-08 2023-02-07 Multinarity Ltd Systems and methods for controlling cursor behavior
US20220253130A1 (en) * 2021-02-08 2022-08-11 Multinarity Ltd Keyboard sensor for augmenting smart glasses sensor
US11475650B2 (en) 2021-02-08 2022-10-18 Multinarity Ltd Environmentally adaptive extended reality display system
US11627172B2 (en) 2021-02-08 2023-04-11 Multinarity Ltd Systems and methods for virtual whiteboards
US11863311B2 (en) 2021-02-08 2024-01-02 Sightful Computers Ltd Systems and methods for virtual whiteboards
US11650626B2 (en) 2021-02-08 2023-05-16 Multinarity Ltd Systems and methods for extending a keyboard to a surrounding surface using a wearable extended reality appliance
US11811876B2 (en) 2021-02-08 2023-11-07 Sightful Computers Ltd Virtual display changes based on positions of viewers
US11797051B2 (en) 2021-02-08 2023-10-24 Multinarity Ltd Keyboard sensor for augmenting smart glasses sensor
US11861061B2 (en) 2021-07-28 2024-01-02 Sightful Computers Ltd Virtual sharing of physical notebook
US11748056B2 (en) 2021-07-28 2023-09-05 Sightful Computers Ltd Tying a virtual speaker to a physical space
US11816256B2 (en) 2021-07-28 2023-11-14 Multinarity Ltd. Interpreting commands in extended reality environments based on distances from physical input devices
US11829524B2 (en) 2021-07-28 2023-11-28 Multinarity Ltd. Moving content between a virtual display and an extended reality environment
US11809213B2 (en) 2021-07-28 2023-11-07 Multinarity Ltd Controlling duty cycle in wearable extended reality appliances
WO2023049248A1 (en) * 2021-09-24 2023-03-30 Apple Inc. Method and device for facilitating interactions with a peripheral device
US11877203B2 (en) 2022-01-25 2024-01-16 Sightful Computers Ltd Controlled exposure to location-based virtual content
US11846981B2 (en) 2022-01-25 2023-12-19 Sightful Computers Ltd Extracting video conference participants to extended reality environment
US11941149B2 (en) 2022-01-25 2024-03-26 Sightful Computers Ltd Positioning participants of an extended reality conference
CN114706489B (en) * 2022-02-28 2023-04-25 北京所思信息科技有限责任公司 Virtual method, device, equipment and storage medium of input equipment
CN114706489A (en) * 2022-02-28 2022-07-05 北京所思信息科技有限责任公司 Virtual method, device, equipment and storage medium of input equipment
CN115633119B (en) * 2022-09-24 2024-02-20 深圳市般若造物科技有限公司 Multifunctional keyboard control method, multifunctional keyboard, control device and storage medium
CN115633119A (en) * 2022-09-24 2023-01-20 深圳市般若造物科技有限公司 Multifunctional keyboard control method, multifunctional keyboard, control device and storage medium
US11948263B1 (en) 2023-03-14 2024-04-02 Sightful Computers Ltd Recording the complete physical and extended reality environments of a user

Similar Documents

Publication Publication Date Title
CN110442245A (en) Display methods, device, terminal device and storage medium based on physical keyboard
CN106575354B (en) Virtualization of tangible interface objects
CN107852443B (en) Information processing apparatus, information processing method, and program
EP2521097B1 (en) System and Method of Input Processing for Augmented Reality
CN107851299B (en) Information processing apparatus, information processing method, and program
CN110163942B (en) Image data processing method and device
CN111158469A (en) Visual angle switching method and device, terminal equipment and storage medium
US20220245859A1 (en) Data processing method and electronic device
CN110456907A (en) Control method, device, terminal device and the storage medium of virtual screen
CN108038726B (en) Article display method and device
JP2010134738A (en) Terminal apparatus, display control method, and display control program
CN111701238A (en) Virtual picture volume display method, device, equipment and storage medium
KR20090001667A (en) Apparatus and method for embodying contents using augmented reality
US20200143600A1 (en) Augmented reality method, system and terminal device of displaying and controlling virtual content via interaction device
CN111383345B (en) Virtual content display method and device, terminal equipment and storage medium
US20240078703A1 (en) Personalized scene image processing method, apparatus and storage medium
US20130057574A1 (en) Storage medium recorded with program, information processing apparatus, information processing system, and information processing method
CN112581571B (en) Control method and device for virtual image model, electronic equipment and storage medium
US8643679B2 (en) Storage medium storing image conversion program and image conversion apparatus
CN111913674A (en) Virtual content display method, device, system, terminal equipment and storage medium
KR20150079387A (en) Illuminating a Virtual Environment With Camera Light Data
CN111161396A (en) Virtual content control method and device, terminal equipment and storage medium
CN111913560A (en) Virtual content display method, device, system, terminal equipment and storage medium
KR101983233B1 (en) Augmented reality image display system and method using depth map
CN111913639A (en) Virtual content interaction method, device, system, terminal equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination