CN107463252B - Virtual model operation method, device, storage medium and MR head display for MR head display - Google Patents

Abstract

The invention relates to the technical field of virtual operation, and provides a virtual model operation method and device of an MR head display, a storage medium and the MR head display. The virtual model operation method of the MR head display comprises the following steps: detecting a currently input gesture; if the detected gesture input currently selects a preset control model, determining the whole target virtual model as an operation object operated by the gesture input currently, wherein the control model is a virtual model which corresponds to the target virtual model and is arranged close to the target virtual model in a preset mode. In the process, even if the target virtual model is in a detached state, a user can input a gesture to select the control model corresponding to the target virtual model, and then the whole target virtual model is determined as an operation object operated by the currently input gesture, so that the whole target virtual model can be operated and controlled, and the convenience of user operation is greatly improved.

Description

Virtual model operation method, device, storage medium and MR head display for MR head display

technical field

The invention relates to the technical field of virtual operation, and in particular, to a virtual model operation method, device, storage medium and MR head display of an MR head display.

Background technique

The MR head-mounted display is a mixed reality head-mounted display device. Users wearing the MR head-mounted display can watch the virtual model superimposed on the real image, and can also interact with these virtual models. However, the operation commands for virtual models in the current MR head-mounted display are very limited, and some operations with complex functions cannot be realized. For example, for a virtual model that can be dissected or disassembled, after the virtual model is disassembled, only each disassembled component can be operated independently, and the entire virtual model (ie, all disassembled parts) cannot be operated at the same time, which brings inconvenience to the user. .

SUMMARY OF THE INVENTION

In view of this, embodiments of the present invention provide a virtual model operation method, device, storage medium and MR head display for an MR head-mounted display, which can simultaneously operate the entire virtual model when the virtual model is disassembled.

A first aspect of the embodiments of the present invention provides a method for operating a virtual model of an MR head-mounted display, including:

Detect the currently input gesture;

If the detected currently input gesture selects a preset manipulation model, the whole of the target virtual model is determined as the operation object operated by the currently input gesture, and the manipulation model is corresponding to the target virtual model, And the virtual model placed close to the target virtual model in a preset manner.

A second aspect of the embodiments of the present invention provides an apparatus for operating a virtual model of an MR head-mounted display, including:

Gesture detection module, used to detect the currently input gesture;

The model selection module is used to determine the entirety of the target virtual model as the operation object operated by the currently input gesture if the detected currently input gesture selects a preset manipulation model, and the manipulation model is the same as the The target virtual model corresponds to a virtual model placed close to the target virtual model in a preset manner.

A third aspect of the embodiments of the present invention provides an MR head-mounted display, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program When implementing the steps of the virtual model operating method provided by the first aspect of the embodiment of the present invention.

A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the implementation as provided in the first aspect of the embodiments of the present invention The steps of the virtual model operation method.

In this embodiment of the present invention, the currently input gesture is detected; if the detected currently input gesture selects a preset manipulation model, the entirety of the target virtual model is determined as the operation object operated by the currently input gesture, The manipulation model is a virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner. In the above process, even if the target virtual model is in a disassembled state, the user can still input a gesture to select a manipulation model corresponding to the target virtual model, and then determine the entirety of the target virtual model as the current input The operation object operated by the gesture, so that the whole of the target virtual model can be operated and controlled, which greatly improves the convenience of the user's operation.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a flowchart of a first embodiment of a method for operating a virtual model of an MR head-mounted display provided by an embodiment of the present invention;

2 is a flowchart of a second embodiment of a method for operating a virtual model of an MR head-mounted display provided by an embodiment of the present invention;

3 is a flowchart of a third embodiment of a method for operating a virtual model of an MR head-mounted display provided by an embodiment of the present invention;

4 is a structural diagram of an embodiment of an apparatus for operating a virtual model of an MR head-mounted display provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram of an MR head-mounted display provided by an embodiment of the present invention.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

Embodiments of the present invention provide a virtual model operation method, device, storage medium and MR head display for an MR head-mounted display, which can simultaneously operate the entire virtual model when the virtual model is disassembled.

In various embodiments of the present invention, the MR head-mounted display can be any type of mixed reality head-mounted display, such as the most representative HoloLens.

Referring to FIG. 1 , a first embodiment of a method for operating a virtual model of an MR head-mounted display in an embodiment of the present invention includes:

101. Detect the currently input gesture;

First, detect the gesture currently input by the user. Various gestures currently input by the user can be detected by the gesture detection device of the MR head-mounted display, and some specified gestures are defined as operation gestures with corresponding functions. For example, the Bloom and Air tap gestures in HoloLens, the Bloom gesture is used to open the menu interface, and the Air tap gesture is used to select an operable object. It should be noted that, the gesture in the embodiment of the present invention may be any user-defined gesture, and is not limited to the above-mentioned Bloom and Air tap gestures.

102. If the detected currently input gesture selects a preset manipulation model, determine the entirety of the target virtual model as the operation object operated by the currently input gesture.

After detecting the currently input gesture, it is determined whether the gesture is a gesture of the selected preset manipulation model. If the detected currently input gesture selects a preset manipulation model, the entirety of the target virtual model is determined as the operation object operated by the currently input gesture. If the detected currently input gesture is not the gesture of the selected manipulation model, it is processed according to the operation rule corresponding to the gesture.

The target virtual model is a virtual model that the user wishes to operate in the virtual environment provided by the MR head-mounted display, which may be any type of detachable or non-detachable virtual model, such as animal models, plant models, or car models. The manipulation model is essentially a virtual model, which corresponds to the target virtual model and is placed close to the target virtual model in a preset manner. In the MR virtual environment, the manipulation model is placed close to the target virtual model, the user can easily distinguish the manipulation model from the background of the MR virtual environment, and intuitively feel the corresponding relationship between the manipulation model and the target virtual model . Specifically, the manipulation model can be a virtual model with a certain size and transparency of any shape and color, such as a virtual model formed by scaling the target virtual model at a certain scale, a translucent spherical model or a tray-shaped model. model etc. The manipulation model can be placed in any direction close to the target virtual model, such as above or below the target virtual model.

The manipulation model is a selectable virtual model, and if it is detected that the manipulation model is selected by the currently input gesture, the whole of the target virtual model is determined as the manipulation object operated by the currently input gesture. Obviously, a corresponding relationship is established between the manipulation model and the target virtual model, and selecting the manipulation model is equivalent to selecting the entirety of the target virtual model as the manipulation object. The manipulation model can be selected by a specified gesture, such as the Air tap gesture commonly used in HoloLens or other custom arbitrary gestures. It can be seen that even if the target virtual model is in a disassembled state, the user can still operate the entire target virtual model by selecting the manipulation model, which greatly improves the convenience of operation.

In general MR application scenarios, the user mainly realizes various functional operations for the target virtual model by clicking various function buttons set in the edge area of the MR display interface. In this way, the user needs to spend a long time to find the specified function. button, the operation efficiency is low. However, in the embodiment of the present invention, the function keys in general MR application scenarios are replaced by the manipulation model, which is more intuitive and has better visual effects. The manipulation model is placed close to the target virtual model, and the user can perceive the corresponding relationship between the manipulation model and the target virtual model at first glance, and can know the specific position of the manipulation model, which effectively improves the operation efficiency. In addition, for the application scenario of selecting the entirety of the specified target virtual model from multiple target virtual models, each virtual model in the multiple target virtual models is provided with a corresponding manipulation model, and the user selects the entirety of the specified target virtual model only It is only necessary to directly select the manipulation model of the specified target virtual model, and the operation is very convenient.

In this embodiment of the present invention, the currently input gesture is detected; if the detected currently input gesture selects a preset manipulation model, the entirety of the target virtual model is determined as the operation object operated by the currently input gesture, The manipulation model is a virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner. In the above process, even if the target virtual model is in a disassembled state, the user can still input a gesture to select a manipulation model corresponding to the target virtual model, and then determine the entirety of the target virtual model as the current input The operation object operated by the gesture, so that the whole of the target virtual model can be operated and controlled, which greatly improves the convenience of the user's operation.

Referring to FIG. 2 , a second embodiment of a method for operating a virtual model of an MR head-mounted display in an embodiment of the present invention includes:

201. Detect the currently input gesture;

Step 201 is the same as step 101 , for details, please refer to the relevant description of step 101 .

202. If the detected currently input gesture selects a preset manipulation model, then determine the entirety of the target virtual model as the operation object operated by the currently input gesture;

The manipulation model is a virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner. Specifically, in the embodiment of the present invention, the manipulation model is in the shape of a tray and is placed close to the lower part of the target virtual model. The tray-shaped manipulation model is similar to a plate in appearance, and it carries the object (target virtual model) on the plate, which can bring intuitive and vivid visual effects to the user.

The manipulation model can be selected using the generic pick gesture, but this approach is less efficient in some cases. For example, for a disassembled target virtual model with a large number of parts, since the manipulation model is placed close to the target virtual model, the user can easily select the parts of the target virtual model by mistake when selecting the manipulation model using the general selection gesture, which greatly affects the Operational efficiency and user experience.

In order to improve the operation efficiency when the user selects the manipulation model, a non-universal selection gesture may be defined for the manipulation model. Considering that the manipulation model in this embodiment is in the shape of a tray, it is preferable to define the palm-up lifting gesture as a gesture for selecting the manipulation model, which is similar to the action of lifting a tray in a user's daily life, which can effectively improve the user experience.

Further, the step of judging whether the detected currently input gesture selects a preset manipulation model includes:

(1) judging whether the gesture of the current input is a lifting gesture with the palm up;

(2) If the currently input gesture is a palm-up gesture, determine that the currently input gesture selects a preset manipulation model.

If the currently input gesture is a palm-up gesture, it is determined that the currently input gesture selects a manipulation model. The above process distinguishes the gesture for selecting the manipulation model from the general selection gesture. When the user wants to select the manipulation model, the manipulation model can be directly selected by the lifting gesture with the palm up. The lifting gesture is selected, so there is no problem of wrong selection, which can effectively improve the operation efficiency and user experience. Further, even if there are multiple target virtual models and multiple corresponding manipulation models in the MR virtual environment, using non-universal selection gestures for manipulation models can filter most of the operable objects, and the user only needs to select from these manipulation models. It is enough to produce one or more control models as desired, which also greatly improves the operation efficiency.

203. Determine a target function key, where the target function key is one or more selected function keys in the function keys corresponding to different operation actions respectively;

In this embodiment, the manipulation model is provided with one or more function buttons corresponding to different operation actions respectively. The operating actions include common operations for operating objects in MR, such as moving, zooming, or rotating. The distribution positions of these function keys on the operation model can be reasonably determined according to the specific shape of the operation model. For example, for the shape of the tray in this embodiment, these function keys can be evenly arranged on the tray body.

After the whole of the target virtual model is determined as the operation object operated by the currently input gesture, a target function button is determined, and the target function button is selected from the one or more function buttons corresponding to different operation actions respectively. One of these function buttons can be selected by the user as the target function button by clicking through a designated gesture.

204. After acquiring the input operation gesture, cause the operation object operated by the currently input gesture to perform an operation action corresponding to the target function button.

After the target function button is determined and the input operation gesture is acquired, the operation object operated by the currently input gesture is made to perform an operation action corresponding to the target function button. The operation gestures are preset gestures used to complete corresponding operation actions, such as gestures such as dragging and circling. In this embodiment, the operation object operated by the currently input gesture is the whole of the target virtual model. If the user clicks and selects the function button for moving, that is, it is determined that the function button for moving is the target function button, but the user makes a drag gesture in the specified direction, then the whole of the target virtual model will move to the specified direction. The operations of direction movement, rotation or scaling are similar to the above examples of movement, and will not be repeated here.

In this embodiment of the present invention, the currently input gesture is detected; if the detected currently input gesture selects a preset manipulation model, the entirety of the target virtual model is determined as the operation object operated by the currently input gesture, The manipulation model is a tray-shaped virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner; target function keys are determined, and the target function keys are one or more corresponding to different The selected function button among the function buttons of the operation action; after acquiring the input operation gesture, the operation object operated by the currently input gesture is made to perform the operation action corresponding to the target function button. In the above process, even if the target virtual model is in a disassembled state, the user can still input a gesture to select a manipulation model corresponding to the target virtual model, and then determine the entirety of the target virtual model as the current input The operation object operated by the gesture, so that the entire target virtual model can be moved, rotated or zoomed, and other operation control, which greatly improves the convenience of user operation.

Referring to FIG. 3, a third embodiment of a method for operating a virtual model of an MR head-mounted display in an embodiment of the present invention includes:

301. Detect the currently input gesture;

Step 301 is the same as step 101 , for details, please refer to the relevant description of step 101 .

302. If the detected currently input gesture selects a preset manipulation model, then determine the entirety of the target virtual model as the operation object operated by the currently input gesture;

The manipulation model is a virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner. Specifically, in the embodiment of the present invention, the manipulation model is in the shape of a cage and surrounds the target virtual model. The cage-shaped manipulation model is similar to a bird cage in appearance, which holds the object (target virtual model) in the cage, and can bring intuitive and vivid visual effects to the user.

As described in the second embodiment of the virtual model manipulation method provided by the present invention, the manipulation model can be selected by using a general selection gesture, but in some cases, the manipulation efficiency of this method is low. For example, for a disassembled target virtual model with a large number of parts, since the manipulation model is placed close to the target virtual model, the user can easily select the parts of the target virtual model by mistake when selecting the manipulation model using the general selection gesture, which greatly affects the Operational efficiency and user experience.

In order to improve the operation efficiency when the user selects the manipulation model, a non-universal selection gesture may be defined for the manipulation model. Considering that the manipulation model in this embodiment is in the shape of a cage, it is preferable to define a grasping gesture with five fingers spread as a gesture for selecting the manipulation model, which is similar to the action of grasping a cage in a user's daily life, which can effectively improve the user experience.

Further, the step of judging whether the detected currently input gesture selects a preset manipulation model includes:

(1) judging whether the gesture of the current input is a grasping gesture with five fingers open;

(2) If the currently input gesture is a grasping gesture with five fingers spread out, it is determined that a preset manipulation model is selected for the currently input gesture.

If the currently input gesture is a grasping gesture with five fingers open, it is determined that the currently input gesture selects a manipulation model. The above process distinguishes the gesture for selecting the manipulation model from the general selection gesture. When the user wants to select the manipulation model, the manipulation model can be directly selected by the grasping gesture with five fingers open, since other controllable components cannot be used. It is selected by grabbing gestures, so there is no problem of wrong selection, which can effectively improve the operation efficiency and user experience. Further, even if there are multiple target virtual models and multiple corresponding manipulation models in the MR virtual environment, using non-universal selection gestures for manipulation models can filter most of the operable objects, and the user only needs to select from these manipulation models. It is enough to produce one or more control models as desired, which also greatly improves the operation efficiency. It should be noted that, for the cage-shaped manipulation model, other selection gestures such as single-finger hooking may also be used, which is not limited herein.

303. Determine the current operation object and/or the current operation action corresponding to the currently input gesture according to the detected currently input gesture;

Steps 303 to 305 are not limited to be executed after step 302, that is, they can be executed at any time. Determine the current operation object and/or the current operation action corresponding to the currently input gesture according to the detected currently input gesture, the gesture detected in step 303 may be an operation gesture for selecting any operation object or any operation action, According to the gesture, the current operation object and/or operation action can be determined, such as the whole of the target virtual model, a disassembled part of the target virtual model, a background button in the MR environment, a movement operation or a rotation operation, etc.

304. Determine prompt information according to the current operation object and/or the current operation action corresponding to the currently input gesture;

After the current operation object and/or the current operation action corresponding to the currently input gesture is determined, prompt information corresponding to the current operation object and/or the current operation action may be determined. The prompt information is mainly a related description of the current operation object and/or the current operation action, and the prompt information can be displayed in various forms such as text, pictures or animation. If the current operation object is a virtual dog model, the prompt information can be an explanatory text about the dog; if the current operation action is to move, the prompt information can be the word "move", if the current operation It is to operate a virtual dog model to rotate, and the prompt information can be the description text about the dog and the word "move" and so on.

305. Display the prompt information in a designated area on the display interface of the MR head-mounted display.

After the prompt information is determined, the prompt information is displayed in a designated area in the display interface of the MR head-mounted display. The designated area may be any reasonable area in the display interface, for example, may be an area to the right of the current operation object and not blocked by interference objects (such as some virtual models). Obviously, the user can obtain the operation prompt according to the prompt information when performing the operation, so as to better complete the desired operation; on the other hand, he can also better understand the relevant description of the current operation object.

In this embodiment of the present invention, the currently input gesture is detected; if the detected currently input gesture selects a preset manipulation model, the entirety of the target virtual model is determined as the operation object operated by the currently input gesture, The manipulation model is a cage-shaped virtual model that corresponds to the target virtual model and is placed close to the target virtual model in a preset manner; according to the detected currently input gesture, it is determined corresponding to the currently input gesture the current operation object and/or the current operation action; determine the prompt information according to the current operation object and/or the current operation action corresponding to the currently input gesture; display all the the prompt information. In the above process, even if the target virtual model is in a disassembled state, the user can still input a gesture to select a manipulation model corresponding to the target virtual model, and then determine the entirety of the target virtual model as the current input The operation object operated by the gesture, so that the whole of the target virtual model can be operated and controlled, which greatly improves the convenience of the user's operation. On the other hand, displaying prompt information in a designated area on the display interface of the MR head-mounted display can effectively guide the user to perform operations and obtain relevant descriptions of the current operating object, which can further improve the user experience.

It should be understood that the size of the sequence numbers of the steps in the above-mentioned various embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention. .

A method for operating a virtual model of an MR head-mounted display has been mainly described above, and a virtual model operating device of an MR head-mounted display will be described in detail below.

Referring to FIG. 4 , an embodiment of a virtual model manipulation device for an MR head-mounted display in an embodiment of the present invention includes:

Gesture detection module 401, used to detect the currently input gesture;

The model selection module 402 is configured to determine the entirety of the target virtual model as the operation object operated by the currently input gesture if a preset manipulation model is selected by the detected currently input gesture, and the manipulation model is A virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner.

Further, the control model is provided with one or more function buttons corresponding to different operation actions, and the virtual model operation device may further include:

A function key determination module 403, configured to determine a target function key, where the target function key is a selected function key among the one or more function keys corresponding to different operation actions;

The action execution module 404 is configured to make the current operation object perform an operation action corresponding to the target function button after acquiring the input operation gesture.

Further, the virtual model operating device may also include:

an operation object determination module 405, configured to determine the current operation object and/or the current operation action corresponding to the currently input gesture according to the detected currently input gesture;

A prompt information determination module 406, configured to determine prompt information according to the current operation object and/or the current operation action corresponding to the currently input gesture;

The display module 407 is configured to display the prompt information in a designated area in the display interface of the MR head-mounted display.

An embodiment of the present invention further provides an MR head-mounted display, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the computer program as shown in FIG. 1 to the steps of any virtual model operation method shown in FIG. 3 .

An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of the virtual model operations shown in FIG. 1 to FIG. 3 is implemented. steps of the method.

FIG. 5 is a schematic diagram of an MR head-mounted display provided by an embodiment of the present invention. As shown in FIG. 5 , the MR head display 5 of this embodiment includes: a processor 50 , a memory 51 , and a computer program 52 stored in the memory 51 and executable on the processor 50 . When the processor 50 executes the computer program 52, the steps in the above embodiments of the virtual model operating methods are implemented, for example, steps 101 to 102 shown in FIG. 1 . Alternatively, when the processor 50 executes the computer program 52, the functions of the modules/units in the foregoing device embodiments are implemented, for example, the functions of the modules 401 to 402 shown in FIG. 4 .

The computer program 52 may be divided into one or more modules/units, which are stored in the memory 51 and executed by the processor 50 to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the MR head-mounted display 5 .

The MR head display 5 may include, but is not limited to, a processor 50 and a memory 51 . Those skilled in the art can understand that FIG. 5 is only an example of the MR head display 5, and does not constitute a limitation on the MR head display 5, and may include more or less components than the one shown, or combine some components, or different components, such as the MR head-mounted display 5 may also include various input and output devices and the like.

The so-called processor 50 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 51 may be an internal storage unit of the MR head display 5 , such as a hard disk or a memory of the MR head display 5 . The memory 51 may also be an external storage device of the MR head display 5, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Further, the memory 51 may also include both an internal storage unit of the MR head-mounted display 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the MR head-mounted display. The memory 51 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the system embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. . Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Excluded are electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the foregoing implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the within the protection scope of the present invention.

Claims (8)

1. a virtual model operation method of an MR head-mounted display, is characterized in that, comprises: Detect the currently input gesture; If the detected currently input gesture selects a preset manipulation model, the whole of the target virtual model is determined as the operation object operated by the currently input gesture, and the manipulation model is corresponding to the target virtual model, and a virtual model placed close to the target virtual model in a preset manner; Wherein, the target virtual model is a detachable or decomposable virtual model, and the manipulation model is provided with one or more function buttons corresponding to different operation actions respectively, and the whole of the target virtual model is determined as the current input After the operation object operated by the gesture, it also includes: Determining a target function button, the target function button is a selected function button among the one or more function buttons corresponding to different operation actions respectively; After the input operation gesture is acquired, the operation object operated by the currently input gesture is made to perform the operation action corresponding to the target function button. 2. The method for operating a virtual model of an MR head-mounted display as claimed in claim 1, wherein the manipulation model is in the shape of a tray and is placed near the target virtual model; The step of judging whether the detected currently input gesture selects a preset manipulation model includes: Judging whether the currently input gesture is a palm-up gesture; If the currently input gesture is a palm-up gesture, it is determined that a preset manipulation model is selected for the currently input gesture. 3. The method for operating a virtual model of an MR head-mounted display according to claim 1, wherein the manipulation model is in the shape of a cage and surrounds the target virtual model; The step of judging whether the detected currently input gesture selects a preset manipulation model includes: Judging whether the currently input gesture is a grasping gesture with five fingers open; If the currently input gesture is a grasping gesture with five fingers spread out, it is determined that a preset manipulation model is selected for the currently input gesture. 4. The method for operating a virtual model of an MR head-mounted display according to any one of claims 1 to 3, further comprising: Determine the current operation object and/or the current operation action corresponding to the currently input gesture according to the detected currently input gesture; Determine the prompt information according to the current operation object and/or the current operation action corresponding to the currently input gesture; The prompt information is displayed in a designated area in the display interface of the MR head-mounted display. 5. A virtual model operating device for an MR head-mounted display, characterized in that it comprises: Gesture detection module, used to detect the currently input gesture; The model selection module is used to determine the entirety of the target virtual model as the operation object operated by the currently input gesture if the detected currently input gesture selects a preset manipulation model, and the manipulation model is the same as the a virtual model corresponding to the target virtual model and placed close to the target virtual model in a preset manner; Wherein, the target virtual model is a detachable or decomposable virtual model, the control model is provided with one or more function buttons corresponding to different operation actions, and the virtual model operation device further includes: a function key determination module, configured to determine a target function key, where the target function key is a selected function key among the one or more function keys corresponding to different operation actions; The action execution module is configured to make the current operation object perform the operation action corresponding to the target function key after acquiring the input operation gesture. 6. The virtual model manipulation device of the MR head-mounted display as claimed in claim 5, further comprising: an operation object determination module, configured to determine the current operation object and/or the current operation action corresponding to the currently input gesture according to the detected currently input gesture; a prompt information determination module, configured to determine prompt information according to the current operation object and/or the current operation action corresponding to the currently input gesture; A display module, configured to display the prompt information in a designated area in the display interface of the MR head-mounted display. 7. An MR head-mounted display comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the computer program as claimed in the right when executing the computer program. The steps of the virtual model operating method described in any one of claims 1 to 4 are required. 8. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the virtual The steps of the model manipulation method.

Images