CN112002017A - Virtual scene generation method, device, terminal, medium and virtual training system - Google Patents

Abstract

The invention belongs to the technical field of virtual and real reality, and particularly relates to a virtual scene generation method, a virtual scene generation device, a virtual scene generation terminal, a virtual scene generation medium and a virtual training system. The virtual generation method comprises the following steps: acquiring use scene information of equipment to be generated; and performing virtual modeling and scene modeling on the equipment to be generated by combining the using scene information of the equipment to be generated to obtain an equipment model and a virtual application scene. The method is applied to power equipment test training, and can generate a virtual reality scene enabling a user to be personally on the scene, so that better immersive use experience is provided for the user.

Description

Virtual scene generation method, device, terminal, medium and virtual training system

technical field

The present invention is in the technical field of virtual reality, in particular to a virtual scene generation method, device, terminal, medium and virtual training system.

Background technique

Safety production is the first priority for the sustainable development of an enterprise. The problems of high risk, high cost, and invisibility in the test and practice of power equipment have hindered the improvement of safety awareness and safety skills.

In recent years, in order to solve the above problems, scientific research institutes and universities have developed training systems based on virtual reality to conduct practical training and teaching drills in the form of virtual reality, but the existing virtual reality training systems still have certain immersive experience. lack.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present invention provide a virtual scene generation method, device, terminal, medium and virtual training system, so as to solve the problem of insufficient immersive experience in the existing virtual reality training system.

A first aspect of the embodiments of the present invention provides a virtual scene generation method, where the virtual scene generation method includes:

Obtain the usage scenario information of the device to be generated;

In combination with the usage scene information of the device to be generated, virtual modeling and scene modeling are performed on the device to be generated to obtain a device model and a virtual application scene.

Optionally, the generating method further includes:

rendering the device model and the virtual application scene;

Distortion correction and distortion scaling for rendered device models and virtual application scenes.

Optionally, rendering the device model and the virtual application scene by using DirectX 3D rendering technology;

Perform distortion correction and distortion scaling on rendered device models and virtual application scenes through Pixel Shader.

Optionally, the rendered device model and the virtual application scene are refreshed at a frequency of more than 60 frames per second.

A second aspect of the embodiments of the present invention provides an apparatus for generating a virtual scene, and the apparatus for generating a virtual scene includes:

The scene acquisition module is used to acquire the usage scene information of the device to be generated;

The virtual generation module is configured to perform virtual modeling and scene modeling on the to-be-generated device in combination with the usage scene information of the to-be-generated device to obtain a device model and a virtual application scenario.

A third aspect of the embodiments of the present invention provides a virtual training system, including the virtual scene generating apparatus and the interactive system provided in the second aspect of the embodiments of the present invention; the interactive system includes:

a display device, configured to display the device model and the virtual application scene acquired by the virtual scene generating device;

A somatosensory interaction device, used for collecting operation instructions and motion information, and transmitting the operation instructions and motion information to the virtual generating device;

The virtual scene generating apparatus is configured to correspondingly change the device model and the virtual application scene according to the operation instruction and the action information.

Optionally, the display device includes: a helmet display device and a 3D display screen.

Optionally, there are multiple helmet display devices, and the somatosensory interaction devices are set in a one-to-one correspondence with the helmet display devices;

The virtual scene generating device is used to correspondingly change the device model and the virtual application scene according to the operation instructions and action information of the multiple somatosensory interaction devices, and transmit the device model and the virtual application scene to the multiple helmet display devices for synchronous display.

A fourth aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program The steps of implementing the method for generating a virtual scene provided by the first aspect of the embodiment of the present invention.

A fifth aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the implementation of the first aspect of the embodiments of the present invention is implemented The steps of the virtual scene generation method.

In this embodiment of the present invention, when the virtual model scene is generated, the usage scene information of the device to be generated is obtained, and the device model is obtained by performing virtual modeling and scene modeling on the device to be generated in combination with the usage scene information of the device to be generated. By combining the device model produced according to the device to be generated and the virtual application scene, the user is immersed in the scene, thereby providing the Better immersive experience for users.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic flowchart of a method for generating a virtual scene according to an embodiment of the present invention;

2 is a schematic structural diagram of an apparatus for generating a virtual scene provided by an embodiment of the present invention;

3 is a schematic structural diagram of an interaction system provided by an embodiment of the present invention;

Fig. 4 is the indoor layout legend of the virtual training system provided by the embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a terminal device provided by an embodiment of the present invention.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

The term "including" and any other variations in the description and claims of the present invention and the above drawings means "including but not limited to", and is intended to cover non-exclusive inclusion. For example, a process, method or system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes Other steps or units inherent in these processes, methods, products or devices. Also, the terms "first," "second," and "third," etc. are used to distinguish between different objects, rather than to describe a particular order.

In order to illustrate the technical solutions of the present invention, the following specific embodiments are used for description.

FIG. 1 is a schematic flowchart of a method for generating a virtual scene according to this embodiment, and the method for generating a virtual scene includes:

Step S101, obtaining usage scene information of the device to be generated;

In this embodiment of the present invention, the device to be generated may be an electric device. Before generating the device model of the electric device, the actual application scenario of the device is also obtained. For example, if installation training is to be performed, the installation environment of the device is obtained. , if you want to perform inspection training, obtain its corresponding inspection environment, and specify emergency training in some special scenarios, then obtain its corresponding special scenarios. By comprehensively acquiring the usage scene information of the generation device, it is beneficial to generate a virtual application scene corresponding to the power device in the subsequent method.

In step S102, virtual modeling and scene modeling are performed on the device to be generated in combination with the usage scene information of the device to be generated to obtain a device model and a virtual application scene.

In this embodiment of the present invention, when virtual reality technology, component modeling technology, and/or three-dimensional modeling technology for power system equipment are used to perform virtual modeling of the device to be generated, the acquired usage scene information of the device to be generated is also imported, and the A virtual application scene corresponding to the device to be generated is generated according to the graphical development environment technology, so as to simulate the real usage scene of the device to be generated in the virtual reality scene, thereby bringing a better immersive training experience to the user.

When generating the virtual model scene in the above-mentioned embodiment, the usage scene information of the device to be generated is obtained, and virtual modeling and scene modeling are performed on the device to be generated in combination with the usage scene information of the device to be generated, and the device model and the scene model are obtained. Virtual application scenarios, so that the same virtual application scenarios as those of the devices to be generated in reality can be obtained. By combining the device models produced according to the devices to be generated and the virtual application scenarios, users can be immersed in the scene, thereby providing users with for a better immersive user experience.

In some embodiments, the virtual scene generation method may further include: rendering the device model and the virtual application scene; and performing distortion correction and distortion scaling on the rendered device model and the virtual application scene.

In the embodiment of the present invention, in order to enable trainers to achieve an ideal visual experience effect, the generated virtual scene is rendered, and various methods are used to reduce the size of the model from the aspects of model construction and texture quality control, including distortion correction and distortion scaling. , so as to achieve a balance between fluency and picture effect without reducing the visual effect, and further improve the immersive experience of trainers.

In some embodiments, the device model and the virtual application scene may be rendered by using the DirectX3D rendering technology; distortion correction and distortion scaling may be performed on the rendered device model and the virtual application scene by using the Pixel Shader.

In the embodiment of the present invention, DirectX 3D is an application program interface (API), which enables games or multimedia programs based on windows to obtain higher execution efficiency, enhances the effects of 3D graphics and sound, and provides a common The hardware driver standard is based on the hardware driver standard, so that the driver program is unified and the complexity of the hardware is reduced. In the embodiment of the present invention, the device model and the virtual application scene are rendered by using the rendering technology; A pixel shader instruction composed of ASCII text, and its arithmetic instruction can be used to perform diffuse reflection and/or specular reflection lighting calculations. Therefore, in this embodiment of the present application, Pixel Shader is used to perform distortion correction on the rendered device model and virtual application scene. Distortion scaling.

In some embodiments, the rendered device model and the virtual application scene are refreshed at a frequency of more than 60 frames per second.

In the embodiment of the present invention, the rendered virtual reality scene (that is, the device model and the virtual application scene) is kept refreshed at a frequency of more than 60 frames per second, which can not only bring a smoother use picture to the user , but also to protect the user's eyes.

In some other embodiments, if the device to which the method is applied is connected to a head-mounted display device for use, the method may further include: dividing and rendering the virtual reality scene after distortion correction and distortion scaling to the head-mounted display device The left and right eye screens can bring users a smoother and more immersive picture experience.

In some other embodiments, if the device to which the method is applied is connected to the virtual reality helmet for use, that is, the user can also acquire sound information through the virtual reality helmet, the method may further include: acquiring the data of the device to be generated in different scenarios. The sound information and/or the sound information of the scene, combined with the sound information of the device to be generated and/or the sound information of the scene in the different scenarios, the device model is obtained by performing virtual modeling on the device to be generated, so as to be more realistic. The simulated practical exercise meets the whole process and full range of simulation simulation, so that the user can obtain a near-real operation experience from the visual and auditory aspects.

FIG. 2 is a schematic structural diagram of an apparatus for generating a virtual scene according to an embodiment of the present invention. Referring to FIG. 2 , the apparatus 20 for generating a virtual scene may include:

The scene acquisition module 21 is configured to acquire usage scene information of the device to be generated.

The virtual generation module 22 is configured to perform virtual modeling and scene modeling on the to-be-generated device in combination with the usage scene information of the to-be-generated device to obtain a device model and a virtual application scenario.

In some embodiments, the virtual scene generation method may further include: rendering the device model and the virtual application scene; and performing distortion correction and distortion scaling on the rendered device model and the virtual application scene.

In some embodiments, the device model and the virtual application scene may be rendered through the DirectX 3D rendering technology; distortion correction and distortion scaling may be performed on the rendered device model and the virtual application scene through the Pixel Shader.

In some embodiments, the rendered device model and the virtual application scene are refreshed at a frequency of more than 60 frames per second.

In some embodiments, if the virtual scene generating apparatus is connected to a head-mounted display device for use, the virtual scene generating apparatus may also be used to: segment and draw the virtual reality scene after distortion correction and distortion scaling to the head-mounted display device. Displays the left and right binocular screens of the device.

In some embodiments, if the virtual scene generation device is connected to a virtual reality helmet for use, that is, the user can also obtain sound information through the virtual reality helmet, the virtual scene generation device may include: a sound acquisition module for acquiring different sound information. The sound information of the device to be generated and/or the sound information of the scene in the scene, and in combination with the sound information of the device to be generated and/or the sound information of the scene in the different scenarios, virtual modeling is performed on the device to be generated to obtain the device Model.

An embodiment of the present invention further provides a virtual training system, where the virtual training system includes the virtual scene generating device and the interaction system provided by the foregoing embodiments of the invention; referring to FIG. 3 , the interaction system 30 may include:

A display device 31, configured to display the device model and the virtual application scene obtained by the virtual scene generating apparatus;

The somatosensory interaction device 32 is used to collect operation instructions and motion information, and transmit the operation instructions and motion information to the virtual generating device;

Correspondingly, the virtual scene generating apparatus is configured to correspondingly change the device model and the virtual application scene according to the operation instruction and the action information.

In the embodiment of the present invention, by using a display device and a somatosensory interaction device, the user is completely immersed in the virtual scene of the power equipment experiment and interacts in a natural way, which greatly improves the user's visual and tactile interaction experience, thereby improving the training of the system. Effect.

In some embodiments, there are multiple headgear display devices, and the somatosensory interaction devices are set in a one-to-one correspondence with the headgear display devices; the virtual scene generation device is configured to generate operation instructions and action information according to the plurality of somatosensory interaction devices. , correspondingly change the device model and the virtual application scene, and transmit the device model and the virtual application scene to multiple helmet display devices for synchronous display.

In the embodiment of the present invention, the virtual scene generating device may be one, and the display device and the somatosensory interaction device may be multiple respectively, so as to adapt to the safety collaborative exercise of multi-person coordination, and enhance the safety operation awareness of individual soldiers. At the same time, it also improves the level of team safety and collaboration. The virtual training system may also include virtual scene generating devices, each virtual scene generating device corresponds to a display device and a somatosensory interaction device, and data synchronization is performed between the virtual scene generating devices through a wired or wireless network.

In some embodiments, the display device includes: a helmet display device and a 3D display screen.

In the embodiment of the present invention, the display device can be divided into a helmet display device and a 3D display screen. The helmet device is provided to the trainees, and the display device can be used by the examiner to intuitively see the training situation of each trainee, so that the trainees can see the training situation. Assessment, analysis and evaluation of operational behavior.

In some embodiments, the somatosensory interaction device may include a simulation operation joystick, a touch interaction device, a fingerprint device, etc., combined with the above-mentioned 3D display screen, so that the trainee can realize both 2D ordinary simulation training and 3D immersive simulation training. .

FIG. 4 is an example of an indoor layout of a virtual training system provided by an embodiment of the present invention. In FIG. 4 , the virtual training system further includes laser locators, which are respectively arranged at four corners of the room and used for locating the equipment. . The display includes each VR eye and a 3D display screen, and the virtual scene generating device may be a computer. A discussion area is set up for users to rest and discuss.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. The module is completed, that is, the internal structure of the virtual scene generating apparatus is divided into different functional units or modules, so as to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

FIG. 5 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in FIG. 5 , in this embodiment, the terminal device 50 includes: a processor 51 , a memory 52 , and a computer program 53 stored in the memory 52 and running on the processor 51 . When the processor 51 executes the computer program 53 , the steps in each embodiment described in the first aspect of the embodiment are implemented, for example, steps S101 to S102 shown in FIG. 1 . Alternatively, when the processor 51 executes the computer program 53, the functions of the modules/units in the above-mentioned embodiment of the virtual scene generating apparatus, for example, the functions of the modules 21 to 22 shown in FIG. 2 are realized.

Exemplarily, the computer program 53 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 52 and executed by the processor 51 to complete the this application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 53 in the terminal device 50 .

The terminal device may be a computing device such as a mobile phone and a tablet computer. The terminal device may include, but is not limited to, a processor 51 and a memory 52 . Those skilled in the art can understand that FIG. 5 is only an example of the terminal device 50, and does not constitute a limitation to the terminal device 50, and may include more or less components than the one shown, or combine some components, or different components For example, the terminal device 50 may further include an input and output device, a network access device, a bus, and the like.

The so-called processor 51 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 52 may be an internal storage unit of the terminal device 50 , such as a hard disk or a memory of the terminal device 50 . The memory 52 may also be an external storage device of the terminal device 50, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) equipped on the terminal device 50. card, flash card (Flash Card) and so on. Further, the memory 52 may also include both an internal storage unit of the terminal device 50 and an external storage device. The memory 52 is used to store the computer program 53 and other programs and data required by the terminal device 50 . The memory 52 can also be used to temporarily store data that has been output or is to be output.

Embodiments of the present invention further provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each embodiment described in the first aspect of the embodiment are implemented, for example, Steps S101 to S102 shown in FIG. 1 .

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed method, device, terminal and storage medium for generating a virtual scene may be implemented in other ways. For example, the embodiment of a virtual scene generating apparatus described above is only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple divisions. Individual units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Excluded are electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims (10)

1. a virtual scene generation method, is characterized in that, comprises: Obtain the usage scenario information of the device to be generated; In combination with the usage scene information of the device to be generated, virtual modeling and scene modeling are performed on the device to be generated to obtain a device model and a virtual application scene. 2. a kind of virtual scene generation method as claimed in claim 1, is characterized in that, also comprises: rendering the device model and the virtual application scene; Distortion correction and distortion scaling for rendered device models and virtual application scenes. 3. a kind of virtual scene generation method as claimed in claim 2, is characterized in that, Render the device model and the virtual application scene by using the DirectX 3D rendering technology; Perform distortion correction and distortion scaling on rendered device models and virtual application scenes through Pixel Shader. 4 . The method for generating a virtual scene according to claim 3 , wherein the rendered device model and the virtual application scene are refreshed at a frequency of more than 60 frames per second. 5 . 5. A device for generating virtual scenes, comprising: The scene acquisition module is used to acquire the usage scene information of the device to be generated; The virtual generation module is configured to perform virtual modeling and scene modeling on the to-be-generated device in combination with the usage scene information of the to-be-generated device to obtain a device model and a virtual application scenario. 6. A virtual training system, characterized in that, comprising the virtual scene generating device and the interactive system as claimed in claim 5; the interactive system comprises: a display device, configured to display the device model and the virtual application scene acquired by the virtual scene generating device; A somatosensory interaction device, used for collecting operation instructions and motion information, and transmitting the operation instructions and motion information to the virtual generating device; The virtual scene generating apparatus is configured to correspondingly change the device model and the virtual application scene according to the operation instruction and the action information. 7. The virtual training system according to claim 6, wherein the display device comprises: a helmet display device and a 3D display screen. 8 . The virtual training system according to claim 7 , wherein there are multiple helmet display devices, and the somatosensory interaction device and the helmet display device are set in one-to-one correspondence; 8 . The virtual scene generating device is used to correspondingly change the device model and the virtual application scene according to the operation instructions and action information of the multiple somatosensory interaction devices, and transmit the device model and the virtual application scene to the multiple helmet display devices for synchronous display. 9. A terminal device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, when the processor executes the computer program, the implementation as claimed in the claims Steps of the method for generating a virtual scene according to any one of 1 to 4. 10. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, a method according to any one of claims 1 to 4 is implemented The steps of the virtual scene generation method.

Images