CN113538705B

CN113538705B - Vulkan-based visual engine for flight simulation

Info

Publication number: CN113538705B
Application number: CN202110811768.7A
Authority: CN
Inventors: 刘德田; 侯健; 彭勇; 李佳益; 梁文杰; 赵建羽; 姚勇; 马潇潇; 谢保川
Original assignee: Pla 66350
Current assignee: Pla 66350
Priority date: 2021-07-19
Filing date: 2021-07-19
Publication date: 2022-09-09
Anticipated expiration: 2041-07-19
Also published as: CN113538705A

Abstract

The invention discloses a Vulkan-based visual engine for flight simulation, which comprises a scene editor, a data middleware, a rendering middleware and an image fusion middleware, wherein the scene editor and the data middleware provide data support for the rendering middleware; the scene editor is used for outputting rendering objects and configuration files; the data middleware analyzes the configuration file to form a source file, converts the rendering object into an internal data format, and finally compiles the source file and the internal data format into a dynamic library; the rendering middleware processes the rendering objects of different types, and checks and determines the execution sequence of the rendering objects; the image fusion middleware is positioned at the tail end of the whole system, integrates the rendering object data and outputs the rendering object data to hardware by means of a driver provided by an operating system. By adopting the Vulkan-based visual engine for flight simulation, the construction period of a training scene is shortened, the use of cross-platform and cross-application scenes is met, and the flexibility and the operation efficiency are relatively balanced.

Description

Vulkan-based visual engine for flight simulation

Technical Field

The invention relates to the technical field of image engines, in particular to a Vulkan-based visual engine for flight simulation.

Background

As shown in fig. 1, the core function of the image engine is to provide easy-to-use scene editing and configuring functions for an application graphics data organizer, automatically and efficiently split a scene into "frames" in the time domain for a service developer to call, further decompose, combine and optimize the frames into basic drawing channels in a rendering module, and finally combine the frames into a picture through an image fusion technology and present the picture to an end user.

Wherein the concept of "scene" is derived from the concept of "scene graph" in the conventional engine. The scene graph adopts a layered tree data structure to organize the relation of graphic image data in the aspects of three-dimension, logic, switch, special effect, drawing control and the like.

The top of the scene graph is a root node, the root node extends downwards, and each group node can contain meta-information including geometry, matrix transformation, rendering control state and the like. At the very bottom of the scene graph, each leaf node contains the actual geometric information that constitutes the scene object. The meta-information (logic, switch, special effect, control, etc.) of the geometric elements is obtained by performing operations of covering, inheriting, merging, etc. layer by layer from top to bottom according to the types. In the program operation period and each rendering period, the traditional engine needs to traverse the whole scene graph, generate a temporary rendering table and rendering unit rendering leaves, and destroy the leaves after execution. This engine design is referred to as a "run-time scenario".

The design method of the engine is faced with the following examination:

1. in the military application field, various software and hardware platforms with huge differences need a graphic engine to realize a 2D/3D image display function from weaponry to a simulation training system, from a combat command system to a portable terminal (a tablet personal computer) and a newly-developed AR/MR terminal.

2. The requirement of the industry user for crossing application scenes is opposite to the requirement for crossing software and hardware platforms. The two methods have the same inherent requirements, namely how to maximally reuse industry knowledge, improve the use efficiency of the knowledge, improve the development speed and efficiency of the application, and reduce the development and maintenance cost of software.

3. The contradiction between the flexibility of the use of the engine and the running efficiency is a group of permanent contradiction from the experience of the development of the graphic engine in the game field, such as the design of a graphic API, the flexibility of the use of the engine and the high efficiency of the running of the application.

Disclosure of Invention

The invention aims to provide an engine mode of a design period scene graph, which is used for solving the two requirements and a contradiction problem.

In order to achieve the purpose, the invention provides the following technical scheme:

A Vulkan-based visual engine for flight simulation comprises a scene editor, a data middleware, a rendering middleware and an image fusion middleware, wherein the scene editor and the data middleware provide data support for the rendering middleware;

the scene editor organizes the logic relation between the rendering objects, completes the sharing of the meta-information between the rendering objects and outputs the rendering objects and the configuration file;

the data middleware analyzes the configuration file to form a syntax tree, optimizes the syntax tree to form a source file aiming at the target output platform, converts the rendering object into an internal data format, and finally compiles the source file and the internal data format into a dynamic library;

the rendering middleware calls a calculation shader, the shader processes rendering objects of different types, and the execution sequence of the rendering objects is checked and determined according to the compiling and building of a source file;

the image fusion middleware is positioned at the tail end of the whole system and is positioned between the engine and the hardware driving layer; and integrating the rendering object data through an image fusion algorithm according to the source file, and outputting the rendering object data to hardware by means of a driver provided by an operating system.

Preferably, the rendering object comprises a ground scene data system, a simulation model, sky, ocean, an atmospheric system, a particle system and a sensor image, and the ground scene data system comprises terrain elevation data and terrain image data.

Preferably, the scene editor comprises six components of interface, interaction, view, logic, style and state/property; the interface component provides specific functions and extensions for the outside and transfers external attributes for the inside; the interaction component is used for collecting rendering object data submitted by an application graphic data organizer; the view component is a visual graphical interface and is used for submitting rendering object data by an application graphical data organizer; the logic component packages and extracts common logic and is used for processing the data state of the rendering object; the style component is used for systematically modifying the current style; the state/attribute component is an inherent manifestation of a series of I/O and logical processes for visualizing the state and attribute values of rendering objects in a particular development application scene.

Preferably, the configuration file is in xml format.

Preferably, the data middleware is further provided with an analysis and verification module, and adopts technical means of verification and iteration to perform consistency verification on the source file according to the configuration file.

The Vulkan-based flight simulation visual engine with the structure has the following advantages:

1. a natural bridge can be erected between design application of graphic images on the airplane airborne equipment and research and development of simulation training equipment, so that some design application under the airborne environment can be conveniently converted into simulation programs in the simulation training equipment, the research and development period of the simulation equipment is shortened, the research and development cost is reduced, and the simulation degree of the simulation training equipment is greatly improved;

2. Therefore, designers of airplane airborne equipment and simulative training equipment are not troublesome due to the difference of hardware and software system environments, and particularly in the aspect of simulation, the simulative training equipment is not limited by the development of a three-dimensional engine of a foreign vision system, and software functions can be cut, changed, optimized and the like more flexibly.

3. Consideration between balance flexibility and operation efficiency enables a training scene to be more visual and vivid and a training effect to be better. In the virtual battlefield, tactical level drilling and strategic level deduction can be carried out, and the battle plan making personnel can evaluate and screen the scheme through virtual scene simulation.

Drawings

FIG. 1 is a diagram illustrating a relationship between a conventional engine structure and various types of users;

FIG. 2 is a block diagram of an engine architecture according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating the relationship between the engine structure and different types of users according to an embodiment of the present invention;

FIG. 4 is a block diagram of a scene editor in an embodiment of the invention;

FIG. 5 is a block diagram of the components of data middleware in an embodiment of the present invention;

FIG. 6 is a block diagram illustrating the components of the image fusion middleware according to an embodiment of the present invention.

Detailed Description

The technical scheme of the invention is further explained by combining the drawings and the embodiment.

In the scheme, the engine adopts a design period scene graph design mode. By introducing data middleware and rendering middleware, a rendering graph is traversed in an application compiling stage, and a basic drawing unit channel is generated, so that the operation period efficiency is improved, and the possibility is provided for cross-platform.

This idea stems from the following four observations:

1. in flight simulation and airborne equipment, the type of picture scene is relatively fixed. The extreme flexibility provided by the run-time scene tree is not sufficient to offset the efficiency penalty it entails.

2. For engine users, since the runtime scenario covers all types of data organization: the precedence relationship of space, logic, dependency, effect and rendering, so the requirement on the user of the engine is high and no subdivision is available. Leading application service developers to master the process from 3D modeling,

The design period scene graph subdivides the requirements on personnel into an application graphic data organizer and an application service developer, and provides different working environments and interfaces for two types of workers respectively, so that the requirements on the skills and knowledge of the workers are reduced, and the production efficiency is improved.

3. Since the 90 s, the JIT (just in time) -type computing organization concept, represented by Java, began to be reviewed again due to the relative hesitation in development of CPU technology in terms of processing speed, the inability of multi-core technology to facilitate the resolution of such problems. In more and more applications, the technical route of aot (aoheadoftime) and the like is beginning to be widely adopted. Namely, the whole life cycle (design, construction and operation) of the application is taken into consideration and fully utilized. In the field of games, a new generation game engine represented by unregealEngine adopts a mode of providing all engine source codes and completely recompiling during application construction to pursue both design and execution efficiency under the support of a business model.

4. In addition to providing a set of efficient and easy-to-use C/C + + compilers meeting the specifications, the whole compiling and building process is exposed by using an intermediate description language and an API (application programming interface), so that the compiling and building system is not in a black box state any more and can be embedded and called. This provides powerful tool support for flexible, controllable architecture design.

The proposal of the design period scene graph can effectively solve two requirements and a contradiction of the application layer, and the design period scene graph specifically comprises the following steps:

2-6, the vision engine for Vulkan-based flight simulation comprises a scene editor, a data middleware, a rendering middleware and an image fusion middleware, wherein the scene editor and the data middleware provide data support for the rendering middleware.

The scene editor organizes the logic relation between the rendering objects, completes the sharing of the meta information between the rendering objects and outputs the rendering objects and the configuration file in the xml format.

The rendering object comprises a ground scene data system, a simulation model, sky, ocean, an atmospheric system, a particle system and heterogeneous image data, the ground scene data system comprises terrain elevation data and terrain image data, and the particle system is used for displaying effects of trails, explosions and the like.

The scenario editor provides a relatively isolated working environment for the application graphical data organizer. The scene editor supports the application of graphic data organizers to organize graphic image material with the scene as the center. Applying the graphical data organizer work content comprises: the system is in charge of acquiring and organizing terrain elevation data and terrain image data (satellite images) of an area covered by a landscape system in a graphic engine, and format conversion is carried out by using a built-in tool and depending on a deployment platform; the three-dimensional model and the related texture mapping of the 3D object in the graphic engine are acquired and organized, and the built-in tool is used for carrying out adjustment work such as format conversion, scaling and positioning on the basis of the deployment platform; and the system is responsible for acquiring and organizing animation data of the 3D objects in the graphic engine and configuring the animation data by using built-in tools. And the converter is responsible for configuring heterogeneous image data sources and selecting a corresponding converter according to types and requirements.

The scenario editor is implemented through component development. The components are relatively independent reusable logic units, a plurality of components interact to form a complete and rich page, and the internal expression of the page is to divide and abstract the page, so that the aims of high reuse and easy maintenance are fulfilled.

The scene editor mainly comprises components such as interfaces, interaction, views, logic, styles, states/attributes and the like, and has the following specific functions:

1. the interface assembly provides specific functions and extension to the outside, passes external attribute to the inside, is invisible to the organizer of graphic data, nevertheless bears the effect of bridge, is the adhesive of each subassembly combination, and consequently the design of interface is very important, and reliable input and output is the stable stone of subassembly, and the interface that the function is abundant will improve the use experience of subassembly also.

2. The interactive component is responsible for the interactive work with the application graphic data organizer, provides operational functions, collects the input of the application graphic data organizer, processes the data, re-presents the new data to the application graphic data organizer, and can quickly respond. In order to meet the requirement of quick response, the scene editor component decomposes the execution tasks of the component in priority, and ensures the quick response of the component.

3. The view component is a visual graphical interface, the use experience of an organizer applying graphical data can be improved due to the good and uniform appearance, and the view layer expression of the component is improved through the componentized visual specification.

4. Common logics are packaged and extracted inside the logic component, and independence of a logic layer is achieved.

5. The pattern component extracts the pattern independently, systematically modifies the current pattern, and reduces the dependence of daily development on the pattern.

6. The state/attribute component is an inherent manifestation of a series of I/O and logical processes that facilitate the visualization of the state and attribute values of rendered objects in the development of a particular application scenario.

The scene editor is realized through component development, the coupling performance of each function can be reduced to a great extent through the component development, data are mutually independent, and the aggregation performance inside the functions is improved. The method has great benefits for developing specific application scenes and reducing the maintenance of codes. The internal structure is sealed and does not affect the overall or other components. The reduction of the coupling improves the extensibility of the graphic engine, reduces the complexity of developing a specific application scene, improves the development efficiency and reduces the development cost. Because all functions are complete, the components used for developing specific application scenes can enable less personnel to contact general logic in the components, and the components are more suitable for development in the specific application scenes, so that the development efficiency is improved, and the generation of errors is reduced.

Data middleware is designed around flexibility/operating efficiency. Technically, the technology of code generation is adopted to meet the requirement of efficiency, and the xml configuration is used to meet the requirement of flexibility. Specifically, the method comprises the following steps: the data middleware reads the xml configuration output by the scene editor; analyzing xml configuration by the data middleware to form a syntax tree; and the data middleware optimizes a data structure aiming at the target output platform through an optimizer and generates the syntax tree into a source file. The data middleware converts different types of rendering object data into an internal data format defined by the platform through a data format converter; and finally, calling a built-in cross-platform compiler to compile into a dynamic library meeting the use requirements of application service developers. The technical difficulty is that the code generation and optimization results are inconsistent with the xml configuration file. In order to solve the problem, a built-in analysis and verification module is designed, and consistency verification is performed by adopting verification and iteration technical means.

Rendering middleware is used to address three requirements:

1. rendering modes and effects of different graphic objects, such as rendering differences between a 2D object and a 3D object, rendering of a landscape, and rendering differences between cloud, fog, rain, snow and other atmospheric effects;

2. The difference of the corresponding graphic APIs on different deployment platforms, for example, on a graphic workstation, a Vulkan graphic API can be adopted, and on-board equipment only an OpenGL graphic API can be adopted;

3. and on different deployment platforms, the data transmission and image caching modes from the main memory of the system to the GPU are different. For example, on a graphics workstation, image data can be cached on a GPU side on a larger scale to improve efficiency, while on-board devices are limited by the hardware environment and can only adopt a frame-by-frame data synchronization manner.

The core working unit of the rendering middleware is a corridor. In a simulation or airborne system, each frame needs to combine and call a plurality of corridors to complete the final graphic image rendering. The mutual dependence relationship between the channels is described by a graph, and the execution sequence is checked and determined when the application is compiled and constructed, so that the balance between efficiency and flexibility is achieved.

According to the stage of GPU use, the corridor can be divided into 3 types:

rendering a front corridor: the method comprises the steps of downloading from a system main memory or reading related rendering data such as vertex data and textures from a cache in a GPU, calling a Compute Shader (computer Shader) to perform corresponding processing on the data, initializing temporary resources and the like;

Rendering a corridor: the kernel of the Shader is a Shader (Shader) which comprises a geometry Shader, a vertex Shader and a fragment Shader;

rendering a channel: the main functions are to clear up temporary resources, save data (to GPU cache or system main memory) or discard data, depending on platform characteristics.

The image fusion middleware is positioned at the extreme end of the whole system and is positioned between the engine and the hardware driving layer. The function of the system is to integrate heterogeneous image data and output the data to hardware such as VR equipment, multi-channel display equipment, airborne equipment and the like by means of a driver provided by an operating system.

From the viewpoint of heterogeneous image data integration, the image fusion middleware firstly reads heterogeneous image data configuration from a scene editor, converts different image types into platform standard image formats specified by the middleware, secondly operates image channels, display areas, position arrangement and the like according to fusion rules, and finally completes generation of a final image through an image fusion algorithm according to the configuration.

From the perspective of supporting multi-type and multi-platform output, the image fusion middleware needs to formulate a platform basic image format for different platform devices and process the problems related to an output platform, such as a multi-channel strong real-time network synchronization technology.

In general, the scene editor is responsible for organizing the spatial information of the image data object, the API generated by the data middleware is responsible for controlling the time domain, the rendering middleware is responsible for presenting the geometric information and the physical information, and finally the image fusion middleware completes the fusion and output of all heterogeneous image data.

The following technical effects are finally achieved:

1. and (4) the control is autonomous. The fully-autonomous source code controllable software supports customization and opening, and provides an SDK (software development kit) and a scene editing environment.

2. Flexible deployment environment. The method supports the operation under two instruction frameworks of Linux and Windows, and X64 and ARM64, and can be expanded to a VxWorks operating system and an MIPS instruction framework. Supporting OpenGL and Vulkan graphics APIs.

3. Multiple output support. The multi-screen and multi-channel large-screen output is supported, the output of popular VR equipment is supported, and the video output of embedded airborne equipment is supported.

4. Friendly scene editing interface. And a blueprint scene editing interface is supported, and the learning and the use are easy.

TABLE 1 Performance index

The above is a specific embodiment of the present invention, but the scope of the present invention should not be limited thereto. Any changes or substitutions which can be easily made by those skilled in the art within the technical scope of the present invention disclosed herein shall be covered by the protection scope of the present invention, and therefore the protection scope of the present invention shall be subject to the protection scope defined by the appended claims.

Claims

1. A Vulkan-based visual engine for flight simulation is characterized in that: the system comprises a scene editor, a data middleware, a rendering middleware and an image fusion middleware, wherein the scene editor and the data middleware provide data support for the rendering middleware;

the rendering middleware calls a calculation shader, the shader processes rendering objects of different types, and the execution sequence of the rendering objects is checked and determined according to the source file during compiling and building;

the image fusion middleware is positioned at the tail end of the whole system and is positioned between the engine and the hardware driving layer; integrating rendering object data through an image fusion algorithm according to a source file, and outputting the rendering object data to hardware by means of a driving program provided by an operating system;

the scene editor comprises six components of an interface, an interaction, a view, logic, a style and a state/attribute; the interface component provides specific functions and extensions for the outside and transfers external attributes for the inside; the interaction component is used for collecting rendering object data submitted by an application graphic data organizer; the view component is a visual graphical interface and is used for submitting rendering object data by an application graphical data organizer; the logic component packages and extracts common logic and is used for processing the data state of the rendering object; the style component is used for systematically modifying the current style; the state/attribute component is an internal embodiment of a series of I/O and logic processing and is used for visualizing the state and attribute values of rendering objects in a specific application scene;

The data middleware is also provided with an analysis and verification module, and adopts verification and iteration technical means to carry out consistency verification on the source file according to the configuration file.

2. The vision engine for Vulkan-based flight simulation of claim 1, wherein: the rendering object comprises a ground scene data system, a simulation model, sky, ocean, an atmospheric system, a particle system and a sensor image, and the ground scene data system comprises terrain elevation data and terrain image data.

3. The vision engine for Vulkan-based flight simulation of claim 1, wherein: the configuration file is in xml format.