CN117876568A - Custom data injection method and device for streaming transmission process - Google Patents

Abstract

The present disclosure relates to a method and apparatus for custom data injection in a streaming process. The method comprises the following steps: acquiring a model vertex index and loading custom data; establishing a corresponding relation table of model vertex indexes and custom data; and writing the custom data into the rendering resource according to the corresponding relation table. According to the scheme, a corresponding relation table of model vertex indexes and custom data is established; the custom data is written into the rendering resource according to the corresponding relation table, so that the flexibility of model rendering and customization and individuation of rendering effects are improved, and the problem that the custom data cannot be directly carried to subsequent use in the streaming transmission process before rendering based on the graphics processor driven rendering method is solved.

Description

Custom data injection method and device for streaming transmission process

The present application claims priority from patent application No. 2023118621725 entitled "custom data injection method and apparatus for streaming process" filed on month 12 and 29 of 2023, the entire contents of which are incorporated herein by reference.

Technical Field

The disclosure relates to the technical field of rendering, and in particular relates to a method and a device for injecting custom data in a streaming process.

Background

At present, the existing method cannot directly carry the custom data to the subsequent use in the Streaming before rendering (Streaming) process based on the graphics processor driven rendering method, so that the flexibility and customization of a rendering pipeline by a user are limited, and the individuation and customization degree of a rendering effect are reduced. It may be necessary to implement a specific rendering effect through complicated additional processing steps, increasing the complexity of development and the difficulty of optimization.

Therefore, how to find a method for custom data injection in a streaming process for improving model rendering flexibility, customizing rendering effects and individuation is a technical problem to be solved by those skilled in the art.

Disclosure of Invention

An objective of the disclosed embodiments is to provide a method and apparatus for injecting custom data in a streaming process, so as to solve the problem that custom data cannot be directly carried to a subsequent use in a streaming process before rendering based on a graphics processor driven rendering method.

In a first aspect, an embodiment of the present disclosure provides a method for injecting custom data in a streaming process, the method including: acquiring a model vertex index and loading custom data; the model vertex index comprises vertex attribute information, index values and mapping information for associating the custom data with model vertices; establishing a corresponding relation table of model vertex indexes and custom data; and writing the custom data into the rendering resource according to the corresponding relation table.

Optionally, obtaining the model vertex index includes: analyzing vertex data of the rendering model, and determining vertex attribute information of each model vertex; assigning an index value to each model vertex; mapping information for associating the custom data with the model vertices is determined.

Optionally, obtaining the model vertex index includes: and if the rendering model is modified, obtaining vertex attribute information, index values and mapping information for associating the custom data with the model vertices in each model vertex in the modified rendering model.

Optionally, writing the custom data into the rendering resource according to the correspondence table includes: constructing a custom data table; the custom data table comprises a corresponding relation table; importing a custom data table into rendering resources used by a rendering pipeline; custom data imported into rendering resources used by the rendering pipeline is integrated with elements in the rendering pipeline other than the custom data table.

Optionally, after writing the custom data into the rendering resource according to the correspondence table, the method further includes: and updating the corresponding relation table according to the run-time attribute dynamically determined in the rendering process.

Optionally, after writing the custom data into the rendering resource according to the correspondence table, the method further includes: monitoring changes in runtime properties during the stream processing phase; and updating the custom data table according to the change of the run-time attribute.

Optionally, after writing the custom data into the rendering resource according to the correspondence table, the method further includes: sending the rendering resources into a subsequent flow of a rendering pipeline; the rendering model is converted into a 2D image.

In a second aspect, an embodiment of the present disclosure provides a custom data injection apparatus for a streaming process, the apparatus comprising: the acquisition module is used for acquiring the model vertex index and loading the custom data; the model vertex index comprises vertex attribute information, index values and mapping information for associating the custom data with model vertices; the building module is used for building a corresponding relation table of the model vertex index and the custom data; and the writing module is used for writing the custom data into the rendering resource according to the corresponding relation table.

Optionally, the obtaining module is configured to obtain a model vertex index, specifically: the acquisition module is used for: analyzing vertex data of the rendering model, and determining vertex attribute information of each model vertex; assigning an index value to each model vertex; mapping information for associating the custom data with the model vertices is determined.

Optionally, the obtaining module is configured to obtain a model vertex index, specifically: the acquisition module is used for: and if the rendering model is modified, obtaining vertex attribute information, index values and mapping information for associating the custom data with the model vertices in each model vertex in the modified rendering model.

Optionally, the writing module is configured to write the custom data into the rendering resource according to the correspondence table, specifically: the writing module is used for: constructing a custom data table; the custom data table comprises a corresponding relation table; importing a custom data table into rendering resources used by a rendering pipeline; custom data imported into rendering resources used by the rendering pipeline is integrated with elements in the rendering pipeline other than the custom data table.

Optionally, the apparatus further comprises: and the first updating module is used for updating the corresponding relation table according to the runtime attribute dynamically determined in the rendering process.

Optionally, the apparatus further comprises: the monitoring module is used for monitoring the change of the attribute during running in the stream processing stage; and the second updating module is used for updating the custom data table according to the change of the attribute in the running process.

Optionally, the apparatus further comprises: the processing module is used for sending the rendering resources into the subsequent flow of the rendering pipeline; the rendering model is converted into a 2D image.

In a third aspect, embodiments of the present disclosure provide an electronic device comprising a memory and a processor; the memory is used for storing a computer program; the processor is configured to execute a computer program stored in the memory, and when the computer program is executed, implement the method of the first aspect described above.

In a fourth aspect, embodiments of the present disclosure provide a computer readable storage medium storing a computer program which, when executed, implements the method of the first aspect described above.

The scheme of the present disclosure at least comprises the following beneficial effects:

acquiring a model vertex index and loading custom data; establishing a corresponding relation table of model vertex indexes and custom data; and writing the custom data into the rendering resource according to the corresponding relation table. According to the scheme, a corresponding relation table of model vertex indexes and custom data is established; the custom data is written into the rendering resource according to the corresponding relation table, so that the flexibility of model rendering and customization and individuation of rendering effects are improved, and the problem that the custom data cannot be directly carried to subsequent use in the streaming transmission process before rendering based on the graphics processor driven rendering method is solved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort for a person of ordinary skill in the art.

Fig. 1 is a flow chart of a custom data injection method of a streaming process according to an embodiment of the present disclosure;

FIG. 2 is a data processing flow diagram of a method for custom data injection for a streaming process provided by an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of a custom data injection device for a streaming process according to an embodiment of the present disclosure;

FIG. 4 is a block diagram of an electronic device provided by an embodiment of the present disclosure;

fig. 5 is a schematic hardware structure of an electronic device according to an embodiment of the disclosure.

Detailed Description

Technical solutions in the embodiments of the present disclosure will be clearly described below with reference to the drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some embodiments of the present disclosure, but not all embodiments. All other embodiments obtained by one of ordinary skill in the art based on the embodiments in this disclosure are within the scope of the present disclosure.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, where appropriate, such that embodiments of the disclosure may be practiced in sequences other than those illustrated and described herein, and that the objects identified by "first," "second," etc. are generally of the same type and are not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The digital three-dimensional scene described in the specification of the present disclosure refers to a highly realistic terrain environment for various simulation, visualization, game development, and other applications. The scene incorporates graphics rendering techniques to achieve the desired visual effect and interactivity, with the following core features:

simulating the real terrain: the scene contains terrain generated from real world data, such as satellite images and terrain measurement data. Topographical features include mountains, valleys, plains, and rivers to present a sense of realism of natural topography.

Dynamic environment element: environmental elements in a scene such as weather, lighting, and seasonal variations are all dynamic. For example, simulating day and night alternations, cloud flow, rain erosion, and seasonal vegetation changes, these elements can respond in real-time to user interactions.

Advanced graphic effects: graphics techniques such as ray tracing, physical-Based-Rendering (PBR), high dynamic range imaging (HDR, high Dynamic Range Imaging), and shadow mapping techniques are employed to provide visual detail and depth.

Interactivity and exploratory: the user is free to move around the scene exploring different terrains and environments. Various modes of interaction are supported, including walking, driving, or flight modes.

And (3) simulating an ecological system: the scene comprises the simulation of an animal and plant ecosystem, and animal behaviors and vegetation growth are based on real-world models and data.

Customizable: the scene provides customizable properties, allows a user to adjust terrain, vegetation, environmental conditions, etc. according to specific requirements, and is suitable for customized simulation requirements.

Application field:

-education and training: can be used for education purposes of geography, environmental science and military training.

-game development: providing a rich and realistic environment for game developers and enhancing the immersion of games.

Urban planning and landscape design: helping designers and planners to visualize new building projects or landscape alterations.

Film and television production: providing high quality background scene for film and television production.

The method and apparatus for injecting custom data in a streaming process according to the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings by means of specific embodiments and application scenarios thereof.

Referring to fig. 1, fig. 1 is a flow chart of a custom data injection method of a streaming process according to an embodiment of the present disclosure. The method may be applied to a terminal device, a server, or other devices having data processing and image processing functions, etc., which is not limited by the present disclosure. Embodiments of the present disclosure will be described below by taking a terminal device as an example. As shown in fig. 1, the method may include the steps of:

Step 101, obtaining a model vertex index and loading custom data.

The model vertex index described above may be generated by an index table component of the streaming process of the graphics processor that maps data of model vertices (simply referred to as vertex data) to their specific locations in the rendering model. In 3D graphics processing, a rendering model is typically composed of a series of model vertices, which are organized by a model vertex index. Such organization may more effectively store and reference data for model vertices, particularly in the presence of duplicate model vertices. Through the index table, the rendering system can process the data of the model vertexes more efficiently, and avoid repeated data storage and calculation, so that the rendering performance is optimized.

Optionally, the obtaining the model vertex index and loading the custom data may be implemented by the following steps:

and analyzing vertex data of the rendering model, and determining information such as the position, normal line, texture coordinates and the like of each model vertex. That is, when the model vertex index is obtained, vertex data of the rendering model may be analyzed first to determine vertex attribute information of each model vertex. In a rendering model, multiple faces (triangles, quadrilaterals, etc.) may share model vertices. In generating the index table, these shared model vertices need to be identified to avoid data duplication. Each unique model vertex is assigned an index value. That is, when the model vertex index is obtained, an index value may also be assigned to each model vertex. The index value may be a continuous sequence of integers starting from 0. A mapping relationship is constructed to map vertex data to their index values. This mapping is essentially a look-up table (or referred to as an index table) from vertex data to index values. The final index table is stored in an appropriate format and location so that it can be accessed and utilized by the graphics processor (graphics processing unit, GPU) rendering flow. In addition, when custom data is injected during streaming based on GPU-driven rendering, the index table may also need to contain additional mapping information that is associated with the custom data (e.g., specific shader parameters or special effects data). This means that in addition to considering the standard properties of model vertices, it is necessary to efficiently correlate these custom data to the corresponding model vertices or set of model vertices in creating the index table, that is, when the model vertex index is obtained, it is necessary to determine mapping information for correlating the custom data to the model vertices. It can be seen that the model vertex index needs to include mapping information for associating custom data with model vertices, in addition to vertex attribute information, index values. According to mapping information included in the model vertex index, some custom data may be associated with one model vertex respectively, or some custom data may be associated with each model vertex in the model vertex set respectively.

The custom data may be specific shader parameters, animation data, physical attributes, etc. for enhancing rendering effects or achieving specific visual effects. Custom data needs to be organized in a format suitable for processing and streaming by a graphics processor, for example, the format of the custom data may be optimized by packaging, compressing or other forms of data to ensure that the custom data can be efficiently transmitted and used.

Step 102, building a corresponding relation table of model vertex indexes and custom data.

Optionally, before loading the custom data, corresponding custom data may be constructed or assigned for each model vertex or model vertex set. That is, the terminal device may assign model vertices to the custom data in advance before loading the custom data into the terminal device. In connection with the foregoing, custom data assigned to model vertices may be characterized by mapping information used to correlate the custom data to model vertices. Further, the respective custom data may be constructed or assigned to each model vertex or model vertex set by calculation or extraction of the custom data from other data sources to ensure that each model vertex or model vertex set has the correct custom data associated with it. The built custom data needs to be able to be correctly identified and processed by the various stages in the rendering pipeline, particularly the vertex shader or geometry shader, etc. may need to use portions of this data. Optimization and error handling mechanisms of the data also need to be considered to ensure the correctness and efficiency of the data during rendering. The corresponding relation table associates the custom data with corresponding model vertexes or model vertex sets, and each model vertex index points to a specific model vertex or model vertex group and is associated with the corresponding custom data.

That is, since the model vertex index includes mapping information for associating the custom data with the model vertices, and there is an association relationship between the mapping information and the custom data, a correspondence table of the model vertex index and the custom data can be established according to the correspondence relationship between the model vertex index and the mapping information for associating the custom data with the model vertices.

And step 103, writing the custom data into the rendering resource according to the corresponding relation table.

Optionally, the foregoing writing of the custom data into the rendering resource may be implemented by:

to ensure that custom data and standard rendering data, such as vertex data, are efficiently transferred and stored in memory accessible to the graphics processor so that the graphics processor can quickly access and process the custom data and standard rendering data during actual rendering, it is first necessary to store the vertex data (location, normal, texture coordinates, etc.) and custom data (e.g., specific shader parameters or animation data) in an efficient format. Therefore, when the custom data is written into the rendering resource according to the correspondence table, the custom data needs to be stored according to the efficient format.

Alternatively, vertex data may be stored in a vertex Buffer, while custom data may be stored in a unified Buffer (unified Buffer) or shader memory Buffer (Shader Storage Buffer).

Second, in writing custom data to rendering resources, the data needs to be optimized and organized to ensure efficient memory usage and fast data access. That is, after storing the custom data in an efficient format, it is also necessary to optimize and organize the custom data.

Alternatively, the optimization and organization of the custom data may be achieved by compression, packing, or organization of the data according to a particular memory layout.

Optionally, custom data and standard rendering data are transferred from the central processor memory (or server memory) to the graphics processor memory. May be implemented by invoking a particular graphics processor graphics library (e.g., openGL or DirectX). In addition, in the process of writing custom data into a rendering resource, it is necessary to ensure that the custom data is ready when needed and that the custom data is written correctly into the rendering resource.

In the embodiment of the disclosure, a model vertex index is obtained and custom data is loaded; establishing a corresponding relation table of model vertex indexes and custom data; and writing the custom data into the rendering resource according to the corresponding relation table. According to the scheme, a corresponding relation table of model vertex indexes and custom data is established; and writing the custom data into the rendering resource according to the corresponding relation table, so that the model rendering flexibility, the customization of rendering effect and individuation are improved, and the problem that the custom data cannot be directly carried to subsequent use in the streaming transmission process before the rendering based on the graphics processor driving rendering method is solved.

Optionally, after the step 103, the method further includes: and updating the corresponding relation table according to the run-time attribute.

Wherein the runtime property may be a property dynamically determined in the rendering flow. That is, the runtime attribute may be a runtime attribute dynamically determined in the rendering process, and the correspondence table may be updated according to the runtime attribute dynamically determined in the rendering process. These attributes may be data generated by user input, scene logic, physical simulation results, or other real-time computing processes. For example, these attributes may include the position, direction, speed, dynamically changing texture information, lighting conditions, etc. of the object.

The runtime attribute may be matched to the rendering resource by:

rendering resources (e.g., textures, vertex data, shaders, etc.) are appropriately bound or mapped according to the runtime properties. I.e. according to the current runtime conditions (e.g. camera position, state of objects in the scene, etc.), appropriate rendering resources are selected and configured. During rendering, runtime properties may change, such as object movement, illumination changes, etc. Thus, the selection and use of rendering resources also need to be adjusted to these changes. For example, an appropriate texture map or shader program is selected based on the position and orientation of the object. Wherein shader programs (particularly vertex shaders and fragment shaders) can dynamically calculate vertex positions, colors, texture coordinates, etc. based on runtime attributes during this process. The runtime properties may be passed to the graphics processor through a unified buffer (unified Buffers) or other type of buffer. The contents of these buffers may be read and used by the shader program during the rendering process. A rendering engine is used to make context-aware decisions such as rendering techniques (e.g., level of detail changes, occlusion culling, etc.) that are used depending on the perspective and scene complexity. Rendering engines may also be used to typically provide an application interface or scripting interface, dynamically interacting with rendering resources at runtime, adjusting rendering configuration according to real-time calculated attributes.

Through the steps, proper rendering resources are selected and configured according to the attribute dynamically generated in the running process so as to adapt to the real-time change and the requirement of the scene.

Optionally, in step 101, the obtaining the model vertex index may be implemented by:

the raw 3D model data is first processed. This includes analyzing model vertex information of the rendering model, such as the position of model vertices, normals, texture coordinates, and the like. If the rendering model is modified (e.g., optimized, reconstructed, animated adjustments, etc.), it is necessary to identify which model vertices are modified, added, or deleted. And comparing the vertex data of the original model with the modified model vertex data. An index value is assigned to each model vertex in the original model and the modified model. These index values reflect the locations of model vertices in the respective model data structures. Creating a mapping table, and recording the index relation between the model vertex index in the original model and the corresponding model vertex in the modified model. For newly added model vertices, assigning new indexes; for deleted model vertices, they are marked in the relationship table accordingly. That is, if the rendering model is modified, vertex attribute information, index values of model vertices (e.g., newly added model vertices or deleted model vertices, etc.) that change are required to be modified accordingly. When the model vertex index is acquired, vertex attribute information, index values and mapping information for associating the custom data with the model vertices in each model vertex in the modified rendering model need to be acquired.

The index relationships are tabulated into a format suitable for storage and fast reading, such as binary format, JSON, XML, etc. The index relationship table is saved in a file system, database, or other data storage solution for use in rendering. The index relation table is compressed and optimized to reduce the storage space and improve the reading efficiency. And verifying the accuracy of the index relation table, and ensuring that all mapping relations are correct and correct. Wherein the index relationship table may be used for the following scenarios: model optimization, when the rendering model is optimized (such as LOD generation and vertex reduction), the mapping relation with the original model is maintained. Animation and morphing, in the animation or morphing process, ensure consistency and data integrity between new and old models. Real-time rendering optimization, in real-time rendering based on a GPU, model data is updated and processed quickly.

Optionally, in step 103, writing the custom data into the rendering resource according to the correspondence table may be implemented by: the custom data table (including the correspondence between the new index and the custom data, that is, the correspondence between the new model vertex index and the custom data) is built and imported into the resources used in the rendering pipeline. That is, after the rendering model is modified, the built custom data table may include vertex attribute information, index values, and mapping information for associating the custom data with the model vertices in the modified rendering model, and a correspondence table of the custom data. It should be appreciated that if the rendering model is not modified, the built custom data table includes a table of correspondence between model vertex indices corresponding to unmodified rendering models and custom data. This does not necessarily involve modifying the original vertex data, but rather making such data available during the rendering process. That is, constructing the custom data table may not involve modifying the vertex data in the correspondence table. The custom data that is written includes additional attributes associated with the model vertices (e.g., color, texture coordinates, normals, additional geometric information) or rendering-specific parameters (e.g., shader parameters). Custom data is written to a Buffer or other type of rendering resource of the graphics processor, e.g., through a vertex Buffer, unified Buffer (unified Buffer), or shader memory Buffer (Shader Storage Buffer), etc. The custom data after writing needs to be integrated with other elements in the rendering pipeline (e.g., shaders) to ensure that the data is used correctly during the rendering process. That is, after the custom data table is imported into the rendering resource used by the rendering pipeline, the custom data therein may also be integrated with elements in the rendering pipeline other than the custom data table. And the original vertex data can be updated or adjusted as required.

Therefore, when the custom data is written into the rendering resource according to the corresponding relation table, the custom data table can be constructed first; then, importing the custom data table into rendering resources used by a rendering pipeline; thereafter, the imported custom data is integrated with elements in the rendering pipeline other than the custom data table. When the rendering model is not modified, the custom data table may include a table of correspondence between model vertex indices of the rendering model that are not modified and the custom data. Alternatively, when the rendering model is modified, the custom data table may include a table of correspondence between model vertex indices of the modified rendering model and the custom data.

Through the steps, the custom data table is correctly written into the rendering resource so as to be used in the rendering process, so that the rendering pipeline can access and process the custom data associated with the model vertex index, and more complex and personalized rendering effects are supported.

Optionally, the custom data table may also be updated during the stream processing phase according to the runtime properties of the resource after step 103. Runtime attributes include the position, orientation, state of an object, lighting conditions of a scene, user input, or other real-time computed results. These runtime properties are dynamically changing at the time of rendering. The rendering system monitors the changes in these runtime properties in real-time. This may be by listening for events, polling status, or acquiring updates from other systems such as the physical engine. And updating the data in the custom data table according to the change of the run-time attribute. That is, after the custom data is written into the rendering resource according to the correspondence table, the change of the runtime attribute may also be monitored at the stream processing stage, and then the custom data table may be updated according to the change of the runtime attribute.

For example, if an object moves to a new location, the illumination or texture data associated with the object may need to be updated. This update may include changing the value of existing data, adding new data items, or removing data from the table that is no longer needed. The updated data needs to be synchronized to the graphics processor memory. This may involve re-writing to a buffer or updating a specific area in the graphics processor memory. A delayed update or batch update strategy may also be employed to reduce performance overhead. Ensuring that updated custom data tables can be correctly read and used by relevant parts (e.g., shaders) in the rendering pipeline. Rendering settings or parameters associated with such data may also be updated, such as changing shader programs or adjusting the configuration of the rendering pipeline. Performance impact needs to be considered in the update process to ensure that the data update and transmission process is as efficient as possible.

Optionally, after step 103, the rendering resources may be further sent to a subsequent flow in the rendering pipeline to convert the rendering model into a 2D image.

Fig. 2 is a data processing flow chart of a method for injecting custom data in a streaming process according to an embodiment of the present disclosure, as shown in fig. 2, including:

Wherein, the Game Thread (Game Thread) part represents a step of constructing a static grid in the Game Thread, from acquiring original grid data to constructing the static grid, to acquiring virtual geometry system (Nanit) resources (which may refer to Nanit virtualization geometry technology of Epic Games), and finally operating an ID buffer array, including:

urban grid constructors: constructing a static grid (FUrbanMeshBuilder:: buildStaticmesh), and obtaining grid data (GetMeshData), wherein the static grid is: building (Staticmesh:: build), obtaining virtual geometry system resources (GetNaniteResource), and an identity data buffer array (FIDBufffArray).

Wherein the join Game Thread (Game Thread) and Render Thread (Render Thread) sections describe how the Game Thread adds primitives and components required for rendering to the scene and creates a Nanit scene proxy, which may involve mapping between new and old indexes and creation of page index arrays for subsequent rendering processes, including:

scene: adding graphic elements (FScene:: addPrimive), static grid component: creating a scene proxy (usatic mereskneproxy), a virtual geometry system scene proxy (Nanite:: FSceneProxy), virtual geometry system resources: virtual geometry system cluster index: new to old (NaniteResources:: naniteClusterIndex:: newToOld =)

Component- > featuredataallo), virtual geometry system resources: virtual geometry system page index array (naniteResources:: nanitePageIndexArray = Component- > PageNdx).

Wherein a rendering Thread (Render Thread) portion relates to a stream manager of the rendering Thread that processes asynchronous updates of new resources and registers stream pages associated with self-characterization data, comprising:

a stream manager (fsstreamingmanager), asynchronous update (asynctpdate), stream manager: new resources (FSstreamingmanager:: processNewResources) are processed, self-characterizing data

(selfeaturedata), manager: register stream page (FSstreamingmanager:: registerStreamingPage), self-profile (SelfFeatureData).

It is understood that the foregoing embodiments are merely examples, and modifications may be made to the foregoing embodiments in actual implementation, and those skilled in the art may understand that the modification methods of the foregoing embodiments without performing any inventive effort fall within the protection scope of the present disclosure, and are not repeated in the embodiments.

All the above optional solutions may be mutually referred to or combined to form an optional embodiment of the disclosure, which is not described herein in detail.

Based on the same inventive concept, the embodiments of the present disclosure further provide a custom data injection device for a streaming process, and since the principle of the problem solved by the custom data injection device for the streaming process is similar to that of the custom data injection method for the streaming process, implementation of the custom data injection device for the streaming process can refer to implementation of the custom data injection method for the streaming process, and repeated descriptions are omitted.

Fig. 3 is a schematic structural diagram of a custom data injection device 300 for a streaming process according to an embodiment of the present disclosure, and as shown in fig. 3, the device 300 includes:

the obtaining module 301 is configured to obtain a model vertex index and load custom data; the model vertex index comprises vertex attribute information, index values and mapping information for associating the custom data with model vertices;

the building module 302 is configured to build a correspondence table between the model vertex indexes and the custom data;

the writing module 303 is configured to write the custom data into the rendering resource according to the correspondence table.

Optionally, the obtaining module 301 is configured to obtain a model vertex index, specifically: the acquisition module 301 is configured to: analyzing vertex data of the rendering model, and determining vertex attribute information of each model vertex; assigning an index value to each model vertex; mapping information for associating the custom data with the model vertices is determined.

Optionally, the obtaining module 301 is configured to obtain a model vertex index, specifically: the acquisition module 301 is configured to: and if the rendering model is modified, obtaining vertex attribute information, index values and mapping information for associating the custom data with the model vertices in each model vertex in the modified rendering model.

Optionally, the writing module 303 is configured to write the custom data into the rendering resource according to the correspondence table, specifically: the writing module 303 is configured to: constructing a custom data table; the custom data table comprises a corresponding relation table; importing a custom data table into rendering resources used by a rendering pipeline; custom data imported into rendering resources used by the rendering pipeline is integrated with elements in the rendering pipeline other than the custom data table.

Optionally, the apparatus 300 further includes: and the first updating module is used for updating the corresponding relation table according to the runtime attribute dynamically determined in the rendering process.

Optionally, the apparatus 300 further includes: the monitoring module is used for monitoring the change of the attribute during running in the stream processing stage; and the second updating module is used for updating the custom data table according to the change of the attribute in the running process.

Optionally, the apparatus 300 further includes: the processing module is used for sending the rendering resources into the subsequent flow of the rendering pipeline; the rendering model is converted into a 2D image.

It should be noted that: the foregoing embodiment provides a custom data injection device for a streaming process, which is only exemplified by the division of the foregoing functional modules, and in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to perform all or part of the functions described above. In addition, the custom data injection device of the streaming process and the custom data injection method embodiment of the streaming process provided in the foregoing embodiments belong to the same concept, and detailed implementation processes of the custom data injection device of the streaming process are detailed in the method embodiment and are not described herein.

The custom data injection device for a streaming process in the embodiments of the present disclosure may be a virtual device, or may be a component, an integrated circuit, or a chip in a server or a terminal. The device may be a mobile electronic device or a non-mobile electronic device. By way of example, the mobile electronic device may be a cell phone, tablet computer, notebook computer, palm top computer, vehicle mounted electronic device, wearable device, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device may be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the disclosure are not limited in particular.

A custom data injection device for a streaming process in embodiments of the present disclosure may be a device with an operating system. The operating system may be an Android operating system, an ios operating system, or other possible operating systems, and the embodiments of the present disclosure are not limited specifically.

The custom data injection device for a streaming transmission process provided in the embodiments of the present disclosure can implement each process implemented by the above method embodiments, and in order to avoid repetition, a description is omitted here.

The embodiment of the disclosure further provides an electronic device, referring to fig. 4, and fig. 4 is a block diagram of the electronic device provided by the embodiment of the disclosure. As shown in fig. 4, the electronic device 400 may include a processor 401, a memory 402, and a program or an instruction stored in the memory 402 and capable of running on the processor 401, where the program or the instruction implements each process of the custom data injection method embodiment of the streaming process when executed by the processor 401, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein. It should be noted that, the electronic device in the embodiment of the present disclosure includes a mobile electronic device and a non-mobile electronic device.

Fig. 5 is a schematic hardware structure of an electronic device implementing an embodiment of the disclosure.

The electronic device 500 includes, but is not limited to: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.

Those skilled in the art will appreciate that the electronic device 500 may further include a power source (e.g., a battery) for powering the various components, and that the power source may be logically coupled to the processor 510 via a power management system to perform functions such as managing charging, discharging, and power consumption via the power management system. The electronic device structure shown in fig. 5 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than shown, or may combine certain components, or may be arranged in different components, which are not described in detail herein.

It should be appreciated that in embodiments of the present disclosure, the input unit 504 may include a graphics processor (Graphics Processing Unit, GPU) 5041 and a microphone 5042, the graphics processor 5041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 507 includes a touch panel 5071 and other input devices 5072. Touch panel 5071, also referred to as a touch screen. Touch panel 5071 may include two parts, a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein. The memory 509 may be used to store software programs as well as various data including, but not limited to, application programs and an operating system. Processor 510 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 510.

The embodiments of the present disclosure further provide a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the respective processes of the foregoing embodiments of the method for injecting custom data in a streaming process, and can achieve the same technical effects, and in order to avoid repetition, a detailed description is omitted herein.

The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic or optical disks, and the like.

The embodiment of the present disclosure further provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction, to implement each process of the foregoing embodiment of the method for injecting custom data in a streaming process, and to achieve the same technical effect, so that repetition is avoided, and no further description is given here.

It should be understood that the chips referred to in the embodiments of the present disclosure may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus and system embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the description of method embodiments in part. In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The orientation or positional relationship indicated by the terms "upper", "lower", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description and to simplify the description, and are not indicative or implying that the apparatus or elements being referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the present disclosure. Unless specifically stated or limited otherwise, the terms "mounted," "connected," and "coupled" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in this disclosure will be understood by those of ordinary skill in the art as the case may be. It should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure is not limited to any single aspect, nor to any single embodiment, nor to any combination and/or permutation of these aspects and/or embodiments. Moreover, each aspect and/or embodiment of the disclosure may be used alone or in combination with one or more other aspects and/or embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present disclosure, and not for limiting the same; although the present disclosure has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the embodiments of the disclosure, and are intended to be included within the scope of the claims and specification of the present disclosure.

Claims (10)

1. A method for custom data injection for a streaming process, the method comprising:

acquiring a model vertex index and loading custom data; the model vertex index comprises vertex attribute information, index values and mapping information for associating the custom data with model vertices;

establishing a corresponding relation table of the model vertex index and the custom data;

and writing the custom data into rendering resources according to the corresponding relation table.

2. The method of claim 1, wherein obtaining the model vertex index comprises:

Analyzing vertex data of the rendering model, and determining vertex attribute information of each model vertex;

assigning an index value to each model vertex;

mapping information for associating the custom data with model vertices is determined.

3. The method of claim 1, wherein obtaining the model vertex index comprises:

and if the rendering model is modified, obtaining vertex attribute information, index values and mapping information for associating the custom data with the model vertices in each model vertex in the modified rendering model.

4. The method of injecting custom data for a streaming process according to claim 1, wherein writing the custom data into a rendering resource according to the correspondence table comprises:

constructing a custom data table; the custom data table comprises the corresponding relation table;

importing the custom data table into rendering resources used by a rendering pipeline;

custom data imported into rendering resources used by a rendering pipeline is integrated with elements in the rendering pipeline other than custom data tables.

5. The method for injecting custom data into a streaming process according to any one of claims 1 to 4, wherein after writing the custom data into a rendering resource according to the correspondence table, further comprising:

And updating the corresponding relation table according to the run-time attribute dynamically determined in the rendering process.

6. The method of injecting custom data for a streaming process according to claim 4, further comprising, after writing the custom data into a rendering resource according to the correspondence table:

monitoring changes in runtime properties during the stream processing phase;

and updating the custom data table according to the change of the run-time attribute.

7. The method of injecting custom data for a streaming process according to claim 1, further comprising, after writing the custom data into a rendering resource according to the correspondence table:

sending the rendering resources into a subsequent flow of a rendering pipeline;

the rendering model is converted into a 2D image.

8. A custom data injection device for a streaming process, the device comprising:

the acquisition module is used for acquiring the model vertex index and loading the custom data; the model vertex index comprises vertex attribute information, index values and mapping information for associating the custom data with model vertices;

the building module is used for building a corresponding relation table of the model vertex index and the custom data;

And the writing module is used for writing the custom data into rendering resources according to the corresponding relation table.

9. An electronic device, the electronic device comprising:

a memory for storing a computer program;

a processor for executing a computer program stored in the memory, and which, when executed, implements the method of any of the preceding claims 1 to 7.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed, implements the method of any of the preceding claims 1 to 7.