CN111866065A - Method and device for returning rendered image - Google Patents

Abstract

The present disclosure provides a method and a device for returning a rendered image, which relate to the field of image processing and can solve the problem of large data transmission overhead in the process of returning a remote rendered image. The provided method for returning a rendered image includes: an application sending a rendering instruction to the rendering; the application sending a data processing request to the rendering when the rendering result needs to be processed; receiving the rendering according to the The rendering result data returned by the data processing request. With the technical solutions provided by the embodiments of the present invention, the application end can notify the rendering end to return and process the rendering result data in a targeted manner through a data processing request, thereby reducing the overhead of data transmission between the application end and the rendering end.

Description

Method and device for returning rendered image

technical field

The present disclosure relates to the field of image processing, and in particular, to a method and apparatus for returning a rendered image.

Background technique

Rendering refers to the use of computer image generation technology, by attaching certain materials, textures, colors and other computational graphic elements to the geometric model of data-based objects in the computer, under certain light source settings, calculate and generate realistic object state scenes. image. In the remote rendering scenario, the application side and the rendering side are separated, and the rendering requirements of the application side can be completed by one or more computers on the rendering side; the rendering requirements can also be sent to the remote server for rendering, and the server executes the corresponding rendering instructions according to the rendering instructions. The rendering task and the rendering result are sent back to the application side. This remote rendering method is also called cloud rendering.

In the process of remote rendering, the application side often needs to process a large number of intermediate rendering results. In related technologies, when the application side needs to process the intermediate rendering results, it will directly send an operation request to the rendering side. The client directly returns the intermediate rendering results to the application side, and after the application side completes the processing, it is sent to the rendering side for further rendering operations; this processing method makes the data transmission overhead between the application side and the rendering side larger, which requires a larger bandwidth.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide a method and an apparatus for returning a rendered image, which can solve the problem of high overhead in data transmission between an application end and a rendering end in the related art.

According to a first aspect of the embodiments of the present disclosure, there is provided a method for returning a rendered image, which can be applied to an application end. The method includes: the application end sends a rendering instruction to the rendering end; when the application end needs to process the rendering result , sending a data processing request to the rendering end; and receiving the rendering result data returned by the rendering end according to the data processing request. Through the implementation of the method, the application end can inform the rendering end to carry out the return and processing of the rendering result data in a targeted manner through the data processing request, thereby reducing the overhead of data transmission between the application end and the rendering end.

In some embodiments, when the application end needs to perform rendering result processing, sending a data processing request to the rendering end may include: the application end application generates an operation request and sends it to the detection module of the rendering end; the The detection module determines the operation type of the operation request, and sends the data processing request to the rendering end according to the operation type.

In some embodiments, the operation request may be a read/write request, and the operation category may be a read operation and/or a write operation (also referred to as a read/write operation for short).

In some embodiments, the data processing request includes: requesting the rendering end to return rendering result data of a specified size at a specified location; or requesting the rendering end to update the rendering result data of the specified content to a specified location.

Through the above rendering image return method, the application side can send different data processing requests for different operation types, which not only satisfies the needs of the application side for processing intermediate rendering results, but also enables the rendering side to perform different processing requests for different data. The data processing of rendering results using the new method solves the problem that the rendering end needs to directly return all rendering results in the related art, and the data transmission overhead is relatively large.

According to a second aspect of the embodiments of the present disclosure, there is provided another method for returning a rendered image, which can be applied to a rendering end. The method includes: the rendering end receives a rendering instruction sent by the application end, and performs image rendering according to the rendering instruction ; The rendering end receives the data processing request sent by the application end, and performs rendering result data processing according to the data processing request. Through the implementation of the technical solution, the rendering end can process the rendering result data in a targeted manner according to the data processing request sent by the application end, and it is no longer necessary to directly return the complete intermediate rendering result data, thereby reducing the data transmission overhead.

In some embodiments, the rendering end performing rendering result data processing according to the data processing request includes: returning rendering result data of a specified size at a specified location to the application end according to the data processing request; or according to the data processing request; The above data processing request updates the rendering result data of the specified content to the specified location.

Through the rendering image return method provided by the above-mentioned embodiment, the rendering end can, in response to the data processing request sent by the application end, return the rendering result data of the specified size at the specified location to the application end in a targeted manner; The data processing request updates the rendering result data of the specified content to the specified location. It is no longer necessary to directly return the complete intermediate rendering result data, which can greatly reduce the data transmission overhead.

According to a third aspect of the embodiments of the present disclosure, there is provided an apparatus for returning a rendered image, which is applied to an application side. The apparatus includes an application program module and a detection module. The application program module is configured to send a rendering instruction to the rendering side. The detection module is configured to send a data processing request to the rendering end when rendering result processing needs to be performed; the detection module is further configured to receive the rendering result data returned by the rendering end according to the data processing request. Through the device, the application end can inform the rendering end to carry out the return and processing of the rendering result data in a targeted manner through the data processing request, which reduces the problem of large data transmission overhead between the application end and the rendering end.

In some embodiments, the application program module can also be used to generate and send an operation request to the detection module; the detection module can also be used to determine the operation type of the operation request, and send the operation request according to the operation type. The data processing request is sent to the rendering end.

In some embodiments, the above operation request may be a read/write request, and the operation category may be a read operation and/or a write operation.

In some embodiments, the data processing request may include: requesting the rendering end to return rendering result data of a specified size at a specified location, or requesting the rendering end to update the rendering result data of the specified content to a specified location.

Through the above rendering image return device, the application side can send different data processing requests for different operation types, which not only satisfies the needs of the application side for processing intermediate rendering results, but also enables the rendering side to perform different processing requests for different data. The data processing of rendering results using the new method solves the problem that the rendering end needs to directly return all rendering results in the related art, and the data transmission overhead is relatively large.

According to a fourth aspect of the embodiments of the present disclosure, there is provided another apparatus for returning a rendered image, which is applied to a rendering end, wherein the apparatus includes a receiving module and an executing module; the receiving module is configured to receive the data sent by the application end. The rendering instruction, the execution module is used to perform image rendering according to the rendering instruction, the receiving module is also used to receive the data processing request sent by the sending module; the execution module is also used to process the request according to the data Perform rendering result data processing. Through the device, the rendering end can process the rendering result data in a targeted manner according to the data processing request sent by the application end, and it is no longer necessary to directly return the complete intermediate rendering result data, thereby reducing the data transmission overhead.

In some embodiments, the execution module performing rendering result data processing according to the data processing request includes: returning rendering result data of a specified size at a specified location to the application according to the data processing request; or according to the data processing request; The data processing request updates the rendering result data of the specified content to the specified location.

With the rendered image returning device provided by the above embodiment, the rendering end can, in response to the data processing request sent by the application end, return the rendering result data of the specified size at the specified location to the application end in a targeted manner; The data processing request updates the rendering result data of the specified content to the specified location. It is no longer necessary to directly return the complete intermediate rendering result data, which can greatly reduce the data transmission overhead.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

Figure 1 is a schematic diagram of a remote rendering system;

2 is a flowchart of a method for returning a rendered image provided by an embodiment of the present disclosure;

3 is a flowchart of another method for returning a rendered image provided by an embodiment of the present disclosure;

4 is a flowchart of another method for returning a rendered image provided by an embodiment of the present disclosure;

5 is a schematic diagram of an application-side read-write operation process provided by an embodiment of the present disclosure;

6 is a structural block diagram of an apparatus for returning a rendered image provided by an embodiment of the present disclosure;

FIG. 7 is a structural block diagram of another apparatus for returning a rendered image provided by an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

Figure 1 is a schematic diagram of a remote rendering system. As shown in FIG. 1 , an application terminal 101 and a rendering terminal 102 in the remote rendering system, wherein the application terminal 101 can also be called a user terminal, which can be a user computer, a mobile phone terminal, a tablet computer, etc.; an image can be installed in the application terminal 101 Rendering applications such as 3D applications etc. The rendering end 102 may be one or more computers or servers deployed in a local area network, or may be one or more computers or servers in the cloud (multiple computers or servers are not shown in the figure). The implementation principle of the rendering system for rendering is as follows: the application terminal 101 sends the image rendering instructions generated by the local 3D application to the rendering terminal 102; the rendering terminal 102 executes the corresponding image rendering instructions to perform image rendering, and after completing the image rendering The result is sent back to the application terminal 101 . In some embodiments, the rendering includes 3D rendering, the application side is also referred to as a 3D application side, and the rendering side is also referred to as a 3D rendering side.

In the related art, in a scenario where the application side 101 needs to process the intermediate rendering result, it will directly send an operation request to the rendering side 102, and the rendering side 102 directly sends the intermediate rendering result to the application side 101 after receiving the operation request from the application side. The application end 101 sends it to the rendering end 102 after the processing is completed. This processing method makes the data transmission overhead between the application side and the rendering side larger, and requires larger bandwidth.

After research and analysis, the rendering results or intermediate rendering results rendered by the application end 101 to the rendering end 102 generally have the following types of operations:

Operation 1: Save, that is, save all or part of the data of the rendering result;

Operation 2: Modify, that is, calculate all or part of the rendering result;

Operation 3: Replace, that is, cover all or part of the rendering result.

Among these three operations, operation 1 and operation 2 need to read the actual content of the rendering result, and need to return the image data, but if it is partially saved or modified, you do not need to return all the data, just return some data. ; The purpose of operation 3 is to cover, and the application side does not actually need to know what the rendering result is. Therefore, the rendering result returned by operation 3 can be omitted.

FIG. 2 is a flowchart of a method for returning a rendered image provided by an embodiment of the present disclosure. As shown in Figure 2, the method includes the following steps:

201: The application side sends a rendering instruction to the rendering side;

202: The application end sends a data processing request to the rendering end when rendering result processing is required;

203: Receive rendering result data returned by the rendering end according to the data processing request.

Through the implementation of this method, the application end can inform the rendering end to carry out the return and processing of the rendering result data in a targeted manner through the data processing request, which reduces the problem of large data transmission overhead between the application end and the rendering end.

In some embodiments, sending a data processing request to the rendering end when the application end needs to process the rendering result in step 202 may include: the application end application generates an operation request and sends it to the detection module of the rendering end; The detection module determines the operation type of the operation request, and sends the data processing request to the rendering end according to the operation type.

In some embodiments, the data processing request may include: requesting the rendering end to return rendering result data of a specified size at a specified location; or requesting the rendering end to update the rendering result data of the specified content to a specified location.

Through the above rendering image return method, the application side can send different data processing requests for different operation types, which not only satisfies the needs of the application side for processing intermediate rendering results, but also enables the rendering side to perform different processing requests for different data. The data processing of rendering results using the new method solves the problem that the rendering end needs to directly return all rendering results in the related art, and the data transmission overhead is relatively large.

FIG. 3 is a flowchart of another method for returning a rendered image provided by an embodiment of the present disclosure. As shown in Figure 3, the method includes the following steps:

301: The rendering terminal receives the rendering instruction sent by the application terminal, and performs image rendering according to the rendering instruction;

302: The rendering end receives the data processing request sent by the application end;

303: The rendering end performs rendering result data processing according to the data processing request.

Through the implementation of the above method, the rendering end can process the rendering result data in a targeted manner according to the data processing request sent by the application end, and it is no longer necessary to directly return the complete intermediate rendering result data, which reduces the data transmission overhead.

In some embodiments, the rendering result data processing performed by the rendering end according to the data processing request in step 303 may include: returning the rendering result data of the specified size at the specified location to the application end according to the data processing request; or according to the data processing request; The above data processing request updates the rendering result data of the specified content to the specified location.

In some embodiments, through the implementation of the above method for returning a rendered image, the rendering end may, in response to the data processing request sent by the application end, return the rendering result data of the specified size at the designated location to the application end in a targeted manner; Or update the rendering result data of the specified content to the specified position according to the data processing request. It is no longer necessary to directly return the complete intermediate rendering result data, which can greatly reduce the data transmission overhead.

Based on the method for returning a rendered image provided by the embodiments corresponding to FIG. 2 and FIG. 3 , another embodiment of the present disclosure provides a method for returning a rendered image, which can be applied to a remote image rendering system. FIG. 4 is a flowchart of another method for returning a rendered image provided by an embodiment of the present disclosure. As shown in Figure 4, the image rendering processing method includes the following steps:

401: The application side sends an image rendering instruction to the rendering side through the 3D application;

402: The rendering end receives the image rendering instruction, and performs image rendering; in some embodiments, the rendering result of the rendering end may be stored in the memory of a graphics processor (Graphic Processing Unit, GPU for short).

403: When the application side needs to process the intermediate rendering result, the application side 3D application generates an operation request and sends it to the detection module of the application side;

404: After receiving the operation request, the application-side detection module judges the operation type of the operation request;

405: The application-side detection module determines the data processing request according to the judgment result and sends it to the rendering end;

405: The rendering end performs corresponding processing according to the received data processing request.

In some embodiments, the operation type of the above operation request may be a read operation and/or a write operation, and may also be referred to as a read-write operation; the detection module may be a read-write detection module.

In some embodiments, in step 404, the detection module determines that the operation type of the operation request is a read operation, then the detection module obtains the read position and size, and the data processing request sent to the rendering end is: requesting the rendering end to return the corresponding position data of the corresponding size. During specific implementation, the data processing request may include location information and data size information. After the rendering end receives the data processing request, it will return the data of the corresponding size in the corresponding position to the application end.

In some embodiments, in step 404, the detection module determines that the operation type of the operation request is a write operation, then the detection module obtains the write location and the written content, and the data processing requirements sent to the rendering are: requesting the rendering end to send the corresponding The content is updated to the corresponding location. During specific implementation, the data processing request may include location information and content information. After receiving the data processing request, the rendering end updates the corresponding content to the corresponding position.

In some embodiments, the above method further includes that the application side performs processing of the intermediate rendering result according to the data returned by the rendering side.

FIG. 5 is a schematic diagram of an application-side read and write operation process provided by an embodiment of the present disclosure. As shown in Figure 5, in some embodiments, the read and write operation process can be divided into the following steps:

501: Send a mapping request; that is, request to map the memory of the GPU to the CPU side, so that the 3D application can access;

502: read and write operations; in some embodiments, the read and write operations may be implemented using a 3D application programming interface (Application Programming Interface, API for short) proprietary read and write functions, or general memory read and write operations;

503: Unmapping; Unmapping also means that the access is completed, and the relevant modifications are submitted to the GPU.

According to the above steps of the read and write operation process, the detection module can use the support of the operating system or the support of the processor memory management unit to determine the operation type of the operation request, and determine the data processing request sent to the rendering end. The following uses the support of the operating system to determine the operation type of the operation request as an example to illustrate a possible implementation:

In step 501, when the 3D application on the application side sends a mapping request, it can request the operating system to map a piece of memory space, the size of which is the size of accessing GPU resources, set the mapped memory to have no read and write permissions, and return the mapped memory address of the segment. for 3D applications.

In step 502, when the 3D application on the application side performs the actual read and write operations, because the above returned mapping address does not have read and write permissions, no matter whether it is reading or writing, a memory access exception will be triggered.

In the memory exception handler, according to the address generated by the exception, compared with the address returned before, the access location can be obtained, and according to the cause of the exception, the detection module can determine whether it is a read access or a write access. In some embodiments, some operating system platforms do not provide information about the cause of the exception, and the detection module needs to use stack backtracking to analyze the CPU instruction executed when the exception occurs, and determine whether it is a read access or a write access according to the instruction type. The detection module can obtain the read and write data size through the 3D API proprietary read and write functions. For the general memory read and write methods, it can be obtained through the stack backtracking analysis function call parameters.

In step 503, when the mapping is canceled, for the write operation, the write position, size and content information obtained in the above steps can be sent to the rendering end.

Through the above methods, on the one hand, for the write operation, unnecessary data transmission in the write operation can be avoided, and the content of the write operation can be directly updated to the corresponding position in the rendering result of the 3D rendering end; Each read operation returns the rendering result of the entire frame, but directly returns the data at the storage location to be read.

FIG. 6 is a structural block diagram of an apparatus for returning a rendered image provided by an embodiment of the present invention. The device can be applied to the application side. As shown in FIG. 6 , the apparatus 600 for rendering and returning an image includes an application module 601 and a detection module 602 . The application module 601 is used for sending rendering instructions to the rendering end, and the detection module 602 is used for sending a data processing request to the rendering end when rendering result processing needs to be performed. In some embodiments, the detection module 601 is further configured to receive rendering result data returned by the rendering end according to the data processing request.

Through this device, the application end can inform the rendering end to return and process the rendering result data in a targeted manner through the data processing request. In some embodiments, the detection module 601 can be used to determine the usage of the rendering result by the operation request. , and according to the usage situation to determine whether the rendering end needs to return the actual image data, and when the data needs to be returned, determine how much image data to return, so as to reduce unnecessary image data return, thereby improving processing efficiency, Reduce transmission overhead.

In some embodiments, the application program module 601 can also be used to generate and send an operation request to the detection module 602; the detection module 601 can also be used to determine the operation type of the operation request, and send the data according to the operation type Process the request to the renderer.

In some embodiments, the above operation request may be a read/write request, and the operation category may be a read operation and/or a write operation. In some embodiments, the data processing request may include: requesting the rendering end to return rendering result data of a specified size at a specified location, or requesting the rendering end to update the rendering result data of the specified content to a specified location.

The application side sends different data processing requests for different operation types, which not only satisfies the needs of the application side for processing intermediate rendering results, but also enables the rendering side to perform different rendering results data processing for different data processing requests. In the technology, the rendering end needs to directly return all rendering results, and the data transmission overhead is relatively large.

FIG. 7 is a structural block diagram of another apparatus for returning a rendered image provided by an embodiment of the present invention. The device can be applied to the rendering side. As shown in FIG. 7 , the apparatus 700 for returning a rendered image includes a receiving module 701 and an executing module 702; the receiving module 701 is configured to receive a rendering instruction sent by an application, and the executing module 702 is configured to perform image rendering according to the rendering instruction. In some embodiments, the receiving module 701 is further configured to receive a data processing request sent by the sending module; the executing module 702 is further configured to perform rendering result data processing according to the data processing request. Through the device, the rendering end can process the rendering result data in a targeted manner according to the data processing request sent by the application end, and it is no longer necessary to directly return the complete intermediate rendering result data, thereby reducing the data transmission overhead.

In some embodiments, the execution module 702 performing rendering result data processing according to the data processing request includes: returning rendering result data of a specified size at a specified location to the application according to the data processing request; or according to the data processing request; The data processing request updates the rendering result data of the specified content to the specified location.

With the rendered image returning device provided by the above embodiment, the rendering end can, in response to the data processing request sent by the application end, return the rendering result data of the specified size at the specified location to the application end in a targeted manner; The data processing request updates the rendering result data of the specified content to the specified location. It is no longer necessary to directly return the complete intermediate rendering result data, which can greatly reduce the data transmission overhead.

Based on the method for returning a rendered image described in the embodiments corresponding to FIG. 2 and FIG. 3 , an embodiment of the present disclosure further provides a computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be read-only. Memory (English: Read Only Memory, ROM), random access memory (English: Random Access Memory, RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, etc. The storage medium stores computer instructions for executing the data transmission methods described in the embodiments corresponding to FIG. 2 and FIG. 3 , and details are not described herein again.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (10)

1. A method for returning a rendered image, wherein the method comprises: an application terminal sends a rendering instruction to the rendering terminal; when the application terminal needs to perform rendering result processing, sending a data processing request to the rendering terminal ; Receive the rendering result data returned by the rendering end according to the data processing request. 2. The method according to claim 1, wherein when the application terminal needs to perform rendering result processing, sending a data processing request to the rendering terminal, comprising: The application side application generates an operation request and sends it to the detection module of the rendering side; The detection module determines the operation type of the operation request, and sends the data processing request to the rendering end according to the operation type. 3. The method according to claim 1 or 2, wherein the data processing request comprises: requesting the rendering end to return rendering result data of a specified size at a specified location; or requesting the rendering end to send back the specified content The rendering result data is updated to the specified position. 4. A method for returning a rendered image, characterized in that the method comprises: a rendering end receives a rendering instruction sent by an application end, and performs image rendering according to the rendering instruction; the rendering end receives a rendering instruction sent by the application end; The data processing request, and the rendering result data processing is performed according to the data processing request. 5 . The method according to claim 4 , wherein the rendering result data processing performed by the rendering end according to the data processing request comprises: returning the rendering result data of a specified size at a specified location according to the data processing request. 6 . to the application end; or update the rendering result data of the specified content to the specified position according to the data processing request. 6. A device for rendering image return, applied to an application terminal, characterized in that the device comprises an application program module and a detection module, the application program module is used to send rendering instructions to the rendering terminal, and the detection module uses When rendering result processing needs to be performed, a data processing request is sent to the rendering end; the detection module is further configured to receive the rendering result data returned by the rendering end according to the data processing request. 7. The apparatus according to claim 6, wherein the application program module is further configured to generate and send an operation request to the detection module; the detection module is further configured to determine the operation category of the operation request, and send the data processing request to the rendering end according to the operation category. 8. The apparatus according to claim 6 or 7, wherein the data processing request comprises: requesting the rendering end to return rendering result data of a specified size at a specified location, or requesting the rendering end to send back the specified content The rendering result data is updated to the specified position. 9. An apparatus for rendering image return, applied to a rendering terminal, characterized in that the apparatus comprises a receiving module and an executing module; the receiving module is used for receiving a rendering instruction sent by an application terminal, and the executing module is used for Image rendering is performed according to the rendering instruction, and the receiving module is further configured to receive a data processing request sent by the sending module; the executing module is further configured to perform rendering result data processing according to the data processing request. 10 . The apparatus according to claim 9 , wherein the execution module performing rendering result data processing according to the data processing request comprises: returning rendering result data of a specified size at a specified location according to the data processing request. 11 . to the application end; or update the rendering result data of the specified content to the specified position according to the data processing request.

Images