WO2021008183A1

WO2021008183A1 - Data transmission method and apparatus, and server

Info

Publication number: WO2021008183A1
Application number: PCT/CN2020/086293
Authority: WO
Inventors: 李超; 王知明; 范志刚
Original assignee: 西安万像电子科技有限公司
Priority date: 2019-07-16
Filing date: 2020-04-23
Publication date: 2021-01-21
Also published as: CN110557624A

Abstract

Provided are a data transmission method and apparatus, and a server, wherein same relate to the technical field of cloud computing, and can solve the problems of servers in current cloud desktop technology being unable to support a large number of 3D image rendering work tasks and the servers being difficult to upgrade. The specific technical solution involves: a data processing apparatus receiving a request from a client and then forwarding the request to a server; when the request is an image rendering request, the server acquiring an instruction corresponding to image rendering, wherein the instruction comprises: a graphics card configuration instruction, a special effect instruction and a drawing instruction; the server determining an image processing unit, corresponding to the client, in the data processing apparatus, and transmitting the instruction to the data processing apparatus, so that the corresponding image processing unit completes image rendering and then transmits image data to the client, wherein in the server, there is no need to perform image rendering according to the instruction.

Description

Data transmission method, device and server

Technical field

The present disclosure relates to the field of cloud computing technology, and in particular to data transmission methods, devices, and servers.

Background technique

Cloud desktop technology refers to the technology in which users remotely access the server through the client, and the server provides the desktop image of the current server host to the remote client in a virtual manner. The server transmits the image to the user through the screen copy, and the user obtains the desktop image sequence. After the image is compressed, it is transmitted to the cloud terminal through the network, and the terminal decodes the desktop image content.

In many existing scenarios, for example, network games, three-dimensional (3D) video, or 3D remote desktop transmission, etc., 3D image transmission and client display are required. In this case, the server side often needs to support a large number of client-side 3D image rendering jobs at the same time, which requires very high graphics processing unit (GPU) resources on the server side, but the existing servers generally do not have such powerful GPU resources, even those without 3D rendering capabilities. If you directly configure and upgrade the server, the server needs to be interrupted. During the interrupted period of time, the server will consume a certain amount of time and cost, and the configuration upgrade process needs to transform the interior of the server rack. This process is also very cumbersome. difficult.

Summary of the invention

The embodiments of the present disclosure provide a data transmission method, device, and server, which can solve the problems that the server cannot support a large number of 3D image rendering tasks in the current cloud desktop technology and the difficulty of upgrading the server is very high. The technical solution is as follows:

According to a first aspect of the embodiments of the present disclosure, there is provided a data transmission method, the method including:

The server receives the client's request forwarded by the data processing device;

When the request is an image rendering request, the server obtains the instructions corresponding to the image rendering; the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

The server determines the image processing unit corresponding to the client in the data processing device;

The server transmits the instruction to the data processing device so that the image processing unit transmits the image data to the client after completing the image rendering according to the instruction; wherein, there is no need to perform image rendering according to the instruction in the server.

The data processing device, which is a device set between the server and the client, completes the image rendering task of the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

In one embodiment, the method further includes:

The server divides the image processing module of the data processing device into N image processing units; among them, each image processing unit can independently complete an image rendering task.

The server divides the image processing module of the data processing device into several image processing units that can complete image tasks independently, and assign them to each client. The image processing unit can be used more efficiently to complete image rendering, avoiding multiple clients competing for resources and The waiting time is longer.

In an embodiment, the server determining the image processing unit corresponding to the image rendering request of the client in the data processing apparatus includes:

The server obtains the identification of the client from the image rendering request;

The server judges whether there is an image processing unit corresponding to the client according to the identifier of the client;

If it does not exist, the server allocates a corresponding image processing unit to the client, and binds the client to the corresponding image processing unit.

By binding the image processing unit with the client, it is convenient to select the corresponding image processing unit for direct processing after the client sends the image rendering request, and the image rendering request of the same client can be handed over to the same image processing unit to improve processing efficiency.

In an embodiment, the server assigns the corresponding image processing unit to the client, including:

The server judges whether there is currently an idle image processing unit;

If it exists, the server selects an idle image processing unit to allocate to the client;

If it does not exist, the server selects an image processing unit to allocate to the client according to a preset rule.

In one embodiment, the server selects an image processing unit to allocate to the client according to preset rules, including:

The server finds the image processing unit with the least number of bound clients and allocates it to the client; or,

The server finds out the image processing unit with the lowest current processing load and assigns it to the client.

In one embodiment, the server acquiring the instruction corresponding to image rendering includes:

The server intercepts the drawing instructions sent by the D3D RT program in the server's operating system to the user mode display driver;

The server intercepts the graphics card configuration instructions and special effect instructions sent by the graphics kernel subsystem in the server's operating system to the display miniport driver.

The server operating system generates instructions in accordance with the client's request for 3D image rendering. By intercepting the instructions, the server can be used to perform 3D image rendering, with minimal changes and no burden on the server.

In one embodiment, the method further includes:

When the request is a request other than the image rendering request, the server processes the request according to the request and then transmits the processing result to the data processing device so that the data processing device forwards it to the client.

For a general cloud desktop client request, what the server can directly process is directly processed in the server without being processed by the data processing device. It only needs to forward the processing result, such as computer desktop image data, to the client through the data processing device.

According to a second aspect of the embodiments of the present disclosure, there is provided a data transmission method, including:

The data processing device receives the client's request;

The data processing device sends the client's request to the server;

When the request is an image rendering request, the data processing device receives the instruction corresponding to the image rendering sent by the server; the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

The data processing device completes image rendering through the image processing module of the data processing device according to the instruction;

The data processing device sends the image data obtained after the image rendering is completed to the client.

The data processing device, a device set between the server and the client, completes the image rendering task of the client, so that the server can support a large number of image rendering requests from the client without a troublesome upgrade.

In an embodiment, the data processing device performs image rendering through the image processing module of the data processing device according to the instruction, including:

The image processing unit corresponding to the client in the image processing module completes the image rendering according to the instruction; where the image processing unit is allocated by the server to the client in the image processing module for correspondingly processing the image rendering request of the client.

In one embodiment, the method further includes:

When the request is a request other than an image rendering request, the data processing device receives the processing result obtained after the server processes the request according to the request;

The data processing device sends the processing result to the client.

According to a third aspect of the embodiments of the present disclosure, a server is provided, including:

The receiving module is used to receive the client's request forwarded by the data processing device;

The obtaining module is used to obtain instructions corresponding to image rendering when the request is an image rendering request; wherein the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

The determining module is used to determine the image processing unit corresponding to the client in the data processing device;

The transmission module is used to transmit the instruction to the data processing device so that the image processing unit transmits the image data to the client after completing the image rendering according to the instruction; wherein, there is no need to perform image rendering according to the instruction in the server.

In one embodiment, the server further includes:

The dividing module is used to divide the image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete the image rendering task.

In an embodiment, the determining module is specifically used for:

Obtain the client's ID from the image rendering request;

Determine whether there is an image processing unit corresponding to the client according to the identifier of the client;

If it does not exist, assign the corresponding image processing unit to the client, and bind the client to the corresponding image processing unit.

In an embodiment, the determining module is specifically used for:

Determine whether there is currently an idle image processing unit;

If it exists, select an idle image processing unit to allocate to the client;

If it does not exist, an image processing unit is selected and assigned to the client according to preset rules.

In an embodiment, the determining module is specifically used for:

Find out the image processing unit with the least number of bound clients and assign it to the client; or,

Find out the image processing unit with the lowest current processing load and assign it to the client.

In an embodiment, the obtaining module is specifically used for:

Intercept the drawing instructions sent by the D3D RT program in the server's operating system to the user mode display driver;

Intercept the graphics card configuration instructions and special effects instructions sent by the graphics kernel subsystem in the server's operating system to the display miniport driver.

In one embodiment, the server further includes:

The processing module is used for when the request is a request other than the image rendering request, the server processes the request according to the request and then calls the transmission module to transmit the processing result to the data processing device so that the data processing device forwards the data processing device to the client.

According to a fourth aspect of the embodiments of the present disclosure, a data processing device includes: a communication module, a processor, an image processing module, and a memory; the data processing device communicates with a client and a server through the communication module; a communication module Including: request receiving module, request forwarding module, instruction receiving module, image acquisition module and image transmission module;

Request receiving module, used to receive client requests;

The request forwarding module is used to send the client's request to the server;

The instruction receiving module is used for when the request is an image rendering request, the data processing device receives the instruction corresponding to the image rendering sent by the server; where the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

The memory is used to cache the instruction corresponding to the image rendering received from the server and the image obtained after the image rendering is completed;

A processor for instructing the image processing module to complete image rendering according to instructions;

The image acquisition module is used to acquire the image obtained after the image rendering is completed;

The image transmission module is used to send the image obtained after the image rendering is completed to the client.

In an embodiment, the processor is specifically configured to:

According to the instruction, the image processing unit corresponding to the client in the image processing module is instructed to complete the image rendering; wherein the image processing unit is allocated by the server to the client in the image processing module for corresponding processing of the image rendering request of the client.

In one embodiment, the image acquisition module is further configured to: when the request is a request other than an image rendering request, acquire an image sent by the server after processing according to the request;

Correspondingly, the image transmission module is also used to: send the image sent by the server and obtained after processing according to the request to the client.

According to the data transmission method, device and server provided by the embodiments of the present disclosure, the data processing device receives the client's request and forwards it to the server; when the request is an image rendering request, the server obtains the instruction corresponding to the image rendering; wherein the instruction includes: graphics card configuration Instructions, special effects instructions and drawing instructions; the server determines the corresponding image processing unit in the data processing device for the client, and transmits the instruction to the data processing device after the corresponding image processing unit completes the image rendering and transmits the image data to the client; where , There is no need to perform image rendering according to instructions in the server. Therefore, the workload of the server is greatly reduced, and even a large number of client-side 3D image rendering tasks can be supported without a graphics processor, and there is no need to upgrade the server.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments in accordance with the disclosure, and together with the specification are used to explain the principle of the disclosure.

FIG. 1 is a schematic diagram of the system architecture of an embodiment of the present disclosure;

Figure 2 is an existing WDDM drive model architecture diagram;

FIG. 3 is a schematic flowchart of a data transmission method provided by an embodiment of the present disclosure;

4 is a diagram of the architecture of the WDDM drive model of the server operating system in an embodiment of the present disclosure;

FIG. 5 is a schematic flowchart of another data transmission method provided by an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a server provided by an embodiment of the present disclosure;

FIG. 7 is a schematic structural diagram of a data processing device provided by an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of an implementation effect provided by an embodiment of the present disclosure.

Detailed ways

Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

A schematic diagram of the system architecture of the embodiment of the present disclosure is shown in FIG. 1, where the server communicates with M (M is a positive integer) clients through a data processing device, and the requests of the M clients and the responses from the server are processed through data. For device forwarding, both the data processing device and the server need to be configured with an operating system that supports the Windows Display Driver Model (WDDM).

WDDM is a display driver architecture supported in the Windows Vista operating system developed by Microsoft. WDDM is an innovation of the display driver structure in the Windows XP operating system. While supporting new solutions, graphics and applications, it can provide users with a desktop experience with better performance and higher reliability.

Figure 2 is a schematic diagram of the architecture of the WDDM drive model. When the application requires image rendering, the operating system will call the relevant interface functions of the D3D RT (Direct 3D runtime) program, and at the same time send the parameters related to image rendering to the D3D RT program. Graphics card configuration parameters), special effects parameters (parameters related to special effects) and drawing parameters. On the one hand, the D3D RT program will interact with the User Mode Display Driver (UMDD) to obtain the status of the graphics card, and pass drawing instructions to the User Mode Display Driver (the drawing instructions are generated based on the drawing parameters). After the display driver translates the received drawing instruction, it will put the translated instruction into the memory to facilitate the display miniport driver to read the translated instruction. At the same time, the user mode display driver will notify The D3D RT program instruction translation is completed. At this time, the D3D RT program will be notified via the graphics kernel subsystem (Direct Graphics kernel subsysterm or Dxgkrnl.sys) to display the miniport driver to obtain the translated instruction from the memory, and access it through direct memory ( Direct Memory Access, DMA)) is passed to the video memory of the graphics card and then executed by the graphics card; on the other hand, the D3D RT program combines graphics card configuration instructions (graphic card configuration instructions generated based on graphics card configuration parameters) and special effect instructions (special effect instructions generated based on special effect parameters) Passed to the graphics kernel subsystem. Among them, the graphics hardware supplier (graphic card manufacturer) must provide a user-mode display driver and a display miniport driver. The user-mode display driver is a dynamic link library loaded at runtime by Microsoft Direct3D (a set of 3D graphics programming interface developed by Microsoft on the Microsoft Windows operating system, which is part of DirectX and is currently widely supported by various graphics card manufacturers) (Dynamic Link Library, DLL). The display miniport driver communicates with the DirectX (Multimedia Programming Interface created by Microsoft Corporation) graphics kernel subsystem. The graphics kernel subsystem is a system file that the operating system completes image display management, and it participates in all image generation management of the system.

The embodiment of the present disclosure provides a data transmission method. As shown in FIG. 3, the method includes the following steps:

101. The server receives a request from the client forwarded by the data processing device.

Specifically, the server and the client communicate through the data processing device. The client's request and the server's response are forwarded through the data processing device. The data processing device is configured with the same operating system as that on the server, for example, both are installed There are operating systems that support WDDM.

102. When the request is an image rendering request, the server obtains an instruction corresponding to the image rendering; where the instruction includes: a graphics card configuration instruction, a special effect instruction, and a drawing instruction.

In an embodiment, the server acquiring the instruction corresponding to the image rendering in step 102 may specifically include:

102a. The server intercepts the drawing instructions sent by the D3D RT program in the server's operating system to the user mode display driver;

102b. The server intercepts the graphics card configuration instructions and special effect instructions sent by the graphics kernel subsystem in the server's operating system to the display miniport driver.

Specifically, FIG. 4 shows the WDDM drive model architecture for the server provided in the embodiments of the disclosure. Compared with FIG. 2, a user is added between the D3D RT program in the server's operating system and the user mode display driver. Mode filter driver (or use virtual driver). At the same time, add a kernel mode filter driver (or use virtual driver) between the graphics kernel subsystem and the display miniport driver. The functions of these two added filter drivers are introduced below:

1) User mode filter driver

The user mode filter driver is used to intercept the drawing instructions transmitted from the D3D RT program to the user mode display driver. The user mode filter driver is equivalent to a bridge between the D3D RT program and the user mode display driver. It monitors the commands between them, but does not change the content of the commands themselves.

Specifically, as shown in the structure shown in Figure 4, when the D3D RT program in the operating system of the server receives the image rendering request sent by the application layer, it will first notify the user mode display driver (such as DrawPrimitive), and the user mode display driver will The received drawing instructions are converted and processed to obtain instructions that the graphics card can recognize, and the graphics memory transferred to the graphics card by DMA is executed by the graphics card. In the embodiments of the present disclosure, the user mode filter driver is used to intercept the drawing instructions before the user mode display driver executes the instruction conversion processing action.

2) Kernel mode filter driver

The kernel mode filter driver is used to intercept the graphics card configuration instructions and special effect instructions sent by the graphics kernel subsystem to the display miniport driver. These instructions are received from the application layer by the D3D RT program and passed to the graphics kernel subsystem.

It should be noted that for a virtualized server, since there is no physical graphics card, for this scenario, when the technical solution of the embodiment of the present disclosure is applied, a virtual server is set in the WDDM drive model architecture in the operating system of the virtualized server. The user mode filter driver is used to act as a user mode display driver in order to intercept drawing instructions, and a virtual display miniport driver is used to act as a display miniport driver in order to configure instructions and special effects instructions for the graphics card.

103. The server determines the image processing unit corresponding to the client in the data processing device.

In the embodiments of the present disclosure, the data processing device has image rendering capabilities, that is, it is equipped with a graphics processing unit (GPU), and several clients are connected to the server through the data processing device. In this case, 3D images from several clients There will also be a lot of rendering requirements, and the data processing device needs to configure several graphics cards as the image processing modules of the data processing device according to the amount of tasks required for rendering.

In one embodiment, the server divides the image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete the image rendering task.

Specifically, the server side logically divides the image processing modules of the data processing device, and divides the image processing modules to obtain N (N is a positive integer) image processing units that can independently complete image rendering tasks. Specifically, it can be Dividing according to the number of graphics cards can also be obtained by dividing the video memory resources of several graphics cards to obtain N image processing units. It is worth mentioning that one image processing unit can provide 3D image rendering services for one or more clients.

By dividing the image processing module of the data processing device into several image processing units that can independently complete image tasks and assign them to each client, the image processing unit can be used more efficiently to complete image rendering, avoiding multiple clients from competing for resources and The waiting time is longer.

Specifically, the identifier of the client may be an IP address, a media access control address (Media Access Control Address, MAC address), and so on. The image processing unit has two states: idle and occupied. The server will set a corresponding number for each image processing unit, and record the corresponding status on the server side in real time. In the occupied state, the server will also record the number of occupied clients and the corresponding identification. In the embodiment of the present disclosure, the server allocates a corresponding image processing unit to the client, and binds the two through the identification of the client and the number of the image processing unit, so that the image processing of the data processing device can be efficiently managed Resources, providing image rendering services for several clients.

It is worth mentioning that after a client is bound to an image processing unit, if the client does not send a 3D image rendering request for a long time, such as more than 30 minutes or 40 minutes, the client and its corresponding image processing unit will be released After the binding is released, if the client sends a 3D image rendering request again, the image processing unit can be reassigned to it.

The server judges whether there is currently an idle image processing unit;

Specifically, selecting an idle image processing unit to allocate to the client includes:

Randomly select an image processing unit from all idle image processing units; or, select the image processing unit with the smallest number from all idle image processing units; or, select the image processing unit with the largest number from all idle image processing units .

In an embodiment, the server selects an image processing unit to allocate to the client according to preset rules, including:

Specifically, the image processing unit with the lowest processing load may refer to the image processing unit with the smallest number of 3D image rendering requests received per unit time. The aforementioned unit time refers to a period of time between a point in time before the current time and the current time, and its duration can be set and adjusted as required.

If there are multiple image processing units with the least number of bound clients and the image processing unit with the lowest current processing load, one can be randomly selected/choose the one with the smallest number/choose the one with the largest number and allocated to the client.

It is worth mentioning that the execution order of step 102 and step 103 can be exchanged. Specifically, when the request is an image rendering request, the server can first execute the step of determining the image processing unit corresponding to the client in the data processing device, and then execute the acquisition The steps of the instructions corresponding to the image rendering.

104. The server transmits the instruction to the data processing device so that the image processing unit transmits the image data to the client after completing the image rendering according to the instruction; wherein, there is no need to perform image rendering according to the instruction in the server.

Specifically, the user mode filter driver sends the intercepted drawing instructions to the data processing device, the kernel mode filter driver sends the intercepted graphics card configuration instructions and special effect instructions to the data processing device, and the data processing device operating system uses the data processing device's operating system according to the aforementioned instructions. The graphics card completes the image rendering. Specifically, the operating system of the data processing device completes the image rendering process through the graphics card according to the instructions related to the image rendering, and only needs to refer to the relevant technical data of the graphics rendering image, which will not be described in detail here.

In one embodiment, the method further includes:

105. When the request is a request other than the image rendering request, the server transmits the processing result to the data processing device after processing the request according to the request, so that the data processing device forwards the data processing device to the client.

In the data transmission method provided by the embodiments of the present disclosure, the data processing device receives the request from the client and forwards it to the server; when the request is an image rendering request, the server obtains the instruction corresponding to the image rendering; wherein the instructions include: graphics card configuration instructions, special effects instructions And drawing instructions; the server determines the image processing unit corresponding to the client in the data processing device; the server transmits the instruction to the data processing device so that the image processing unit transmits the image data to the client after rendering the image according to the instruction; You need to perform image rendering according to instructions. Therefore, the workload of the server is greatly reduced, and even a large number of client-side 3D image rendering tasks can be supported without a graphics processor, and there is no need to upgrade the server.

The embodiment of the present disclosure provides another data transmission method. As shown in FIG. 5, the method includes the following steps:

201. The data processing device receives a request from a client.

202. The data processing device sends the client's request to the server.

203. When the request is an image rendering request, the data processing device receives an instruction corresponding to the image rendering sent by the server; where the instruction includes: a graphics card configuration instruction, a special effect instruction, and a drawing instruction.

204. The data processing device completes image rendering through the image processing module of the data processing device according to the instruction.

In the embodiment of the present disclosure, the data processing device has image rendering capability, that is, it is equipped with a graphics processing unit (GPU). In practical applications, several clients are connected to the server through the data processing device. In this case, The client's 3D image rendering needs will be very large, and the data processing device needs to configure several graphics cards as the image processing modules of the data processing device according to the amount of rendering required tasks.

205. The data processing device sends the image data obtained after the image rendering is completed to the client.

In one embodiment, step 204 specifically includes:

The image processing unit corresponding to the client in the image processing module completes the image rendering according to the instruction; where the image processing unit is allocated by the server to the client in the image processing module for correspondingly processing the image rendering request of the client. By dividing the image processing unit, the image processing resources of the data processing device can be efficiently used.

The user mode display driver in the operating system of the data processing device converts the drawing instructions into instructions that can be recognized by the graphics card and is executed by the graphics card corresponding to the image processing unit; the display miniport driver in the operating system of the data processing device obtains the graphics card configuration Instructions and special effects instructions are executed by the graphics card corresponding to the image processing unit.

Specifically, after the data processing device receives the instruction sent by the server, the Microsoft Windows operating system installed on the data processing device supports the WDDM drive model architecture, and the user mode display driver in the data processing device operating system converts the received drawing instructions Obtain instructions that can be recognized by the graphics card and transfer them to the graphics memory of the graphics card through direct memory access (DMA). Combining the received graphics configuration instructions and special effects instructions, the graphics card executes the instructions to complete the image rendering.

Exemplarily, the image rendering process performed by the data processing device is implemented based on the WDDM model framework shown in FIG. 2. After the data processing device receives instructions (including configuration instructions for the graphics card, special effects instructions, and drawing instructions) sent by the server through the network , Where the drawing instruction is composed of one or more functions containing specific parameters, and these functions can be reproduced in the operating system of the data processing device to complete the final 3D image rendering work. For example, the received drawing commands are: func1 (parameter 1, parameter 2, parameter 3); func2 (parameter 1, parameter 2); func3 (parameter 1, parameter 2, parameter 3, parameter 4); among them, func1, func2 , Func3 are three exemplary function names given to distinguish different functions. According to the above examples, the data processing device can execute the three functions in the drawing instruction to perform 3D image rendering (that is, the function reproduction process).

In an embodiment, the method may further include:

206. When the request is a request other than an image rendering request, the data processing device receives the processing result obtained by the server after processing the request according to the request.

207. The data processing device sends the processing result to the client.

For general cloud desktop client requests, what the server can directly process is directly processed in the server, and the server can forward the processing result, such as computer desktop image data, to the client through the data processing device.

In the data transmission method provided by the embodiments of the present disclosure, the data processing device receives the client's request and forwards it to the server; when the request is an image rendering request, the data processing device receives the instruction corresponding to the image rendering sent by the server, where the instruction includes: a graphics card Configuration instructions, special effect instructions, and drawing instructions; the data processing device completes image rendering through the image processing module of the data processing device according to the instructions; the data processing device sends the image data obtained after the image rendering is completed to the client. Therefore, the workload of the server is greatly reduced, and even a large number of client-side 3D image rendering tasks can be supported without a graphics processor, and there is no need to upgrade the server.

Based on the data transmission method described in the above corresponding embodiment, the following are device embodiments of the disclosure, which can be used to execute the above method embodiments of the disclosure.

An embodiment of the present disclosure provides a server. As shown in FIG. 6, the server 60 includes:

The receiving module 601 is used to receive the client's request forwarded by the data processing device;

The obtaining module 602 is configured to obtain instructions corresponding to image rendering when the request is an image rendering request; the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

The determining module 603 is used to determine the image processing unit corresponding to the client in the data processing device;

The transmission module 604 is configured to transmit the instruction to the data processing device so that the image processing unit transmits the image data to the client after completing the image rendering according to the instruction; wherein, there is no need to perform image rendering according to the instruction in the server.

In an embodiment, as shown in FIG. 6, the server 60 may further include:

The dividing module 600 is used to divide the image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete an image rendering task.

In an embodiment, the determining module 603 is specifically configured to:

Obtain the client's ID from the image rendering request;

In an embodiment, the determining module 603 is specifically configured to:

Determine whether there is currently an idle image processing unit;

If it exists, select an idle image processing unit to allocate to the client;

If it does not exist, select an image processing unit to assign to the client according to preset rules, including:

In an embodiment, as shown in FIG. 6, the server 60 may further include:

The processing module 605 is configured to, when the request is a request other than the image rendering request, call the transmission module 603 to transmit the processing result to the data processing device after processing the request according to the request, so that the data processing device forwards the data processing device to the client.

For general cloud desktop client requests, what the server can directly process is directly processed in the server, without being processed by the data processing device, just forward the processing result, such as computer desktop image data, to the client through the data processing device.

In an embodiment, the obtaining module 602 is specifically configured to:

In the server provided by the embodiment of the present disclosure, the data processing device receives the request from the client and forwards it to the server; when the request is an image rendering request, the server obtains the instructions corresponding to the image rendering; wherein the instructions include: graphics card configuration instructions, special effects instructions and drawing Instruction; the server determines the image processing unit corresponding to the client in the data processing device; the server transmits the instruction to the data processing device so that the image processing unit transmits the image data to the client after the image rendering is completed according to the instruction; Instruct image rendering. Therefore, the workload of the server is greatly reduced, and even a large number of client-side 3D image rendering tasks can be supported without a graphics processor, and there is no need to upgrade the server.

An embodiment of the present disclosure provides a data processing device. As shown in FIG. 7, the data processing device 70 includes:

The communication module 701, the processor 702, the image processing module 703, and the memory 704; the data processing device 70 communicates with the client and the server respectively through the communication module 701; the communication module 701 includes: a request receiving module 7011, a request forwarding module 7012 , Instruction receiving module 7013, image acquisition module 7014 and image transmission module 7015;

The request receiving module 7011 is used to receive the client's request;

The request forwarding module 7012 is used to send the client's request to the server;

The instruction receiving module 7013 is configured to receive the instruction corresponding to the image rendering sent by the server by the data processing device when the request is an image rendering request; wherein the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

The memory 704 is configured to cache the instruction corresponding to the image rendering received from the server and the image obtained after the image rendering is completed;

The processor 702 is configured to instruct the image processing module 703 to complete image rendering according to instructions;

The image acquisition module 7014 is used to acquire an image obtained after image rendering is completed;

The image transmission module 7015 is used to send the image obtained after the image rendering is completed to the client.

In an embodiment, the processor 702 is specifically configured to:

According to the instruction, the image processing unit corresponding to the client in the image processing module is instructed to complete the image rendering; wherein the image processing unit is allocated by the server to the client in the image processing module for corresponding processing of the image rendering request of the client. The user mode display driver in the operating system of the data processing device converts the drawing instructions into instructions that can be recognized by the graphics card and is executed by the graphics card corresponding to the image processing unit; the display miniport driver in the operating system of the data processing device obtains the graphics card configuration Instructions and special effects instructions are executed by the graphics card corresponding to the image processing unit.

Exemplarily, the data processing device is configured with an operating system that supports WDDM, and after receiving the instruction corresponding to the image rendering sent by the server, the image rendering is completed through the graphics card configured by itself according to the instruction execution flow in the operating system.

In an embodiment, the image acquisition module 7014 is further configured to: when the request is a request other than an image rendering request, acquire an image sent by the server after processing according to the request;

Correspondingly, the image transmission module 7015 is also used to: send the image sent by the server and obtained after processing according to the request to the client.

Based on the data processing device shown in FIG. 7, FIG. 8 is a detailed schematic diagram of implementation effects provided by the embodiments of the disclosure.

In the data processing device provided by the embodiment of the present disclosure, the data processing device receives the request from the client and forwards it to the server; when the request is an image rendering request, the data processing device receives the instruction corresponding to the image rendering sent by the server, where the instruction includes: a graphics card Configuration instructions, special effect instructions, and drawing instructions; the data processing device completes image rendering through the image processing module of the data processing device according to the instructions; the data processing device sends the image data obtained after the image rendering is completed to the client. Therefore, the workload of the server is greatly reduced, and even a large number of client-side 3D image rendering tasks can be supported without a graphics processor, and there is no need to upgrade the server.

Based on the data transmission method described in the embodiment corresponding to FIG. 3/5, the embodiment of the present disclosure also provides a computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a read-only memory (Read Only). Memory, ROM), random access memory (Random Access Memory, RAM), CD-ROM, magnetic tape, floppy disk and optical data storage device, etc. The storage medium stores computer instructions for executing the data transmission method described in the embodiment corresponding to FIG. 3/5, which will not be repeated here.

After considering the specification and practicing the disclosure disclosed herein, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The description and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

Claims

A data transmission method, characterized by comprising:

The server receives the client's request forwarded by the data processing device;

When the request is an image rendering request, the server obtains instructions corresponding to the image rendering; wherein the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

Determining, by the server, an image processing unit corresponding to the client in the data processing device;

The server transmits the instruction to the data processing device so that the image processing unit transmits the image data to the client after the image rendering is completed according to the instruction; wherein, there is no need to perform the image processing in the server. The instructions are for image rendering.
The method according to claim 1, wherein the method further comprises:

The server divides the image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete an image rendering task.
The method according to claim 2, wherein the server determining the image processing unit corresponding to the image rendering request of the client in the data processing device comprises:

The server obtains the identification of the client from the image rendering request;

The server judges whether there is an image processing unit corresponding to the client according to the identifier of the client;

If it does not exist, the server allocates a corresponding image processing unit to the client, and binds the client to the corresponding image processing unit.
The method according to claim 3, wherein the server assigns a corresponding image processing unit to the client, comprising:

The server judges whether there is currently an idle image processing unit;

If it exists, the server selects an idle image processing unit to allocate to the client;

If it does not exist, the server selects an image processing unit to allocate to the client according to a preset rule.
The method according to claim 4, wherein the server selecting an image processing unit to allocate to the client according to a preset rule comprises:

The server finds out the image processing unit with the least number of bound clients and allocates it to the client; or,

The server finds out the image processing unit with the lowest current processing load and allocates it to the client.
The method according to claim 1, wherein the server acquiring the instruction corresponding to the image rendering comprises:

The server intercepts the drawing instruction sent by the D3D RT program in the operating system of the server to the user mode display driver;

The server intercepts the graphics card configuration instruction and the special effect instruction sent by the graphics kernel subsystem in the server's operating system to the display miniport driver.
The method of claim 1, wherein the method further comprises:

When the request is a request other than the image rendering request, the server transmits the processing result to the data processing device after processing the request according to the request so that the data processing device can forward the data processing device to the client end.
A data transmission method, characterized by comprising:

The data processing device receives the client's request;

The data processing device sends the client's request to the server;

When the request is an image rendering request, the data processing device receives an instruction corresponding to the image rendering sent by the server; wherein the instruction includes: a graphics card configuration instruction, a special effect instruction, and a drawing instruction;

The data processing device completes image rendering through the image processing module of the data processing device according to the instruction;

The data processing device sends the image data obtained after the image rendering is completed to the client.
The method according to claim 8, wherein the data processing device performs image rendering through the image processing module of the data processing device according to the instruction, comprising:

The image processing unit corresponding to the client in the image processing module completes the image rendering according to the instruction; wherein, the image processing unit is allocated by the server in the image processing module for the client to correspond to Processing the image rendering request of the client.
The method according to claim 8, wherein the method further comprises:

When the request is a request other than the image rendering request, the data processing device receives a processing result obtained by the server after processing according to the request;

The data processing device sends the processing result to the client.
A server, characterized in that it comprises:

The receiving module is used to receive the client's request forwarded by the data processing device;

The obtaining module is configured to obtain instructions corresponding to the image rendering when the request is an image rendering request; wherein the instructions include: graphics card configuration instructions, special effects instructions, and drawing instructions;

A determining module, configured to determine an image processing unit corresponding to the client in the data processing device;

The transmission module is configured to transmit the instruction to the data processing device so that the image processing unit transmits the image data to the client after rendering the image according to the instruction; wherein, there is no need in the server Perform image rendering according to the instructions.
The server according to claim 1, wherein the server further comprises:

The dividing module is used to divide the image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete an image rendering task.
The server according to claim 12, wherein the determining module is specifically configured to:

Obtaining the identification of the client from the image rendering request;

Judging whether there is an image processing unit corresponding to the client according to the identifier of the client;

If it does not exist, assign a corresponding image processing unit to the client, and bind the client to the corresponding image processing unit.
The server according to claim 13, wherein the determining module is specifically configured to:

Determine whether there is currently an idle image processing unit;

If it exists, select an idle image processing unit to allocate to the client;

If it does not exist, an image processing unit is selected and allocated to the client according to a preset rule.
The server according to claim 14, wherein the determining module is specifically configured to:

Find out the image processing unit with the least number of bound clients and allocate it to the client; or,

The image processing unit with the lowest current processing load is found and allocated to the client.
The server according to claim 1, wherein the obtaining module is specifically configured to:

Intercepting the drawing instruction sent by the D3D RT program in the operating system of the server to the user mode display driver;

Intercept the graphics card configuration instructions and the special effect instructions sent by the graphics kernel subsystem in the operating system of the server to the display miniport driver.
The server according to claim 1, wherein the server further comprises:

The processing module is configured to, when the request is a request other than the image rendering request, the server processes the request according to the request and then calls the transmission module to transmit the processing result to the data processing device so that The data processing device forwards to the client.
A data processing device, which is characterized by comprising: a communication module, a processor, an image processing module, and a memory; the data processing device communicates with a client and a server through the communication module; the communication module Including: request receiving module, request forwarding module, instruction receiving module, image acquisition module and image transmission module;

The request receiving module is configured to receive a request from the client;

The request forwarding module is configured to send the client's request to the server;

The instruction receiving module is configured to: when the request is an image rendering request, the data processing device receives an instruction corresponding to the image rendering sent by the server; wherein the instruction includes: a graphics card configuration instruction and a special effect instruction And drawing instructions;

The memory is configured to cache the instruction corresponding to the image rendering received from the server and the image obtained after the image rendering is completed;

The processor is configured to instruct the image processing module to complete image rendering according to the instruction;

The image acquisition module is used to acquire the image obtained after the image rendering is completed;

The image transmission module is configured to send the image obtained after the image rendering is completed to the client.
The data processing device according to claim 18, wherein the processor is specifically configured to:

According to the instruction, the image processing unit corresponding to the client in the image processing module is instructed to complete the image rendering; wherein, the image processing unit is allocated by the server to the client in the image processing module for Correspondingly process the image rendering request of the client.
The data processing device according to claim 18, wherein the image acquisition module is further configured to: when the request is a request other than the image rendering request, collect the request sent by the server according to the request The image obtained after processing;

Correspondingly, the image transmission module is further configured to: send the image obtained after processing according to the request sent by the server to the client.