WO2017088484A1 - Procédé, appareil et système de rendu hors écran en temps réel basés sur l'informatique en nuage - Google Patents

Procédé, appareil et système de rendu hors écran en temps réel basés sur l'informatique en nuage Download PDF

Info

Publication number
WO2017088484A1
WO2017088484A1 PCT/CN2016/088767 CN2016088767W WO2017088484A1 WO 2017088484 A1 WO2017088484 A1 WO 2017088484A1 CN 2016088767 W CN2016088767 W CN 2016088767W WO 2017088484 A1 WO2017088484 A1 WO 2017088484A1
Authority
WO
WIPO (PCT)
Prior art keywords
rendering
resource
user
client
time
Prior art date
Application number
PCT/CN2016/088767
Other languages
English (en)
Chinese (zh)
Inventor
罗涛
杨磊
张微
陈乐吉
杨学亮
周益
曾锦平
邱泳天
Original Assignee
成都赫尔墨斯科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 成都赫尔墨斯科技有限公司 filed Critical 成都赫尔墨斯科技有限公司
Publication of WO2017088484A1 publication Critical patent/WO2017088484A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols

Definitions

  • the present invention relates to the field of cloud rendering, and in particular, to a cloud computing-based real-time off-screen rendering method, apparatus and system.
  • cloud computing related services such as Iaas (infrastructure as a service), Paas (platform as a service), and Saas (software as a service) have emerged.
  • the general principle is to use cloud computing's powerful virtualization technology to pool computing, storage and network resources, to achieve arbitrary segmentation of resources, to achieve high availability of computing, storage and network resources, and overall improvement The utilization of resources.
  • 3D large-scale high-dynamic scene rendering services have been the demand of many industries.
  • the popularity of cloud computing services and the demand for large-scale 3D scene rendering have naturally created cloud rendering services.
  • the existing cloud rendering is roughly divided into two categories: one is online rendering, and the other is offline rendering. Both of these types use the way of cloud rendering, and the implementation of the prior art can only provide a delay rendering knot. If it is mainly used for movie special effects and animation rendering, it can not meet the real-time interoperability requirements of 3D large-scale high-dynamic scene rendering pictures on the mobile device side. The displayed 3D special effects are poor in quality and cannot provide users with high dynamics in the client terminal. A good user experience for 3D real-time scene services.
  • a Chinese invention patent (Application Publication No.: CN103313120A) discloses a method for displaying a picture, the method comprising the steps of: acquiring picture information of a rendering entity; transmitting the picture information to a cloud, and using the cloud according to the picture information The rendering entity is rendered to obtain a rendering result; the rendering result returned by the cloud is obtained, and the screen is displayed according to the rendering result.
  • the invention reduces the need for the processing capability of the portable client terminal by performing the rendering operation in the cloud, and can be applied to various types of terminals, even the low-end client terminal is equally applicable, and with the strong processing capability of the cloud, Cloud rendering display of 3D scenes can be performed on the client terminal device.
  • the cost is The time will increase, ie the delay will increase. Moreover, the number of clients supporting simultaneous online access will also be reduced. Once the communication load of the cloud reaches a critical value, the rendering task of the excess client will be suspended, and the delay is more serious.
  • the object of the present invention is to overcome the above-mentioned deficiencies in the prior art, and provide a real-time off-screen rendering method based on cloud computing, which can support a plurality of simultaneous client devices and can carry a rendering service request of a large number of client devices. Moreover, all online rendering tasks are independent of each other, which not only reduces the communication load of the cloud, but also reduces the latency of the cloud rendering service.
  • a cloud computing-based real-time off-screen rendering method including:
  • the rendering resource is rendered in real time according to the cloud rendering service request, and the rendering result generated in real time is sent to the client through the direct link;
  • the client After receiving the rendering result in real time, the client displays the screen in real time according to the rendering result.
  • the optimal rendering resource is enabled; otherwise, no rendering resource information is available to the client.
  • an address of the rendering resource is obtained, and a communication port is allocated to the client, and an address of the rendering resource is bound to a communication port of the client. And creating the rendering resource and the direct link of the client.
  • the rendering resource after the direct link is successfully created, according to the rendering entity described in the cloud rendering service request, the rendering resource first downloads resource data of the rendering entity;
  • the rendering resource After the resource data of the rendering entity is downloaded, the rendering resource renders the resource data of the rendering entity in real time according to the rendering content described in the cloud rendering request, and is in a frame buffer of the rendering resource. Generate rendering results in real time;
  • the rendering result in the frame buffer is obtained in real time, and is compression-encoded, and then sent to the client through the direct link.
  • the transmission frequency of the rendering result is adjusted in real time according to the communication status on the direct link and the operation of the client.
  • the client before receiving the cloud rendering service request sent by the client, the client is first sent to send a user verification request;
  • the identification information is returned to the client, and a heartbeat connection with the client is established. Otherwise, the identification information is not returned to the client.
  • the heartbeat packet sent by the client after returning the identification information to the client, if the heartbeat packet sent by the client is received, determining that the heartbeat connection is successfully established, and adding the user to the online list of the user; If the heartbeat packet sent by the client is not received, it is determined that the heartbeat connection establishment fails, and the user is offline;
  • the heartbeat connection is successfully established, if the offline request sent by the client is received, the heartbeat connection is disconnected, the user is removed from the online list of the user, and the created Direct link and enabled rendering resources.
  • a heartbeat packet sent by the client after establishing a heartbeat connection with the client, receiving a heartbeat packet sent by the client according to a certain frequency to monitor a login status of the user, and each time after receiving one of the heartbeat packets, immediately Start timing at zero;
  • timing time is greater than or equal to the drop threshold time, the user is removed from the online list of users, and the created direct link and the enabled rendering resource are released.
  • a certain number of rendering resources are opened in advance, and in the online trough period of the client, a certain number of rendering resources are closed.
  • a cloud computing-based real-time off-screen rendering apparatus including:
  • a rendering resource unit for providing a rendering resource for rendering services
  • a resource management unit configured to manage and enable the rendering resource
  • a service scheduling unit configured to create a direct link between the rendering resource and the client.
  • the resource management unit includes:
  • a rendering resource list sub-unit configured to save an address of each rendering resource in the rendering resource unit, and perform statistics on the working state of each rendering resource according to a certain frequency, and establish a rendering resource list according to the statistical result;
  • a rendering resource enabling subunit configured to query whether there is a rendering resource available in the rendering resource list, and enable an optimal rendering resource when the rendering resource is available; otherwise, return no available resource information to the client end;
  • a rendering resource address sending subunit configured to send an address of the rendering resource enabled sub-unit enabled rendering resource to the service scheduling unit, to be used by the service scheduling unit to create the direct link.
  • the service scheduling unit includes:
  • a communication link management subunit configured to allocate a communication port to the client, and after receiving the address of the rendering resource sent by the rendering resource address sending subunit, communicate the address of the rendering resource with the client Port binding to create the direct link of the rendering resource to the client.
  • the rendering resource includes:
  • a resource data downloading subunit configured to download resource data of the rendering entity according to the rendering entity described in the cloud rendering service request, and perform downloading of the resource data
  • a rendering execution subunit configured to perform real-time rendering on the resource data according to the rendering content described in the cloud rendering request
  • a frame buffer subunit configured to cache a rendering result generated by the rendering execution subunit in real time
  • a compression coding subunit configured to acquire a rendering result in the frame buffer subunit in real time, and compress and encode the rendering result
  • a rendering result sending subunit configured to send, in real time, a rendering result processed by the compression encoding subunit to the client through the direct link, and according to the communication status and location of the direct link
  • the operation of the client adjusts the transmission frequency of the rendering result in real time.
  • the device further includes:
  • a storage unit configured to pre-store the rendering entity resource data of the user
  • a database management unit configured to manage rendering entity resource data of the user, and when the rendering resource requests to download resource data, send a storage address of the resource data in the storage unit to the rendering resource, where The rendering resource downloads the resource data.
  • the device comprises a user management unit, and
  • the communication relay unit is further configured to receive a user verification request and an offline request sent by the client;
  • the database management unit is further configured to manage identity information of the user, and store user identity information pre-bound by the user;
  • the service scheduling unit is further configured to forward the user verification request and the offline request to the user management unit.
  • the user management unit includes:
  • An identity information verification subunit configured to parse user identity information included in the user authentication request, and send the user identity information to the database management unit for comparison, and verify whether the user identity information is pre-bound User identity information is consistent;
  • a heartbeat connection subunit configured to return identification information to the client according to whether the user identity information is consistent with pre-bound user identity information, and establish a heartbeat connection with the client; and return the identification information
  • a user online list management sub-unit configured to establish a user online list, and when the heartbeat connection is successfully established, add the user to the online list of the user; and when receiving the offline request sent by the client, Removing the user from the online list of users releases the created direct link and enabled rendering resources.
  • the heartbeat connection subunit is further configured to start timing from zero after receiving one of the heartbeat packets, and drop the user when the timing time is greater than the offline threshold time Sending line information to the user online list management sub-unit, causing the user online list management sub-unit to remove the user from the online list of users, releasing the created direct link and enabled Render resources.
  • the device further includes:
  • the intelligent management unit is configured to open a certain amount of rendering resources in advance during the online peak period of the user, and close a certain amount of rendering resources during the online trough period of the user.
  • a cloud computing-based real-time off-screen rendering system including a client, a public network, and a plurality of user resource pools, wherein the user resource pools are respectively bound with a unique public network.
  • IP the client accesses the user resource pool by linking to the public network IP;
  • the user resource pool includes a router, a rendering resource server, a resource management server, and a service scheduling server;
  • the router is configured to receive, by the client, a cloud rendering service request, and forward the cloud rendering service request to the service scheduling server;
  • the service scheduling server is configured to create a direct link between the rendering resource and the client, and forward the cloud rendering service request to the rendering resource server and the resource management server;
  • the rendering resource server is configured to provide a rendering resource for rendering services
  • the resource management server is configured to manage and enable the rendering resource.
  • the resource management server saves an address of each rendering resource in the rendering resource server, and performs statistics on the working state of each rendering resource according to a certain frequency, and establishes a rendering resource list according to the statistical result;
  • the resource management server enables a rendering resource and sends an address of the rendering resource to the service scheduling server for use by the service scheduling server to create the direct link.
  • the service scheduling server receives an address sent by the resource management server, and allocates to the client a communication port, and binds the address of the rendering resource to the communication port of the client. And creating the rendering resource and the direct link of the client.
  • the rendering resource downloads resource data of the rendering entity according to a rendering entity described in the cloud rendering service request; and downloads according to the rendering content described in the cloud rendering request.
  • the resource data is rendered in real time, and the rendering result generated in real time in the frame buffer;
  • the rendering resource performs compression encoding on the rendering result in the frame buffer, and sends the compression-encoded rendering result to the client through the direct link, and according to the communication status of the direct link And the operation of the client, adjusting the sending frequency of the rendering result in real time.
  • the system includes a database server and a storage server; the user resource pool shares the database server and the storage server;
  • the storage server is configured to pre-store the rendering entity resource data of the user
  • the database management server is configured to manage the rendering entity resource data of the user, and send the storage address of the resource data in the storage server to the rendering resource when the rendering resource requests to download the resource data, The resource data is downloaded for the rendering resource.
  • the system includes: a user management server, and,
  • the router is further configured to receive a user verification request sent by the client;
  • the database management server is further configured to manage identity information of the user, and store user identity information pre-bound by the user;
  • the service scheduling server is further configured to forward the user verification request to the user management server.
  • the user management server receives a user authentication request forwarded by the service scheduling server, and parses user identity information included in the user verification request, and sends the user identity information to the database.
  • the management unit performs an comparison to verify whether the user identity information is consistent with the pre-bound user identity information;
  • the user management server returns the identification information to the client according to whether the user identity information is consistent with the pre-bound user identity information, so that the client establishes a heartbeat connection with the user management server;
  • the user management server After returning the identification information to the client, the user management server receives the heartbeat packet sent by the client, determines that the heartbeat connection is successfully established, and adds the user to the online list of users; If the user management server does not receive the heartbeat connection, it is determined that the heartbeat connection fails, and the user is offline;
  • the user management server receives the offline request sent by the client, disconnects the heartbeat connection, removes the user from the online list of the user, and releases the The direct link is created and the enabled rendering resources are deactivated.
  • the user management server after receiving the heartbeat packet, starts timing from zero;
  • the user management server removes the user from the online list of users, and releases the created direct link and disables the enabled rendering Dye resources.
  • the system comprises:
  • the intelligent management server is configured to open a certain number of rendering resources in advance during a user online peak period, and close a certain number of rendering resources during a user online trough period.
  • the invention After accepting the cloud rendering service request sent by the client, the invention enables the rendering resource and creates a direct link of the client terminal and the rendering resource, so that the rendering resource directly processes the cloud rendering service request, and the rendering resource is real-time rendered and generated in real time.
  • the rendering result is directly sent to the client, so that the client terminal receives the rendering result in real time and displays the image according to the rendering result, which can not only support multiple online client devices, but also can carry a large number of rendering service requests of the client device, and all online
  • the rendering tasks are independent of each other, reducing the communication load of the cloud and reducing the latency of the cloud rendering service.
  • the present invention dynamically adjusts the transmission frequency of the rendering result in real time and compresses the rendering result to different degrees, and further reduces the delay of the rendering service.
  • Real-time interoperability between users and large 3D scenes is achieved, and the client terminal device provides users with a good user experience of highly dynamic 3D real-time scene services.
  • the present invention is also capable of recording the time rule of the user requesting the cloud rendering service, and according to the time rule, reasonably controlling the standby quantity of the rendering resource, reducing energy consumption, saving energy and environmental protection while ensuring sufficient user resources.
  • FIG. 1 is a schematic diagram showing the basic flow of a cloud rendering service of the method of the present invention
  • FIG. 2 is a schematic flowchart of a method for enabling rendering resources according to the method of the present invention
  • FIG. 3 is a schematic flowchart of real-time rendering of a rendering resource of the method of the present invention.
  • FIG. 4 is a schematic diagram of a basic flow of verifying user identity information by the method of the present invention.
  • FIG. 5 is a schematic flow chart of managing a heartbeat connection by the method of the present invention.
  • FIG. 6 is a schematic flow chart of determining a user login status of the method of the present invention.
  • Figure 7 is a schematic structural view of a first embodiment of the apparatus of the present invention.
  • Figure 8 is a schematic structural view of a second embodiment of the apparatus of the present invention.
  • Figure 9 is a schematic structural view of a third embodiment of the apparatus of the present invention.
  • Figure 10 is a schematic structural view of a fourth embodiment of the apparatus of the present invention.
  • Figure 11 is a schematic view showing the structure of a fifth embodiment of the apparatus of the present invention.
  • Figure 12 is a schematic structural view of a sixth embodiment of the apparatus of the present invention.
  • Figure 13 is a schematic structural view of a first embodiment of the system of the present invention.
  • Figure 14 is a schematic structural view of a second embodiment of the system of the present invention.
  • Figure 15 is a schematic view showing the structure of a third embodiment of the system of the present invention.
  • FIG. 1 A basic flow diagram of the cloud rendering service of the method of the present invention shown in FIG. 1; on a client with an application having a 3D scene, the user performs certain operations on the application of the 3D scene, such as rotating, moving, and scaling the rendering entity. Operation, etc., when these related operations are recognized by the 3D scene application, a cloud rendering service request including related rendering content and rendering entity information is generated. Therefore, after the user completes the related operations on the client, the client sends a relevant cloud rendering service request to request the cloud rendering service.
  • the real-time off-screen rendering method based on cloud computing of the invention comprises:
  • the rendering resource is rendered in real time according to the cloud rendering service request, and the real-time generated rendering result is sent to the client through the direct link.
  • the client After the rendering result is received in real time, the client displays the picture in real time according to the rendering result.
  • the present invention makes all online rendering tasks independent of each other, reduces the communication load of the cloud, and reduces the delay of the cloud rendering service.
  • FIG. 2 A schematic diagram of a process for enabling a rendering resource according to the method of the present invention shown in FIG. 2; after receiving a cloud rendering service request sent by a client, first accessing a rendering resource list to query whether a rendering resource is available.
  • the optimal rendering resource is enabled, that is, the running load is determined to be low.
  • the physical machine, and the use of relatively low-frequency rendering resources on the physical machine which is beneficial to balance the use of rendering resources, rationally plan the enabling of rendering resources, and avoid the idleness of rendering resources.
  • the address of the rendering resource is also obtained, and a communication port is allocated to the client, and the address of the rendering resource is bound to the communication port of the client to create a direct link between the rendering resource and the client.
  • the cloud rendering service request is suspended, and the rendering service is provided when there are available resources.
  • the schematic diagram of the real-time rendering of the resource rendering is performed in conjunction with the method of the present invention shown in FIG. 3; after the direct link is successfully created, the rendering resource first downloads the resource data of the rendering entity according to the rendering entity described in the cloud rendering service request.
  • the rendering resource After the resource data of the rendering entity is downloaded, according to the rendering content described in the cloud rendering request, the rendering resource renders the resource data of the rendering entity in real time, and generates the rendering result in real time in the frame buffer of the rendering resource.
  • the rendering result in the frame buffer is obtained in real time, and is compressed and encoded, and then sent to the client through the direct link.
  • the rendering resource is a resource with GPU hardware, and the rendering of the data is mainly performed by the GPU.
  • the process of data rendering can be either a traditional rendering pipeline process or a programmable rendering pipeline process, and the final rendering results are placed in the frame buffer.
  • the transmission frequency of the rendering result is adjusted in real time according to the communication status on the direct link and the operation of the client.
  • the transmission frequency of the rendering result is reduced, so as to avoid data congestion and communication overload when the client receives, affecting the user experience, and when the user performs related operations on the client, according to The related operation adjusts the sending frequency in real time, and after the client completes the related operation, sends the rendering result to the client at the adjusted sending frequency, so that the user obviously feels the rendering effect.
  • the present invention reduces the delay of the rendering service, realizes real-time interoperability between the user and the large 3D scene, and provides the user with a good user experience of the highly dynamic 3D real-time scene service in the client terminal device.
  • FIG. 4 A basic flow diagram for verifying user identity information in conjunction with the method of the present invention shown in FIG. 4; before receiving a cloud rendering service request sent by a client, the receiving client sends a user authentication request.
  • the identity information of the household is consistent.
  • the identification information is returned to the client, and a heartbeat connection with the client is established. Otherwise, the identification information is not returned to the client.
  • FIG. 5 A schematic diagram of a process for managing a heartbeat connection according to the method of the present invention shown in FIG. 5; after receiving the identification information to the client, if the heartbeat packet sent by the client is received, it is determined that the heartbeat connection is successfully established, and the user is added to the online list of the user; If the heartbeat packet sent by the client is not received, it is determined that the heartbeat connection establishment fails, and the user is offline;
  • the heartbeat connection is successfully established, if the offline request sent by the client is received, the heartbeat connection is disconnected, the user is removed from the online list of the user, and the created direct link and the enabled rendering resource are released. .
  • FIG. 6 A schematic diagram of a process for determining a user login status in conjunction with the method of the present invention shown in FIG. 6; after establishing a heartbeat connection with a client, receiving a heartbeat packet sent by the client at a certain frequency to monitor the login status of the user, and each time a heartbeat packet is received , immediately start timing from zero.
  • timing is greater than or equal to the drop threshold time, the user is considered to be offline, the user is removed from the user online list, and the created direct link and the enabled rendering resource are released.
  • the real-time off-screen rendering method based on cloud computing according to the invention also opens a certain number of rendering resources in advance according to the rule that the client joins the online list of the user, and turns off a certain number of renderings in the online trough period of the client. Resources.
  • the present invention also provides a real-time off-screen rendering device based on cloud computing.
  • a schematic diagram of a first embodiment of the apparatus of the present invention shown in FIG. 7; the real-time off-screen rendering apparatus based on cloud computing of the present invention comprises: a communication relay unit, a rendering resource unit, a resource management unit, and a service scheduling unit.
  • the communication relay unit is configured to receive a cloud rendering service request sent by the client, and receive the received The cloud rendering service request is forwarded to the business scheduling unit.
  • the service scheduling unit forwards the cloud rendering service request to the resource management unit, the resource management unit manages and enables the rendering resource, the rendering resource is provided by the rendering resource unit, and the service scheduling unit creates a direct link between the rendering resource and the client, and renders by rendering the resource. service.
  • FIG. 8 A schematic structural diagram of a second embodiment of the apparatus of the present invention shown in FIG. 8; wherein the resource management unit comprises: a rendering resource list subunit, a rendering resource enabling subunit, and a rendering resource address transmitting subunit.
  • the rendering resource list sub-unit saves the address of each rendering resource in the rendering resource unit, and statistics the working state of each rendering resource according to a certain frequency, and establishes a rendering resource list according to the statistical result.
  • the rendering resource enable subunit is used to query whether there are available rendering resources in the rendering resource list, and enable the optimal rendering resource when there are available rendering resources, otherwise, no available resource information is returned to the client.
  • the rendering resource address sending subunit is configured to send the address of the rendering resource enabled by the rendering resource enabling subunit to the service scheduling unit for use by the service scheduling unit to create the direct link.
  • the service scheduling unit includes a communication link management sub-unit.
  • the communication link management sub-unit allocates a communication port to the client, and after receiving the address of the rendering resource sent by the rendering resource address sending sub-unit Bind the address of the rendering resource to the communication port of the client to create a direct link between the rendering resource and the client.
  • FIG. 9 is a schematic structural diagram of a third embodiment of the apparatus of the present invention.
  • the cloud-based real-time off-screen rendering apparatus of the present invention further includes a storage unit and a database management unit.
  • the storage unit is configured to pre-store the rendering entity resource data of the user
  • the database management unit is configured to manage the rendering entity resource data of the user, and send the resource data in the storage unit when the rendering resource requests to download the resource data.
  • the rendering resource includes a resource data downloading subunit, a rendering execution subunit, a frame buffer subunit, a compression encoding subunit, and a rendering result sending subunit.
  • the resource data downloading subunit is configured to request to download resource data of the rendering entity according to the rendering entity described in the cloud rendering service request, and perform downloading of the resource data.
  • the frame buffer sub-unit is used to cache the rendering result generated by the rendering execution sub-unit in real time.
  • the compression coding sub-unit is configured to acquire the rendering result in the frame buffer sub-unit in real time, and compress and encode the rendering result.
  • the rendering result sending subunit is configured to send the rendering result processed by the compression encoding subunit in real time to the client through the direct link, and adjust the rendering result in real time according to the communication status of the direct link and the operation of the client. Send frequency.
  • FIG. 10 is a schematic structural diagram of a fourth embodiment of the apparatus of the present invention.
  • the cloud-based real-time off-screen rendering apparatus of the present invention further includes a user management unit, and the communication relay unit is further configured to receive user authentication sent by the client.
  • the request and the offline request ;
  • the database management unit is further configured to manage the identity information of the user, and store the user identity information pre-bound by the user;
  • the service scheduling unit is further configured to forward the user verification request and the offline request to the user management unit.
  • FIG. 11 A schematic structural diagram of a fifth embodiment of the apparatus of the present invention shown in FIG. 11; the user management unit includes:
  • the identity information verification sub-unit is configured to parse the user identity information included in the user verification request, and send the user identity information to the database management unit for comparison, and verify whether the user identity information is consistent with the pre-bound user identity information.
  • the heartbeat connection sub-unit according to whether the user identity information is consistent with the pre-bound user identity information, returns the identification information to the client, and establishes a heartbeat connection with the client; and after returning the identification information to the client, the receiving client sends the If the heartbeat packet is received, the heartbeat connection is successfully determined. If the heartbeat packet is not received, the heartbeat connection is determined to be failed, and the user is offline; and after the heartbeat connection is successfully established, when the offline request sent by the client is received, Disconnect the heartbeat connection.
  • the user online list management sub-unit is configured to establish a user online list, and when the heartbeat connection is successfully established, add the user to the online list of the user; and when receiving the offline request sent by the client, move the user from the online list of the user In addition, releases the created direct link and enabled render resources.
  • the heartbeat connection subunit is further configured to start timing from zero after receiving a heartbeat packet, and send the user drop information to the user online list management subunit when the timing time is greater than the drop threshold time. To remove the user from the user's online list, freeing the created direct link and enabled rendering resources.
  • FIG. 12 A schematic diagram of a structure of a sixth embodiment of the apparatus of the present invention shown in FIG. 12; the real-time off-screen rendering apparatus based on cloud computing of the present invention further includes an intelligent management unit.
  • the intelligent management unit is configured to open a certain number of rendering resources in advance according to the rule that the user joins the online list of the user, and close a certain number of rendering resources during the online trough period of the user.
  • the present invention further provides a real-time off-screen rendering system based on cloud computing, combined with the structural schematic diagram of the first embodiment of the system of the present invention shown in FIG. 13; the real-time off-screen rendering system based on cloud computing of the present invention
  • the client, the public network, and the user resource pool are bound to each other.
  • the user resource pool is bound to a public IP address.
  • the client accesses the user resource pool by linking to the public network IP address.
  • the user resource pool includes a router, a rendering resource server, a resource management server, and a service scheduling server.
  • the router is configured to receive a request for sending a cloud rendering service by the client, and forward the cloud rendering service request to the service scheduling server.
  • a service scheduling server is configured to create a direct link between the rendering resource and the client, and forward the cloud rendering service request to the rendering resource server and the resource management server.
  • a rendering resource server that provides rendering resources for rendering services.
  • a resource management server for managing and enabling rendering resources.
  • the resource management server saves the address of each rendering resource in the rendering resource server, and statistics the working state of each rendering resource according to a certain frequency, and establishes a rendering resource list according to the statistical result.
  • the resource management server receives the cloud rendering service request forwarded by the service scheduling server, and queries whether the rendering resource is available in the rendering resource list. If yes, the optimal rendering resource is enabled, otherwise the available resource information is returned to the client.
  • the resource management server enables the rendering resource and sends the address of the rendering resource to the service scheduling server for use by the service scheduling server to create a direct link.
  • the service scheduling server receives the address sent by the resource management server, and allocates a communication port to the client, and binds the address of the rendering resource to the communication port of the client to create a direct link between the rendering resource and the client.
  • FIG. 14 A schematic diagram of the structure of the second embodiment of the system of the present invention shown in FIG. 14; the real-time off-screen rendering system based on cloud computing of the present invention further includes a database server and a storage server.
  • the user resource pool shares the database server and the storage server, and the storage server is configured to pre-store the rendering entity resource data of the user.
  • a database management server configured to manage the rendering entity resource data of the user, and send the storage address of the resource data in the storage server to the rendering resource when the rendering resource requests to download the resource data, For rendering resources to download resource data.
  • the real-time off-screen rendering system based on cloud computing of the present invention comprises: a user management server and an intelligent management server.
  • the router is configured to receive a user authentication request sent by the client.
  • a database management server for managing user identity information and storing user identity information pre-bound by the user.
  • a service scheduling server for forwarding user authentication requests to the user management server.
  • the user management server receives the user authentication request forwarded by the service scheduling server, and parses the user identity information included in the user authentication request, sends the user identity information to the database management unit for comparison, and verifies whether the user identity information is associated with the pre-bound user identity. The information is consistent;
  • the user management server returns the identification information to the client according to whether the user identity information is consistent with the pre-bound user identity information, so that the client establishes a heartbeat connection with the user management server;
  • the user management server After returning the identification information to the client, the user management server receives the heartbeat packet sent by the client, determines that the heartbeat connection is successfully established, and adds the user to the online list of the user; if the user management server does not receive the heartbeat connection, the heartbeat is determined. The connection failed and the user is offline;
  • the user management server receives the offline request sent by the client, disconnects the heartbeat connection, removes the user from the online list of the user, and releases the created direct link and disables the enabled one. Render resources.
  • the client sends the heartbeat packet to the user management server according to a certain frequency.
  • the user management server starts timing from zero.
  • the User Management Server removes the user from the user's online list and releases the created direct link and deactivates the enabled rendering resources.
  • the intelligent management server is configured to open a certain number of rendering resources in advance according to the rule that the user joins the online list of the user, and close a certain amount of rendering resources in the online trough period of the user.
  • the user uses the client to send a user authentication request to the user resource pool corresponding to the user through the TCP/IP protocol.
  • the router forwards the user authentication request to the service scheduling server, and then the service scheduling server forwards it to the user management server.
  • the user management server parses the user identity information in the user authentication request, and compares the user identity information with the user information pre-bound in the database management server, and verifies whether the user identity information is pre-bound User identity information is consistent.
  • the user management server returns unique identification information to the client through the service scheduling server and router.
  • the client establishes a heartbeat connection with the user management server according to the identification information, and the user management server receives the heartbeat packet sent by the client, indicating that the heartbeat connection is successfully established, and the user is added to the user online list.
  • the client can send the corresponding cloud rendering service request to the user resource pool through a real-time operation.
  • the router After receiving the cloud rendering service request, the router immediately forwards the user authentication request to the service scheduling server. It is then forwarded by the service scheduling server to the resource management server.
  • the resource management server After receiving the cloud rendering service request, the resource management server queries whether there is a rendering resource available in the resource list, and selects a rendering resource with a relatively low activation and use frequency on a physical machine with a lower running load to enable, in response to the cloud. Render a service request.
  • the resource management server will forward the address of the enabled rendering resource.
  • the service scheduling server allocates a communication port to the client, and binds the address of the rendering resource to the communication port, thereby establishing a direct link between the rendering resource and the client.
  • the rendering resource requests the database management server to download the resource data of the rendering entity teapot according to the rendering entity teapot described in the cloud rendering service request, and the database management server renders the resource data of the entity teapot in the storage server.
  • the storage address is sent to the rendering resource, and the rendering resource downloads the resource data of the rendering entity teapot according to the storage address of the resource data of the rendering entity teapot in the storage server.
  • the rendering content requested by the cloud rendering service such as the rotation, movement and scaling of the rendering entity teapot
  • the rendering entity teapot is rendered in real time and in the frame buffer of the rendering resource.
  • Produce rendering results The rendering result in the frame buffer is extracted in real time, and the rendering result is compression-encoded, and after being compression-encoded, it is directly sent to the client through the direct link.
  • the user management server if the time counted by the user management server after receiving one heartbeat packet is greater than or equal to the drop threshold time, the user is removed from the online list of the user, and the release is created. Direct link and enabled rendering resources. If the client sends an offline request, the user management server disconnects the heartbeat and removes the user from the online list of users, releasing the created direct link and the enabled rendering resources.
  • the rendering resource In the step of transmitting the compression-encoded rendering result to the client, the rendering resource also adjusts the transmission frequency of the rendering result in real time according to the communication status on the direct link and the operation of the client.
  • the real-time operation of the client is correspondingly described in the rendered content of the cloud rendering service request.
  • the invention can not only support multiple online client devices, but also can carry a large number of rendering service requests of the client device, and all the online rendering tasks are independent of each other, reducing the cloud's Communication load reduces the latency of cloud rendering services.
  • the present invention dynamically adjusts the transmission frequency of the rendering result in real time and compresses the rendering result to different degrees, further reducing the delay of the rendering service.
  • Real-time interoperability between users and large 3D scenes can be realized, and the user terminal equipment provides users with a good user experience of highly dynamic 3D real-time scene services.
  • One aspect of the invention relates to a non-transitory computer readable medium comprising instruction set encoding adapted for use in the above method, which is operated and/or controlled by a dedicated and integrated software package.
  • the present invention also includes algorithms, computer programs, computer readable media and/or software, implantable and/or executable files on a general purpose computer or workstation equipped with a conventional processor, and for performing one or more Method and/or one or more of the hardware disclosed herein.
  • a computer program or computer readable medium typically includes a set of instructions configured to perform the above when executed by a suitable processing device (eg, a signal processing device such as a microcontroller, microprocessor or DSP device) Method, run, and/or algorithm.
  • a suitable processing device eg, a signal processing device such as a microcontroller, microprocessor or DSP device
  • the computer readable medium can include any medium that can be read by a signal processing device to execute code stored thereon, such as a floppy disk, optical disk, magnetic tape, or hard disk drive.
  • code can contain object code, source code, and/or binary code.
  • the code is generally numeric and is typically processed by a central processing unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

La présente invention concerne un procédé, un appareil et un système de restitution hors écran en temps réel basés sur l'informatique en nuage. Le procédé consiste : après réception d'une demande de service de rendu en nuage envoyé par un client, à permettre une ressource de rendu et à créer une liaison de connexion directe entre un terminal de client et la ressource de rendu de telle sorte que la ressource de rendu traite directement la demande de service de rendu en nuage ; et à réaliser, au moyen de la ressource de rendu, un rendu en temps réel et à envoyer directement un résultat de rendu généré en temps réel au client de sorte à permettre au client de recevoir le résultat de rendu en temps réel et à afficher une image en fonction du résultat de rendu. Plusieurs dispositifs de client qui sont en même temps en ligne peuvent être pris en charge, des demandes de service de rendu en masse des dispositifs de client peuvent être transportées et toutes les tâches de rendu en ligne sont mutuellement indépendantes de telle sorte que la charge de communication d'une extrémité en nuage soit réduite et que le retard temporel d'un service de rendu en nuage soit réduit.
PCT/CN2016/088767 2015-11-24 2016-07-06 Procédé, appareil et système de rendu hors écran en temps réel basés sur l'informatique en nuage WO2017088484A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510827310.5 2015-11-24
CN201510827310.5A CN105488837B (zh) 2015-11-24 2015-11-24 基于云计算的实时离屏渲染方法、装置及系统

Publications (1)

Publication Number Publication Date
WO2017088484A1 true WO2017088484A1 (fr) 2017-06-01

Family

ID=55675801

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/088767 WO2017088484A1 (fr) 2015-11-24 2016-07-06 Procédé, appareil et système de rendu hors écran en temps réel basés sur l'informatique en nuage

Country Status (2)

Country Link
CN (1) CN105488837B (fr)
WO (1) WO2017088484A1 (fr)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110264550A (zh) * 2019-06-19 2019-09-20 深圳市瑞云科技有限公司 一种基于cg实时渲染的烟花汇演效果模拟展示系统及方法
CN111260540A (zh) * 2020-01-13 2020-06-09 成都卓影科技股份有限公司 5g网络下的2d-3d的2.5d转换引擎
CN112132942A (zh) * 2020-09-30 2020-12-25 深圳星寻科技有限公司 一种三维场景漫游实时渲染方法
CN112486620A (zh) * 2020-12-03 2021-03-12 广州虎牙科技有限公司 跨进程离屏渲染方法、装置、电子设备及存储介质
CN113382221A (zh) * 2021-05-14 2021-09-10 异现实科技(成都)有限公司 单实例多终端云渲染系统及其方法
CN113572971A (zh) * 2021-06-30 2021-10-29 杭州群核信息技术有限公司 基于云渲染平台的实时高动态范围图像的曝光控制方法和系统
CN113946399A (zh) * 2021-10-28 2022-01-18 上海数慧系统技术有限公司 一种空间数据加载方法及装置
CN115865518A (zh) * 2023-01-30 2023-03-28 天云融创数据科技（北京）有限公司 一种基于大数据的云平台数据处理方法及系统
CN116415321A (zh) * 2022-12-28 2023-07-11 浙江华东工程数字技术有限公司 一种云端cad处理系统及其构建方法
CN116563085A (zh) * 2023-07-06 2023-08-08 湖南马栏山视频先进技术研究院有限公司 一种离线渲染的大规模并行处理方法及系统

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105488837B (zh) * 2015-11-24 2019-06-04 成都云创互联信息技术有限公司 基于云计算的实时离屏渲染方法、装置及系统
CN105677240B (zh) 2015-12-30 2019-04-23 上海联影医疗科技有限公司 数据删除方法及系统
CN106202927A (zh) * 2016-05-31 2016-12-07 武汉联影医疗科技有限公司 医学影像的渲染方法和系统
CN107483390B (zh) * 2016-06-08 2020-10-16 成都云创互联信息技术有限公司 一种云渲染网络部署子系统、系统及云渲染平台
CN106127844A (zh) * 2016-06-22 2016-11-16 民政部零研究所 移动终端用户实时交互访问远程3d场景的渲染交互方法
CN106126148B (zh) * 2016-06-30 2020-05-26 联想(北京)有限公司 显示控制方法及电子设备
CN108074210B (zh) * 2016-11-10 2021-05-11 成都云创互联信息技术有限公司 一种用于云渲染的对象获取系统和方法
CN106598514B (zh) * 2016-12-01 2020-06-09 惠州Tcl移动通信有限公司 一种终端设备中切换虚拟现实模式的方法及系统
CN108648257B (zh) * 2018-04-09 2020-12-29 腾讯科技（深圳）有限公司 全景画面的获取方法、装置、存储介质及电子装置
CN108924587A (zh) * 2018-06-27 2018-11-30 深圳市炬力北方微电子有限公司 一种基于云端服务器的渲染方法及系统
CN109218402A (zh) * 2018-08-02 2019-01-15 胡雄风 端云一体化解决方案
CN111080768A (zh) * 2019-10-16 2020-04-28 重庆灏漫科技有限公司 多类终端采集建模系统和建模方法
CN110913202B (zh) * 2019-11-26 2022-01-07 深圳英伦科技股份有限公司 一种三维显示云渲染方法和系统
CN111494936A (zh) * 2020-02-12 2020-08-07 阿里巴巴集团控股有限公司 画面渲染方法、设备、系统及存储介质
WO2021190651A1 (fr) * 2020-03-27 2021-09-30 华为技术有限公司 Procédé de réglage de la qualité de rendu et dispositif associé
CN112565818B (zh) * 2020-11-30 2022-01-11 中国矿业大学(北京) 二三维一体化的操作站平台构建方法和电子设备
CN113453073B (zh) * 2021-06-29 2023-01-06 北京百度网讯科技有限公司 一种图像渲染方法、装置、电子设备及存储介质
CN116449961B (zh) * 2023-06-14 2023-10-10 蔚来汽车科技(安徽)有限公司 输入事件控制方法、系统、电子设备、存储介质及车辆
CN117395454A (zh) * 2023-12-08 2024-01-12 南京维赛客网络科技有限公司 应用云渲染端更换虚拟场景资源的方法、系统及存储介质
CN117456113B (zh) * 2023-12-26 2024-04-23 山东山大华天软件有限公司 一种云端离线渲染交互应用实现方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102571900A (zh) * 2010-12-08 2012-07-11 中国电信股份有限公司 一种实现远端实时操控的系统和方法
CN102592315A (zh) * 2011-01-12 2012-07-18 上海库达数字信息技术有限公司 基于gpu云集群的3d渲染平台
CN103247067A (zh) * 2013-04-02 2013-08-14 深圳市瑞云科技有限公司 一种远程渲染方法及云渲染平台
WO2014093641A1 (fr) * 2012-12-13 2014-06-19 Microsoft Corporation Diffusion asynchrone de vidéo rendue dans le nuage
CN105488837A (zh) * 2015-11-24 2016-04-13 成都赫尔墨斯科技有限公司 基于云计算的实时离屏渲染方法、装置及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102571900A (zh) * 2010-12-08 2012-07-11 中国电信股份有限公司 一种实现远端实时操控的系统和方法
CN102592315A (zh) * 2011-01-12 2012-07-18 上海库达数字信息技术有限公司 基于gpu云集群的3d渲染平台
WO2014093641A1 (fr) * 2012-12-13 2014-06-19 Microsoft Corporation Diffusion asynchrone de vidéo rendue dans le nuage
CN103247067A (zh) * 2013-04-02 2013-08-14 深圳市瑞云科技有限公司 一种远程渲染方法及云渲染平台
CN105488837A (zh) * 2015-11-24 2016-04-13 成都赫尔墨斯科技有限公司 基于云计算的实时离屏渲染方法、装置及系统

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110264550A (zh) * 2019-06-19 2019-09-20 深圳市瑞云科技有限公司 一种基于cg实时渲染的烟花汇演效果模拟展示系统及方法
CN111260540B (zh) * 2020-01-13 2023-06-13 成都卓影科技股份有限公司 5g网络下的2d-3d的2.5d转换引擎
CN111260540A (zh) * 2020-01-13 2020-06-09 成都卓影科技股份有限公司 5g网络下的2d-3d的2.5d转换引擎
CN112132942A (zh) * 2020-09-30 2020-12-25 深圳星寻科技有限公司 一种三维场景漫游实时渲染方法
CN112132942B (zh) * 2020-09-30 2022-03-18 深圳星寻科技有限公司 一种三维场景漫游实时渲染方法
CN112486620A (zh) * 2020-12-03 2021-03-12 广州虎牙科技有限公司 跨进程离屏渲染方法、装置、电子设备及存储介质
CN113382221A (zh) * 2021-05-14 2021-09-10 异现实科技(成都)有限公司 单实例多终端云渲染系统及其方法
CN113572971A (zh) * 2021-06-30 2021-10-29 杭州群核信息技术有限公司 基于云渲染平台的实时高动态范围图像的曝光控制方法和系统
CN113946399A (zh) * 2021-10-28 2022-01-18 上海数慧系统技术有限公司 一种空间数据加载方法及装置
CN113946399B (zh) * 2021-10-28 2023-07-21 上海数慧系统技术有限公司 一种空间数据加载方法及装置
CN116415321A (zh) * 2022-12-28 2023-07-11 浙江华东工程数字技术有限公司 一种云端cad处理系统及其构建方法
CN115865518B (zh) * 2023-01-30 2023-05-16 天云融创数据科技（北京）有限公司 一种基于大数据的云平台数据处理方法及系统
CN115865518A (zh) * 2023-01-30 2023-03-28 天云融创数据科技（北京）有限公司 一种基于大数据的云平台数据处理方法及系统
CN116563085A (zh) * 2023-07-06 2023-08-08 湖南马栏山视频先进技术研究院有限公司 一种离线渲染的大规模并行处理方法及系统
CN116563085B (zh) * 2023-07-06 2023-09-01 湖南马栏山视频先进技术研究院有限公司 一种离线渲染的大规模并行处理方法及系统

Also Published As

Publication number Publication date
CN105488837A (zh) 2016-04-13
CN105488837B (zh) 2019-06-04

Similar Documents

Publication Publication Date Title
WO2017088484A1 (fr) Procédé, appareil et système de rendu hors écran en temps réel basés sur l'informatique en nuage
WO2022028089A1 (fr) Procédé et dispositif de traitement d'image, serveur et support
CN111399964B (zh) 一种基于视频串流技术的云桌面平台
CN103023872B (zh) 一种云游戏服务平台
WO2019019370A1 (fr) Procédé de traitement pour diffusion en direct de données audio et vidéo, support d'informations et terminal mobile
CN106303674B (zh) 数据传输方法、装置和智能电视系统
US20230096900A1 (en) Cloud gaming processing method, apparatus, and device, and storage medium
WO2014082505A1 (fr) Procédé et système de transcodage, et dispositif de fichiers distribués
WO2017107911A1 (fr) Procédé et dispositif de lecture de vidéo avec plate-forme vidéo en nuage
CN107370714A (zh) 面向云渲染的高效通讯方法
CN102307153A (zh) 一种虚拟桌面传输设备和方法
US11863832B2 (en) Start-up performance improvement for remote computer application
WO2014139269A1 (fr) Procédé et appareil pour lire et traiter des vidéos sur la base d'un bureau virtuel
CN104349177B (zh) 一种在桌面云下转向播放多媒体文件的方法、虚拟机及系统
WO2022021536A1 (fr) Procédé et système pour jeu de scène 3d natif en nuage
CN104144351A (zh) 应用虚拟化平台的视频播放方法和装置
US11889133B2 (en) Burst traffic processing method, computer device and readable storage medium
WO2015120766A1 (fr) Système et procédé d'optimisation de vidéo
WO2023045551A1 (fr) Procédé et système de commande d'appareil de prise de vues, dispositif électronique et support d'enregistrement
WO2014121477A1 (fr) Procédé, dispositif et système de redirection de vidéo, et support lisible par ordinateur
CN115445194A (zh) 游戏的渲染方法、装置、设备及存储介质
CN103220347B (zh) Crp云交互方法
CN103763307B (zh) 一种带宽优化方法及系统
CN110113298A (zh) 数据传输方法、装置、信令服务器和计算机可读介质
US20210021658A1 (en) Technologies for streaming device role reversal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16867711

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16867711

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16867711

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04.12.2018)

122 Ep: pct application non-entry in european phase

Ref document number: 16867711

Country of ref document: EP

Kind code of ref document: A1