CN114390046A - Remote asset file extremely-fast transmission method and transmission system based on Redis database - Google Patents
Remote asset file extremely-fast transmission method and transmission system based on Redis database Download PDFInfo
- Publication number
- CN114390046A CN114390046A CN202210043130.8A CN202210043130A CN114390046A CN 114390046 A CN114390046 A CN 114390046A CN 202210043130 A CN202210043130 A CN 202210043130A CN 114390046 A CN114390046 A CN 114390046A
- Authority
- CN
- China
- Prior art keywords
- file
- data server
- transmission
- rendering
- redis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000009877 rendering Methods 0.000 claims abstract description 70
- 230000008569 process Effects 0.000 claims description 8
- 238000009434 installation Methods 0.000 claims description 3
- 230000005577 local transmission Effects 0.000 claims 1
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000002950 deficient Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a remote asset file extremely-fast transmission method and system based on a Redis database. The transmission system includes: the system comprises a central platform database, a transmission server connected with the central platform database, a rendering client connected with the transmission server, and a data server connected with the rendering client. According to the invention, the data server is established locally at the client, so that the user only needs to interact with the local data server most of the time when transmitting the file, thereby avoiding a large amount of time consumed by network access and further achieving the purposes of quickly transmitting and quickly submitting the task.
Description
Technical Field
The invention relates to the technical field of file transmission, in particular to a remote asset file extremely-speed transmission method and system based on a Redis database.
Background
In the CG production process, a large number of assets (files) such as maps are required in addition to the main scene files, and if only small scenes are available, rendering is not problematic locally; but how can up to hundreds of TB asset files achieve a fast rendering output locally for a movie project? Obviously, this is impossible, so the cloud rendering platform is generated by the user, and the user can quickly render the output result only by submitting the prepared file to the cloud rendering platform.
The current cloud rendering process is as follows: 1. making a scene by a user; 2. analyzing the scene file by using a client of the cloud rendering platform to obtain assets such as a map required by the scene; 3. uploading a scene file and an asset file; 4. submitting a task and starting rendering after the file is uploaded successfully; 5. and downloading a rendering result.
As shown in fig. 1, in practical applications, since a user has a large amount of assets to upload, it takes a lot of time to transfer files in the step 3, and multiple scenes of the user may use duplicate asset files, which may cause several problems: 1. repeated files are uploaded, and the comparison time between the local files and the server files is long; 2. the file transfer time is further lengthened due to the bandwidth limitation of the client network. Therefore, the experience of the user is very bad, the transmission time is far longer than the rendering time, and the advantages of cloud rendering cannot be fully embodied.
Accordingly, the prior art is deficient and needs improvement.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the remote asset file extremely-fast transmission method and system based on the Redis database are provided, a data node is established locally at a client, and then a user only needs to interact with a local data center most of the time when transmitting assets, so that a large amount of time consumed by network access is avoided, and the purposes of fast transmission and fast task submission are achieved.
The technical scheme of the invention is as follows: a remote asset file extremely-speed transmission method based on a Redis database is provided, and comprises the following steps.
S1: deploying a redis service on the data server, recording an IP address of the data server and a port used by the redis service, and closing a firewall of the data server.
S2: installing a rendering client on a rendering project submitting machine; finding a redis.ini-default file under the installation directory of the rendering and submitting machine client, and renaming the file as redis.ini.
S3: and opening the redis.ini file, filling the IP address of the data server and the port used by the redis service into the redis.ini file, and then saving and closing the redis.ini file.
S4: and starting the rendering client, and submitting the rendering task on the rendering client.
S5: and uploading the rendering task by the rendering client.
S6: the rendering client acquires basic information of a file A which needs to be transmitted currently; and the file path of the file A which needs to be transmitted at present is sent to a data server for query.
S7: if the rendering client inquires the record of the file A which needs to be transmitted currently in the data server, reading the basic information of the file A, and entering the step S8; if the record of the file A which needs to be transmitted currently is not inquired in the data server by the rendering client, the file A is not transmitted or the information of the file A is expired; at this time, the rendering client communicates with the transmission server to determine whether the file a has been transmitted, and the process proceeds to step S10.
S8: the basic information of the file A in the step S6 is verified against the basic information of the file A read from the data server in the step S7.
S9: if the information is consistent, the file A is indicated to be transmitted, the transmission can be directly skipped, and the time in the basic information of the file A in the data server is refreshed; if the information is inconsistent, the file A is not transmitted or the information of the file A is expired; at this time, the local transfer engine of the user needs to communicate with the transfer server to confirm whether the document a has been transferred, and then the process goes to step S10.
S10: if the file A is transmitted, skipping the data transmission of the file A and refreshing the basic information of the file A in the data server; if the file a is not transmitted, that is, the file a does not exist in the platform storage, the rendering client establishes a network connection with the central platform database through the transmission server, and the rendering client starts to upload the file a, and then the process proceeds to step S11.
S11: after the file a is completely transmitted through S10, the basic information of the file a needs to be recorded in the data server.
S12: after the transmission of the file A is finished, the current transmission process of the file A is finished; when the next file is transmitted, the steps of the steps S6-S12 are carried out until the uploading of all the files needing to be uploaded is finished.
In the field of cloud rendering, 90% of files of one item of a user are shared and basically rarely change, the changed files are shot files generally, and the proportion of the shot files to the total files is small. In the existing transmission mode, after a user synchronizes a file to a rendering platform, most of the time is spent in the file comparison between the local file and the rendering platform, and the length of the time mainly depends on the quality of a network. The data server is established locally at the client, so that the user only needs to interact with the local data server most of the time when transmitting the file, thereby avoiding a large amount of time consumption of network access and further achieving the purposes of fast transmission and fast task submission.
The data server adopts computer memory configuration larger than 16G, CPU is in i3 and above, operating system is in windows7 and above, or linux system supporting redis service.
The basic information of the file A comprises: time, size, path.
The basic information of file a in the data server cannot be removed.
Preheating the data in the data server for 4-9 days. The evening data in the data server needs to be preheated for 4-9 days before the local data server can completely reach the expected result, because most users can cover more than 80% of assets in 4-9 days when submitting tasks. So after one week, the transmission experience of the user can be improved by at least 1 order of magnitude
The data server is connected in a local area network where the rendering project submitter is located.
The present invention also provides a transmission system comprising: the system comprises a central platform database, a transmission server connected with the central platform database, a rendering client connected with the transmission server, and a data server connected with the rendering client.
By adopting the scheme, the invention provides the remote asset file extremely-fast transmission method and the transmission system based on the Redis database, the data server is established locally at the client, so that the user only needs to interact with the local data server most of the time when transmitting the file, a large amount of network access time is avoided, and the aims of fast transmission and fast task submission are achieved.
Drawings
Fig. 1 is a functional block diagram of a conventional transmission system;
FIG. 2 is a comparison of the transfer speeds of the raylync engine and the aspera engine under the transfer system of FIG. 1;
FIG. 3 is a functional block diagram of the transmission system of the present invention;
FIG. 4 is a graph comparing the transmission speed of the raylync engine using the method of the present invention with the transmission speed of the aspera engine in the transmission mode of FIG. 1.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
Referring to fig. 3, the present embodiment provides a transmission system, including: the system comprises a central platform database 10, a transmission server 20 connected with the central platform database 10, a rendering client 30 connected with the transmission server 20, and a data server 40 connected with the rendering client 30.
The embodiment also provides a remote asset file extremely-fast transmission method based on the Redis database, which comprises the following steps.
S1: the redis service is deployed on the data server 40, the IP address of the data server 40 and the port used by the redis service are recorded, and the firewall of the data server 40 is closed.
S2: installing a rendering client 30 in a rendering project submitting machine; finding a redis.ini-default file under the installation directory of the rendering and submitting machine client, and renaming the file as redis.ini.
S3: the redis. ini file is opened, the IP address of the data server 40 and the port used by the redis service are filled in the redis. ini file, and then saved and closed.
S4: the rendering client 30 is started and the rendering task is submitted on the rendering client 30.
S5: rendering client 30 uploads the rendering tasks.
S6: the rendering client 30 obtains the basic information of the file a which needs to be transmitted currently; and sends the file path of the file a which needs to be transmitted currently to the data server 40 for query.
S7: if the rendering client 30 inquires the record of the file a which needs to be transmitted currently in the data server 40, reading the basic information of the file a, and proceeding to step S8; if the rendering client 30 does not inquire the record of the file a needing to be transmitted currently in the data server 40, it indicates that the file a has not been transmitted, or the information of the file a has expired; at this time, the rendering client 30 communicates with the transfer server 20 to confirm whether the file a has been transferred, and the process proceeds to step S10.
S8: the basic information of the file A in the step S6 is verified against the basic information of the file A read from the data server 40 in the step S7.
S9: if the information is consistent, the file A is indicated to have been transmitted, the transmission can be directly skipped, and the time in the basic information of the file A in the data server 40 is refreshed; if the information is inconsistent, the file A is not transmitted or the information of the file A is expired; at this time, the local transfer engine of the user needs to communicate with the transfer server 20 to confirm whether the document a has been transferred, and then the process goes to step S10.
S10: if the file A is already transmitted, skipping the data transmission of the file A and refreshing the basic information of the file A in the data server 40; if the file a is not transmitted, that is, the file a does not exist on the platform storage, the rendering client 30 establishes a network connection with the central platform database 10 through the transmission server 20, and the rendering client 30 starts to upload the file a, and then the process proceeds to step S11.
S11: after the file a is completely transmitted through S10, the basic information of the file a needs to be recorded in the data server 40.
S12: after the transmission of the file A is finished, the current transmission process of the file A is finished; when the next file is transmitted, the steps of the steps S6-S12 are carried out until the uploading of all the files needing to be uploaded is finished.
In the field of cloud rendering, 90% of files of one item of a user are shared and basically rarely change, the changed files are shot files generally, and the proportion of the shot files to the total files is small. In the existing transmission mode, after a user synchronizes a file to a rendering platform, most of the time is spent in the file comparison between the local file and the rendering platform, and the length of the time mainly depends on the quality of a network. The data server 40 is established locally at the client, so that the user only needs to interact with the local data server 40 most of the time when transmitting the file, thereby avoiding a large amount of time consumption of network access and further achieving the purposes of quickly transmitting and quickly submitting tasks.
The data server 40 adopts a computer memory configuration larger than 16G, a CPU (central processing unit) is at least i3 and above, an operating system is at least in a windows7 and above, or a linux system supporting redis service.
The basic information of the file A comprises: time, size, path.
The basic information of file a in the data server 40 cannot be removed.
The data in the data server 40 needs to be preheated for 4-9 days. The evening data in the data server 40 needs to be preheated for 4-9 days before the local data server 40 can fully achieve the expected result, because most users can cover more than 80% of the assets in 4-9 days when submitting tasks. So after one week, the user's transmission experience will be improved by at least 1 order of magnitude.
The data server 40 is connected in a local area network where the rendering item rendering machine is located.
Referring to fig. 2 and 4, the user transmission speed of the present invention can be increased by 1 order of magnitude, so as to achieve the purpose of fast transmission and fast task submission, and greatly improve the user experience.
In summary, the invention provides a remote asset file very fast transmission method and transmission system based on a Redis database, a data server is established locally at a client, so that a user only needs to interact with the local data server most of the time when transmitting a file, thereby avoiding a large amount of time consumed by network access and further achieving the purposes of fast transmission and fast task submission.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A remote asset file extremely-fast transmission method based on a Redis database is characterized by comprising the following steps:
s1: deploying a redis service on a data server, recording an IP address of the data server and a port used by the redis service, and closing a firewall of the data server;
s2: installing a rendering client on a rendering project submitting machine; finding a redis.ini-default file under the installation directory of the rendering presenter client, and renaming the file as redis.ini;
s3: opening the redis.ini file, filling the IP address of the data server and the port used by the redis service into the redis.ini file, and then storing and closing the redis.ini file;
s4: starting a rendering client and submitting a rendering task on the rendering client;
s5: the rendering client uploads a rendering task;
s6: the rendering client acquires basic information of a file A which needs to be transmitted currently; inquiring a file path of a file A which needs to be transmitted currently in a data server;
s7: if the rendering client inquires the record of the file A which needs to be transmitted currently in the data server, reading the basic information of the file A, and entering the step S8; if the record of the file A which needs to be transmitted currently is not inquired in the data server by the rendering client, the file A is not transmitted or the information of the file A is expired; at this time, the rendering client communicates with the transmission server to determine whether the file A is transmitted, and the process goes to step S10;
s8: verifying the basic information of the file a in the step S6 with the basic information of the file a read from the data server in the step S7;
s9: if the information is consistent, the file A is indicated to be transmitted, the transmission can be directly skipped, and the time in the basic information of the file A in the data server is refreshed; if the information is inconsistent, the file A is not transmitted or the information of the file A is expired; at this time, the local transmission engine of the user needs to communicate with the transmission server to determine whether the file a is transmitted, and then step S10 is performed;
s10: if the file A is transmitted, skipping the data transmission of the file A and refreshing the basic information of the file A in the data server; if the file A is not transmitted, namely the file A does not exist in the platform storage, the rendering client establishes network connection with the central platform database through the transmission server, the rendering client starts to upload the file A, and the step S11 is entered;
s11: after the file a is transmitted in S10, the basic information of the file a needs to be recorded in the data server;
s12: after the transmission of the file A is finished, the current transmission process of the file A is finished; when the next file is transmitted, the steps of the steps S6-S12 are carried out until the uploading of all the files needing to be uploaded is finished.
2. The method of claim 1, wherein the configuration of the computer memory adopted by the data server is larger than 16G, the CPU is at least i3, the operating system is at least in a windows7 version or above, or a linux system supporting Redis services.
3. The extremely-fast remote asset file transmission method based on Redis database according to claim 1, wherein the basic information of the file A comprises: time, size, path.
4. The method for extremely-fast file transfer of remote assets based on Redis database as claimed in claim 1, wherein the basic information of file A in the data server is not removable.
5. The extremely-fast remote asset file transmission method based on Redis database as claimed in claim 1, wherein the data in the data server needs to be preheated for 4-9 days.
6. The Redis database-based offsite asset file very-fast transmission method according to claim 1, wherein the data server is connected in a local area network where a rendering project submitting machine is located.
7. A transmission system, comprising: the system comprises a central platform database, a transmission server connected with the central platform database, a rendering client connected with the transmission server, and a data server connected with the rendering client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210043130.8A CN114390046A (en) | 2022-01-14 | 2022-01-14 | Remote asset file extremely-fast transmission method and transmission system based on Redis database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210043130.8A CN114390046A (en) | 2022-01-14 | 2022-01-14 | Remote asset file extremely-fast transmission method and transmission system based on Redis database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114390046A true CN114390046A (en) | 2022-04-22 |
Family
ID=81202358
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210043130.8A Pending CN114390046A (en) | 2022-01-14 | 2022-01-14 | Remote asset file extremely-fast transmission method and transmission system based on Redis database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114390046A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103247067A (en) * | 2013-04-02 | 2013-08-14 | 深圳市瑞云科技有限公司 | Remote rendering method and cloud rendering platform |
| WO2014137938A1 (en) * | 2013-03-04 | 2014-09-12 | Vmware, Inc. | Cross-file differential content synchronization |
| CN104052803A (en) * | 2014-06-09 | 2014-09-17 | 国家超级计算深圳中心(深圳云计算中心) | Decentralized distributed rendering method and system |
| CN106254489A (en) * | 2016-08-16 | 2016-12-21 | 王淼 | A kind of cloud rendering system without file transmission and method thereof |
| CN110417890A (en) * | 2019-07-30 | 2019-11-05 | 深圳市瑞云科技有限公司 | A method of promoting the biography verification of enterprise's rendering client file second |
| CN114020703A (en) * | 2021-11-11 | 2022-02-08 | 中国电子科技集团公司第二十八研究所 | Large file uploading method used in narrow-band network environment |
-
2022
- 2022-01-14 CN CN202210043130.8A patent/CN114390046A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014137938A1 (en) * | 2013-03-04 | 2014-09-12 | Vmware, Inc. | Cross-file differential content synchronization |
| CN103247067A (en) * | 2013-04-02 | 2013-08-14 | 深圳市瑞云科技有限公司 | Remote rendering method and cloud rendering platform |
| CN104052803A (en) * | 2014-06-09 | 2014-09-17 | 国家超级计算深圳中心(深圳云计算中心) | Decentralized distributed rendering method and system |
| CN106254489A (en) * | 2016-08-16 | 2016-12-21 | 王淼 | A kind of cloud rendering system without file transmission and method thereof |
| CN110417890A (en) * | 2019-07-30 | 2019-11-05 | 深圳市瑞云科技有限公司 | A method of promoting the biography verification of enterprise's rendering client file second |
| CN114020703A (en) * | 2021-11-11 | 2022-02-08 | 中国电子科技集团公司第二十八研究所 | Large file uploading method used in narrow-band network environment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8412729B2 (en) | Sharing of presets for visual effects or other computer-implemented effects | |
| US20070073766A1 (en) | System, Method, and Computer-Readable Medium for Mobile Media Management | |
| US8352931B2 (en) | Data push service method and system using data pull model | |
| EP1701280A1 (en) | File server for translating user identifier | |
| US20030187868A1 (en) | Data acquisition system | |
| CN101167071A (en) | Methods, systems, and computer-readable media for synchronizing modifiable documents with multiple clients | |
| CA2438284A1 (en) | A method in transmission of information | |
| US11709863B2 (en) | Synchronizing online and offline transfer of data to cloud storage system | |
| CN109033393A (en) | Paster processing method, device, storage medium and electronic equipment | |
| CN114924845A (en) | Mirror image delay loading method and system suitable for edge AI scene | |
| CN114390046A (en) | Remote asset file extremely-fast transmission method and transmission system based on Redis database | |
| CN110677443A (en) | Data transmitting and receiving method, transmitting end, receiving end, system and storage medium | |
| US11157462B2 (en) | Namespace data object name persistence after decoupling of transportable storage device from data server | |
| US20060224461A1 (en) | Systems and methods for providing client based photo user interface | |
| CN104717242A (en) | Method of editing shared file by multiple users, server and user side | |
| WO2023138353A1 (en) | Picture processing method, electronic device and computer readable medium | |
| CN114253839A (en) | Wireless debugging method, system and device | |
| WO2017036308A1 (en) | File uploading method, access method, device and apparatus | |
| CN110636092B (en) | Content access system, method and content server | |
| CN112347046A (en) | Method for acquiring creation time of file in distributed system | |
| EP3780625A1 (en) | Recorded data processing method and relevant device | |
| CN107528797B (en) | Data processing method, device and system | |
| US20040212698A1 (en) | Method and device for image management and image management system | |
| US20110029522A1 (en) | Photo-image Discovery Device Database Management | |
| CN110874874A (en) | Automatic driving data acquisition system and working method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220422 |
|
| RJ01 | Rejection of invention patent application after publication |