CN113923208B - Processing method for downloading massive drive test data - Google Patents
Processing method for downloading massive drive test data Download PDFInfo
- Publication number
- CN113923208B CN113923208B CN202111154386.8A CN202111154386A CN113923208B CN 113923208 B CN113923208 B CN 113923208B CN 202111154386 A CN202111154386 A CN 202111154386A CN 113923208 B CN113923208 B CN 113923208B
- Authority
- CN
- China
- Prior art keywords
- directory
- information
- file
- drive test
- test data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention relates to a processing method for downloading mass drive test data, which comprises the following steps: calling a virtual file system interface to create a first virtual directory and a second virtual directory; creating a directory mapping relation between a first virtual directory and a basic drive test data sharing directory of a local sharing server; generating shared directory file list information; generating first virtual directory file list information; creating a corresponding relation between the second virtual directory and the remote cloud storage platform; acquiring list information of a selectable drive test data file; generating second virtual directory file list information; acquiring a first directory access instruction; generating a first data file downloading instruction; if the first file path information is matched with the path of the first virtual directory, downloading a basic drive test data file from the basic drive test data sharing directory; and if the first file path information is matched with the path of the second virtual directory, downloading the optional drive test data file from the remote cloud storage platform. The invention can reduce the network flow and concurrency.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a processing method for downloading mass drive test data.
Background
In the field of automatic driving or unmanned driving technology processing, massive drive test data collected, analyzed and processed every day can exceed the level of Terabytes (TB), and if each company builds a data center to process the data storage and call of such volume, the data operation and maintenance cost is very high. With the maturation and development of cloud storage technology, many companies start renting public clouds to store and call massive drive test data, and the data deployment mode can greatly reduce the operation and maintenance cost of independent data maintenance of each company. But this also presents additional problems for the user, such as high network traffic being overwhelming, excessive network bandwidth resource overhead, etc.
Disclosure of Invention
The invention aims at overcoming the defects of the prior art and provides a processing method, electronic equipment and a computer readable storage medium for downloading mass drive test data, wherein the mass drive test data stored on a remote cloud storage platform are divided into two types: the necessary basic drive test data and the optional drive test data; the method comprises the steps that a local shared server is deployed locally, and the latest basic drive test data packets are downloaded in batches from a remote cloud storage platform at regular time; and creating a first virtual directory corresponding to the local shared server shared directory and a second virtual directory corresponding to the remote cloud storage platform at the user side. According to the invention, when a user needs to acquire the basic drive test data file list and download the basic drive test data file, the user does not need to be connected with a remote cloud storage platform, a large amount of network bandwidth resources can be released, and the network flow and the network concurrency can be greatly reduced.
To achieve the above objective, a first aspect of an embodiment of the present invention provides a processing method for downloading massive drive test data, where the method includes:
calling a virtual file system interface to create a first virtual directory and a second virtual directory;
creating a directory mapping relation between the first virtual directory and a basic drive test data sharing directory of a local sharing server; invoking a file access interface of the local sharing server to acquire the basic drive test data file list information under the root directory and under each level of sub-directory of the basic drive test data sharing directory, and generating shared directory file list information; performing first virtual directory file list conversion processing on the shared directory file list information to generate corresponding first virtual directory file list information;
creating a corresponding relation between the second virtual directory and the remote cloud storage platform; calling a data access API interface of the remote cloud storage platform, and acquiring optional drive test data file list information from the remote cloud storage platform; performing second virtual directory file list conversion processing on the optional drive test data file list information to generate corresponding second virtual directory file list information;
acquiring a first directory access instruction input by a user; the first directory access instruction includes first directory path information;
displaying corresponding virtual directory file list information to a user according to the first directory path information, acquiring file information selected by the user from the displayed virtual directory file list information, and generating a first data file downloading instruction; the first data file downloading instruction comprises first file path information and first file name information;
if the first file path information is matched with the path of the first virtual directory, performing basic drive test data sharing directory path conversion processing on the first file path information to generate corresponding sharing directory path information; the shared directory path information and the first file name information form corresponding shared directory data file information; and invoking a file access interface of the local sharing server, and downloading a basic drive test data file matched with the shared directory data file information from the basic drive test data shared directory;
and if the first file path information is matched with the path of the second virtual directory, calling a data access API interface of the remote cloud storage platform, and downloading the optional drive test data file matched with the first file name information from the remote cloud storage platform.
Preferably, the method further comprises:
the local sharing server calls a server data access API interface provided by the remote cloud storage platform at a specified downloading time point every day, and downloads the latest complete basic drive test data file package from the remote cloud storage platform; and storing each latest basic drive test data file in the new complete basic drive test data file packet under a corresponding root directory or each level of sub-directory in the basic drive test data sharing directory according to a preset basic drive test data file-storage directory correspondence.
Preferably, the shared directory file list information includes a plurality of first data file information sets; the first data file information set at least comprises first data file path information, first data file name information, first data file type information and first data file size information; the first data file path information includes first root directory path information; the first root directory path information is specifically a preset shared directory root directory identifier;
the selectable drive test data file list information comprises a plurality of second data file information sets; the second data file information set includes at least second data file path information, second data file name information, second data file type information, and second data file size information; the second data file path information is null.
Preferably, the performing a first virtual directory file list conversion process on the shared directory file list information to generate corresponding first virtual directory file list information specifically includes:
taking the root directory path information of the first virtual directory as a first root directory identifier;
replacing the first root directory path information of each of the first data file information sets in the shared directory file list information using the first root directory identifier;
and taking the shared directory file list information with the root directory path replaced as the first virtual directory file list information.
Preferably, the performing a second virtual directory file list conversion process on the optional drive test data file list information to generate corresponding second virtual directory file list information specifically includes:
taking root directory path information of the second virtual directory as a second root directory identifier;
setting the second root directory path information of each second data file information set in the selectable drive test data file list information by using the second root directory identifier;
and taking the optional drive test data file list information with the root directory path set as the second virtual directory file list information.
Preferably, the displaying corresponding virtual directory file list information to the user according to the first directory path information, and obtaining file information selected by the user from the displayed virtual directory file list information, to generate a first data file downloading instruction, specifically includes:
if the path information of the first directory is matched with the path information of the first virtual directory, displaying the list information of the first virtual directory file to a user; acquiring file information selected by a user from the displayed first virtual directory file list information, and generating a first data file downloading instruction;
if the path information of the first directory is matched with the path information of the second virtual directory, displaying the list information of the second virtual directory file to a user; and acquiring file information selected by a user from the displayed second virtual directory file list information, and generating the first data file downloading instruction.
Preferably, the performing basic drive test data sharing directory path conversion processing on the first file path information to generate corresponding sharing directory path information specifically includes:
in the first file path information, a character string matched with root directory path information of the first virtual directory is marked as a third root directory identifier;
replacing the third root directory identifier of the first file path information with a preset shared directory root directory identifier;
and taking the first file path information with the root directory replaced as the shared directory path information.
A second aspect of an embodiment of the present invention provides an electronic device, including: memory, processor, and transceiver;
the processor is configured to couple to the memory, and read and execute the instructions in the memory, so as to implement the method steps described in the first aspect;
the transceiver is coupled to the processor and is controlled by the processor to transmit and receive messages.
A third aspect of the embodiments of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the method of the first aspect described above.
The embodiment of the invention provides a processing method, electronic equipment and a computer readable storage medium for downloading massive drive test data, which are used for dividing the massive drive test data stored on a remote cloud storage platform into two types: the necessary basic drive test data and the optional drive test data; the method comprises the steps that a local shared server is deployed locally, and the latest basic drive test data packets are downloaded in batches from a remote cloud storage platform at regular time; and creating a first virtual directory corresponding to the local shared server shared directory and a second virtual directory corresponding to the remote cloud storage platform at the user side. According to the invention, the user does not feel the difference operation of local or remote data download in the working mode of the virtual directory, so that the working efficiency is improved; when a user needs to acquire the basic drive test data file list and download the basic drive test data file, the user does not need to be connected with a remote cloud storage platform and directly downloads the basic drive test data file from a shared directory of a local shared server, a large amount of network bandwidth resources are released, and network traffic and network concurrency are greatly reduced.
Drawings
Fig. 1 is a schematic diagram of a processing method for downloading massive drive test data according to a first embodiment of the present invention;
fig. 2 is a schematic structural diagram of an electronic device according to a second embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The first embodiment of the invention provides a processing method for downloading mass drive test data, which is suitable for a mass drive test data management system, wherein the mass drive test data management system comprises a client, a local sharing server and a remote cloud storage platform, the client is respectively connected with the local sharing server and the remote cloud storage platform, the client and the local sharing server are both in a local network, and the local sharing server is connected with the remote cloud storage platform; the embodiment of the invention creates a first virtual directory and a second virtual directory corresponding to a local shared server and a remote cloud storage platform at a client, and downloads corresponding file list information and basic data files from the shared directory based on the mapping relation between the first virtual directory and the shared directory of the local shared server when a user needs to acquire a basic drive test data file list and download the drive test basic data files; when a user needs to acquire a selectable drive test data file list and download the selectable drive test data file, downloading corresponding file list information and the selectable data file from the remote cloud storage platform based on the corresponding relation between the second virtual directory and the remote cloud storage platform; fig. 1 is a schematic diagram of a processing method for downloading mass drive test data according to a first embodiment of the present invention, where, as shown in fig. 1, the method mainly includes the following steps:
and step 1, calling a virtual file system interface to create a first virtual directory and a second virtual directory.
Here, the client uses a virtual file system (Virtual File Systems, VFS) interface in the operating system to locally pre-establish two virtual directories, namely a first, human virtual directory; the virtual directory is created to save the serial instruction operation of the user on data downloading, so that the data downloading operation of the user is complicated, and the working efficiency and the use experience of the user are improved.
Step 2, creating a directory mapping relation between the first virtual directory and a basic drive test data sharing directory of a local sharing server; calling a file access interface of a local sharing server, obtaining basic drive test data file list information under a root directory of a basic drive test data sharing directory and under each level of sub-directory, and generating shared directory file list information; performing first virtual directory file list conversion processing on the shared directory file list information to generate corresponding first virtual directory file list information;
wherein the shared directory file list information includes a plurality of first data file information sets; the first data file information set at least comprises first data file path information, first data file name information, first data file type information and first data file size information; the first data file path information includes first root directory path information; the first root directory path information is specifically a preset shared directory root directory identifier.
Here, in the embodiment of the present invention, a local shared server provides a shared directory, that is, a basic drive test data shared directory, for storing a shared basic drive test data file, where the directory includes a root directory and a multi-level subdirectory; in addition, the local sharing server also provides a group of file access interfaces formed by a series of network directory file access instructions, wherein the file access instructions are similar to file system instructions of an operating system and at least comprise directory traversal instructions, file selection instructions, file copying instructions and the like;
when creating a directory mapping relation between a first virtual directory and a basic drive test data sharing directory of a local sharing server, the embodiment of the invention carries out directory mapping configuration on the first virtual directory and the basic drive test data sharing directory through a network mapping interface of an operating system, thereby establishing the directory mapping relation between the first virtual directory and the basic drive test data sharing directory;
after the directory mapping relation is successfully created, the embodiment of the invention calls a directory traversing instruction of a file access interface of a local sharing server to traverse the sub-directory and the file information under the directory of the basic drive test data sharing directory under the root directory and the basic drive test data files under each level of sub-directory, so as to obtain the root directory of the basic drive test data sharing directory and the basic drive test data file list information under each level of sub-directory to generate shared directory file list information;
after the shared directory file list information is obtained, because the root directory information of the path information of each file is also the root directory identifier of the basic drive test data shared directory, if the shared directory file list information is directly used as the directory traversal basis of the first virtual directory, the root directory identifier of the basic drive test data shared directory needs to be changed into the absolute path identifier of the first virtual directory at the local client; therefore, the file list information corresponding to the absolute path of the first virtual directory, namely the first virtual directory file list information, can be obtained only by performing the first virtual directory file list conversion processing on the shared directory file list information;
further, performing a first virtual directory file list conversion process on the shared directory file list information to generate corresponding first virtual directory file list information, which specifically includes: taking root directory path information of the first virtual directory as a first root directory identifier; replacing the first root directory path information of each first data file information set in the shared directory file list information by using the first root directory identifier; and taking the shared directory file list information with the root directory path replaced as first virtual directory file list information.
Step 3, creating a corresponding relation between the second virtual directory and the remote cloud storage platform; calling a data access API interface of a remote cloud storage platform, and acquiring list information of the optional drive test data file from the remote cloud storage platform; performing second virtual directory file list conversion processing on the information of the selectable drive test data file list to generate corresponding second virtual directory file list information;
wherein the selectable drive test data file list information includes a plurality of second data file information sets; the second data file information set includes at least second data file path information, second data file name information, second data file type information, and second data file size information; the second data file path information is null.
The method comprises the steps that massive drive test data are stored on a remote cloud storage platform, the massive drive test data are divided into two main types of basic drive test data and optional drive test data according to the use frequency of the data, the basic drive test data basically belong to necessary data under different use scenes, and the optional drive test data are optional under different use scenes; the total data amount of the optional drive test data is far greater than that of the basic drive test data under the conventional condition;
the remote cloud storage platform provides a group of data access application program interfaces (Application Programming Interface, API), the data access API interfaces are similar to common service applications, and directory information in any similar file access interfaces is not included, so that the interaction times between the client and the remote cloud storage platform can be reduced, and the data downloading efficiency is improved; the data access API interface at least comprises a selectable drive test data file list application API, a selectable drive test data file downloading API and the like;
because a plurality of remote cloud storage platforms and a plurality of clients may exist in the system, the corresponding relation between the second virtual directory and the remote cloud storage platforms is bound;
after binding the corresponding relation between the second virtual directory and the remote cloud storage platform, the embodiment of the invention calls an application API of a selectable road test data file list in a data access API interface, and obtains selectable road test data file list information from the remote cloud storage platform; the list information lists file lists of all optional drive test data in the cloud, and second data file path information of each second data file information set in the optional drive test data file list information is empty because a directory structure is not adopted;
after obtaining the route test data file list information, because the route information of each file, namely the route information of the second data file, is empty and cannot be used as the directory traversal basis of the second virtual directory, the route information of each second data file in the route test data file list information needs to be replaced by the absolute route identifier of the second virtual directory at the local client to obtain the file list information corresponding to the absolute route of the second virtual directory, namely the second virtual directory file list information;
further, performing a second virtual directory file list conversion process on the optional drive test data file list information to generate corresponding second virtual directory file list information, which specifically includes: taking root directory path information of the second virtual directory as a second root directory identifier; setting second root directory path information of each second data file information set in the selectable drive test data file list information by using the second root directory identifier; and the optional drive test data file list information which completes the setting of the root directory path is used as second virtual directory file list information.
Step 4, acquiring a first directory access instruction input by a user;
the first directory access instruction comprises first directory path information;
here, the first directory access instruction is a directory location instruction in the operating system; such as cd instructions of the DOS operating system or Linux operating system.
Step 5, displaying corresponding virtual directory file list information to a user according to the first directory path information, and obtaining file information selected by the user from the displayed virtual directory file list information to generate a first data file downloading instruction;
the first data file downloading instruction comprises first file path information and first file name information;
after obtaining the first directory access instruction, the client judges whether the first directory path information carried by the first directory access instruction is related to the first and second virtual directories, and if so, the client uses the corresponding first and second virtual directory file list information to output directory traversal results;
the method specifically comprises the following steps: step 51, if the path information of the first directory matches the path information of the first virtual directory, displaying the list information of the first virtual directory file to the user; acquiring file information selected by a user from the displayed first virtual directory file list information, and generating a first data file downloading instruction;
here, as long as the path information of the first virtual directory is included in the first directory path information, it is explained that the first directory path information matches the path information of the first virtual directory; the client takes the first virtual directory file list information as the current directory, namely the directory traversing result of the first virtual directory, and displays the directory traversing result; when in display, a text display mode similar to an instruction line can be adopted, and a tree-shaped and linked list display mode of a graphical interface can also be adopted; the user can select the basic drive test data file to be downloaded from the displayed directory traversal result, namely the first virtual directory file list information;
step 52, if the path information of the first directory matches the path information of the second virtual directory, displaying the list information of the second virtual directory file to the user; and acquiring file information selected by a user from the displayed second virtual directory file list information, and generating a first data file downloading instruction.
Here, as long as the path information of the second virtual directory is included in the first directory path information, it is explained that the first directory path information matches the path information of the second virtual directory; the client takes the second virtual directory file list information as the current directory, namely the directory traversing result of the second virtual directory, and displays the directory traversing result; when in display, a text display mode similar to an instruction line can be adopted, and a tree-shaped and linked list display mode of a graphical interface can also be adopted; the user may thus choose the optional drive test data file to be downloaded from the displayed directory traversal result, i.e., the second virtual directory file list information.
Step 6, if the first file path information is matched with the path of the first virtual directory, performing basic drive test data sharing directory path conversion processing on the first file path information to generate corresponding sharing directory path information; the shared directory path information and the first file name information form corresponding shared directory data file information; and calling a file access interface of the local sharing server, and downloading the basic drive test data file matched with the shared directory data file information from the basic drive test data sharing directory.
Here, as long as the first file path information in the first data file downloading instruction includes the path information of the first virtual directory, it means that the first file path information matches the path of the first virtual directory; at this time, the absolute path information of the first virtual directory at the local client, that is, the root directory path information of the first virtual directory, is included in the first file path information, which is not matched with the root directory information of the basic drive test data sharing directory, so that the basic drive test data sharing directory path conversion processing is performed on the first file path information to generate corresponding shared directory path information;
further, the method for converting the basic drive test data shared directory path to the first file path information generates corresponding shared directory path information, specifically includes: in the first file path information, a character string matched with the root directory path information of the first virtual directory is marked as a third root directory identifier; replacing a third root directory identifier of the first file path information by using a preset shared directory root directory identifier; taking the first file path information with the root directory replaced as shared directory path information;
after path information conversion is completed, the embodiment of the invention calls a file selection instruction of a file access interface of a local sharing server to locate a basic drive test data file matched with shared directory data file information, and copies the located basic drive test data file to a local appointed storage position of a client from a basic drive test data sharing directory by using a file copying instruction so as to complete a processing flow of downloading the basic drive test data file matched with the shared directory data file information from the basic drive test data sharing directory.
And 7, if the path information of the first file is matched with the path of the second virtual directory, calling a data access API interface of the remote cloud storage platform, and downloading the optional drive test data file matched with the first file name information from the remote cloud storage platform.
Here, as long as the path information of the second virtual directory is included in the first file path information in the first data file downloading instruction, it means that the first file path information matches the path of the second virtual directory; at this time, because the remote cloud storage platform adopts the directory structure, operations related to path conversion in step 6 are not needed, and only the first file name information is used as the uploading parameter of the optional drive test data file downloading API to send an optional drive test data file downloading request to the remote cloud storage platform; after receiving a data file downloading request of the drive test with the first file name information, the remote cloud storage platform retrieves the data file of the drive test corresponding to the first file name information from the cloud storage system and sends the data file of the drive test to the client as return data of the data file downloading request of the drive test; after receiving the optional drive test data file sent by the remote cloud storage platform, the client saves the optional drive test data file in a local designated storage position of the client, so that the processing flow of downloading the optional drive test data file matched with the first file name information from the remote cloud storage platform is completed.
In addition, the embodiment of the invention further comprises the following steps: the local sharing server can call a server data access API interface provided by the remote cloud storage platform at a specified downloading time point every day, and the latest complete basic drive test data file package is downloaded from the remote cloud storage platform; and storing each latest basic drive test data file in the new complete basic drive test data file packet under a corresponding root directory or each level of sub-directory in the basic drive test data sharing directory according to a preset basic drive test data file-storage directory corresponding relation.
Here, the above-described designated download time point is generally set to a time point of the evening, for example, 12 pm or the like, for data use without delaying the time of the work.
Fig. 2 is a schematic structural diagram of an electronic device according to a second embodiment of the present invention. The electronic device may be a terminal device or a server for implementing the method of the embodiment of the present invention, or may be a terminal device or a server for implementing the method of the embodiment of the present invention, which is connected to the foregoing terminal device or server. As shown in fig. 2, the electronic device may include: a processor 301 (e.g., a CPU), a memory 302, a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiving actions of the transceiver 303. The memory 302 may store various instructions for performing the various processing functions and implementing the processing steps described in the method embodiments previously described. Preferably, the electronic device according to the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to implement communication connections between the elements. The communication port 306 is used for connection communication between the electronic device and other peripheral devices.
The system bus 305 referred to in fig. 2 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, or the like. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. The communication interface is used to enable communication between the database access apparatus and other devices (e.g., clients, read-write libraries, and read-only libraries). The Memory may comprise random access Memory (Random Access Memory, RAM) and may also include Non-Volatile Memory (Non-Volatile Memory), such as at least one disk Memory.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a graphics processor (Graphics Processing Unit, GPU), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
It should be noted that, the embodiments of the present invention also provide a computer readable storage medium, where instructions are stored, when the computer readable storage medium runs on a computer, to cause the computer to perform the method and the process provided in the above embodiments.
The embodiment of the invention also provides a chip for running the instructions, and the chip is used for executing the processing steps described in the embodiment of the method.
The embodiment of the invention provides a processing method, electronic equipment and a computer readable storage medium for downloading massive drive test data, which are used for dividing the massive drive test data stored on a remote cloud storage platform into two types: the necessary basic drive test data and the optional drive test data; the method comprises the steps that a local shared server is deployed locally, and the latest basic drive test data packets are downloaded in batches from a remote cloud storage platform at regular time; and creating a first virtual directory corresponding to the local shared server shared directory and a second virtual directory corresponding to the remote cloud storage platform at the user side. According to the invention, the user does not feel the difference operation of local or remote data download in the working mode of the virtual directory, so that the working efficiency is improved; when a user needs to acquire the basic drive test data file list and download the basic drive test data file, the user does not need to be connected with a remote cloud storage platform and directly downloads the basic drive test data file from a shared directory of a local shared server, a large amount of network bandwidth resources are released, and network traffic and network concurrency are greatly reduced.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of function in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. The processing method for downloading mass drive test data is characterized by comprising the following steps:
calling a virtual file system interface to create a first virtual directory and a second virtual directory;
creating a directory mapping relation between the first virtual directory and a basic drive test data sharing directory of a local sharing server; invoking a file access interface of the local sharing server to acquire the basic drive test data file list information under the root directory and under each level of sub-directory of the basic drive test data sharing directory, and generating shared directory file list information; performing first virtual directory file list conversion processing on the shared directory file list information to generate corresponding first virtual directory file list information;
creating a corresponding relation between the second virtual directory and the remote cloud storage platform; calling a data access API interface of the remote cloud storage platform, and acquiring optional drive test data file list information from the remote cloud storage platform; performing second virtual directory file list conversion processing on the optional drive test data file list information to generate corresponding second virtual directory file list information;
acquiring a first directory access instruction input by a user; the first directory access instruction includes first directory path information;
displaying corresponding virtual directory file list information to a user according to the first directory path information, acquiring file information selected by the user from the displayed virtual directory file list information, and generating a first data file downloading instruction; the first data file downloading instruction comprises first file path information and first file name information;
if the first file path information is matched with the path of the first virtual directory, performing basic drive test data sharing directory path conversion processing on the first file path information to generate corresponding sharing directory path information; the shared directory path information and the first file name information form corresponding shared directory data file information; and invoking a file access interface of the local sharing server, and downloading a basic drive test data file matched with the shared directory data file information from the basic drive test data shared directory;
if the first file path information is matched with the path of the second virtual directory, calling a data access API interface of the remote cloud storage platform, and downloading an optional drive test data file matched with the first file name information from the remote cloud storage platform;
the local sharing server calls a server data access API interface provided by the remote cloud storage platform at a specified downloading time point every day, and downloads the latest complete basic drive test data file package from the remote cloud storage platform; and storing each latest basic drive test data file in the new complete basic drive test data file packet under a corresponding root directory or each level of sub-directory in the basic drive test data sharing directory according to a preset basic drive test data file-storage directory correspondence.
2. The method of claim 1, wherein,
the shared directory file list information includes a plurality of first data file information sets; the first data file information set at least comprises first data file path information, first data file name information, first data file type information and first data file size information; the first data file path information includes first root directory path information; the first root directory path information is specifically a preset shared directory root directory identifier;
the selectable drive test data file list information comprises a plurality of second data file information sets; the second data file information set includes at least second data file path information, second data file name information, second data file type information, and second data file size information; the second data file path information is null.
3. The method for processing the downloaded mass drive test data according to claim 2, wherein the performing a first virtual directory file list conversion process on the shared directory file list information to generate corresponding first virtual directory file list information specifically includes:
taking the root directory path information of the first virtual directory as a first root directory identifier;
replacing the first root directory path information of each of the first data file information sets in the shared directory file list information using the first root directory identifier;
and taking the shared directory file list information with the root directory path replaced as the first virtual directory file list information.
4. The method for processing the downloaded mass drive test data according to claim 2, wherein the performing a second virtual directory file list conversion process on the selectable drive test data file list information generates corresponding second virtual directory file list information, and specifically includes:
taking root directory path information of the second virtual directory as a second root directory identifier;
setting second root directory path information of each second data file information set in the selectable drive test data file list information by using the second root directory identifier;
and taking the optional drive test data file list information with the root directory path set as the second virtual directory file list information.
5. The method for processing the downloaded mass drive test data according to claim 1, wherein displaying corresponding virtual directory file list information to a user according to the first directory path information, and obtaining file information selected by the user from the displayed virtual directory file list information, and generating a first data file downloading instruction, specifically comprising:
if the path information of the first directory is matched with the path information of the first virtual directory, displaying the list information of the first virtual directory file to a user; acquiring file information selected by a user from the displayed first virtual directory file list information, and generating a first data file downloading instruction;
if the path information of the first directory is matched with the path information of the second virtual directory, displaying the list information of the second virtual directory file to a user; and acquiring file information selected by a user from the displayed second virtual directory file list information, and generating the first data file downloading instruction.
6. The method for processing the downloaded mass drive test data according to claim 1, wherein the performing basic drive test data shared directory path conversion processing on the first file path information generates corresponding shared directory path information, and specifically includes:
in the first file path information, a character string matched with root directory path information of the first virtual directory is marked as a third root directory identifier;
replacing the third root directory identifier of the first file path information with a preset shared directory root directory identifier;
and taking the first file path information with the root directory replaced as the shared directory path information.
7. An electronic device, comprising: memory, processor, and transceiver;
the processor being adapted to be coupled to the memory, read and execute the instructions in the memory to implement the method steps of any one of claims 1-6;
the transceiver is coupled to the processor and is controlled by the processor to transmit and receive messages.
8. A computer readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the instructions of the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111154386.8A CN113923208B (en) | 2021-09-29 | 2021-09-29 | Processing method for downloading massive drive test data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111154386.8A CN113923208B (en) | 2021-09-29 | 2021-09-29 | Processing method for downloading massive drive test data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113923208A CN113923208A (en) | 2022-01-11 |
| CN113923208B true CN113923208B (en) | 2023-07-18 |
Family
ID=79237254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111154386.8A Active CN113923208B (en) | 2021-09-29 | 2021-09-29 | Processing method for downloading massive drive test data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113923208B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114553931B (en) * | 2022-02-23 | 2024-03-08 | 广州小鹏汽车科技有限公司 | Processing method, device, vehicle and system for shared storage space of vehicle |
| CN116233115B (en) * | 2023-02-14 | 2023-09-19 | 中国科学院计算机网络信息中心 | Efficient continuous controlled sharing distribution method and system for mass data files |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111045985A (en) * | 2019-11-25 | 2020-04-21 | 北京百度网讯科技有限公司 | File storage processing method, server, electronic device and storage medium |
| CN111667605A (en) * | 2020-06-10 | 2020-09-15 | 北京百度网讯科技有限公司 | Automatic driving test data storage method and device and electronic equipment |
| CN112214469A (en) * | 2020-10-29 | 2021-01-12 | 北京红山信息科技研究院有限公司 | Drive test data processing method, device, server and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2439578B (en) * | 2006-06-30 | 2011-11-09 | Data Equation Ltd | Data processing |
-
2021
- 2021-09-29 CN CN202111154386.8A patent/CN113923208B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111045985A (en) * | 2019-11-25 | 2020-04-21 | 北京百度网讯科技有限公司 | File storage processing method, server, electronic device and storage medium |
| CN111667605A (en) * | 2020-06-10 | 2020-09-15 | 北京百度网讯科技有限公司 | Automatic driving test data storage method and device and electronic equipment |
| CN112214469A (en) * | 2020-10-29 | 2021-01-12 | 北京红山信息科技研究院有限公司 | Drive test data processing method, device, server and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113923208A (en) | 2022-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10171377B2 (en) | Orchestrating computing resources between different computing environments | |
| CN113923208B (en) | Processing method for downloading massive drive test data | |
| CN111741134B (en) | System and method for quickly constructing virtual machine in large-scale scene of network shooting range | |
| CN109040207B (en) | Method, device and equipment for accessing user mode network file system | |
| CN104539681A (en) | Distributed GIS accelerating system and GIS service processing method | |
| CN110865840B (en) | Application management method, device, server and storage medium | |
| CN113709810B (en) | Method, equipment and medium for configuring network service quality | |
| CN109104368B (en) | Connection request method, device, server and computer readable storage medium | |
| CN112579399B (en) | Cloud service testing method and device, electronic equipment and computer storage medium | |
| CN112422450B (en) | Computer equipment, and flow control method and device for service request | |
| CN114500537A (en) | Access method, system, storage medium and electronic device of container service | |
| CN114465937A (en) | Network card testing method, device, server, medium, and computer program product | |
| CN112261125B (en) | Centralized unit cloud deployment method, device and system | |
| US20220278944A1 (en) | Method for allocating resources of a network infrastructure | |
| CN112994915B (en) | SD-WAN service arrangement method, system, equipment and storage medium | |
| CN112003743B (en) | Service data processing method and device | |
| CN111338758A (en) | Resource management method and device and electronic equipment | |
| CN114070889B (en) | Configuration method, traffic forwarding device, storage medium, and program product | |
| CN116755994A (en) | Cloud platform performance testing method and system, electronic equipment and storage medium | |
| CN115484158A (en) | Communication configuration method and device of vehicle-mounted equipment based on eSIM card | |
| CN112399378B (en) | Internet of vehicles-based over-the-air downloading method, system, equipment and medium | |
| CN114356387A (en) | Process automation method, equipment and system for realizing process automation | |
| CN111026502B (en) | Business debugging architecture creating method and device, electronic equipment and storage medium | |
| CN114257503A (en) | Method, server, system and storage medium for accelerating domain name deployment | |
| CN114979128A (en) | Cross-region communication method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |