CN111241172A - Data storage method, device, terminal and computer readable medium - Google Patents
Data storage method, device, terminal and computer readable medium Download PDFInfo
- Publication number
- CN111241172A CN111241172A CN201911368979.7A CN201911368979A CN111241172A CN 111241172 A CN111241172 A CN 111241172A CN 201911368979 A CN201911368979 A CN 201911368979A CN 111241172 A CN111241172 A CN 111241172A
- Authority
- CN
- China
- Prior art keywords
- data
- target data
- type
- thread
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data storage method, a data storage device, a terminal and a computer readable medium, wherein the method comprises the following steps: receiving a data storage request sent by a terminal; according to the data storage request, acquiring the target data from the terminal, and acquiring the target data type of the target data from a configuration file of the terminal; converting the target data from a target data type to a preset data type through a data processing cluster; the target data of the preset data type is transmitted to the storage system to be stored, and the data transmission of a plurality of devices can be completed by only adopting one data interface, so that the maintenance of the plurality of data interfaces is avoided, and manpower and material resources are saved.
Description
Technical Field
The present application relates to the field of data transmission technologies, and in particular, to a data storage method, an apparatus, a terminal, and a computer-readable medium.
Background
In the process of building a digital factory, various devices all need to transmit data to a database for storage, and because the data types of different devices are different, a corresponding service interface needs to be developed for each data type, which results in a large number of service interfaces, and thus the maintenance complexity of the service interfaces is high.
Disclosure of Invention
An object of the embodiments of the present application is to provide a data storage method, so as to solve the problem that maintaining multiple data interfaces is complicated. The specific technical scheme is as follows:
in a first aspect, a data storage method is provided, the method including:
receiving a data storage request sent by a terminal;
acquiring target data from the terminal according to the data storage request, and acquiring a target data type of the target data from a configuration file of the terminal;
converting the target data from the target data type to a preset data type through a data processing cluster;
and transmitting the target data of the preset data type to a storage system for storage.
Optionally, the converting, by the data processing cluster, the target data from the target data type to a preset data type includes:
according to the data storage request, the target data is transmitted to the data processing cluster through a thread;
and converting the target data from the target data type to the preset data type through the data processing cluster.
Optionally, the delivering the target data to the data processing cluster through a thread according to the data storage request includes:
determining the quantity of the acquired target data according to the data storage request;
establishing a corresponding number of threads according to the number of the target data;
activating the thread to cause the thread to pass the target data to the data processing cluster;
after the thread finishes transmitting the target data, acquiring feedback information of the thread, and sending the feedback information to the terminal, wherein the feedback information is used for informing the terminal of the transmission condition of the target data;
and after the feedback information is sent to the terminal, the thread is recovered.
Optionally, the activating the thread to cause the thread to deliver the target data to the data processing cluster includes:
checking whether the target data transmitted by the thread is correct;
under the condition that the target data transmitted by the thread is correct, acquiring the actual data type of the target data, and confirming whether the actual data type of the target data is the same as the target data type in the configuration file;
and under the condition that the actual data type of the target data is the same as the target data type in the configuration file, converting the target data from the target data type to the preset data type.
Optionally, the converting the target data from the target type to the preset data type includes:
converting the target data type of the acquired target data into an Object;
and converting the Object into the preset data type through a built-in Object.
Optionally, the verifying whether the target data passed by the thread is correct includes:
acquiring target data transmitted by the thread;
comparing the target data with standard data according to the type of the target data;
and if the target data is the same as the standard data, determining that the target data transmitted by the thread is correct.
In a second aspect, a data storage device, the device comprising:
the receiving module is used for receiving a data storage request sent by a terminal;
the acquisition module is used for acquiring target data from the terminal according to the data storage request and acquiring a target data type of the target data from a configuration file of the terminal;
the conversion module is used for converting the target data from the target data type to a preset data type through the data processing cluster;
and the transmission module is used for transmitting the target data of the preset data type to a storage system for storage.
Optionally, the conversion module includes:
the transfer unit is used for transferring the target data to the data processing cluster through a thread according to the data storage request;
and the conversion unit is used for converting the target data from the target data type to the preset data type through the data processing cluster.
In a third aspect, an electronic device is provided, which includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing any of the method steps described herein when executing the program stored in the memory.
In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out any of the method steps.
The embodiment of the application has the following beneficial effects:
the embodiment of the application provides a data interface, the data interface writes the type of data into a configuration file in advance, then the data interface converts the target data in the configuration file from the target data type into a preset data type and transmits the preset data type to a storage system, and by adopting the method, data transmission of a plurality of devices can be completed by only adopting one data interface, so that the maintenance of the plurality of data interfaces is avoided, and manpower and material resources are saved. Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
Fig. 1 is a flowchart of a data storage method according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for transforming data types according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for a thread to transfer target data according to an embodiment of the present disclosure;
fig. 4 is a flowchart of a method for processing target data by a data processing cluster according to an embodiment of the present application;
fig. 5 is a system diagram of a data storage method according to an embodiment of the present application
FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
1. The system comprises a terminal 2, a server 3, a data processing cluster 4 and a storage system.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application provides a data storage method, which can be applied to a server, wherein the server can adopt one data interface to be in butt joint with a plurality of devices, so that the situation that one data interface is arranged for each device is avoided, the number of the data interfaces is reduced, and the complexity of interface maintenance is reduced.
A data storage method provided in an embodiment of the present application will be described in detail below with reference to specific embodiments, as shown in fig. 1, the specific steps are as follows:
step 101: and receiving a data storage request sent by the terminal.
In the embodiment of the invention, different terminals are connected with the same data interface. When the terminal stores data in the storage system through the server, a data storage request is sent to the server, and the server receives the data storage request sent by the terminal.
Step 102: and acquiring the target data from the terminal according to the data storage request, and acquiring the target data type of the target data from the configuration file of the terminal.
In the embodiment of the invention, the server sends a data storage acquisition request to the terminal according to the received data storage request sent by the terminal, and the terminal sends target data corresponding to the data storage acquisition request to the server after receiving the data storage acquisition request. The embodiment of the invention also comprises a configuration file, wherein the configuration file comprises the terminal identification of each terminal and the data type corresponding to each terminal, the configuration file is stored in the server, the server acquires the terminal identification from the data storage request, and then the corresponding target data type is found according to the terminal identification.
Step 103: and converting the target data from the target data type to a preset data type through the data processing cluster.
In the embodiment of the invention, the server converts the target data type into the uniform data type from the target data type through the data processing cluster, and then converts the uniform data type into the preset data type, wherein the storage system can only store the data of the preset data type, so that the target data type of the target data is converted into the preset data type.
Specifically, the server converts the acquired target data from the target data type to the uniform data type Object through the data processing cluster, and then converts the uniform data type Object to the preset data type through the built-in Object. For example, the predetermined data type is a binary data type.
Step 104: and transmitting the target data of the preset data type to a storage system for storage.
In the embodiment of the invention, the server transmits the target data of the preset data type to the storage system for storage.
Optionally, as shown in fig. 2, the converting, by the data processing cluster, the target data from the target data type to the preset data type includes:
step 201: and according to the data storage request, transmitting the target data to the data processing cluster through the thread.
In the embodiment of the invention, after acquiring the target data according to the data storage request, the server establishes a plurality of threads, wherein the threads are used for respectively transmitting the target data to the data processing cluster.
Step 202: and converting the target data from the target data type to a preset data type through the data processing cluster.
In the embodiment of the invention, after the data processing cluster receives the target data sent by the working thread, the target data type is converted into the uniform data type, and the uniform bookkeeping type is converted into the preset data type.
Specifically, the data processing cluster converts the acquired target data from the target data type to the uniform data type Object through a preset protocol according to the acquired target data and the corresponding target data type, and then converts the uniform data type Object to the preset data type through the built-in Object.
Optionally, as shown in fig. 3, the delivering the target data to the data processing cluster by the thread according to the data storage request includes:
step 301: and determining the quantity of the acquired target data according to the data storage request.
In the embodiment of the invention, a server receives data storage requests sent by one or more terminals, acquires one or more target data according to the received one or more data storage requests, and then determines the quantity of the target data.
In the embodiment of the invention, a server receives a plurality of data storage requests sent by a plurality of terminals, the server acquires a plurality of target data according to the received plurality of data storage requests, and then the quantity of the plurality of target data is determined.
Step 302: and establishing a corresponding number of threads according to the number of the target data.
In the embodiment of the invention, the server establishes a corresponding number of threads according to the number of the target data, wherein the number of the threads is the same as the number of all the target data, so that each thread only transmits one piece of target data. After the server establishes the threads, the target data type corresponding to each thread is determined, and the target data of the corresponding target data type is transmitted through each thread.
Step 303: the thread is activated to cause the thread to pass the target data to the data processing cluster.
In the embodiment of the invention, after the corresponding number of threads are established, the server activates the threads with the preset number, and transmits the target data to the data processing cluster through the activated threads.
Step 304: and after the thread transmits the data, acquiring feedback information of the thread, and sending the feedback information to the terminal, wherein the feedback information is used for informing the terminal of the transmission condition of the target data.
In the embodiment of the invention, after the data transmission of the thread is finished, the server acquires the feedback information of the thread and sends the feedback information to the terminal so that the terminal can know the progress of data transmission, wherein the feedback information is used for informing the terminal of the transmission condition of the target data, and specifically, the transmission condition comprises the success of the target data transmission to the storage system or the failure of the target data transmission to the storage system.
Step 305: and after the feedback information is sent to the terminal, recovering the thread.
In the embodiment of the invention, after the server feedback information is sent to the terminal, the thread is recovered so as to release the storage space of the server.
The process of the thread transferring the target data to the data processing cluster is as follows: the method comprises the steps that a server determines the quantity of acquired target data according to a data storage request, then corresponding threads are established according to the quantity of the target data, after the threads are established, the server suspends all the threads to wait for tasks, due to the fact that the capacity of the server for transmitting and processing data is limited, the server activates threads with preset quantity every time, other threads are queued to wait for activation, the server transmits the target data to a data processing cluster through the activated threads, after the activated threads transmit the data, the server recovers the threads to release the storage space of the server, activates the next threads with preset quantity to start transmitting the data, and recovers the threads after the transmission is finished until all the threads are recovered.
The server establishes a plurality of threads, so that the plurality of threads can transmit data to the data processing cluster at the same time, and the data transmission efficiency is improved.
For example, a total of one hundred terminals are provided, a server receives data storage requests sent by one hundred terminals, then determines that the number of target data is one hundred, the server establishes one hundred threads, the server can only transmit and process ten target data each time, therefore, the server activates ten threads, each thread transmits one data, the remaining ninety threads wait in line for processing, after the ten threads finish transmitting data, the server recovers ten threads to release storage space, then establishes ten threads to start transmission of another ten data, and recovers the ten threads after transmission is finished until the one hundred threads are recovered.
Optionally, as shown in fig. 4, activating the thread to make the thread deliver the target data to the data processing cluster includes:
step 401: checking whether the target data passed by the thread is correct.
In the embodiment of the present invention, the thread transmits the target data to the service monitoring module of the data processing cluster, and after receiving the target data, the service monitoring module checks whether the target data is correct, specifically, the service monitoring module obtains a target data type of the target data, and compares the target data with standard data of the same data type in the database to determine integrity and correctness of the target data, such as whether a data length of the target data is correct.
In the embodiment of the present invention, if the format of the target data is the same as that of the standard data, and if the data lengths of the target data and the standard data are the same, it indicates that the target data received by the data processing cluster is correct, then step 402 is performed; and if the target data is different from the standard data, the target data received by the data processing cluster is incorrect, and the incorrect target data is rejected.
In an example, in the embodiment of the present invention, a Cyclic Redundancy Check (CRC) is used to Check the integrity and the correctness of the target data, and the present invention does not limit a specific checking manner.
Step 402: and under the condition that the target data transmitted by the thread is correct, acquiring the actual data type of the target data, and confirming whether the actual data type of the target data is the same as the target data type in the configuration file.
In the embodiment of the present invention, in a case that the service monitoring module determines that the target data transmitted by the thread is correct, the service monitoring module transmits the target data to the authentication module of the data processing cluster, and the authentication module obtains the actual data type of the target data and compares the actual data type with the target data type in the configuration file.
In the embodiment of the invention, if the actual data type of the target data is the same as the target data type in the configuration file, the target data is further proved to be correct; and if the actual data type of the target data is different from the target data type in the configuration file, rejecting the target data, and setting the next access time of the target data, specifically, setting the next access time of the target data as time for informing the terminal of sending the data storage request next time. The authentication module eliminates wrong target data, so that the safety of the data can be ensured, meanwhile, the next access time of the wrong target data is set, the data access storage request quantity of the server can be reduced, the high concurrent data condition is relieved, and the server is prevented from being crashed due to the fact that the data access storage request quantity is too high.
Step 403: and under the condition that the actual data type of the target data is the same as the target data type in the configuration file, converting the target data from the target data type into a preset data type.
In the embodiment of the present invention, in a case that the authentication module determines that the actual data type of the target data is the same as the target type in the configuration file, the authentication module transmits the target data to the information processing module of the data processing cluster, and the information processing module converts the target data from the target data type to the preset data type.
Specifically, the information processing module converts the acquired target data from a target data type to a uniform data type Object through a preset protocol according to the acquired target data and a corresponding target data type, and then converts the uniform data type Object to a preset data type through a built-in Object.
In the embodiment of the invention, after converting the target data from the target data type to the preset data type, the information processing module sends the target data to the updating module of the data processing cluster, and on one hand, the updating module sends the feedback information to the terminal through the thread so that the terminal can know the transmission condition of the target data, and on the other hand, the target data is transmitted to the storage system for storage through the unified data access layer.
Specifically, after receiving the target data sent to the data processing cluster by the preset number of threads, the update module sends feedback information to the terminal through the threads on one hand, so that the terminal knows the delivery condition of the target data, wherein the feedback information is used for notifying the terminal of the delivery condition of the preset number of target data. After the updating module sends the feedback information to the terminal through the thread, the server recovers the thread to release the storage space of the server, and on the other hand, the updating module transmits the preset amount of target data to the storage system for storage through the unified data access layer. The embodiment of the invention adopts the distributed transaction to update the information, and can reduce the coupling degree among the modules.
In the embodiment of the present invention, the service monitoring module, the authentication module, the information processing module, and the update module of the data processing cluster may exist in the same server, or may exist in different servers, respectively. In the embodiment of the present invention, the data interface, the thread, and the data processing cluster may exist in the same server, or the data interface and the thread may exist in one server, and the data processing cluster exists in another server.
In the embodiment of the present invention, a data transmission system is further provided, in which a data interface and a thread exist in one server, and a data processing cluster exists in another server, as shown in fig. 5, the system includes a terminal 1, a server 2, a data processing cluster 3, and a storage system 4, where the terminal 1 is configured to send a data storage request to the server 2, and receive feedback data returned from the server 2; the server 2 is used for acquiring target data from the terminal 1, acquiring target data types of all the target data from the configuration file, transmitting the acquired target data and the target data types to the data processing cluster 3 through threads, and sending feedback data received from the data processing cluster 3 to the terminal 1; (ii) a The data processing cluster 3 is used for receiving target data from the server 2, transmitting the processed target data to the storage system 4, and sending feedback information received from the storage system 4 to the server 2; the storage system 4 is configured to receive the processed target data transmitted by the data processing cluster 3, and send the feedback information to the data processing cluster 3.
Based on the same technical concept, an embodiment of the present application further provides a data transmission apparatus, as shown in fig. 6, the apparatus includes:
a receiving module 601, configured to receive a data storage request sent by a terminal;
an obtaining module 602, configured to obtain the target data from the terminal according to the data storage request, and obtain a target data type of the target data from a configuration file of the terminal;
a conversion module 603, configured to convert, by the data processing cluster, the target data from a target data type to a preset data type;
a transmitting module 604, configured to transmit the target data of the preset data type to a storage system for storage.
Optionally, the conversion module 603 includes:
the transfer unit is used for transferring the target data to a data processing cluster through a thread according to the data storage request;
and the conversion unit is used for converting the target data from the target data type to a preset data type through the data processing cluster.
Optionally, the transmission unit includes:
the acquisition subunit is used for determining the number of the acquired target data according to the data storage request;
the establishing subunit is used for establishing threads with corresponding quantity according to the quantity of the target data;
an activation subunit to activate the thread to cause the thread to pass the target data to a data processing cluster;
and the sending subunit is configured to, after the data transmission of the thread is finished, obtain feedback information of the thread, and send the feedback information to the terminal, where the feedback information is used to notify the terminal of the transmission condition of the target data.
And the recovery subunit is used for recovering the thread after the feedback information is sent to the terminal.
Optionally, the activating subunit includes:
a checking section for checking whether or not target data passed by the thread is correct;
a confirming part, configured to, in a case that target data delivered by the thread is correct, obtain an actual data type of the target data, and confirm whether the actual data type of the target data is the same as a target data type in the configuration file;
and the conversion part is used for converting the target data from the target data type to a preset data type under the condition that the actual data type of the target data is the same as the target type in the configuration file.
Optionally, the transformation part comprises:
a first conversion sub-part, which is used for converting the target data type of the acquired target data into an Object;
and the second conversion subsection is used for converting the Object into a preset data type through the built-in Object.
Optionally, the inspection part comprises:
the acquisition subpart is used for acquiring target data transmitted by the thread;
a comparison subsection for comparing the target data with standard data according to the type of the target data;
and the determining subsection is used for determining that the target data transmitted by the thread is correct if the target data is the same as the standard data.
The embodiment of the application provides a data interface, the data interface writes the type of data into a configuration file in advance, then the data interface converts the target data in the configuration file from the target data type into a preset data type and transmits the preset data type to a storage system, and by adopting the method, data transmission of a plurality of devices can be completed by only adopting one data interface, so that the maintenance of the plurality of data interfaces is avoided, and manpower and material resources are saved. Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.
Based on the same technical concept, an embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704,
a memory 703 for storing a computer program;
the processor 701 is configured to implement the above method steps when executing the program stored in the memory 703.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
In yet another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above data storage methods.
In yet another embodiment, the present invention further provides a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the data storage methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is merely exemplary of the present application and is presented to enable those skilled in the art to understand and practice the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of data storage, the method comprising:
receiving a data storage request sent by a terminal;
acquiring target data from the terminal according to the data storage request, and acquiring a target data type of the target data from a configuration file of the terminal;
converting the target data from the target data type to a preset data type through a data processing cluster;
and transmitting the target data of the preset data type to a storage system for storage.
2. The method of claim 1, wherein the converting, by the data processing cluster, the target data from the target data type to a preset data type comprises:
according to the data storage request, the target data is transmitted to the data processing cluster through a thread;
and converting the target data from the target data type to the preset data type through the data processing cluster.
3. The method of claim 2, wherein said passing the target data to the data processing cluster by a thread in accordance with the data storage request comprises:
determining the quantity of the acquired target data according to the data storage request;
establishing a corresponding number of threads according to the number of the target data;
activating the thread to cause the thread to pass the target data to the data processing cluster;
after the thread finishes transmitting the target data, acquiring feedback information of the thread, and sending the feedback information to the terminal, wherein the feedback information is used for informing the terminal of the transmission condition of the target data;
and after the feedback information is sent to the terminal, the thread is recovered.
4. The method of claim 3, wherein said activating the thread to cause the thread to pass the target data to the data processing cluster comprises:
checking whether the target data transmitted by the thread is correct;
under the condition that the target data transmitted by the thread is correct, acquiring the actual data type of the target data, and confirming whether the actual data type of the target data is the same as the target data type in the configuration file;
and under the condition that the actual data type of the target data is the same as the target data type in the configuration file, converting the target data from the target data type to the preset data type.
5. The method of claim 4, wherein converting the target data from the target type to the preset data type comprises:
converting the target data type of the acquired target data into an Object;
and converting the Object into the preset data type through a built-in Object.
6. The method of claim 4, wherein verifying that the target data passed by the thread is correct comprises:
acquiring target data transmitted by the thread;
comparing the target data with standard data according to the type of the target data;
and if the target data is the same as the standard data, determining that the target data transmitted by the thread is correct.
7. A data storage device, characterized in that the device comprises:
the receiving module is used for receiving a data storage request sent by a terminal;
the acquisition module is used for acquiring target data from the terminal according to the data storage request and acquiring a target data type of the target data from a configuration file of the terminal;
the conversion module is used for converting the target data from the target data type to a preset data type through the data processing cluster;
and the transmission module is used for transmitting the target data of the preset data type to a storage system for storage.
8. The apparatus of claim 7, wherein the conversion module comprises:
the transfer unit is used for transferring the target data to the data processing cluster through a thread according to the data storage request;
and the conversion unit is used for converting the target data from the target data type to the preset data type through the data processing cluster.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1-6 when executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368979.7A CN111241172B (en) | 2019-12-26 | 2019-12-26 | Data storage method, device, terminal and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368979.7A CN111241172B (en) | 2019-12-26 | 2019-12-26 | Data storage method, device, terminal and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111241172A true CN111241172A (en) | 2020-06-05 |
| CN111241172B CN111241172B (en) | 2023-05-26 |
Family
ID=70864011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911368979.7A Active CN111241172B (en) | 2019-12-26 | 2019-12-26 | Data storage method, device, terminal and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111241172B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307064A (en) * | 2020-10-29 | 2021-02-02 | 上海达梦数据库有限公司 | Data management system, method and storage medium |
| CN114640632A (en) * | 2022-03-25 | 2022-06-17 | 北京奇艺世纪科技有限公司 | Data aggregation method, system, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1940133A1 (en) * | 2006-12-31 | 2008-07-02 | Societé Française du Radiotéléphone | System and method for reachability management through at least one communication network |
| CN107277050A (en) * | 2017-07-27 | 2017-10-20 | 维沃移动通信有限公司 | A kind of data processing method, server, terminal and computer-readable recording medium |
| CN109413154A (en) * | 2018-09-26 | 2019-03-01 | 平安普惠企业管理有限公司 | Conversion method of data format, device, computer equipment and storage medium |
| WO2019075978A1 (en) * | 2017-10-16 | 2019-04-25 | 平安科技（深圳）有限公司 | Data transmission method and apparatus, computer device, and storage medium |
-
2019
- 2019-12-26 CN CN201911368979.7A patent/CN111241172B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1940133A1 (en) * | 2006-12-31 | 2008-07-02 | Societé Française du Radiotéléphone | System and method for reachability management through at least one communication network |
| CN107277050A (en) * | 2017-07-27 | 2017-10-20 | 维沃移动通信有限公司 | A kind of data processing method, server, terminal and computer-readable recording medium |
| WO2019075978A1 (en) * | 2017-10-16 | 2019-04-25 | 平安科技（深圳）有限公司 | Data transmission method and apparatus, computer device, and storage medium |
| CN109413154A (en) * | 2018-09-26 | 2019-03-01 | 平安普惠企业管理有限公司 | Conversion method of data format, device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 李日永;金芳;郑绪刚;: "测控数据传输服务系统设计与实现" * |
| 高梦超;胡庆宝;程耀东;周旭;李海波;杜然;: "基于众包的社交网络数据采集模型设计与实现" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307064A (en) * | 2020-10-29 | 2021-02-02 | 上海达梦数据库有限公司 | Data management system, method and storage medium |
| CN114640632A (en) * | 2022-03-25 | 2022-06-17 | 北京奇艺世纪科技有限公司 | Data aggregation method, system, device, equipment and storage medium |
| CN114640632B (en) * | 2022-03-25 | 2023-12-15 | 北京奇艺世纪科技有限公司 | Data aggregation method, system, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111241172B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109995866B (en) | Distributed file verification method and device, computer device and storage medium | |
| CN104156863A (en) | OTA website order processing system and method | |
| CN111241172A (en) | Data storage method, device, terminal and computer readable medium | |
| CN111325501B (en) | Logistics state management method, device, equipment and storage medium | |
| CN111400777B (en) | Network storage system, user authentication method, device and equipment | |
| CN110175833B (en) | Computer device, method, system and medium for binding bank card information | |
| US20160337210A1 (en) | Method and system for trouble ticketing | |
| CN111556125B (en) | Access request distribution method, load balancing equipment and electronic equipment | |
| CN112887196A (en) | Message sending method, system, device, equipment and readable storage medium | |
| US11341842B2 (en) | Metering data management system and computer readable recording medium | |
| CN109936609B (en) | Terminal chain type upgrading method and device and upgrading management server | |
| CN111125240A (en) | Distributed transaction implementation method and device, electronic equipment and storage medium | |
| CN116186661A (en) | Method for summarizing service data, service data summarizing device and storage medium | |
| CN111125168A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN114116262B (en) | Processing method, device, medium and equipment for distributed asynchronous data communication | |
| CN107800754B (en) | Data transmission task processing method and device, and data transmission task monitoring method and device | |
| CN115357266A (en) | CAN bus-based airborne equipment upgrading method and device and storage medium | |
| CN114172894A (en) | Data transmission method and device, server and computer equipment | |
| CN110098860B (en) | Unmanned aerial vehicle data transmission system and data transmission method based on block chain technology | |
| CN115037653A (en) | Service flow monitoring method and device, electronic equipment and storage medium | |
| CN114371860A (en) | Controller batch upgrading method and device | |
| CN112596867A (en) | Suspension transaction processing method and distributed database system | |
| CN111666132B (en) | Distributed transaction implementation method, device, computer system and readable storage medium | |
| CN111371900B (en) | Method and system for monitoring health state of synchronous link | |
| CN117407330A (en) | Communication method, device, equipment and storage medium based on CAN bus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |