CN113010490A - Data storage method and device and electronic equipment - Google Patents
Data storage method and device and electronic equipment Download PDFInfo
- Publication number
- CN113010490A CN113010490A CN202110199865.5A CN202110199865A CN113010490A CN 113010490 A CN113010490 A CN 113010490A CN 202110199865 A CN202110199865 A CN 202110199865A CN 113010490 A CN113010490 A CN 113010490A
- Authority
- CN
- China
- Prior art keywords
- storage
- node
- data
- target
- target database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 143
- 230000002452 interceptive effect Effects 0.000 claims description 34
- 230000006870 function Effects 0.000 claims description 22
- 230000003993 interaction Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 9
- 230000015556 catabolic process Effects 0.000 abstract description 5
- 238000006731 degradation reaction Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 17
- 230000004044 response Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000007405 data analysis Methods 0.000 description 7
- 238000013144 data compression Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2291—User-Defined Types; Storage management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the disclosure discloses a data storage method, a data storage device and electronic equipment. One embodiment of the method comprises: receiving initial data to be stored in a target database; sending the initial data to the processing node to enable the processing node to construct the initial data into a target file suitable for storage in a target database; and issuing a storage instruction to the storage node so that the storage node stores the target file into the target database. Thus, the performance degradation of the target database caused when data is written can be reduced.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a data storage method and device and electronic equipment.
Background
A database can be considered as a repository for storing and managing data. In practical application, data can be written into the database or queried from the database.
In the related art, data is directly written into a database. In writing data, it takes a long time to construct the data in a database into a file suitable for storage. Therefore, directly writing data into the database requires a long time to operate the database, and further, may cause a decrease in performance (e.g., query performance) of the database.
Disclosure of Invention
This disclosure is provided to introduce concepts in a simplified form that are further described below in the detailed description. This disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Embodiments of the present disclosure provide a data processing storage, an apparatus, and an electronic device, which may reduce performance degradation of a target database caused when data is written.
In a first aspect, an embodiment of the present disclosure provides a data storage method, where the method includes: receiving initial data to be stored in a target database; sending the initial data to the processing node to enable the processing node to construct the initial data into a target file suitable for storage in a target database; and issuing a storage instruction to the storage node so that the storage node stores the target file into the target database.
In a second aspect, embodiments of the present disclosure provide a data storage device, the device including: the receiving module is used for receiving initial data to be stored in a target database; the building module is used for sending the initial data to the processing node so that the processing node builds the initial data into a target file suitable for being stored in a target database; and the storage module is used for issuing a storage instruction to the storage node so that the storage node stores the target file into the target database.
In a third aspect, an embodiment of the present disclosure provides a data storage system, including: the interactive node receives initial data to be stored in a target database and sends the initial data to the processing node; a processing node that constructs the initial data into a target file suitable for storage to a target database; and the storage node responds to the storage instruction issued by the interactive node and stores the target file into the target database.
In a fourth aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the data storage method of the first aspect.
In a fifth aspect, embodiments of the present disclosure provide a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the steps of the data storage method according to the first aspect.
According to the data storage method and device and the electronic equipment provided by the embodiment of the disclosure, the interactive node can receive initial data to be stored in the target database and send the initial data to the processing node. The processing node may construct the initial data into a target file suitable for storage to a target database. The interactive node can also issue a storage instruction to the storage node, and further, the storage node can store the target file into a target database. It can be seen that after the initial data is received, the interactive node does not directly write the initial data into the target database, but the processing node constructs the initial data into a target file suitable for being stored in the target database, and then the storage node stores the target file into the target database.
The initial data is firstly constructed into the target file suitable for being stored in the target database, and then the target file is stored in the target database, so that the operations of index construction, data analysis, data compression and the like in the target database can be avoided. Thus, when storing data into the target database, it is possible to avoid operating the target database for a long time. This can reduce the performance degradation of the target database caused when data is written. In particular, when the target database is used for data analysis, the data query performance requirement on the target database is high. Therefore, when data is written into the target database, the target database is prevented from being operated for a long time, and the query performance of the target database can be ensured. Thus, the efficiency of the target database for data analysis can be ensured.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
FIG. 1 is a flow diagram of some embodiments of a data storage method according to the present disclosure;
FIG. 2 is a flow diagram of some embodiments of a data storage method according to the present disclosure;
FIG. 3 is a flow diagram of some embodiments of a data storage method according to the present disclosure;
FIG. 4 is a flow diagram of some embodiments of a data storage method according to the present disclosure;
FIG. 5 is a schematic structural diagram of some embodiments of a data storage device according to the present disclosure;
FIG. 6 is a timing diagram of some embodiments of a data storage system according to the present disclosure;
FIG. 7 is an exemplary system architecture to which the data storage methods of some embodiments of the present disclosure may be applied;
fig. 8A and 8B are schematic diagrams of system architectures in accordance with some embodiments of the present disclosure;
fig. 9 is a schematic diagram of a basic structure of an electronic device provided in accordance with some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Referring to fig. 1, a flow diagram of some embodiments of a data storage method according to the present disclosure is shown. As shown in fig. 1, the data storage method includes the following steps:
In this embodiment, an execution subject of the data storage method (e.g., the interactive node 701 shown in fig. 7) may receive initial data to be stored to a target database.
The initial data may be various data received through various ways. For example, the initial data may be data generated on a received line. In general, the initial data received by the interactive node may include a plurality of pieces of data.
Alternatively, the target database may be a database for data analysis. For example, the target database may be a clickwouse.
In this embodiment, the execution body may send initial data to the processing node. Further, the processing node may construct the initial data as a target file suitable for storage to a target database.
In some scenarios, the processing node may perform data parsing, data compression, index construction, and the like on the initial data to construct the initial data into a target file suitable for storage to a target database.
And 103, issuing a storage instruction to the storage node to enable the storage node to store the target file into the target database.
In this embodiment, the execution subject may issue a storage instruction to the storage node. Further, the storage node may store the target file in a target database.
The target file can be directly stored in the target database after being processed by the processing node, so that when the storage node stores the target file in the target database, operations such as data analysis, data compression, index construction and the like do not need to be carried out in the target database.
In this embodiment, after receiving the initial data, the interactive node does not directly write the initial data into the target database, but the processing node first constructs the initial data into a target file suitable for being stored in the target database, and then the storage node stores the target file in the target database.
Therefore, the processing node performs data analysis, data compression, index construction and other operations on the initial data, and constructs the initial data into a target file suitable for being stored in a target database. Therefore, when the storage node stores the target file in the target database, the operations such as data analysis, data compression, index construction and the like do not need to be executed in the database. Thus, when storing data into the target database, it is possible to avoid operating the target database for a long time. Further, the performance degradation of the target database caused when data is written can be reduced.
Construction of
In some embodiments, the processing node may construct the initial data as a target file according to the flow shown in FIG. 2. The flow includes step 201 and step 202.
The data processing object is packaged with data processing information for constructing initial data into a target file.
The data processing information may be various information for processing the initial data. Alternatively, the data processing information may be information for data parsing, data compression, index construction, and the like of the initial data. The data processing object may be an object generated by packaging the data processing information in various computer languages.
Alternatively, the data processing object is initialized by the processing node based on initialization data sent by the interactive node (i.e., the execution body).
The initialization data is data for initializing a data processing target. In some scenarios, the initialization data may be data that initializes parameters and/or functions encapsulated in the data processing object. Therefore, by initializing the data processing object, the parameters and/or functions for constructing the initial data into the target file can be updated according to actual requirements.
In some scenarios, the processing node may perform data parsing, data compression, index construction, and the like on the initial data by using the data processing information encapsulated in the data processing object, and construct the initial data as a target file suitable for storage in a target database.
By calling the data processing object, the initial data is constructed into the target file suitable for being stored in the target database, and the initial data can be more simply and conveniently constructed into the target file.
In some embodiments, the data processing information includes parameters and/or functions that construct the initial data into the target file.
At this time, the processing node may perform step 202 described above in the following manner.
In particular, the initial data is constructed as an object file suitable for storage into an object database using the parameters and/or functions described above.
Accordingly, according to specific requirements, corresponding parameters and/or functions can be packaged in the data processing object so as to construct initial data into an object file suitable for being stored in an object database. Thus, the initial data may be more flexibly structured into a target file suitable for storage to a target database.
In some embodiments, the target database and the predetermined directory are located on the same device. The target file is stored in a predetermined directory by the processing node. In some scenarios, after constructing the initial data as a target file, the processing node may deposit the target file into a predetermined directory.
Here, the interaction node, the processing node, the storage node, the target database, and the predetermined directory may be provided in the same device.
At this time, the execution body may execute the step 103 according to the flow shown in fig. 3. The flow includes step 301.
Step 301, in response to receiving the storage completion information fed back by the processing node, issuing a storage instruction to the storage node, so that the storage node stores the target file acquired from the predetermined directory into the target database.
After storing the target file in the predetermined directory, the processing node may feed back storage completion information to the execution main body. Further, the execution body may issue a storage instruction to the storage node. Still further, the storage node may obtain a target file from a local predetermined directory, and store the target file in the target database.
Therefore, when the target database and the preset directory are arranged on the same device, the target file can be locally transferred from the preset directory to the target database through the cooperation of the interaction node, the processing node and the storage node. Thus, a new method of storing data into a target database is provided.
In some embodiments, the target database and the predetermined file system are deployed on different devices. The target file is uploaded to a predetermined file system by the processing node. In some scenarios, after the initial data is constructed as a target file, the processing node may upload the target file into a predetermined file system.
Here, the interaction node and the processing node may be located in the same device, and the storage node and the target database may be located in the same device.
At this time, the execution body may execute the step 103 according to the flow shown in fig. 4. The flow includes step 401.
After uploading the target file to the predetermined file system, the processing node may feed back upload completion information to the execution main body. Further, the execution body may issue a storage instruction to the storage node. Still further, the storage node may obtain the target file from a remote predetermined file system, and store the target file in the target database.
Therefore, when the target database and the preset file system are arranged on different devices, the target file can be transferred from the preset file system to the target database in a remote mode through the cooperation of the interaction node, the processing node and the storage node. Thus, a new method of storing data into a target database is provided.
In some embodiments, the execution body may send the initial data to the processing node as follows.
Specifically, initial data is sent to the processing node in response to the data storage condition being satisfied.
The data storage condition may be a condition for storing data into the target database. For example, the data storage conditions include: the generation time of the initial data is within a predetermined time range.
It can be seen that the interactive node sends the initial data to the processing node when the data storage condition is satisfied. Therefore, the target file is stored in the target database through the cooperation of the interaction node, the processing node and the storage node. Thus, when the data storage condition is satisfied, the target data is stored into the target database.
Optionally, the data storage conditions include at least one of: the number of the initial data is greater than or equal to a preset number threshold; the time length for which data are not stored in the target database is greater than or equal to a preset time length threshold value.
Thus, the interactive node may send the initial data to the processing node when the amount of initial data is greater than or equal to the amount threshold. Further, a large amount of initial data can be constructed into a target file at one time and stored in a target database. When the duration of time for which data is not stored in the target database is greater than or equal to the duration threshold, the interactive node may send the initial data to the processing node. Further, data may be stored in the target database at certain time intervals.
Thus, the number of times data is stored into the target database can be reduced. To some extent, performance degradation of the target database may be reduced.
With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a data storage device, which correspond to the method embodiment shown in fig. 1, and which may be applied in various electronic devices in particular.
As shown in fig. 5, the data storage device of the present embodiment includes: a receiving module 501, a building module 502 and a storing module 503. The receiving module 501 is configured to: initial data to be stored to a target database is received. The building block 502 is configured to: the initial data is sent to the processing node to cause the processing node to construct the initial data into a target file suitable for storage to a target database. The storage module 503 is used for: and issuing a storage instruction to the storage node so that the storage node stores the target file into the target database.
In this embodiment, specific processing of the receiving module 501, the constructing module 502, and the storing module 503 of the data storage device and technical effects thereof can refer to related descriptions of step 101, step 102, and step 103 in the corresponding embodiment of fig. 1, which are not described herein again.
In some embodiments, the initial data is constructed as a target file by the processing node by: calling a data processing object, wherein data processing information for constructing initial data into a target file is encapsulated in the data processing object; the initial data is constructed into a target file suitable for storage to a target database using the data processing information.
In some embodiments, the target database and the predetermined directory are located on the same device; storing the target file into a preset directory by the processing node; the storage module 503 is further configured to: and responding to the received storage completion information fed back by the processing node, and issuing a storage instruction to the storage node so that the storage node stores the target file acquired from the preset directory into the target database.
In some embodiments, the target database and the predetermined file system are located on different devices; uploading the target file to a preset file system by the processing node; the storage module 503 is further configured to: and responding to the received uploading completion information fed back by the processing node, and issuing a storage instruction to the storage node so that the storage node stores the target file acquired from the preset file system into the target data.
In some embodiments, the data processing object is initialized by the processing node based on initialization data sent by the interaction node.
In some embodiments, the build module 502 is further to: in response to the data storage condition being satisfied, initial data is sent to the processing node.
In some embodiments, the data storage conditions include at least one of: the number of the initial data is greater than or equal to a preset number threshold; the time length for which data are not stored in the target database is greater than or equal to a preset time length threshold value.
In some embodiments, the data processing information includes parameters and/or functions that construct the initial data into the target file; the processing node constructs the initial data into a target file by: the initial data is constructed into an object file suitable for storage into an object database using the parameters and/or functions described above.
Referring to FIG. 6, a timing diagram of some embodiments of a data storage system according to the present disclosure is shown. The data storage system comprises an interaction node, a processing node and a storage node. The timing diagram shown in fig. 6 includes the following steps:
At step 602, the processing node constructs the initial data into a target file suitable for storage to a target database.
In this embodiment, after receiving the initial data to be stored in the target database, the interaction node may send the initial data to the processing node. Further, the processing node may construct the initial data as a target file suitable for storage to a target database. The interactive node can also issue a storage instruction to the storage node. Further, the storage node may store the target file in a target database.
In some embodiments, a processing node may invoke a data processing object to construct initial data into a target file suitable for storage to a target database using data processing information encapsulated in the data processing object.
In some embodiments, the target database and the predetermined directory are provided in the same device. After storing the target file in the predetermined directory, the processing node may feed back storage completion information to the interaction node. Further, the interactive node may issue a storage instruction to the storage node in response to the storage completion information. Still further, the storage node may store the target file retrieved from the predetermined directory in the target database in response to the storage instruction.
In some embodiments, the target database and the predetermined file system are located on different devices. After uploading the target file to the predetermined file system, the processing node may feed back upload completion information to the interaction node. Further, the interactive node may issue a storage instruction to the processing node in response to the upload completion information. Still further, the storage node may store a target file obtained from a predetermined file system in the target database in response to the storage instruction.
In some embodiments, the data processing information includes parameters and/or functions that construct the initial data into the target file. The processing node may construct the initial data into an object file suitable for storage into an object database using the parameters and/or functions described above.
It should be noted that, for the interaction process of the interaction node, the processing node, and the storage node in the data storage system described in the present disclosure, reference may be made to the description in the data storage method described in the present disclosure, and details are not repeated here.
With further reference to fig. 7, fig. 7 illustrates an exemplary system architecture in which the data storage methods of some embodiments of the present disclosure may be applied. As shown in fig. 7, the system architecture may include an interaction node 701, a processing node 702, and a storage node 703.
Interacting node 701 may interact with processing node 702 or storage node 703. Processing node 702 may construct the initial data into a target file suitable for storage to a target database. The storage node 703 may store the target file to a target database.
In practical applications, specific locations of the interactive node 701, the processing node 702, the storage node 703 and the predetermined directory (or the predetermined file system) may be set according to specific requirements.
In some scenarios, referring to the schematic shown in FIG. 8A, the predetermined directory and the target database are located on the same device. After receiving the initial data to be stored to the target database, the interactive node 701 may send the initial data to the processing node 702. Further, the processing node 702 may construct the initial data into a target file suitable for storing into a target database, then store the target file into a predetermined directory, and feed back the storage completion information to the interaction node 701. Still further, the interactive node 701 may issue a storage instruction to the storage node 703. Finally, the storage node 703 may store the target file obtained from the predetermined directory in the target database.
According to actual requirements, the interactive node 701, the processing node 702 and the storage node 703 may also be disposed in the same device as the predetermined directory and the target database.
Therefore, the target file can be locally transferred from the preset directory to the target database.
In some scenarios, referring to the schematic shown in FIG. 8B, the predetermined file system and the target database are located on different devices. Compared with the schematic shown in fig. 8A, the processing node 702 may upload the target file to a predetermined file system, and feed back upload completion information to the interactive node 701. The storage node 703 may store the target file stored in the predetermined file system into the target database in response to the storage instruction issued by the interactive node.
According to actual requirements, the interaction node 701 and the processing node 702 may be deployed in the same device, and the storage node 702 and the target database may be deployed in the same device.
Therefore, the target file can be remotely transferred from the preset directory to the target database.
In practice, the interacting node 701, the processing node 702 and the storage node 703 may be hardware or software. When the interactive node 701, the processing node 702, and the storage node 703 are hardware, they may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When interacting node 701, processing node 702, and storage node 703 are software, they may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be noted that the data storage method provided by the embodiment of the present disclosure may be executed by the interactive node 701, and accordingly, a data storage device may be disposed in the interactive node 701.
It should be understood that the number of interacting nodes 701, processing nodes 702, and storage nodes 703 in FIG. 7 is illustrative only. There may be any number of interacting nodes 701, processing nodes 702, and storage nodes 703, as desired for the implementation.
Referring now to fig. 9, shown is a schematic diagram of an electronic device (e.g., a server deploying the interactive node 701 shown in fig. 7) suitable for use to implement some embodiments of the present disclosure. The electronic device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure. The electronic device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 9, the electronic device may include a processing means (e.g., a central processing unit, a graphic processor, etc.) 901, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage means 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing apparatus 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
Generally, the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 907 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 908 including, for example, magnetic tape, hard disk, etc.; and a communication device 909. The communication means 909 may allow the electronic device to perform wireless or wired communication with other devices to exchange data. While fig. 9 illustrates an electronic device having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 9 may represent one device or may represent multiple devices as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. The computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure when executed by the processing apparatus 901.
It should be noted that the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be included in the electronic device or may exist separately without being incorporated in the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving initial data to be stored in a target database; sending the initial data to the processing node to enable the processing node to construct the initial data into a target file suitable for storage in a target database; and issuing a storage instruction to the storage node so that the storage node stores the target file into the target database.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in some embodiments of the present disclosure may be implemented by software or hardware. The names of these modules do not in some cases constitute a definition of the module itself, and for example, the receiving module may also be described as a module that receives initial data to be stored in the target database.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure in the embodiments of the present disclosure is not limited to the particular combination of the above-described features, but also encompasses other embodiments in which any combination of the above-described features or their equivalents is possible without departing from the scope of the present disclosure. For example, the above features may be interchanged with other features disclosed in this disclosure (but not limited to) those having similar functions.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (16)
1. A data storage method is applied to an interactive node and comprises the following steps:
receiving initial data to be stored in a target database;
sending the initial data to a processing node to cause the processing node to construct the initial data into a target file suitable for storage to the target database;
and issuing a storage instruction to a storage node to enable the storage node to store the target file into the target database.
2. The method of claim 1, wherein the initial data is constructed by the processing node as the target file by:
calling a data processing object, wherein data processing information for constructing initial data into a target file suitable for being stored in the target database is packaged in the data processing object;
building the initial data into the target file suitable for storage to the target database using the data processing information.
3. The method of claim 1, wherein the target database and the predetermined directory are located on the same device; the target file is stored into the preset directory by the processing node; and
the issuing of the storage instruction to the storage node to enable the storage node to store the target file in the target database includes:
and responding to the received storage completion information fed back by the processing node, issuing the storage instruction to the storage node so that the storage node stores the target file acquired from the preset directory into the target database.
4. The method of claim 1, wherein the target database and the predetermined file system are located on different devices; the target file is uploaded to the preset file system by the processing node; and
the issuing of the storage instruction to the storage node to enable the storage node to store the target file in the target database includes:
and responding to the received uploading completion information fed back by the processing node, and issuing the storage instruction to the storage node so that the storage node stores the target file acquired from the preset file system into the data.
5. The method of claim 2, wherein the data processing object is initialized by the processing node based on initialization data sent by the interaction node.
6. The method of claim 1, wherein sending the initial data to a processing node comprises:
and responding to the satisfaction of the data storage condition, and sending the initial data to the processing node.
7. The method of claim 6, wherein the data storage condition comprises at least one of:
the number of the initial data is greater than or equal to a preset number threshold;
the time length for which data are not stored in the target database is greater than or equal to a preset time length threshold value.
8. The method of claim 2, wherein the data processing information includes parameters and/or functions to construct initial data into a target file;
said constructing said initial data into said target file suitable for storage to said target database using said data processing information:
using the parameters and/or the functions, the initial data is constructed into the target file suitable for storage to the target database.
9. A data storage device, applied to an interactive node, comprising:
the receiving module is used for receiving initial data to be stored in a target database;
a construction module for sending the initial data to a processing node to cause the processing node to construct the initial data into a target file suitable for storage to the target database;
and the storage module is used for issuing a storage instruction to the storage node so that the storage node stores the target file into the target database.
10. A data storage system, comprising:
the interactive node receives initial data to be stored in a target database and sends the initial data to the processing node;
the processing node constructs the initial data into a target file suitable for storing to the target database;
and the storage node responds to a storage instruction issued by the interactive node and stores the target file into the target database.
11. The system of claim 10, wherein the processing node invokes a data processing object, wherein the data processing object encapsulates data processing information that constructs initial data into the target file, and wherein the initial data is constructed into the target file suitable for storage into the target database using the data processing information.
12. The system of claim 10, wherein the target database and the predetermined directory are located on the same device;
the processing node feeds back storage completion information to the interactive node after storing the target file to the preset directory;
the interactive node responds to the storage completion information and sends the storage instruction to the storage node;
and the storage node responds to the storage instruction and stores the target file acquired from the preset directory into the target database.
13. The system of claim 10, wherein the target database and the predetermined file system are located on different devices;
the processing node feeds back uploading completion information to the data interaction node after uploading the target file to the preset file system;
the interactive node responds to the uploading completion information and sends the storage instruction to the processing node;
and the storage node responds to the storage instruction and stores the target file acquired from the preset file system into the target database.
14. The system of claim 10, wherein the data processing information includes parameters and/or functions to construct initial data into a target file;
the processing node, using the parameters and/or the functions, constructs the initial data into the target file suitable for storage to the target database.
15. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-8.
16. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110199865.5A CN113010490A (en) | 2021-02-22 | 2021-02-22 | Data storage method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110199865.5A CN113010490A (en) | 2021-02-22 | 2021-02-22 | Data storage method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113010490A true CN113010490A (en) | 2021-06-22 |
Family
ID=76406989
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110199865.5A Pending CN113010490A (en) | 2021-02-22 | 2021-02-22 | Data storage method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113010490A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109213756A (en) * | 2018-10-22 | 2019-01-15 | 北京锐安科技有限公司 | Data storage, search method, device, server and storage medium |
| CN109525632A (en) * | 2018-09-30 | 2019-03-26 | 珠海派诺科技股份有限公司 | Gateway data uploads database connection digital control method, device, equipment and medium |
| CN109582644A (en) * | 2018-12-21 | 2019-04-05 | 深圳前海微众银行股份有限公司 | File memory method, device, equipment and computer readable storage medium |
| CN109783538A (en) * | 2018-12-28 | 2019-05-21 | 陆明 | Data processing system |
| CN110175209A (en) * | 2019-04-12 | 2019-08-27 | 中国人民财产保险股份有限公司 | Incremental data synchronization method, system, equipment and storage medium |
| CN110196836A (en) * | 2019-03-29 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of date storage method and device |
| CN110365540A (en) * | 2019-07-30 | 2019-10-22 | 上海朝旭电子科技有限公司 | A kind of security system data processing equipment |
| CN111241137A (en) * | 2020-01-08 | 2020-06-05 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111858067A (en) * | 2020-07-31 | 2020-10-30 | 厦门网宿有限公司 | Data processing method and device |
| CN111914031A (en) * | 2020-08-11 | 2020-11-10 | 中国科学院地质与地球物理研究所 | Satellite-borne magnetic field data processing system based on big data |
| CN112328678A (en) * | 2020-09-29 | 2021-02-05 | 深圳开源互联网安全技术有限公司 | Document data processing method and device, electronic equipment and storage medium |
-
2021
- 2021-02-22 CN CN202110199865.5A patent/CN113010490A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109525632A (en) * | 2018-09-30 | 2019-03-26 | 珠海派诺科技股份有限公司 | Gateway data uploads database connection digital control method, device, equipment and medium |
| CN109213756A (en) * | 2018-10-22 | 2019-01-15 | 北京锐安科技有限公司 | Data storage, search method, device, server and storage medium |
| CN109582644A (en) * | 2018-12-21 | 2019-04-05 | 深圳前海微众银行股份有限公司 | File memory method, device, equipment and computer readable storage medium |
| CN109783538A (en) * | 2018-12-28 | 2019-05-21 | 陆明 | Data processing system |
| CN110196836A (en) * | 2019-03-29 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of date storage method and device |
| CN110175209A (en) * | 2019-04-12 | 2019-08-27 | 中国人民财产保险股份有限公司 | Incremental data synchronization method, system, equipment and storage medium |
| CN110365540A (en) * | 2019-07-30 | 2019-10-22 | 上海朝旭电子科技有限公司 | A kind of security system data processing equipment |
| CN111241137A (en) * | 2020-01-08 | 2020-06-05 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111858067A (en) * | 2020-07-31 | 2020-10-30 | 厦门网宿有限公司 | Data processing method and device |
| CN111914031A (en) * | 2020-08-11 | 2020-11-10 | 中国科学院地质与地球物理研究所 | Satellite-borne magnetic field data processing system based on big data |
| CN112328678A (en) * | 2020-09-29 | 2021-02-05 | 深圳开源互联网安全技术有限公司 | Document data processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108920222B (en) | Business processing method and device based on rule engine | |
| CN113722055A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN110830427A (en) | Method and device for message encoding and message decoding in netty environment | |
| CN112418389A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN111612434B (en) | Method, apparatus, electronic device and medium for generating processing flow | |
| CN112732302A (en) | Configuration updating method and device of application program and electronic equipment | |
| CN109669720B (en) | Chain type asynchronous request processing method and device based on Promise and electronic equipment | |
| CN112507676B (en) | Method and device for generating energy report, electronic equipment and computer readable medium | |
| CN113010490A (en) | Data storage method and device and electronic equipment | |
| CN114785770A (en) | Mirror layer file sending method and device, electronic equipment and computer readable medium | |
| CN115022328A (en) | Server cluster, server cluster testing method and device and electronic equipment | |
| CN111625692B (en) | Feature extraction method, device, electronic equipment and computer readable medium | |
| CN114398035A (en) | Method, apparatus, device and computer readable medium for providing service using component | |
| CN114157658A (en) | Mirror image warehouse deployment method and device, electronic equipment and computer readable medium | |
| CN114035861A (en) | Cluster configuration method and device, electronic equipment and computer readable medium | |
| JP2017073123A (en) | Bulk data transfer device and method | |
| CN113760487A (en) | Service processing method and device | |
| CN113342837B (en) | Data transmission method, device, electronic equipment and computer readable medium | |
| CN110619079A (en) | Method and device for pushing information | |
| CN112445517B (en) | Inlet file generation method, device, electronic equipment and computer readable medium | |
| CN114356295B (en) | Three-dimensional model construction method and device and electronic equipment | |
| CN112311833B (en) | Data updating method and device | |
| CN113472565B (en) | Method, apparatus, device and computer readable medium for expanding server function | |
| CN115705193A (en) | Distributed compiling method, device, equipment and medium | |
| CN116737122A (en) | Processing method, processing device, terminal equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |