CN114546270B - Data storage method and device and electronic equipment - Google Patents
Data storage method and device and electronic equipment Download PDFInfo
- Publication number
- CN114546270B CN114546270B CN202210138857.4A CN202210138857A CN114546270B CN 114546270 B CN114546270 B CN 114546270B CN 202210138857 A CN202210138857 A CN 202210138857A CN 114546270 B CN114546270 B CN 114546270B
- Authority
- CN
- China
- Prior art keywords
- data
- condition
- memory queue
- quotation
- snapshot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000013500 data storage Methods 0.000 title claims abstract description 47
- 238000004590 computer program Methods 0.000 claims description 14
- 230000001960 triggered effect Effects 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 13
- 230000003139 buffering effect Effects 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0877—Cache access modes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data storage method, a data storage device and electronic equipment, comprising the following steps: under the condition that first line condition data is acquired, caching the first line condition data into a memory queue; and under the condition of triggering a storage condition, the first line condition data in the memory queue are written into a storage medium in batches, and the first line condition data cached in the memory queue comprise data of at least one line condition source at different moments.
Description
Technical Field
The present disclosure relates to the field of data storage, and in particular, to a data storage method, apparatus, and electronic device.
Background
The data is recorded in a certain format on a storage medium inside or outside the computer, the data stream reflects the data flowing in the system, features dynamic data, and the data storage reflects the data stationary in the system, features static data. Common storage media are magnetic disks and tapes, and the data storage organization varies from storage medium to storage medium.
In some scenes, thousands of stocks can be delivered in one second and the price can be obtained, the generated data is stored in real time by adopting a real-time data storage mode, when the amount of the quotation data is large, the data is more, the data stream is written into the storage medium in real time, so that the consumption of hardware resources is larger, and the occupation of a central processing unit (Central Processing Unit, CPU) is higher.
Disclosure of Invention
The embodiment of the application aims to provide a data storage method, a data storage device and electronic equipment, so as to solve the problems of high hardware resource consumption and high CPU occupancy rate.
In a first aspect, an embodiment of the present application provides a data storage method, including:
under the condition that first line condition data is acquired, caching the first line condition data into a memory queue; and under the condition of triggering a storage condition, the first line condition data in the memory queue are written into a storage medium in batches, and the first line condition data cached in the memory queue comprise data of at least one line condition source at different moments.
In a second aspect, embodiments of the present application provide a data storage device, including:
the buffer module is used for buffering the first line condition data to the memory queue under the condition that the first line condition data is acquired; and the writing module is used for writing the first line condition data in the memory queue into the storage medium in batches under the condition of triggering the storage condition, wherein the first line condition data cached in the memory queue comprises data of at least one line condition source at different moments.
In a third aspect, embodiments of the present application provide an electronic device including a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete communication with each other through a bus; the memory is used for storing a computer program; the processor is configured to execute a program stored in the memory, to implement the steps of the data storage method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the data storage method according to the first aspect.
As can be seen from the technical solutions provided in the embodiments of the present application, when first line condition data is received, the first line condition data is cached to a memory queue, and when any one of storage conditions is triggered, the first line condition data in the memory queue is written into a storage medium in batch, and the first line condition data cached in the memory queue includes data of at least one line condition source at different moments. Therefore, the first line condition data cached in the memory queue comprises data of at least one line condition source at different moments, and the first line condition data cached in the memory queue is not written into the storage medium in batches until any one of the storage conditions is triggered, so that the number and frequency of writing the line condition data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupation rate of a CPU is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
Fig. 1 is a schematic flow chart of a first method for storing data according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a second flow chart of a data storage method according to an embodiment of the present application;
FIG. 3 is an exemplary diagram of market snapshot data provided in an embodiment of the present application;
FIG. 4 is a third flow chart of a data storage method according to an embodiment of the present disclosure;
FIG. 5 is an exemplary diagram of disk drive data provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of the module composition of a data storage device according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a data storage method, a data storage device and electronic equipment, which solve the problems of high hardware resource consumption and high CPU occupancy rate.
In order to better understand the technical solutions in the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
As shown in fig. 1, an embodiment of the present application provides a data storage method, where an execution body of the method may be a terminal device, that is, the data storage method provided in the embodiment of the present application may be implemented by hardware or software installed on the terminal device. The data storage method specifically comprises the following steps:
in step S101, when the first line condition data is acquired, the first line condition data is cached in the memory queue.
In particular, the first ticker data is data from at least one ticker source, which may be securities, valuable securities, such as stocks, etc., in particular received via a message queue or a user datagram protocol (User Datagram Protocol, udp) message. The first market data comprises but is not limited to market snapshot data, and the market data comprises but is not limited to maximum price, minimum price, volume and the like in a data caching period, and the market data comprises but is not limited to time-sharing trend of a certain stock after being opened, and each transaction data of buying and selling a disc.
The market data acquired in one data caching period is cached to the memory queue firstly instead of being directly written into the storage medium in real time, and the memory queue can be divided into a market snapshot memory queue and a disk memory queue according to different types of first market data, and the data of each type are stored in the corresponding memory queue, so that the data can be distinguished conveniently. For example, the first line condition data includes first line condition snapshot data and first disk port data, the memory queue includes a line condition snapshot memory queue and a disk port memory queue, and caching the first line condition data into the memory queue includes: caching the first line snapshot data into a line snapshot memory queue, and caching the first disk mouth data into a disk mouth memory queue, wherein the first disk mouth data is divided into purchased disk mouth data and sold disk mouth data.
Further, the storage structure of the data of the memory queue may be divided into a sequential queue structure and a chain queue structure. The sequential queue structure refers to sequentially storing first line condition data in the memory queue by using a group of memory units with continuous addresses, and can specifically define a structure array with a specified size as the memory queue. The chained queue structure uses a chained list form to store the values of each element in the memory queue. In the embodiment of the application, the memory queue with the chained queue structure is selected to cache the first line condition data, so that the problem of overflow caused by the fact that the memory queue is full is avoided, and the stability of the cached data of the memory queue is higher.
In step S103, under the condition of triggering the storage condition, the first line condition data in the memory queue is written into the storage medium in batches, where the first line condition data cached in the memory queue includes data of at least one line condition source at different moments.
In particular, the storage condition includes, but is not limited to, the data length of the first line condition data in the memory queue reaching a preset value and/or the end of the current buffering time period. That is, the length of data allowed to be cached in the memory queue can be preset as a preset value, and the duration of one caching period of the cached data in the memory queue, under the condition that any one of the above storage conditions is triggered, the first line data cached in the memory queue is written into the storage medium in batches, so that the number and the frequency of writing the line data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupation rate of the CPU and the memory is reduced. The preset value and the duration of one buffering time period may be set according to actual situations, which are not limited herein.
Further, the first line data in the memory queue can be written into the storage medium in batches under the condition that all conditions in the storage conditions are met, so that the number and frequency of writing the line data into the storage medium are further reduced, the consumption of hardware resources is further reduced, and the occupation rate of the CPU and the memory is reduced.
The storage areas of the storage medium can be divided according to different types of the first quotation data, so that the data of each type can be stored in a classified mode, management of the quotation data is facilitated, and data management efficiency is improved. For example, the first line condition data includes first line condition snapshot data and first disk port data, the memory queue includes a line condition snapshot memory queue and a disk port memory queue, the storage medium is divided into a first storage area and a second storage area, the first line condition snapshot data in the line condition snapshot memory queue is stored in the first storage area, the first disk port data in the disk port memory queue is stored in the second storage area, and the space of the first storage area and the space of the second storage area can be equally divided, or can be divided according to the quantity of the first line condition snapshot data and the data quantity of the first disk port data. The storage medium includes, but is not limited to, semiconductor memory, magnetic surface memory, and the like.
As can be seen from the technical solutions provided in the embodiments of the present application, when first line condition data is received, the first line condition data is cached to a memory queue, and when any one of storage conditions is triggered, the first line condition data in the memory queue is written into a storage medium in batch, and the first line condition data cached in the memory queue includes data of at least one line condition source at different moments. Therefore, the first line condition data cached in the memory queue comprises data of at least one line condition source at different moments, and the first line condition data cached in the memory queue is not written into the storage medium in batches until any one of the storage conditions is triggered, so that the number and frequency of writing the line condition data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupation rate of a CPU is reduced.
As shown in fig. 2, an embodiment of the present application provides a data storage method, where the execution body of the method may be a terminal device, that is, the data storage method provided in the embodiment of the present application may be implemented by hardware or software installed on the terminal device. The data storage method specifically comprises the following steps:
in step S201, when the first line condition data is acquired, the first line condition data is cached in the memory queue.
In step S202, if the time stamp of the second market snapshot data is greater than the time stamp of the first market snapshot data and the volume of the second market snapshot data is greater than the volume of the first market snapshot data under the condition that the second market snapshot data is obtained, the first market snapshot data in the memory queue is updated to the second market snapshot data, and the second market snapshot data corresponds to the same market source with the first market snapshot data.
Specifically, due to network reasons and reasons of non-order preservation of the message queues, the real-time order of the market data may be disordered, and the time of generating the market data is early but the time of arriving at the service is late, so that the market snapshot data in the memory queues need to be updated, and the market snapshot data stored in the memory queues are guaranteed to be new data before triggering the storage conditions.
Further, the first market snapshot data and the second market snapshot data both include the highest price, the lowest price, the starting price, the latest price and the volume of the price, the second market snapshot data may include at least one market snapshot data, and for the same market source (like a stock), if the second market snapshot data is new data compared with the market snapshot data in the memory queue, the market snapshot data in the memory queue needs to be updated.
In one possible implementation manner, the second market snapshot data is multiple, if there are multiple time stamps in the second market snapshot data that are greater than the time stamps of the first market snapshot data, and there are multiple second market snapshot data in the second market snapshot data that have greater volume of transaction than the volume of transaction in the first market snapshot data, updating the first market snapshot data in the memory queue to the second market snapshot data includes: and updating the first quotation snapshot data in the memory queue into quotation snapshot data with the largest timestamp in the second quotation snapshot data. Or updating the first quotation data in the memory queue into quotation snapshot data with the largest volume of transaction in the second quotation snapshot data. Or updating the first quotation data in the memory queue into quotation snapshot data with the largest transaction amount and the largest timestamp in the second quotation snapshot data.
Specifically, comparing the obtained timestamp of any one of the second market snapshot data with the timestamp of the market snapshot data in the memory queue, and if the second market snapshot data has the market snapshot data with the timestamp larger than the timestamp of the market snapshot data in the memory queue and the second market snapshot data has the market snapshot data with the volume larger than the volume of the first market snapshot data, updating the first market snapshot data in the memory queue into the market snapshot data with the maximum timestamp in the second market snapshot data.
For example, as shown in fig. 3, the second market snapshot data includes three messages, each message represents one second market snapshot data, the larger the message sequence number is, the larger the timestamp of the second market snapshot data is, the largest timestamp and the largest transaction amount of the second market snapshot data with the message sequence number of 3 in the three messages are determined to be new data, if the timestamp of the first market snapshot data in the memory queue does not exceed the timestamp of the second market snapshot data with the message sequence number of 3 and the transaction amount does not exceed the transaction amount of the second market snapshot data, the first market snapshot data in the memory queue is updated to be the second market snapshot data with the message sequence of 3, the highest price of 13, the lowest price of 9.5, the open price of 11, the latest price of 13 and the transaction amount of 2100.
In step S203, in the case of triggering the storage condition, the second market snapshot data in the memory queue is written into the storage medium in batch.
It should be noted that, the step S201 and the step S203 have the same or similar implementation manner as the step S101 and the step S103, which may be referred to each other, and the embodiments of the present application are not repeated herein.
According to the technical scheme provided by the embodiment of the application, the first line condition data cached in the memory queue comprises the data of at least one line condition source at different moments, and the first line condition data cached in the memory queue is written into the storage medium in batches until any one of the storage conditions is triggered, so that the number and frequency of writing the line condition data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupation rate of a CPU is reduced. In addition, the first line condition data in the memory queue is updated, so that the line condition data written into the storage medium is guaranteed to be new data, the data is prevented from being outdated and redundant, and the occupancy rate of the storage space of the storage medium is reduced.
As shown in fig. 4, an embodiment of the present application provides a data storage method, where the execution body of the method may be a terminal device, that is, the data storage method provided in the embodiment of the present application may be implemented by hardware or software installed on the terminal device. The data storage method specifically comprises the following steps:
in step S401, when the first line condition data is acquired, the first line condition data is cached in the memory queue.
Specifically, the first line of information includes first disc data including a message number (indicating a size of a time stamp), a type of a transaction (e.g., buy and sell), a price of the transaction (e.g., buy and sell), a listing amount, and a gear of a market source (e.g., stock), etc.
In step S402, if the second disk port data is received, if the timestamp of the second disk port data is greater than the timestamp of the first disk port data, the first disk port data in the memory queue is updated to the second disk port data, and the first disk port data and the second disk port data correspond to the same market.
Specifically, the second slot data includes a message number (indicating a size of a timestamp) carried, a type of a transaction (e.g., buy and sell), a price of the transaction (e.g., buy and sell prices), and a gear of a market source (e.g., stock), etc. The corresponding message serial numbers (time stamps) are respectively arranged for different buying prices and selling prices, and the larger the message serial number is, the larger the time stamp is. For the same stock, the same transaction type contains a plurality of data, and then the data with the largest time stamp in the same transaction type is selected to update the first disc data in the memory queue.
For example, as shown in fig. 5, data with message number 1 to 4 is obtained through the message queue as the second disk port data, wherein the transaction type represented by message number 1 and message number 4 is buy, the transaction type represented by message number 2 and message number 3 is sell, the data corresponding to message number 4 is selected from message number 1 and message number 4 as the most time stamped buy data, the data corresponding to message number 3 is selected from message number 2 and message number 3 as the most time stamped sell data, and if message number 4 (time stamp) is greater than the time stamp of the corresponding buy data in the memory queue, the data corresponding to message number 4 is used to update the buy data in the memory queue. If the message sequence number 3 (time stamp) is greater than the corresponding time stamp of the sold data in the memory queue, the sold data in the memory queue is updated with the data corresponding to the message sequence number 3.
In step S403, when the storage condition is triggered, the second disk data in the memory queue is written to the storage medium in batch.
It should be noted that, the step S401 and the step S403 have the same or similar implementation manner as the step S101 and the step S103, which are not described herein.
According to the technical scheme provided by the embodiment of the application, the first line condition data cached in the memory queue comprises the data of at least one line condition source at different moments, and the first line condition data cached in the memory queue is written into the storage medium in batches until any one of the storage conditions is triggered, so that the number and frequency of writing the line condition data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupation rate of a CPU is reduced. In addition, the first line condition data in the memory queue is updated, so that the line condition data written into the storage medium is guaranteed to be new data, the data is prevented from being outdated and redundant, and the occupancy rate of the storage space of the storage medium is reduced.
The embodiment of the present application further provides a data storage device, based on the same technical concept, according to the data storage method provided in the foregoing embodiment, and fig. 6 is a schematic block diagram of the data storage device provided in the embodiment of the present application, where the data storage device is configured to perform the data storage method described in fig. 1 to 5, and as shown in fig. 6, the data storage device includes: the system comprises a buffer memory module 601 and a writing module 602, wherein the buffer memory module 601 is used for buffering first line condition data to a memory queue under the condition that the first line condition data is acquired; the writing module 602 is configured to write, in batch, first line condition data in the memory queue to the storage medium under a condition that a storage condition is triggered, where the first line condition data cached in the memory queue includes data of at least one line condition source at different moments.
As can be seen from the technical solutions provided in the embodiments of the present application, when first line condition data is received, the first line condition data is cached to a memory queue, and when any one of storage conditions is triggered, the first line condition data in the memory queue is written into a storage medium in batch, and the first line condition data cached in the memory queue includes data of at least one line condition source at different moments. Therefore, the first line condition data cached in the memory queue comprises data of at least one line condition source at different moments, and the first line condition data cached in the memory queue is not written into the storage medium in batches until any one of the storage conditions is triggered, so that the number and frequency of writing the line condition data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupation rate of a CPU and a memory is reduced.
In one possible implementation, the first line condition data includes first line condition snapshot data and first disk port data, and the memory queue includes: the cache module 601 is further configured to cache the first line condition snapshot data to the line condition snapshot memory queue and the first disk port data to the disk port memory queue.
In one possible implementation, the storage medium includes a first storage area and a second storage area, and the writing module 602 is further configured to store the first line condition snapshot data in the line condition snapshot memory queue to the first storage area, and store the first disk port data in the disk port memory queue to the second storage area.
In one possible implementation, the first quotation data includes first quotation snapshot data, further including: the first updating module is configured to update the first market snapshot data in the memory queue to the second market snapshot data if the timestamp of the second market snapshot data is greater than the timestamp of the first market snapshot data and the volume of the second market snapshot data is greater than the volume of the first market snapshot data under the condition that the second market snapshot data is obtained, where the second market snapshot data corresponds to the same market source with the first market snapshot data. The writing module 602 is further configured to write the second market snapshot data in the memory queue to the storage medium in batch when the storage condition is triggered.
In one possible implementation manner, the second market snapshot data is multiple, if there are multiple timestamps in the second market snapshot data that are greater than the timestamps of the first market snapshot data, and there are multiple second market snapshot data in the second market snapshot data that have greater volume of transaction than volume of transaction in the first market snapshot data, the first updating module is further configured to update the first market snapshot data in the memory queue to market snapshot data with the largest timestamps in the multiple second market snapshot data.
In one possible implementation, the first line of emotion data includes first disk mouth data, and further includes: and the second updating module is used for updating the first disk port data in the memory queue into the second disk port data if the time stamp of the second disk port data is larger than the time stamp of the first disk port data under the condition that the second disk port data is received, and the first disk port data and the second disk port data correspond to the same quotation source.
In one possible implementation, the writing module 602 is further configured to write the second disk port data in the memory queue to the storage medium in batch when the storage condition is triggered.
In one possible implementation, the storage condition includes that the data length of the first line condition data in the memory queue reaches a preset value, and/or that the current buffering time period ends.
The data storage device provided in the embodiment of the present application can implement each process in the embodiment corresponding to the data storage method, and in order to avoid repetition, a detailed description is omitted here.
It should be noted that, the data storage device provided in the embodiment of the present application and the data storage method provided in the embodiment of the present application are based on the same application concept, so the implementation of this embodiment may refer to the implementation of the foregoing data storage method, and the repetition is not repeated.
The embodiment of the present application further provides an electronic device, based on the same technical concept, for executing the data storage method provided in the foregoing embodiment of the present application, and fig. 7 is a schematic structural diagram of an electronic device implementing various embodiments of the present application, as shown in fig. 7. The electronic device may vary considerably in configuration or performance and may include one or more processors 701 and memory 702, where the memory 702 may store one or more stored applications or data. Wherein the memory 702 may be transient storage or persistent storage. The application programs stored in the memory 702 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for use in an electronic device.
Still further, the processor 701 may be arranged to communicate with the memory 702 and execute a series of computer executable instructions in the memory 702 on an electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input/output interfaces 705, and one or more keyboards 706.
In this embodiment, the electronic device includes a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete communication with each other through a bus; a memory for storing a computer program; and the processor is used for executing the program stored in the memory to realize the steps in the method embodiment.
There is also provided in this embodiment a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method embodiments.
It will be apparent to those skilled in the art that embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, the electronic device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash memory (flashRAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transshipment) such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
It will be apparent to those skilled in the art that embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (7)
1. A data storage method, the data storage method comprising:
under the condition that first line condition data is acquired, caching the first line condition data into a memory queue, wherein the memory queue is of a chain type queue structure;
under the condition of triggering a storage condition, first line condition data in the memory queue are written into a storage medium in batches, and the first line condition data cached in the memory queue comprise data of at least one line condition source at different moments;
the first line condition data includes first line condition snapshot data and first disk port data, and the memory queue includes: the caching the first line condition data to the memory queue comprises:
caching the first line condition snapshot data to the line condition snapshot memory queue, and caching the first disk port data to the disk port memory queue;
under the condition that second quotation snapshot data are obtained, if the time stamp of the second quotation snapshot data is larger than the time stamp of the first quotation snapshot data and the volume of the incomes in the second quotation snapshot data is larger than the volume of the incomes in the first quotation snapshot data, updating the first quotation snapshot data in the memory queue into the second quotation snapshot data, wherein the second quotation snapshot data and the first quotation snapshot data correspond to the same quotation source;
the batch writing of the first line condition data in the memory queue to the storage medium under the condition of triggering the storage condition comprises:
and under the condition that the storage condition is triggered, the second quotation snapshot data in the memory queue are written into the storage medium in batches.
2. The data storage method of claim 1, wherein the storage medium comprises a first storage area and a second storage area, and wherein the batch writing of the first line condition data in the memory queue to the storage medium comprises:
storing first line condition snapshot data in the line condition snapshot memory queue to the first storage area, and storing first disk port data in the disk port memory queue to the second storage area.
3. The method of claim 1, wherein the second market snapshot data is a plurality of, and if there are a plurality of timestamps in the second market snapshot data that are greater than the timestamps in the first market snapshot data, and there are a plurality of second market snapshot data in the second market snapshot data that have a greater volume of traffic than the volume of traffic in the first market snapshot data, updating the first market snapshot data in the memory queue to the second market snapshot data comprises:
and updating the first quotation snapshot data in the memory queue into the second quotation snapshot data with the largest timestamp in the second quotation snapshot data.
4. A data storage method according to any one of claims 1-3, wherein the first line condition data comprises first disk port data, and after said caching the first line condition data into a memory queue, the method further comprises:
under the condition that second disk port data are received, if the time stamp of the second disk port data is larger than that of the first disk port data, updating the first disk port data in the memory queue into the second disk port data, wherein the first disk port data and the second disk port data correspond to the same quotation source;
the batch writing of the first line condition data in the memory queue to the storage medium under the condition of triggering the storage condition comprises:
and under the condition of triggering a storage condition, writing the second disk port data in the memory queue into a storage medium in batches.
5. The method according to claim 1, wherein the storage condition includes that a data length of the first scenario data in the memory queue reaches a preset value and/or that a current buffering time period ends.
6. A data storage device, the device comprising:
the buffer module is used for buffering the first line condition data to a memory queue under the condition that the first line condition data is acquired, and the memory queue is of a chain type queue structure;
the writing module is used for writing first line condition data in the memory queue into the storage medium in batches under the condition of triggering a storage condition, wherein the first line condition data cached in the memory queue comprises data of at least one line condition source at different moments;
the first line condition data includes first line condition snapshot data and first disk port data, and the memory queue includes: the cache module is also used for caching the first line condition snapshot data to the line condition snapshot memory queue and caching the first disk port data to the disk port memory queue;
the first updating module is used for updating the first quotation snapshot data in the memory queue into the second quotation snapshot data if the time stamp of the second quotation snapshot data is larger than the time stamp of the first quotation snapshot data and the volume of the second quotation snapshot data is larger than the volume of the first quotation snapshot data under the condition that the second quotation snapshot data is acquired, and the second quotation snapshot data and the first quotation snapshot data correspond to the same quotation source;
and the writing module is used for writing the second quotation snapshot data in the memory queue into the storage medium in batches under the condition of triggering the storage condition.
7. An electronic device comprising a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete communication with each other through a communication bus; the memory is used for storing a computer program; the processor is configured to execute a program stored in the memory, and implement the steps of the data storage method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210138857.4A CN114546270B (en) | 2022-02-15 | 2022-02-15 | Data storage method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210138857.4A CN114546270B (en) | 2022-02-15 | 2022-02-15 | Data storage method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114546270A CN114546270A (en) | 2022-05-27 |
| CN114546270B true CN114546270B (en) | 2024-02-09 |
Family
ID=81675930
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210138857.4A Active CN114546270B (en) | 2022-02-15 | 2022-02-15 | Data storage method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114546270B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102611454A (en) * | 2012-01-29 | 2012-07-25 | 上海锅炉厂有限公司 | Dynamic lossless compressing method for real-time historical data |
| CN108664520A (en) * | 2017-04-01 | 2018-10-16 | 北京京东尚科信息技术有限公司 | Safeguard method, apparatus, electronic equipment and the readable storage medium storing program for executing of data consistency |
| CN109379305A (en) * | 2018-11-28 | 2019-02-22 | 上海银赛计算机科技有限公司 | A kind of data distributing method, device, server and storage medium |
| CN109617950A (en) * | 2018-11-26 | 2019-04-12 | 杭州涂鸦信息技术有限公司 | A kind of data-updating method, device, equipment and readable storage medium storing program for executing |
| CN109710182A (en) * | 2018-12-14 | 2019-05-03 | 北京向上一心科技有限公司 | Market data processing method, device and controller and medium |
| CN112000675A (en) * | 2019-05-27 | 2020-11-27 | 招商证券股份有限公司 | Market data updating method and device, terminal equipment and storage medium |
| CN112486888A (en) * | 2020-12-11 | 2021-03-12 | 盛立金融软件开发（杭州）有限公司 | Market data transmission method, device, equipment and medium |
| CN112799595A (en) * | 2021-02-02 | 2021-05-14 | 联想(北京)有限公司 | Data processing method, device and storage medium |
| CN113204580A (en) * | 2021-05-12 | 2021-08-03 | 中华人民共和国广东海事局 | Overwater target position information query method, system, device, equipment and medium |
| CN113434516A (en) * | 2021-08-03 | 2021-09-24 | 深圳华锐金融技术股份有限公司 | Market data storage method and device, computer equipment and readable storage medium |
| WO2021189283A1 (en) * | 2020-03-25 | 2021-09-30 | 深圳市欢太科技有限公司 | Method and device for data processing, electronic device, and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514111A (en) * | 2012-06-21 | 2014-01-15 | 华为技术有限公司 | Method and device for replacing data in Cache |
-
2022
- 2022-02-15 CN CN202210138857.4A patent/CN114546270B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102611454A (en) * | 2012-01-29 | 2012-07-25 | 上海锅炉厂有限公司 | Dynamic lossless compressing method for real-time historical data |
| CN108664520A (en) * | 2017-04-01 | 2018-10-16 | 北京京东尚科信息技术有限公司 | Safeguard method, apparatus, electronic equipment and the readable storage medium storing program for executing of data consistency |
| CN109617950A (en) * | 2018-11-26 | 2019-04-12 | 杭州涂鸦信息技术有限公司 | A kind of data-updating method, device, equipment and readable storage medium storing program for executing |
| CN109379305A (en) * | 2018-11-28 | 2019-02-22 | 上海银赛计算机科技有限公司 | A kind of data distributing method, device, server and storage medium |
| CN109710182A (en) * | 2018-12-14 | 2019-05-03 | 北京向上一心科技有限公司 | Market data processing method, device and controller and medium |
| CN112000675A (en) * | 2019-05-27 | 2020-11-27 | 招商证券股份有限公司 | Market data updating method and device, terminal equipment and storage medium |
| WO2021189283A1 (en) * | 2020-03-25 | 2021-09-30 | 深圳市欢太科技有限公司 | Method and device for data processing, electronic device, and storage medium |
| CN112486888A (en) * | 2020-12-11 | 2021-03-12 | 盛立金融软件开发（杭州）有限公司 | Market data transmission method, device, equipment and medium |
| CN112799595A (en) * | 2021-02-02 | 2021-05-14 | 联想(北京)有限公司 | Data processing method, device and storage medium |
| CN113204580A (en) * | 2021-05-12 | 2021-08-03 | 中华人民共和国广东海事局 | Overwater target position information query method, system, device, equipment and medium |
| CN113434516A (en) * | 2021-08-03 | 2021-09-24 | 深圳华锐金融技术股份有限公司 | Market data storage method and device, computer equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114546270A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200265009A1 (en) | Blockchain-based data migration method and apparatus | |
| CN108055343B (en) | Data synchronization method and device for computer room | |
| US20160142369A1 (en) | Service addressing in distributed environment | |
| CN113034233B (en) | Method, apparatus, medium and program product for allocating resources in a reading application | |
| CN102437929A (en) | Method and device for de-queuing data in queue manager | |
| CN110716813A (en) | Data stream processing method and device, readable storage medium and processor | |
| KR20220125198A (en) | Data additional writing method, apparatus, electronic device, storage medium and computer programs | |
| CN106657182B (en) | Cloud file processing method and device | |
| US20060248015A1 (en) | Adjusting billing rates based on resource use | |
| AU2019253882A1 (en) | Systems and methods for storing data in an integrated array and linked list based structure | |
| CN114546270B (en) | Data storage method and device and electronic equipment | |
| CN107276998B (en) | OpenSSL-based performance optimization method and device | |
| CN106570005A (en) | Database cleaning method and device | |
| CN110764930B (en) | Request or response processing method and device based on message mode | |
| CN115840654B (en) | Message processing method, system, computing device and readable storage medium | |
| CN105264608A (en) | Data storage method, memory controller and central processing unit | |
| US10901972B2 (en) | Table partition configuration method, apparatus and system for database system | |
| CN114116676A (en) | Data migration method and device, electronic equipment and computer readable storage medium | |
| CN110377262B (en) | Data storage method and device, storage medium and processor | |
| CN110874268B (en) | Data processing method, device and equipment | |
| CN111177077B (en) | Data storage and data query method, device, equipment and medium | |
| CN107360441B (en) | User live broadcast room speech control method and device | |
| CN111352897A (en) | Real-time data storage method, equipment and storage medium | |
| CN112291310B (en) | Method and device for counting connection number | |
| CN116126546B (en) | Performance optimization method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |