CN114546270A - Data storage method and device and electronic equipment - Google Patents
Data storage method and device and electronic equipment Download PDFInfo
- Publication number
- CN114546270A CN114546270A CN202210138857.4A CN202210138857A CN114546270A CN 114546270 A CN114546270 A CN 114546270A CN 202210138857 A CN202210138857 A CN 202210138857A CN 114546270 A CN114546270 A CN 114546270A
- Authority
- CN
- China
- Prior art keywords
- data
- quotation
- memory queue
- snapshot
- condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000006399 behavior Effects 0.000 claims abstract description 42
- 230000001960 triggered effect Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 13
- 230000003139 buffering effect Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0877—Cache access modes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data storage method, a data storage device and electronic equipment, wherein the data storage method comprises the following steps: under the condition of acquiring first behavior data, caching the first behavior data into a memory queue; and under the condition of triggering a storage condition, writing the first quotation data in the memory queue into a storage medium in batch, wherein the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments.
Description
Technical Field
The present application relates to the field of data storage, and in particular, to a data storage method and apparatus, and an electronic device.
Background
Data is recorded in a format on a storage medium internal or external to the computer, the data stream reflects data flowing in the system and represents characteristics of dynamic data, and the data storage reflects data that is static in the system and represents characteristics of static data. Common storage media are magnetic disks and magnetic tapes, and the organization of data storage varies with the storage media.
In some scenes, a single stock has thousands of transaction quantities and prices in one second, the generated data is stored in real time in a real-time data storage mode, when the amount of the quotation data is large, the data is more, hardware resource consumption is large due to the fact that data streams are written into a storage medium in real time, and a Central Processing Unit (CPU) occupies a large amount.
Disclosure of Invention
The embodiment of the application aims to provide a data storage method, a data storage device and electronic equipment, so as to solve the problems of high hardware resource consumption and high CPU occupancy rate.
In a first aspect, an embodiment of the present application provides a data storage method, including:
under the condition of acquiring first behavior data, caching the first behavior data into a memory queue; and under the condition of triggering a storage condition, writing the first quotation data in the memory queue into a storage medium in batch, wherein the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments.
In a second aspect, an embodiment of the present application provides a data storage device, including:
the cache module is used for caching the first behavior data to a memory queue under the condition of acquiring the first behavior data; and the writing module is used for writing the first quotation data in the memory queue into a storage medium in batch under the condition of triggering the storage condition, wherein the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; the memory is used for storing a computer program; the processor is configured to execute the program stored in the memory to implement the steps of the data storage method according to the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data storage method according to the first aspect are implemented.
As can be seen from the above technical solutions provided in the embodiments of the present application, when first travel information is received, the first travel information is buffered in a memory queue, and when any one of the storage conditions is triggered, the first travel information in the memory queue is written in a storage medium in batch, where the first travel information buffered in the memory queue includes data of at least one travel information source at different times. Therefore, the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments, and the first quotation data cached in the memory queue is written into the storage medium in batch until any one of the storage conditions is triggered, so that the writing frequency and frequency of the quotation data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupancy rate of a CPU is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a first schematic flowchart of a data storage method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a second data storage method according to an embodiment of the present application;
fig. 3 is an exemplary diagram of market snapshot data provided in an embodiment of the present application;
fig. 4 is a schematic flow chart of a data storage method according to an embodiment of the present application;
fig. 5 is an exemplary diagram of the data of the disk port provided in the embodiment of the present application;
FIG. 6 is a block diagram illustrating an exemplary module of a data storage device according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a data storage method, a data storage device and electronic equipment, and solves the problems of high hardware resource consumption and high CPU occupancy rate.
In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.
For example, as shown in fig. 1, an execution subject of the data storage method provided in the embodiment of the present application may be a terminal device, that is, the data storage method provided in the embodiment of the present application may be implemented by hardware or software installed on the terminal device. The data storage method may specifically include the steps of:
in step S101, when the first behavior data is acquired, the first behavior data is cached in the memory queue.
Specifically, the first behavior data is data from at least one source of a quotation, which may be a security, a value document, such as a stock, etc., and is specifically received via a message queue or a User data Protocol (udp) message. The first quotation data comprises, but is not limited to, quotation snapshot data, quotation intersection data and the like, the quotation snapshot data comprises, but is not limited to, highest price, lowest price, volume of trades and the like in one data caching period, and the intersection data comprises, but is not limited to, each piece of trading data of time-sharing tendency of a certain stock after opening an order, buying an order and selling an order.
The market data acquired in a data caching period is cached in the memory queue instead of being directly written into the storage medium in real time, and the memory queue can be divided into a market snapshot memory queue and a disk port memory queue according to different types of the first market data, and the data of each type is stored in the corresponding memory queue, so that the data can be distinguished conveniently. For example, the first behavior data includes first behavior snapshot data and first disk port data, the memory queue includes a behavior snapshot memory queue and a disk port memory queue, and caching the first behavior data in the memory queue includes: caching the first quotation snapshot data to a quotation snapshot memory queue, and caching the first disk port data to a disk port memory queue, wherein the first disk port data is divided into bought disk port data and sold disk port data.
Further, the storage structure of the data of the memory queue can be divided into a sequential queue structure and a chained queue structure. The sequential queue structure refers to a group of memory units with continuous addresses for sequentially storing first transaction data in a memory queue, and specifically, a structure array with a specified size can be defined as the memory queue. The chain queue structure is used for storing the value of each element in the memory queue in a chain table mode. In the embodiment of the application, the memory queue with the chain queue structure is selected to cache the first behavior data, so that the problem of overflow caused by the fullness of the memory queue is avoided, and the stability of the cache data of the memory queue is high.
In step S103, under the condition that the storage condition is triggered, the first quotation data in the memory queue is written into the storage medium in batch, where the first quotation data cached in the memory queue includes data of at least one quotation source at different times.
Specifically, the storage condition includes, but is not limited to, that the data length of the first line condition data in the memory queue reaches a preset value and/or the current buffer time period ends. That is to say, the length of data allowed to be cached in the memory queue may be preset as a preset value, and the duration of one caching period for caching data in the memory queue, and when any one of the storage conditions is triggered, the first line situation data cached in the memory queue is written into the storage medium in batch, so that the number of times and frequency of writing the line situation data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupancy rates of the CPU and the memory are reduced. The preset value and the duration of one buffering time period may be set according to actual conditions, and the embodiment of the present application is not limited herein.
Further, when all the conditions in the storage conditions are satisfied, the first quotation data in the memory queue is written into the storage medium in batch, so that the times and frequency of writing the quotation data into the storage medium are further reduced, the consumption of hardware resources is further reduced, and the occupancy rates of a CPU and a memory are reduced.
The storage area of the storage medium can be divided according to different types of the first market data, so that various types of data can be stored in a classified mode, management of the market data is facilitated, and data management efficiency is improved. For example, the first line condition data includes first line condition snapshot data and first disk port data, the memory queue includes a line condition snapshot memory queue and a disk port memory queue, the storage medium is divided into a first storage area and a second storage area, the first line condition snapshot data in the line condition snapshot memory queue is stored in the first storage area, the first disk port data in the disk port memory queue is stored in the second storage area, a space of the first storage area and a space of the second storage area may be equally divided, or may be divided according to a data amount of the first line condition snapshot data and the first disk port data, which is not limited herein. Storage media include, but are not limited to, semiconductor memory, magnetic surface memory, and the like.
As can be seen from the above technical solutions provided in the embodiments of the present application, when first travel information is received, the first travel information is buffered in a memory queue, and when any one of the storage conditions is triggered, the first travel information in the memory queue is written in a storage medium in batch, where the first travel information buffered in the memory queue includes data of at least one travel information source at different times. Therefore, the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments, and the first quotation data cached in the memory queue is written into the storage medium in batch until any one of the storage conditions is triggered, so that the writing frequency and frequency of the quotation data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupancy rate of a CPU is reduced.
For example, as shown in fig. 2, an execution subject of the data storage method provided in the embodiment of the present application may be a terminal device, that is, the data storage method provided in the embodiment of the present application may be implemented by hardware or software installed on the terminal device. The data storage method may specifically include the steps of:
in step S201, when the first behavior data is acquired, the first behavior data is cached in the memory queue.
In step S202, under the condition that the second market snapshot data is obtained, if the timestamp of the second market snapshot data is greater than the timestamp of the first market snapshot data and the volume of traffic in the second market snapshot data is greater than the volume of traffic in the first market snapshot data, the first market snapshot data in the memory queue is updated to the second market snapshot data, and the second market snapshot data and the first market snapshot data correspond to the same market source.
Specifically, due to the reason of the network and the reason of the non-order-preserving of the message queue, the real-time order of the market data may be out of order, and some market data are generated earlier but arrive at the service later, so that the market snapshot data in the memory queue needs to be updated, thereby ensuring that the market snapshot data stored in the memory queue is new data before the storage condition is triggered.
Further, the first quotation snapshot data and the second quotation snapshot data both include a highest bargain price, a lowest bargain price, an opening price, a latest price and a volume of bargain, and the second quotation snapshot data may include at least one quotation snapshot data, and for the same quotation source (like a stock), if the second quotation snapshot data is new data compared with the quotation snapshot data in the memory queue, the quotation snapshot data in the memory queue needs to be updated.
In a possible implementation manner, the updating the first quotation snapshot data in the memory queue to the second quotation snapshot data includes, if a plurality of timestamps are greater than the timestamp of the first quotation snapshot data and a plurality of second quotation snapshot data whose volume of traffic is greater than the volume of traffic in the first quotation snapshot data exist in the second quotation snapshot data: and updating the first quotation snapshot data in the memory queue into the quotation snapshot data with the maximum timestamp in the plurality of second quotation snapshot data. Or updating the first quotation data in the memory queue into the quotation snapshot data with the largest volume of traffic in the plurality of second quotation snapshot data. Or updating the first quotation data in the memory queue into the quotation snapshot data with the largest transaction amount and the largest timestamp in the plurality of second quotation snapshot data.
Specifically, a timestamp of any one piece of acquired quotation snapshot data in the second quotation snapshot data is compared with a timestamp of the quotation snapshot data in the memory queue, and if the quotation snapshot data with the timestamp larger than the timestamp of the quotation snapshot data in the memory queue exists in the second quotation snapshot data and the quotation snapshot data with the volume larger than the volume in the first quotation snapshot data exists in the second quotation snapshot data, the first quotation snapshot data in the memory queue is updated to the quotation snapshot data with the largest timestamp in the second quotation snapshot data.
For example, as shown in fig. 3, the second market condition snapshot data includes three messages, each of which represents one second market condition snapshot data, the larger the message number is, the larger the timestamp of the second market condition snapshot data is, and of the three messages, the timestamp and the volume of the second market condition snapshot data with the message number of 3 are the largest, the three messages are determined as new data, and if the timestamp of the first market condition snapshot data in the memory queue does not exceed the timestamp of the second market condition snapshot data with the message number of 3 and the volume of.
In step S203, in the case of triggering the storage condition, writing the second market snapshot data in the memory queue into the storage medium in batch.
It should be noted that step S201 and step S203 have the same or similar implementation manner as step S101 and step S103, which can be referred to each other, and the embodiments of the present application are not described herein again.
As can be seen from the above technical solutions provided in the embodiments of the present application, the first line situation data cached in the memory queue includes data of at least one market situation source at different times, and the first line situation data cached in the memory queue is written into the storage medium in batch until any one of the storage conditions is triggered, so that the number of times and frequency of writing the market situation data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupancy rate of the CPU is reduced. In addition, the first quotation data in the memory queue is updated, so that the quotation data written into the storage medium is ensured to be new data, the data obsolescence and redundancy are avoided, and the occupancy rate of the storage space of the storage medium is reduced.
For example, as shown in fig. 4, an execution subject of the data storage method provided in the embodiment of the present application may be a terminal device, that is, the data storage method provided in the embodiment of the present application may be implemented by hardware or software installed on the terminal device. The data storage method may specifically include the steps of:
in step S401, when the first behavior data is acquired, the first behavior data is cached in the memory queue.
Specifically, the first behavior data includes first intersection data including a message number (indicating the size of a timestamp), a type of a deal (e.g., buy and sell), a price of the deal (e.g., buy and sell prices), a volume of the listing, and a gear of a behavior source (e.g., stock), which are carried.
In step S402, under the condition that the second disk port data is received, if the timestamp of the second disk port data is greater than the timestamp of the first disk port data, the first disk port data in the memory queue is updated to the second disk port data, where the first disk port data and the second disk port data correspond to the same market source.
Specifically, the second data includes the carried message sequence number (representing the size of the timestamp), the type of transaction (e.g., buy and sell), the price of the transaction (e.g., buy price and sell price), and the rank of the market source (e.g., stock), etc. The corresponding message serial numbers (timestamps) are respectively arranged for different purchase prices and sale prices, and the larger the message serial number is, the larger the timestamp is. And aiming at the same stock, if the same transaction type contains a plurality of data, selecting the data with the largest timestamp in the same transaction type to update the first disk data in the memory queue.
For example, as shown in fig. 5, data with message numbers 1 to 4 are acquired through the message queue as second socket data, where the transaction type represented by the message number 1 and the message number 4 is buy, the transaction type represented by the message number 2 and the message number 3 is sell, data corresponding to the message number 4 is selected from the message number 1 and the message number 4 as buy data with the largest timestamp, data corresponding to the message number 3 is selected from the message number 2 and the message number 3 as sell data with the largest timestamp, and if the message number 4 (timestamp) is greater than the timestamp of the corresponding buy data in the memory queue, the buy data in the memory queue is updated with the data corresponding to the message number 4. And if the message sequence number 3 (timestamp) is greater than the timestamp of the corresponding sold data in the memory queue, updating the sold data in the memory queue by using the data corresponding to the message sequence number 3.
In step S403, when the storage condition is triggered, writing the second disk port data in the memory queue to the storage medium in batch.
It should be noted that step S401 and step S403 have the same or similar implementation manner as step S101 and step S103, and are not described herein again in this embodiment of the application.
As can be seen from the above technical solutions provided in the embodiments of the present application, the first line situation data cached in the memory queue includes data of at least one market situation source at different times, and the first line situation data cached in the memory queue is written into the storage medium in batch until any one of the storage conditions is triggered, so that the number of times and frequency of writing the market situation data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupancy rate of the CPU is reduced. In addition, the first quotation data in the memory queue is updated, so that the quotation data written into the storage medium is ensured to be new data, the data obsolescence and redundancy are avoided, and the occupancy rate of the storage space of the storage medium is reduced.
Corresponding to the data storage method provided by the foregoing embodiment, based on the same technical concept, an embodiment of the present application further provides a data storage device, fig. 6 is a schematic diagram of module composition of the data storage device provided by the embodiment of the present application, where the data storage device is configured to execute the data storage method described in fig. 1 to fig. 5, and as shown in fig. 6, the data storage device includes: the cache module 601 and the write-in module 602, where the cache module 601 is configured to cache the first behavior data in the memory queue when the first behavior data is acquired; the writing module 602 is configured to write the first quotation data in the memory queue to the storage medium in batch under the condition that the storage condition is triggered, where the first quotation data cached in the memory queue includes data of at least one quotation source at different times.
As can be seen from the above technical solutions provided in the embodiments of the present application, when first travel information is received, the first travel information is buffered in a memory queue, and when any one of the storage conditions is triggered, the first travel information in the memory queue is written in a storage medium in batch, where the first travel information buffered in the memory queue includes data of at least one travel information source at different times. Therefore, the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments, and the first quotation data cached in the memory queue is written into the storage medium in batch until any one of the storage conditions is triggered, so that the writing frequency and frequency of the quotation data into the storage medium are reduced, the consumption of hardware resources is reduced, and the occupancy rates of a CPU and a memory are reduced.
In a possible implementation manner, the first behavior data includes first behavior snapshot data and first disk port data, and the memory queue includes: the cache module 601 is further configured to cache the first lineup snapshot data to the lineup snapshot memory queue, and cache the first disk port data to the disk port memory queue.
In a possible implementation manner, the storage medium includes a first storage area and a second storage area, and the writing module 602 is further configured to store the first quotation snapshot data in the quotation snapshot memory queue to the first storage area, and store the first disk port data in the disk port memory queue to the second storage area.
In one possible implementation manner, the first behavior data includes first behavior snapshot data, and further includes: and the first updating module is used for updating the first quotation snapshot data in the memory queue into the second quotation snapshot data if the timestamp of the second quotation snapshot data is greater than the timestamp of the first quotation snapshot data and the volume of volume in the second quotation snapshot data is greater than the volume of volume in the first quotation snapshot data under the condition of acquiring the second quotation snapshot data, wherein the second quotation snapshot data and the first quotation snapshot data correspond to the same quotation source. The writing module 602 is further configured to write the second market snapshot data in the memory queue into the storage medium in batch under the condition that the storage condition is triggered.
In a possible implementation manner, the second market condition snapshot data are multiple, and if multiple timestamps greater than the timestamp of the first market condition snapshot data exist in the second market condition snapshot data and multiple second market condition snapshot data with a volume greater than the volume in the first market condition snapshot data exist in the second market condition snapshot data, the first updating module is further configured to update the first market condition snapshot data in the memory queue to the market condition snapshot data with the largest timestamp in the multiple second market condition snapshot data.
In one possible implementation manner, the first behavior data includes first portal data, and further includes: and the second updating module is used for updating the first disk port data in the memory queue into the second disk port data if the timestamp of the second disk port data is greater than the timestamp of the first disk port data under the condition of receiving the second disk port data, wherein the first disk port data and the second disk port data correspond to the same market situation source.
In a possible implementation manner, the writing module 602 is further configured to write the second data in the memory queue to the storage medium in a batch manner if the storage condition is triggered.
In a possible implementation manner, the storage condition includes that a data length of the first line condition data in the memory queue reaches a preset value, and/or the current cache time period ends.
The data storage device provided in the embodiment of the present application can implement each process in the embodiment corresponding to the data storage method, and is not described here again to avoid repetition.
It should be noted that the data storage device provided in the embodiment of the present application and the data storage method provided in the embodiment of the present application are based on the same application concept, and therefore specific implementation of the embodiment may refer to implementation of the data storage method, and repeated details are not repeated.
Based on the same technical concept, the embodiment of the present application further provides an electronic device, which is configured to execute the data storage method, and fig. 7 is a schematic structural diagram of an electronic device implementing the embodiments of the present application, as shown in fig. 7. Electronic devices may vary widely in configuration or performance and may include one or more processors 701 and memory 702, where one or more stored applications or data may be stored in memory 702. Memory 702 may be, among other things, transient storage or persistent storage. The application program stored in memory 702 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for the electronic device.
Still further, the processor 701 may be configured to communicate with the memory 702 to execute a series of computer-executable instructions in the memory 702 on the electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input-output interfaces 705, one or more keyboards 706.
Specifically, in this embodiment, the electronic device includes a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; a memory for storing a computer program; and the processor is used for executing the program stored in the memory and realizing the steps in the method embodiment.
The embodiment also provides a computer readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps in the above method embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, an electronic device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media (transmyedia) such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A data storage method, characterized in that the data storage method comprises:
under the condition of acquiring first behavior data, caching the first behavior data into a memory queue;
and under the condition of triggering a storage condition, writing the first quotation data in the memory queue into a storage medium in batch, wherein the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments.
2. The data storage method according to claim 1, wherein the first behavior data includes first behavior snapshot data and first disk port data, and the memory queue includes: the caching of the first behavior data into the memory queue comprises:
caching the first quotation snapshot data to the quotation snapshot memory queue, and caching the first disk port data to the disk port memory queue.
3. The data storage method according to claim 2, wherein the storage medium comprises a first storage area and a second storage area, and the writing the first pieces of behavior data in the memory queue to the storage medium in bulk comprises:
and storing the first quotation snapshot data in the quotation snapshot memory queue to the first storage area, and storing the first disk port data in the disk port memory queue to the second storage area.
4. The data storage method according to claim 1, wherein the first behavior data comprises first behavior snapshot data, and after the caching of the first behavior data in a memory queue, the method further comprises:
under the condition of acquiring second quotation snapshot data, if the timestamp of the second quotation snapshot data is greater than the timestamp of the first quotation snapshot data and the volume of transaction in the second quotation snapshot data is greater than the volume of transaction in the first quotation snapshot data, updating the first quotation snapshot data in the memory queue into the second quotation snapshot data, wherein the second quotation snapshot data and the first quotation snapshot data correspond to the same quotation source;
under the condition that the storage condition is triggered, writing the first behavior data in the memory queue into the storage medium in batches comprises the following steps:
and under the condition that the storage condition is triggered, writing the second market condition snapshot data in the memory queue into a storage medium in batch.
5. The data storage method according to claim 4, wherein the second market condition snapshot data is a plurality of second market condition snapshot data, and if a plurality of timestamps greater than the timestamp of the first market condition snapshot data exist in the second market condition snapshot data and a plurality of second market condition snapshot data whose volume of traffic is greater than the volume of traffic in the first market condition snapshot data exist in the second market condition snapshot data, updating the first market condition snapshot data in the memory queue to the second market condition snapshot data comprises:
and updating the first quotation snapshot data in the memory queue into second quotation snapshot data with the largest timestamp in the plurality of second quotation snapshot data.
6. The data storage method of any one of claims 1 to 5, wherein the first transaction data comprises a first port data, and after the buffering the first transaction data into a memory queue, the method further comprises:
under the condition that second disk port data are received, if the time stamp of the second disk port data is larger than that of the first disk port data, updating the first disk port data in the memory queue into the second disk port data, wherein the first disk port data and the second disk port data correspond to the same market situation source;
under the condition that the storage condition is triggered, writing the first behavior data in the memory queue into the storage medium in batches comprises the following steps:
and writing the second disk port data in the memory queue into a storage medium in batch under the condition of triggering the storage condition.
7. The data storage method according to claim 1, wherein the storage condition comprises that the data length of the first line condition data in the memory queue reaches a preset value, and/or the current buffer time period ends.
8. A data storage device, the device comprising:
the cache module is used for caching the first behavior data to a memory queue under the condition of acquiring the first behavior data;
and the writing module is used for writing the first quotation data in the memory queue into a storage medium in batch under the condition of triggering the storage condition, wherein the first quotation data cached in the memory queue comprises data of at least one quotation source at different moments.
9. The data storage device of claim 8, wherein the first behavior data comprises first behavior snapshot data and first disk port data, and the memory queue comprises: the cache module is further configured to cache the first quotation snapshot data to the quotation snapshot memory queue, and cache the first disk port data to the disk port memory queue.
10. An electronic device comprising a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a communication bus; the memory is used for storing a computer program; the processor, executing the program stored in the memory, implementing the steps of the data storage method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210138857.4A CN114546270B (en) | 2022-02-15 | 2022-02-15 | Data storage method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210138857.4A CN114546270B (en) | 2022-02-15 | 2022-02-15 | Data storage method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114546270A true CN114546270A (en) | 2022-05-27 |
| CN114546270B CN114546270B (en) | 2024-02-09 |
Family
ID=81675930
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210138857.4A Active CN114546270B (en) | 2022-02-15 | 2022-02-15 | Data storage method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114546270B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102611454A (en) * | 2012-01-29 | 2012-07-25 | 上海锅炉厂有限公司 | Dynamic lossless compressing method for real-time historical data |
| US20130346701A1 (en) * | 2012-06-21 | 2013-12-26 | Huawei Technologies Co., Ltd. | Replacement method and apparatus for cache |
| CN108664520A (en) * | 2017-04-01 | 2018-10-16 | 北京京东尚科信息技术有限公司 | Safeguard method, apparatus, electronic equipment and the readable storage medium storing program for executing of data consistency |
| CN109379305A (en) * | 2018-11-28 | 2019-02-22 | 上海银赛计算机科技有限公司 | A kind of data distributing method, device, server and storage medium |
| CN109617950A (en) * | 2018-11-26 | 2019-04-12 | 杭州涂鸦信息技术有限公司 | A kind of data-updating method, device, equipment and readable storage medium storing program for executing |
| CN109710182A (en) * | 2018-12-14 | 2019-05-03 | 北京向上一心科技有限公司 | Market data processing method, device and controller and medium |
| CN112000675A (en) * | 2019-05-27 | 2020-11-27 | 招商证券股份有限公司 | Market data updating method and device, terminal equipment and storage medium |
| CN112486888A (en) * | 2020-12-11 | 2021-03-12 | 盛立金融软件开发（杭州）有限公司 | Market data transmission method, device, equipment and medium |
| CN112799595A (en) * | 2021-02-02 | 2021-05-14 | 联想(北京)有限公司 | Data processing method, device and storage medium |
| CN113204580A (en) * | 2021-05-12 | 2021-08-03 | 中华人民共和国广东海事局 | Overwater target position information query method, system, device, equipment and medium |
| CN113434516A (en) * | 2021-08-03 | 2021-09-24 | 深圳华锐金融技术股份有限公司 | Market data storage method and device, computer equipment and readable storage medium |
| WO2021189283A1 (en) * | 2020-03-25 | 2021-09-30 | 深圳市欢太科技有限公司 | Method and device for data processing, electronic device, and storage medium |
-
2022
- 2022-02-15 CN CN202210138857.4A patent/CN114546270B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102611454A (en) * | 2012-01-29 | 2012-07-25 | 上海锅炉厂有限公司 | Dynamic lossless compressing method for real-time historical data |
| US20130346701A1 (en) * | 2012-06-21 | 2013-12-26 | Huawei Technologies Co., Ltd. | Replacement method and apparatus for cache |
| CN108664520A (en) * | 2017-04-01 | 2018-10-16 | 北京京东尚科信息技术有限公司 | Safeguard method, apparatus, electronic equipment and the readable storage medium storing program for executing of data consistency |
| CN109617950A (en) * | 2018-11-26 | 2019-04-12 | 杭州涂鸦信息技术有限公司 | A kind of data-updating method, device, equipment and readable storage medium storing program for executing |
| CN109379305A (en) * | 2018-11-28 | 2019-02-22 | 上海银赛计算机科技有限公司 | A kind of data distributing method, device, server and storage medium |
| CN109710182A (en) * | 2018-12-14 | 2019-05-03 | 北京向上一心科技有限公司 | Market data processing method, device and controller and medium |
| CN112000675A (en) * | 2019-05-27 | 2020-11-27 | 招商证券股份有限公司 | Market data updating method and device, terminal equipment and storage medium |
| WO2021189283A1 (en) * | 2020-03-25 | 2021-09-30 | 深圳市欢太科技有限公司 | Method and device for data processing, electronic device, and storage medium |
| CN112486888A (en) * | 2020-12-11 | 2021-03-12 | 盛立金融软件开发（杭州）有限公司 | Market data transmission method, device, equipment and medium |
| CN112799595A (en) * | 2021-02-02 | 2021-05-14 | 联想(北京)有限公司 | Data processing method, device and storage medium |
| CN113204580A (en) * | 2021-05-12 | 2021-08-03 | 中华人民共和国广东海事局 | Overwater target position information query method, system, device, equipment and medium |
| CN113434516A (en) * | 2021-08-03 | 2021-09-24 | 深圳华锐金融技术股份有限公司 | Market data storage method and device, computer equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114546270B (en) | 2024-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108055343B (en) | Data synchronization method and device for computer room | |
| WO2020082871A1 (en) | Method, device and system for executing blockchain transactions in parallel | |
| CN111209352B (en) | Data processing method and device, electronic equipment and storage medium | |
| US9128966B2 (en) | Determining a storage location based on frequency of use | |
| CN105684020B (en) | Order book management equipment in hardware platform | |
| CN110119304B (en) | Interrupt processing method and device and server | |
| EP4141641A2 (en) | Method and apparatus for writing data in append mode, device and storage medium | |
| CN111506580B (en) | Transaction storage method based on centralized block chain type account book | |
| CN111459948B (en) | Transaction integrity verification method based on centralized block chain type account book | |
| CN113946552A (en) | Data processing method and electronic device | |
| CN108062235A (en) | Data processing method and device | |
| CN109345221A (en) | The checking method and device of resource circulation | |
| CN112597151A (en) | Data processing method, device, equipment and storage medium | |
| CN103634350A (en) | Storage system and implementation method thereof | |
| CN111913807A (en) | Event processing method, system and device based on multiple storage areas | |
| CN107018158B (en) | Method and device for screening internet resources | |
| CN105264608A (en) | Data storage method, memory controller and central processing unit | |
| CN114546270A (en) | Data storage method and device and electronic equipment | |
| CN116737084A (en) | Queue statistics method and device, electronic equipment and storage medium | |
| US10901972B2 (en) | Table partition configuration method, apparatus and system for database system | |
| CN114785662B (en) | Storage management method, device, equipment and machine-readable storage medium | |
| CN114116676A (en) | Data migration method and device, electronic equipment and computer readable storage medium | |
| CN114513469A (en) | Traffic shaping method and device for distributed system and storage medium | |
| CN110874268B (en) | Data processing method, device and equipment | |
| CN112269721A (en) | Method, system, equipment and readable storage medium for performance data statistics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |