CN114625805A - Method, device, equipment and medium for configuration of return test - Google Patents
Method, device, equipment and medium for configuration of return test Download PDFInfo
- Publication number
- CN114625805A CN114625805A CN202210525772.1A CN202210525772A CN114625805A CN 114625805 A CN114625805 A CN 114625805A CN 202210525772 A CN202210525772 A CN 202210525772A CN 114625805 A CN114625805 A CN 114625805A
- Authority
- CN
- China
- Prior art keywords
- data information
- cache
- original market
- sstable
- market data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000012360 testing method Methods 0.000 title description 7
- 238000013102 re-test Methods 0.000 claims abstract description 44
- 238000012163 sequencing technique Methods 0.000 claims abstract description 20
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 17
- 101100217298 Mus musculus Aspm gene Proteins 0.000 claims abstract description 12
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 9
- 230000001360 synchronised effect Effects 0.000 claims description 5
- 239000002699 waste material Substances 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Abstract
The application discloses a retest configuration method, a retest configuration device and a retest configuration medium, wherein the retest configuration method comprises the following steps: acquiring original market quotation data information, classifying the original market quotation data information, sequencing each kind of original market quotation data information, and storing each kind of sequenced original market quotation data information to each level of cache in a preset database; based on the strategy identification and the preset retest parameter which are obtained in advance, determining each hash value by utilizing an SHA1 algorithm, and then respectively sending and storing each sort of original market data information and each hash value which are sequenced in each primary cache to each secondary cache; and (3) respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in a preset database to obtain the original market data information. Through the technical scheme, the time for data retest can be effectively shortened, the efficiency of data retest is improved, and the waste of server resources is reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a medium for configuration of a retest.
Background
At present, in the prior art, the number of issued quotations of the quotation that the TPS system can process per second is huge. It is an important index for measuring the processing capacity of the system. The program needs to process the trade mark of 50000TPS to 100000TPS per second, a considerable amount of simulated trade orders are needed, the conventional test program cannot achieve the high-performance processing capability, the whole test time is long, the processing delay occurs, and even the system is crashed. In order to solve the problem of Processing large data volume by backtesting, a method for enhancing the performance of hardware devices is usually adopted, such as improving the performance of a memory and a Central Processing Unit (CPU). To solve the issue, the data of the market is usually placed in the memory of the test program to prevent the data from being delayed due to the I/O operation when the data is read. With the increase of the retest years and the asset targets, the existing method for enhancing the performance of hardware equipment causes the investment of the hardware equipment to be increased continuously by a user, so that the cost burden is increased; in addition, the conventional data storage framework also causes problems of slow reading speed and short server resources due to frequent reading and writing of data of large data blocks, even causes the memory of the operating system to be used up under severe conditions, and brings unpredictable loss to mechanisms.
Therefore, how to reduce the time for data retest, improve the efficiency of data retest, and reduce the waste of server resources in the retest configuration process is a problem to be solved in the field.
Disclosure of Invention
In view of this, the present invention provides a method, an apparatus, a device, and a medium for configuration of data recovery, which can effectively reduce the time for data recovery, improve the efficiency of data recovery, and reduce the waste of server resources. The specific scheme is as follows:
in a first aspect, the present application discloses a backtesting configuration method, including:
acquiring original market quotation data information, classifying the original market quotation data information, sequencing each kind of original market quotation data information, and storing each kind of sequenced original market quotation data information to each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache;
determining each hash value by utilizing an SHA1 algorithm based on a pre-obtained strategy identifier and a preset retest parameter, and then respectively sending and storing each kind of sequenced original market data information and each hash value in each primary cache to each secondary cache;
and (3) respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database to obtain the original market data information.
Optionally, the acquiring the original market data information includes:
creating a Rocksdb database based on an LRU algorithm; the Rocksdb database comprises a first-level cache, a second-level cache and a third-level cache;
and synchronizing the remote original market data information by using a preset information acquisition rule so as to acquire the synchronized original market data information.
Optionally, the sorting each kind of original market data information respectively includes:
establishing a custom iterator based on the Rocksdb database, and determining each timestamp by using the original market data information;
and determining a time axis based on each timestamp and the user-defined iterator, and sequencing each type of original market data information according to the time axis.
Optionally, the determining a time axis based on each timestamp and the custom iterator, and sorting each original market data information according to the time axis respectively includes:
taking each timestamp as a key of the custom iterator, and determining a preset sequencing method based on a character string mode exchange standard code of the custom iterator key;
and sequencing each type of original market data information by the preset sequencing method and utilizing the time axis.
Optionally, after the reading of each SSTable model in the second-level cache is sequentially performed by using the third-level cache in the preset database, the method further includes:
if the SSTable model corresponding to the original market data information does not exist in the secondary cache in the reading process, the original market data information in the primary cache is obtained by using the secondary cache, the original market data information is landed to each SSTable model, and then the SSTable models are read in a segmented mode based on the time axis and a preset sorting method.
Optionally, after the step of respectively dropping each kind of original market data information in each secondary cache to each SSTable model, the step of further includes:
and respectively adding a lock identifier for each SSTable model after the disk is dropped so as to read the next segment of the SSTable model after the SSTable model is read and meets the lock identifier.
Optionally, after obtaining the original market data information, the method further includes:
sending the original market data information in the third-level cache to a preset first-in first-out queue to obtain market information;
and establishing a strategy list by using a preset list determining method, and sending the quotation information to the strategy list so as to determine a strategy method corresponding to the quotation information.
In a second aspect, the present application discloses a backtesting configuration apparatus, comprising:
the system comprises an information acquisition module, a data processing module and a data processing module, wherein the information acquisition module is used for acquiring original market quotation data information, classifying the original market quotation data information, sequencing each type of original market quotation data information respectively, and storing each type of sequenced original market quotation data information into each level of cache in a preset database respectively; the preset database comprises a first-level cache, a second-level cache and a third-level cache;
a hash value determining module, configured to determine, based on a policy identifier obtained in advance and a preset retest parameter, each hash value by using an SHA1 algorithm, and then send and store each piece of original market data information and each hash value sorted in each primary cache to each secondary cache respectively;
and the model reading module is used for respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database so as to obtain the original market data information.
In a third aspect, the present application discloses an electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the foregoing back test configuration method.
In a fourth aspect, the present application discloses a computer storage medium for storing a computer program; wherein the computer program realizes the steps of the previously disclosed backtesting configuration method when executed by a processor.
The method comprises the steps of obtaining original market data information, classifying the original market data information, sequencing each kind of original market data information, and storing each kind of sequenced original market data information to each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache; determining each hash value by utilizing an SHA1 algorithm based on a pre-obtained strategy identifier and a preset retest parameter, and then respectively sending and storing each kind of sequenced original market data information and each hash value in each primary cache to each secondary cache; and (3) respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database to obtain the original market data information. According to the method and the device, hash function calculation is carried out on the database, the data are sequenced and dropped to each SSTable model, the SSTable models are read to obtain original market data information, the time of data retest can be effectively shortened, the efficiency of data retest is improved, and the waste of server resources is further reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a backtesting configuration method disclosed herein;
FIG. 2 is a flow chart of a backtesting configuration method disclosed herein;
FIG. 3 is a detailed topology diagram of a backtesting configuration method disclosed in the present application;
FIG. 4 is a detailed flowchart of a retest configuration method disclosed herein;
FIG. 5 is a schematic diagram of a configuration device for back test according to the present disclosure;
fig. 6 is a block diagram of an electronic device provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the present context, since in the prior art, the data implementation of the market is usually placed in the memory in the test program to prevent the data from being delayed due to the I/O operation when the data is read. With the increase of the retest years and the asset targets, the existing method for enhancing the performance of hardware equipment causes the investment of hardware equipment to be increased by a user, so that the cost burden is increased; in addition, the conventional data storage framework also causes problems of slow reading speed and short server resources due to frequent reading and writing of data of large data blocks, even causes the memory of the operating system to be used up under severe conditions, and brings unpredictable loss to mechanisms. Therefore, how to reduce the time for data retest, improve the efficiency of data retest, and reduce the waste of server resources in the retest configuration process is a problem to be solved in the field.
Referring to fig. 1, an embodiment of the present invention discloses a retest configuration method, which may specifically include:
step S11: acquiring original market quotation data information, classifying the original market quotation data information, sequencing each kind of original market quotation data information, and storing each kind of sequenced original market quotation data information to each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache.
In the embodiment, a Rocksdb database is created based on an LRU algorithm; the Rocksdb database comprises a first-level cache, a second-level cache and a third-level cache, and then remote original market data information is synchronized by using a preset information acquisition rule so as to acquire the synchronized original market data information and perform classification operation on the original market data information, wherein the classification operation is performed according to attributes such as contracts and quarterly time periods.
After the original quotation data information is classified, a custom iterator is built based on the Rocksdb database, each timestamp is determined by the original quotation data information, then each timestamp is used as a key of the custom iterator, a preset sorting method is determined based on a character string mode exchange standard code of the custom iterator key, each kind of original quotation data information is sorted by the preset sorting method and the time axis, each kind of original quotation data information is sorted according to the time axis, and each kind of original quotation data information after sorting is stored in each level of cache in a preset database.
Step S12: determining each hash value by using an SHA1 algorithm based on a policy identifier and a preset retest parameter which are obtained in advance, and then respectively sending and storing each kind of original market data information and each hash value which are sequenced in each primary cache to each secondary cache.
Step S13: and (3) respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database to obtain the original market data information.
In this embodiment, each SSTable model in the secondary cache is sequentially read by using the tertiary cache in the preset database, and if the SSTable model corresponding to the original market data information does not exist in the secondary cache in the reading process, the original market data information in the primary cache is obtained by using the secondary cache, and is landed to each SSTable model, and then each SSTable model is read in a segmented manner based on the time axis and the preset sorting method.
In this embodiment, after original market data information is obtained, the original market data information in the third-level cache is sent to a preset first-in first-out queue to obtain market information, a policy list is established by using a preset list determining method, and the market information is sent to the policy list to determine a policy method corresponding to the market information.
In the embodiment, original market data information is obtained, the original market data information is classified, each type of original market data information is sorted respectively, and each type of original market data information after being sorted is stored in each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache; determining each hash value by utilizing an SHA1 algorithm based on a pre-obtained strategy identifier and a preset retest parameter, and then respectively sending and storing each kind of sequenced original market data information and each hash value in each primary cache to each secondary cache; and respectively dropping each kind of original market data information in each second-level cache to each SSTable model, and sequentially reading each SSTable model in the second-level cache by using a third-level cache in the preset database to obtain the original market data information. According to the method and the device, hash function calculation is carried out on the database, the data are sequenced and dropped to each SSTable model, the SSTable models are read to obtain original market data information, the time of data retest can be effectively shortened, the efficiency of data retest is improved, and the waste of server resources is further reduced.
Referring to fig. 2, an embodiment of the present invention discloses a retest configuration method, which may specifically include:
step S21: acquiring original market quotation data information, classifying the original market quotation data information, sequencing each kind of original market quotation data information, and storing each kind of sequenced original market quotation data information to each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache.
Step S22: based on the strategy identification and the preset retest parameter which are obtained in advance, determining each hash value by utilizing an SHA1 algorithm, and then respectively sending and storing each sort of original market data information and each hash value which are sequenced in each primary cache to each secondary cache.
Step S23: and respectively landing each original market data information in each secondary cache to each SSTable model, and adding lock identification for each SSTable model after landing.
In this embodiment, after each piece of original market data information in each secondary cache is respectively landed to each SSTable model, a lock identifier is respectively added to each SSTable model after landing, so that after a lock flag is encountered during reading of the SSTable model, the SSTable model is read for a next segment.
Step S24: and sequentially reading each SSTable model in the second-level cache by utilizing the third-level cache in the preset database so as to obtain original market data information.
In the embodiment, original market data information is obtained, the original market data information is classified, each type of original market data information is sorted respectively, and each type of original market data information after being sorted is stored in each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache; determining each hash value by utilizing an SHA1 algorithm based on a pre-obtained strategy identifier and a preset retest parameter, and then respectively sending and storing each kind of sequenced original market data information and each hash value in each primary cache to each secondary cache; respectively landing each original market data information in each secondary cache to each SSTable model, and adding a lock identifier for each SSTable model after landing; and sequentially reading each SSTable model in the second-level cache by utilizing the third-level cache in the preset database so as to obtain original market data information. According to the method and the device, hash function calculation is carried out on the database, the data are sequenced and dropped to each SSTable model, the SSTable models are read to obtain original market data information, the time of data retest can be effectively shortened, the efficiency of data retest is improved, and the waste of server resources is further reduced.
For example, as shown in fig. 3, if the SSTable model corresponding to the original market data information does not exist in the secondary cache during reading, the secondary cache is used to obtain the original market data information in the primary cache, and the original market data information is landed to each SSTable model, then each SSTable model is read in segments based on the time axis and the preset sorting method, after the SSTable model of the slot 1 is read and meets the lock flag, the SSTable model of the slot 2 is read, and after the SSTable model of the slot 3 is read and meets the lock flag, the SSTable model of the slot 2 is read.
For example, as shown in fig. 4, a policy list is established through a manual method, where the policy list includes a plurality of policy functions, for example, an interface function for monitoring data acquisition and an interface function for monitoring receipt, and the simulated transaction information issued by an event publisher and corresponding data issue information are sent to an event production processor, then are pressed into a First Input First Output (FIFO) queue, and are sent to the policy list through an event consumption processor, so as to obtain a policy method corresponding to the simulated transaction information issued by the event publisher.
Referring to fig. 5, an embodiment of the present invention discloses a retest configuration device, which may specifically include:
the information acquisition module 11 is configured to acquire original market data information, perform a classification operation on the original market data information, sort each type of original market data information, and store each type of original market data information after being sorted to each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache;
a hash value determining module 12, configured to determine, based on a policy identifier obtained in advance and a preset retest parameter, each hash value by using an SHA1 algorithm, and then send and store each piece of original market data information and each hash value sorted in each primary cache to each secondary cache respectively;
and the model reading module 13 is configured to separately drop each type of original market data information in each secondary cache to each SSTable model, and sequentially read each SSTable model in the secondary cache by using a third-level cache in the preset database to obtain the original market data information.
In the embodiment, original market data information is obtained, the original market data information is classified, each type of original market data information is sorted respectively, and each type of original market data information after being sorted is stored in each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache; determining each hash value by utilizing an SHA1 algorithm based on a pre-obtained strategy identifier and a preset retest parameter, and then respectively sending and storing each kind of sequenced original market data information and each hash value in each primary cache to each secondary cache; and (3) respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database to obtain the original market data information. According to the method and the device, hash function calculation is carried out on the database, the data are sequenced and dropped to each SSTable model, the SSTable models are read to obtain original market data information, the time of data retest can be effectively shortened, the efficiency of data retest is improved, and the waste of server resources is further reduced.
In some specific embodiments, the information obtaining module 11 may specifically include:
the database creating module is used for creating a Rocksdb database based on an LRU algorithm; the Rocksdb database comprises a first-level cache, a second-level cache and a third-level cache;
and the data information synchronization module is used for synchronizing the remote original market data information by using a preset information acquisition rule so as to acquire the synchronized original market data information.
In some specific embodiments, the information obtaining module 11 may specifically include:
the timestamp determining module is used for establishing a custom iterator based on the Rocksdb database and determining each timestamp by using the original market data information;
and the time axis determining module is used for determining a time axis based on each timestamp and the user-defined iterator and sequencing each original market data information according to the time axis.
In some specific embodiments, the information obtaining module 11 may specifically include:
the sorting method determining module is used for taking each timestamp as a key of the self-defined iterator and determining a preset sorting method based on a character string mode exchange standard code of the key of the self-defined iterator;
and the data information sequencing module is used for sequencing each type of original market data information respectively by the preset sequencing method and by utilizing the time shaft.
In some embodiments, the model reading module 13 may specifically include:
a data information destaging module, configured to, if an SSTable model corresponding to the original market data information does not exist in the secondary cache in the reading process, obtain the original market data information in the primary cache by using the secondary cache, and destage the original market data information to each SSTable model;
and the segmented reading module is used for reading the SSTable models in a segmented manner based on the time axis and a preset sorting method.
In some embodiments, the model reading module 13 may specifically include:
and the lock identifier adding module is used for respectively adding lock identifiers to the SSTable models after the disk is dropped so as to read the SSTable models for the next segment after the SSTable models are read and encounter lock marks.
In some embodiments, the model reading module 13 may specifically include:
the data information sending module is used for sending the original market data information in the third-level cache to a preset first-in first-out queue to obtain market information;
and the strategy method determining module is used for establishing a strategy list by using a preset list determining method and sending the quotation information to the strategy list so as to determine a strategy method corresponding to the quotation information.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The retest configuration device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. The memory 22 is used for storing a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the configuration method for configuration for loop back test executed by the configuration device disclosed in any of the foregoing embodiments.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the memory 22 is a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., the resources stored thereon may include an operating system 221, a computer program 222, data 223, etc., and the data 223 may include various data. The storage means may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device on the electronic device 20 and the computer program 222, and may be Windows Server, Netware, Unix, Linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the method for configuration of a backtest performed by the electronic device 20 disclosed in any of the foregoing embodiments.
Further, embodiments of the present application disclose a computer-readable storage medium, where the computer-readable storage medium includes a Random Access Memory (RAM), a Memory, a Read-Only Memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic disk, or an optical disk or any other form of storage medium known in the art. Wherein the computer program when executed by a processor implements the foregoing method of configuration for loop back testing. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The steps of a retest configuration or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing describes in detail a method, apparatus, device and medium for configuration of a retest provided by the present invention, and specific examples are applied herein to explain the principles and embodiments of the present invention, and the descriptions of the foregoing examples are only used to help understanding the method and its core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A backtesting configuration method, comprising:
acquiring original market quotation data information, classifying the original market quotation data information, sequencing each kind of original market quotation data information, and storing each kind of sequenced original market quotation data information to each level of cache in a preset database; the preset database comprises a first-level cache, a second-level cache and a third-level cache;
determining each hash value by utilizing an SHA1 algorithm based on a pre-obtained strategy identifier and a preset retest parameter, and then respectively sending and storing each kind of sequenced original market data information and each hash value in each primary cache to each secondary cache;
and (3) respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database to obtain the original market data information.
2. The method of claim 1, wherein the obtaining of the raw market data information comprises:
creating a Rocksdb database based on an LRU algorithm; the Rocksdb database comprises a first-level cache, a second-level cache and a third-level cache;
and synchronizing the remote original market data information by using a preset information acquisition rule so as to acquire the synchronized original market data information.
3. The backtesting configuration method according to claim 2, wherein the sorting each of the raw market data messages respectively comprises:
establishing a custom iterator based on the Rocksdb database, and determining each timestamp by using the original market data information;
and determining a time axis based on each timestamp and the user-defined iterator, and sequencing each type of original market data information according to the time axis.
4. The backtesting configuration method of claim 3, wherein determining a time axis based on each timestamp and the custom iterator, and sorting each original market data information according to the time axis comprises:
taking each timestamp as a key of the custom iterator, and determining a preset sequencing method based on a character string mode exchange standard code of the custom iterator key;
and sequencing each type of original market data information by the preset sequencing method and utilizing the time axis.
5. The retest configuration method according to claim 3, wherein after the step of sequentially reading each SSTable model in the secondary cache by using the tertiary cache in the preset database, the method further comprises:
if the SSTable model corresponding to the original market data information does not exist in the secondary cache in the reading process, the original market data information in the primary cache is obtained by using the secondary cache, the original market data information is landed to each SSTable model, and then the SSTable models are read in a segmented mode based on the time axis and a preset sorting method.
6. The retest configuration method according to claim 5, wherein said after dropping each of the original market data information in each of the secondary caches to each SSTable model respectively, further comprises:
and respectively adding a lock identifier for each SSTable model after the disk is dropped so as to read the next segment of the SSTable model after the SSTable model is read and meets the lock identifier.
7. The retrospective configuration method according to any one of claims 3 to 6, further comprising, after obtaining the raw market data information:
sending the original market data information in the third-level cache to a preset first-in first-out queue to obtain market information;
and establishing a strategy list by using a preset list determining method, and sending the market information to the strategy list so as to determine a strategy method corresponding to the market information.
8. A backtesting configuration apparatus, comprising:
the system comprises an information acquisition module, a data processing module and a data processing module, wherein the information acquisition module is used for acquiring original market quotation data information, classifying the original market quotation data information, sequencing each type of original market quotation data information respectively, and storing each type of sequenced original market quotation data information into each level of cache in a preset database respectively; the preset database comprises a first-level cache, a second-level cache and a third-level cache;
a hash value determining module, configured to determine, based on a policy identifier obtained in advance and a preset retest parameter, each hash value by using an SHA1 algorithm, and then send and store each piece of original market data information and each hash value sorted in each primary cache to each secondary cache respectively;
and the model reading module is used for respectively dropping each original market data information in each secondary cache to each SSTable model, and sequentially reading each SSTable model in the secondary cache by using a tertiary cache in the preset database so as to obtain the original market data information.
9. A backtesting configuration apparatus, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the method of configuration of the echo in any of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the backtesting configuration method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210525772.1A CN114625805B (en) | 2022-05-16 | 2022-05-16 | Return test configuration method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210525772.1A CN114625805B (en) | 2022-05-16 | 2022-05-16 | Return test configuration method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114625805A true CN114625805A (en) | 2022-06-14 |
| CN114625805B CN114625805B (en) | 2022-09-20 |
Family
ID=81907303
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210525772.1A Active CN114625805B (en) | 2022-05-16 | 2022-05-16 | Return test configuration method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114625805B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117745432A (en) * | 2024-02-19 | 2024-03-22 | 上海大智慧信息科技有限公司 | Quantitative back-testing system and method based on micro-service architecture |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070244788A1 (en) * | 2004-11-08 | 2007-10-18 | Crescent Technology Limited | Method of Storing Data Used in Backtesting a Computer Implemented Investment Trading Strategy |
| CN103218365A (en) * | 2012-01-20 | 2013-07-24 | 阿里巴巴集团控股有限公司 | SS Table file data processing method and system |
| CN107797797A (en) * | 2017-11-03 | 2018-03-13 | 上海宽全智能科技有限公司 | Quantify back to survey with quantifying method of commerce and device, storage medium, equipment and system |
| CN110689436A (en) * | 2019-09-30 | 2020-01-14 | 北京九章云极科技有限公司 | Strategy retest method and strategy retest system |
| CN111445319A (en) * | 2020-02-17 | 2020-07-24 | 深圳壹账通智能科技有限公司 | Voucher generation method and device, computer equipment and storage medium |
| CN111737125A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Method and device for generating market data of quantitative transaction and server |
| CN112087470A (en) * | 2020-09-27 | 2020-12-15 | 山东云海国创云计算装备产业创新中心有限公司 | Market data transmission method and related device |
| CN112862013A (en) * | 2021-03-31 | 2021-05-28 | 中国工商银行股份有限公司 | Problem diagnosis method and device for quantitative transaction strategy |
| US20210182247A1 (en) * | 2019-12-16 | 2021-06-17 | Chicago Mercantile Exchange Inc. | Validation of system functionality subsequent to modification using parallel regression testing |
| CN113268509A (en) * | 2021-05-28 | 2021-08-17 | 北京大米科技有限公司 | Layered cache updating method and device, storage medium and electronic equipment |
| WO2021238260A1 (en) * | 2020-05-29 | 2021-12-02 | 广东浪潮智慧计算技术有限公司 | Pre-read data caching method and apparatus, device, and storage medium |
| WO2022041812A1 (en) * | 2020-08-28 | 2022-03-03 | 平安科技（深圳）有限公司 | Data processing method, related device and computer-readable storage medium |
-
2022
- 2022-05-16 CN CN202210525772.1A patent/CN114625805B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070244788A1 (en) * | 2004-11-08 | 2007-10-18 | Crescent Technology Limited | Method of Storing Data Used in Backtesting a Computer Implemented Investment Trading Strategy |
| CN103218365A (en) * | 2012-01-20 | 2013-07-24 | 阿里巴巴集团控股有限公司 | SS Table file data processing method and system |
| CN107797797A (en) * | 2017-11-03 | 2018-03-13 | 上海宽全智能科技有限公司 | Quantify back to survey with quantifying method of commerce and device, storage medium, equipment and system |
| CN110689436A (en) * | 2019-09-30 | 2020-01-14 | 北京九章云极科技有限公司 | Strategy retest method and strategy retest system |
| US20210182247A1 (en) * | 2019-12-16 | 2021-06-17 | Chicago Mercantile Exchange Inc. | Validation of system functionality subsequent to modification using parallel regression testing |
| CN111445319A (en) * | 2020-02-17 | 2020-07-24 | 深圳壹账通智能科技有限公司 | Voucher generation method and device, computer equipment and storage medium |
| WO2021238260A1 (en) * | 2020-05-29 | 2021-12-02 | 广东浪潮智慧计算技术有限公司 | Pre-read data caching method and apparatus, device, and storage medium |
| CN111737125A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Method and device for generating market data of quantitative transaction and server |
| WO2022041812A1 (en) * | 2020-08-28 | 2022-03-03 | 平安科技（深圳）有限公司 | Data processing method, related device and computer-readable storage medium |
| CN112087470A (en) * | 2020-09-27 | 2020-12-15 | 山东云海国创云计算装备产业创新中心有限公司 | Market data transmission method and related device |
| CN112862013A (en) * | 2021-03-31 | 2021-05-28 | 中国工商银行股份有限公司 | Problem diagnosis method and device for quantitative transaction strategy |
| CN113268509A (en) * | 2021-05-28 | 2021-08-17 | 北京大米科技有限公司 | Layered cache updating method and device, storage medium and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 徐光林: "回测检验在商业银行市场风险度量中的应用研究", 《金融理论与实践》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117745432A (en) * | 2024-02-19 | 2024-03-22 | 上海大智慧信息科技有限公司 | Quantitative back-testing system and method based on micro-service architecture |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114625805B (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11645183B1 (en) | User interface for correlation of virtual machine information and storage information | |
| CN104731690B (en) | Adaptive metrology collection, storage and warning threshold | |
| US9043660B2 (en) | Data store capable of efficient storing of keys | |
| CN110058958B (en) | Method, apparatus and computer program product for managing data backup | |
| US20200401562A1 (en) | Parallel processing of filtered transaction logs | |
| CN111881011A (en) | Log management method, platform, server and storage medium | |
| WO2022252512A1 (en) | Root cause analysis method and apparatus, electronic device, medium, and program | |
| CN112989156A (en) | Big data based policy and enterprise matching method and system | |
| CN110704194A (en) | Method and system for managing memory data and maintaining data in memory | |
| US20200409566A1 (en) | Key Value Store Using Progress Verification | |
| US20230396633A1 (en) | Method and Apparatus for Detecting Security Event, and Computer-Readable Storage Medium | |
| CN114169401A (en) | Data processing and prediction model training method and device | |
| CN114625805B (en) | Return test configuration method, device, equipment and medium | |
| CN114817651B (en) | Data storage method, data query method, device and equipment | |
| US11080239B2 (en) | Key value store using generation markers | |
| CN109471860B (en) | Large-scale charging pile data processing method and device for electric vehicle charging network | |
| CN114518848B (en) | Method, device, equipment and medium for processing stored data | |
| CN111190790A (en) | Cloud computing cluster monitoring method and system based on peak prediction | |
| CN115269519A (en) | Log detection method and device and electronic equipment | |
| US20210117096A1 (en) | Method, device and computer program product for backuping data | |
| CN116628042A (en) | Data processing method, device, equipment and medium | |
| CN113626516A (en) | Data increment synchronization method and system | |
| CN111782588A (en) | File reading method, device, equipment and medium | |
| CN115328878B (en) | Log data storage system, method, device and medium | |
| CN110138723A (en) | The determination method and system of malice community in a kind of mail network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A backtesting configuration method, device, equipment, and medium Effective date of registration: 20231124 Granted publication date: 20220920 Pledgee: Zhejiang Xiaoshan Rural Commercial Bank Co.,Ltd. Yisheng Sub branch Pledgor: Hangzhou era Yitong software Limited by Share Ltd. Registration number: Y2023980067447 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |