CN112783633A - Data updating system and method based on resource mutual exclusion scheduling model - Google Patents
Data updating system and method based on resource mutual exclusion scheduling model Download PDFInfo
- Publication number
- CN112783633A CN112783633A CN202110187032.7A CN202110187032A CN112783633A CN 112783633 A CN112783633 A CN 112783633A CN 202110187032 A CN202110187032 A CN 202110187032A CN 112783633 A CN112783633 A CN 112783633A
- Authority
- CN
- China
- Prior art keywords
- data
- resource
- market
- product
- updating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computer Security & Cryptography (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
The invention provides a data updating system and method based on a resource mutual exclusion scheduling model, which can be applied to the field of data processing and the field of finance, and the system comprises: the resource caching module acquires market pushing data and stores the market pushing data into a preset caching queue according to the receiving time of the market pushing data and the data type of the market pushing data; the coordination thread module acquires the resource types of the products and constructs the characteristic data corresponding to the products according to the resource types; acquiring market data in a cache queue, and constructing a product set according to the resource type and the characteristic data of the market data; mutually exclusive scheduling is carried out according to the resource types of the product sets, the data of all the conditions are distributed to different execution queues, and the resource updating data of all the products are calculated through the working threads corresponding to the execution queues; and the synthesis calculation module acquires the resource updating data calculated by each working thread and updates the corresponding resources in each product through the resource updating data.
Description
Technical Field
The invention relates to the field of computer resource scheduling, which can be applied to the field of data processing and the field of finance, in particular to a data updating system and a data updating method based on a resource mutual exclusion scheduling model.
Background
Often, the market and product category data are in a multi-stack relationship, for example, a product price generation service may trigger updates to prices i run on multiple financial products each time a market is received. As shown in fig. 1, the market Q1 triggers the updating of the prices of product P1, product P2 and even product PN.
While the parameter resources required for different kinds of products are not exactly the same. As shown in FIG. 2, the quotation Q1 triggers the price update of financial products P1 and P2, and the price update of financial products P1 and P2 requires partial common resources (black boxes with white background) and resources unique to the product types (white boxes with black background). The parameter resources have the possibility of real-time change, and the latest parameter resources need to be obtained by reading the table in real time in order to ensure the reliability and accuracy of the resources and not to excessively depend on NOS cache.
Typical treatments are exemplified by: receiving a quotation Q1, in order to generate the structure of financial products P1 and P2, it is necessary to obtain the parameters of the common resource table A and the table N, and also the parameters of the non-common resource table B, C, E, F, G (refer to FIG. 3). Acquiring non-common resources of the processed product takes a relatively high processing time.
The above treatment has several disadvantages:
the market data is pushed irregularly, as shown in fig. 3, and each push invokes a different thread of work to process. The size, type and the like of processing market data of each working thread are different, and the processing market data are coupled with a push processing function.
The resources which need to be accessed and occupied by each working thread are irregularly circulated, a phenomenon that a large number of intersection sets of resource types are used exists among the threads, and each quotation is used as a dimension to update the corresponding product price. Computer resources are inefficient to use.
Disclosure of Invention
The invention aims to provide a data updating system and method based on a resource mutual exclusion scheduling model, which can improve the execution efficiency of processing large-scale different market data sources and the use efficiency of corresponding computer resources.
To achieve the above object, the present invention provides a data updating system based on a resource mutual exclusion scheduling model, the system comprising: the system comprises a resource cache module, a coordination thread module and a synthesis calculation module; the resource cache module is used for acquiring market pushing data and storing the market pushing data into a preset cache queue according to the receiving time of the market pushing data and the data type of the market pushing data; the coordination thread module is in communication connection with the resource cache module and is used for acquiring the resource types of the products and constructing the characteristic data corresponding to the products according to the resource types; acquiring market data in a cache queue, and constructing a product set according to the resource type of the market data and the characteristic data; mutually exclusive scheduling is carried out according to the resource types of the product sets, all the behavior data are distributed to different execution queues, and the resource updating data of all the products are calculated through the working threads corresponding to the execution queues; and the synthesis calculation module is in communication connection with the coordination thread module and is used for acquiring resource updating data calculated by each working thread and updating corresponding resources in each product through the resource updating data.
In the data updating system based on the resource mutual exclusion scheduling model, preferably, the coordination thread module further includes a data conversion unit and a construction unit; the data conversion unit is used for converting each product into product identification data in a structured data format through a preset characteristic data dictionary according to the characteristic data; the construction unit is used for extracting associated product identification data according to the resource type of the market data and the characteristic data, and constructing a product identification data set according to the provided product identification data.
In the data updating system based on the resource mutual exclusion scheduling model, preferably, the resource cache module includes a comparing unit, and the comparing unit is configured to, when a plurality of pieces of market data with the same data type exist at the same receiving time, cover the market data received first with the market data received later.
In the data updating system based on the resource mutual exclusion scheduling model, preferably, the coordination thread module includes a mutual exclusion scheduling module, the mutual exclusion scheduling module is configured to compare resource types of the products, and when the resource types of two or more products are the same, the resource updating data is calculated through the same job site.
The invention also provides a data updating method based on the resource mutual exclusion scheduling model, which comprises the following steps: acquiring market pushing data, and storing the market pushing data into a preset cache queue according to the receiving time of the market pushing data and the data type of the market pushing data in the market pushing data; acquiring the resource types of products, constructing characteristic data corresponding to the products according to the resource types, and constructing a product set according to the resource types of the market data in the cache queue and the characteristic data; mutually exclusive scheduling is carried out according to the resource types of the product sets, all the behavior data are distributed to different execution queues, and the resource updating data of all the products are calculated through the working threads corresponding to the execution queues; and updating the corresponding resources in each product through the resource updating data.
In the above data updating method based on the resource mutual exclusion scheduling model, preferably, the constructing a product set according to the resource type of the market data in the cache queue and the characteristic data includes: converting each product into product identification data in a structured data format through a preset characteristic data dictionary according to the characteristic data; and extracting associated product identification data according to the resource type of the market data and the characteristic data, and constructing a product identification data set according to the provided product identification data.
In the data updating method based on the resource mutual exclusion scheduling model, preferably, storing the market data into a preset buffer queue includes: when a plurality of pieces of market data with the same data type exist at the same receiving time, the market data received later covers the market data received first.
In the data updating method based on the resource mutual exclusion scheduling model, preferably, the distributing each item of situation data to different execution queues, and the calculating the resource updating data of each product through the work thread corresponding to the execution queue further includes: comparing the resource types of the products, and calculating resource updating data through the same working site when the resource types of two or more products are the same.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method when executing the computer program.
The present invention also provides a computer-readable storage medium storing a computer program for executing the above method.
The invention has the beneficial technical effects that: by using the market resource cache queue, all market can be received in time, and then the influence of market quantity fluctuation is shielded by quantitatively acquiring market data through the coordination thread, so that the fluctuation is stabilized, and the use of resources is balanced. The resource data characteristics of the parameter resource components depended on in the calculation production process of each financial product are calculated through the coordination thread, and the mutual exclusion analysis scheduling is carried out on the data of the components, so that the reasonability and the concentration degree of the resources acquired by the working thread are enhanced, the frequency of acquiring the resources by the total working thread and the occupied memory are greatly reduced, the use tension of the memory in operation is relieved, and the processing efficiency of the platform is improved. Each execution queue, occupied resources and updated products are fixed through rule calculation, so that the data can be submitted in a mode of a data set with higher concentration degree during updating, instead of being submitted in a mode of one or scattered data sets in a common scheme, and the I/O efficiency of the updated database is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
fig. 1 is a schematic diagram of a many-to-many correspondence relationship between a market quotation and a product according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a product occupying resources according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating processing a market quotation according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a data updating system based on a resource mutual exclusion scheduling model according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a resource caching module according to an embodiment of the present invention;
FIG. 6 is a block diagram of a thread coordination module according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating an application of a thread coordination module according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating a principle of mutually exclusive scheduling of resources of a thread coordinating module according to an embodiment of the present invention;
FIG. 9 is a flowchart illustrating a data updating method based on a resource mutual exclusion scheduling model according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following detailed description of the embodiments of the present invention will be provided with reference to the drawings and examples, so that how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented. It should be noted that, unless otherwise specified, the embodiments and features of the embodiments of the present invention may be combined with each other, and the technical solutions formed are within the scope of the present invention.
Additionally, the steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions and, although a logical order is illustrated in the flow charts, in some cases, the steps illustrated or described may be performed in an order different than here.
Referring to fig. 4, a data updating system based on a resource mutual exclusion scheduling model provided by the present invention includes: the system comprises a resource cache module, a coordination thread module and a synthesis calculation module; the resource cache module is used for acquiring market pushing data and storing the market pushing data into a preset cache queue according to the receiving time of the market pushing data and the data type of the market pushing data; the coordination thread module is in communication connection with the resource cache module and is used for acquiring the resource types of the products and constructing the characteristic data corresponding to the products according to the resource types; acquiring market data in a cache queue, and constructing a product set according to the resource type of the market data and the characteristic data; mutually exclusive scheduling is carried out according to the resource types of the product sets, all the behavior data are distributed to different execution queues, and the resource updating data of all the products are calculated through the working threads corresponding to the execution queues; and the synthesis calculation module is in communication connection with the coordination thread module and is used for acquiring resource updating data calculated by each working thread and updating corresponding resources in each product through the resource updating data.
In an embodiment of the present invention, the resource cache module includes a comparing unit, and the comparing unit is configured to, when a plurality of pieces of market data with the same data type exist at the same receiving time, overwrite the market data received first with the market data received later. Specifically, as shown in fig. 5, in actual work, the resource caching module has the following processing steps:
and each market pushing data is directly received and put into a market resource cache queue, and the market sending platform is informed of successful receiving of the pushing data every time the pushing data is successfully received. If the receiving fails and the sending fails, the market information sending platform is informed to request to resend, if the market information sending platform fails three times continuously, the batch of market information is not received and abnormal monitoring is carried out. And the market resource buffer queue performs subsequent processing according to a first-in first-out principle. The market receiving module and the product price generating module are decoupled by the queue, so that the resource caching module is ensured to correctly receive each price, and the receiving timeliness of each market is ensured. It is worth to be noted that the market resource cache queue is unique, each quotation is marked by the time mark received by the queue, the prices in the queue are ordered according to the receiving time sequence, each international market also has only one quotation at a certain time, if a plurality of quotations are provided at the same time, the quotation received first is covered by the later received quotation, the dequeued data does not exist in the market resource cache queue, and the captured market data does not overlap.
Referring to fig. 6, the thread coordinating module further includes a data converting unit and a constructing unit; the data conversion unit is used for converting each product into product identification data in a structured data format through a preset characteristic data dictionary according to the characteristic data; the construction unit is used for extracting associated product identification data according to the resource type of the market data and the characteristic data, and constructing a product identification data set according to the provided product identification data. Furthermore, the coordination thread module comprises a mutual exclusion scheduling module, the mutual exclusion scheduling module is used for comparing the resource types of the products, and when the resource types of two or more products are the same, the resource updating data is calculated through the same working site. Referring to fig. 7, the processing flow of the thread coordinating module in actual operation is as follows:
the method comprises the following steps: the coordination thread acquires the resource types related to each financial product and constructs a piece of resource data characteristic data for each financial product.
Specifically, the method comprises the following steps:
financial product P1: { product attribute resources, commodity season resources, product financial calendar resources, and customer information resources };
financial product P2: { product attribute resources, gradient point difference resources, public transaction calendar resources, customer information resources, currency pair information resources };
financial product P3: { product financial calendar resources, customer information resources, product attribute resources, commodity season resources };
step two: because the resource types are unstructured data (such as product attributes, commodity dates, gradient point differences, financial product calendars, and the like), the analysis and the processing are not facilitated. And attaching independent labels to each type of resources, and establishing a data dictionary of the resource types to convert the data dictionary into structured data.
Specifically, the method comprises the following steps:
as shown in table 1, the topic plate data dictionary is as follows:
TABLE 1
In the second step, the resource data of the financial products are converted as follows:
financial product P1: {1, 2, 3, 7 };
financial product P2: {1, 5, 6, 7, 4 };
financial product P3: {3, 7, 1, 2 };
and D, sequencing the data values of the dictionary from small to large, and converting the resource data of the fusion products in the step three-B into:
financial product P1: {1, 2, 3, 7 };
financial product P2: {1, 4, 5, 6, 7 };
financial product P3: {1, 2, 3, 7 };
it should be noted that, both the first step and the second step of the coordination thread are operated when the product price generation system is initialized, and the corresponding relationship between the product and the resource is established and then placed in the cache. The corresponding relation between the product and the resource is not changed in general, if the corresponding relation is changed, the step one and the step two are triggered to be executed again, and the corresponding old relation data in the cache is covered.
Step three: and the coordination thread acquires quantitative market resources from the market resource cache queue and constructs a related financial product set. The coordination thread acquires a certain number of market resources (the average number of data received in market per second is N) from the market resource cache queue, wherein N is a configured system parameter and is related to the system processing aging, and the coordination thread processing speed is guaranteed to be slightly prior to the queue growth speed. Therefore, the data processed by the coordination processing thread each time is averaged, and the condition of resource waste or resource insufficiency caused by the non-centralized data amount is avoided.
Specifically, the method comprises the following steps:
market Q1: { financial product P1, financial product P2, financial product P3, financial product P4 };
market Q2: { financial product P2, financial product P5, financial product P6 };
market conditions QM: { financial product P1, financial product P3, financial product P8, financial product PN };
relates to a product set: { financial product P1, financial product P2, financial product P3, financial product P4, financial product P5, financial product PN }.
It is worth to be noted that the corresponding relation between the quotation and the product is similar to the corresponding relation between the product and the resource, generally, the relation is not changed, and the relation is many-to-many, and the quotation and the product can be directly generated and fixed in the cache from the parameter table of the database according to the rule when the product price generating system is initialized; then, each time the coordination thread processes the market resources, the parameters are directly read to generate the market to be processed and the corresponding product thereof (i.e. the above example).
Step four: the coordination thread carries out mutual exclusion scheduling according to the constructed resource characteristics, and forwards the quotations to different execution queues in a classified manner, so that the resource characteristics required by the financial products processed by the working threads corresponding to each execution queue are consistent, the consumption caused by excessive resource acquisition of the working threads due to resource characteristic intersection of the financial products is avoided, the same resource characteristics of the financial products can be processed by the same working thread, the difference set of any two resource characteristic sets of the financial products is not empty, and the situation that the mutual exclusion of partial resources exists between the two resource characteristic sets of the financial products and the two working threads are required to process the financial products is indicated.
Referring to fig. 8, mutually exclusive analysis and division of resource features are performed through the coordination threads, unprocessed market resources are distributed to different work thread queues to be executed, and compared with the situation before non-scheduling, each work thread may need to acquire resources 1, 2, 3, 4, 5, 6, and 7, the resource concentration degree that the work thread after scheduling needs to acquire is enhanced, and the total number of resources to be read is greatly reduced, so that the situation of use tension of the memory during operation is alleviated, and the processing efficiency of financial market quotation is improved. Referring to fig. 8 again, the working thread obtains a financial product set to be synthesized, obtains related resource data, and constructs a parameter data set for each financial product, where the resource parameter data set does not change during the life cycle of the working thread. And continuously acquiring market data from the queue to be executed of the working thread, and calculating according to the business rule. Because each working thread corresponds to the updating of the price of a specific product, not only the total number of the obtained resources is reduced, but also the data set concentration is enhanced (one thread of the original scheme corresponds to relatively more products and relatively more data sets are established) compared with the conventional original scheme described in the technical background according to the mode of a product data set during updating, the frequency of database interaction and commit is reduced, and the database access and the overall efficiency are improved.
Referring to fig. 9, the present invention further provides a data updating method based on the resource mutual exclusion scheduling model, where the method includes:
step S901, acquiring market pushing data, and storing the market pushing data into a preset cache queue according to the receiving time of the market pushing data and the data type of the market pushing data;
step S902, acquiring resource types of products, constructing characteristic data corresponding to the products according to the resource types, and constructing a product set according to the resource types of market data in a cache queue and the characteristic data;
step S903 is used for carrying out mutual exclusion scheduling according to the resource types of the product sets, distributing each behavior data to different execution queues, and calculating the resource updating data of each product through the working thread corresponding to the execution queues;
step S904 updates the corresponding resource in each product through the resource update data.
In the above embodiment, the step S902 of building a product set according to the resource type of the market data in the buffer queue and the characteristic data includes: converting each product into product identification data in a structured data format through a preset characteristic data dictionary according to the characteristic data; and extracting associated product identification data according to the resource type of the market data and the characteristic data, and constructing a product identification data set according to the provided product identification data.
In an embodiment of the present invention, storing the market data in a preset buffer queue includes: when a plurality of pieces of market data with the same data type exist at the same receiving time, the market data received later covers the market data received first. In another embodiment, the distributing each item of situation data to different execution queues, and the calculating the resource update data of each product through the work thread corresponding to the execution queue further includes: comparing the resource types of the products, and calculating resource updating data through the same working site when the resource types of two or more products are the same.
The invention has the beneficial technical effects that: by using the market resource cache queue, all market can be received in time, and then the influence of market quantity fluctuation is shielded by quantitatively acquiring market data through the coordination thread, so that the fluctuation is stabilized, and the use of resources is balanced. The resource data characteristics of the parameter resource components depended on in the calculation production process of each financial product are calculated through the coordination thread, and the mutual exclusion analysis scheduling is carried out on the data of the components, so that the reasonability and the concentration degree of the resources acquired by the working thread are enhanced, the frequency of acquiring the resources by the total working thread and the occupied memory are greatly reduced, the use tension of the memory in operation is relieved, and the processing efficiency of the platform is improved. Each execution queue, occupied resources and updated products are fixed through rule calculation, so that the data can be submitted in a mode of a data set with higher concentration degree during updating, instead of being submitted in a mode of one or scattered data sets in a common scheme, and the I/O efficiency of the updated database is improved.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method when executing the computer program.
The present invention also provides a computer-readable storage medium storing a computer program for executing the above method.
As shown in fig. 10, the electronic device 600 may further include: communication module 110, input unit 120, audio processing unit 130, display 160, power supply 170. It is noted that the electronic device 600 does not necessarily include all of the components shown in FIG. 10; furthermore, the electronic device 600 may also comprise components not shown in fig. 10, which may be referred to in the prior art.
As shown in fig. 10, the central processor 100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, the central processor 100 receiving input and controlling the operation of the various components of the electronic device 600.
The memory 140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 100 may execute the program stored in the memory 140 to realize information storage or processing, etc.
The input unit 120 provides input to the cpu 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display an object to be displayed, such as an image or a character. The display may be, for example, an LCD display, but is not limited thereto.
The memory 140 may be a solid state memory such as Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 140 may also be some other type of device. Memory 140 includes buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application/function storage section 142, and the application/function storage section 142 is used to store application programs and function programs or a flow for executing the operation of the electronic device 600 by the central processing unit 100.
The memory 140 may also include a data store 143, the data store 143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by the electronic device. The driver storage portion 144 of the memory 140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging application, address book application, etc.).
The communication module 110 is a transmitter/receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter/receiver) 110 is coupled to the central processor 100 to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132 to implement general telecommunications functions. Audio processor 130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, an audio processor 130 is also coupled to the central processor 100, so that recording on the local can be enabled through a microphone 132, and so that sound stored on the local can be played through a speaker 131.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A data updating system based on a resource mutual exclusion scheduling model, the system comprising: the system comprises a resource cache module, a coordination thread module and a synthesis calculation module;
the resource cache module is used for acquiring market pushing data and storing the market pushing data into a preset cache queue according to the receiving time of the market pushing data and the data type of the market pushing data;
the coordination thread module is in communication connection with the resource cache module and is used for acquiring the resource types of the products and constructing the characteristic data corresponding to the products according to the resource types; acquiring market data in a cache queue, and constructing a product set according to the resource type of the market data and the characteristic data; mutually exclusive scheduling is carried out according to the resource types of the product sets, all the behavior data are distributed to different execution queues, and the resource updating data of all the products are calculated through the working threads corresponding to the execution queues;
and the synthesis calculation module is in communication connection with the coordination thread module and is used for acquiring resource updating data calculated by each working thread and updating corresponding resources in each product through the resource updating data.
2. The resource mutual exclusion scheduling model based data updating system of claim 1, wherein the coordination thread module further comprises a data conversion unit and a construction unit;
the data conversion unit is used for converting each product into product identification data in a structured data format through a preset characteristic data dictionary according to the characteristic data;
the construction unit is used for extracting associated product identification data according to the resource type of the market data and the characteristic data, and constructing a product identification data set according to the provided product identification data.
3. The resource mutual exclusion scheduling model-based data updating system of claim 1, wherein the resource caching module comprises a comparing unit, and the comparing unit is configured to, when a plurality of pieces of market data with the same data type exist at the same receiving time, overwrite the market data received first with the market data received later.
4. The system of claim 1, wherein the thread module comprises a mutual exclusion scheduling module, and the mutual exclusion scheduling module is configured to compare resource types of the products, and when the resource types of two or more products are the same, calculate the resource update data through the same job site.
5. A data updating method based on a resource mutual exclusion scheduling model is characterized in that the method comprises the following steps:
acquiring market pushing data, and storing the market pushing data into a preset cache queue according to the receiving time of the market pushing data and the data type of the market pushing data in the market pushing data;
acquiring the resource types of products, constructing characteristic data corresponding to the products according to the resource types, and constructing a product set according to the resource types of the market data in the cache queue and the characteristic data;
mutually exclusive scheduling is carried out according to the resource types of the product sets, all the behavior data are distributed to different execution queues, and the resource updating data of all the products are calculated through the working threads corresponding to the execution queues;
and updating the corresponding resources in each product through the resource updating data.
6. The method for updating data based on the resource mutual exclusion scheduling model of claim 5, wherein the constructing the product set according to the resource types of the market data in the buffer queue and the characteristic data comprises:
converting each product into product identification data in a structured data format through a preset characteristic data dictionary according to the characteristic data;
and extracting associated product identification data according to the resource type of the market data and the characteristic data, and constructing a product identification data set according to the provided product identification data.
7. The method for updating data based on the resource mutual exclusion scheduling model of claim 5, wherein the storing the market data into a preset buffer queue comprises: when a plurality of pieces of market data with the same data type exist at the same receiving time, the market data received later covers the market data received first.
8. The method as claimed in claim 5, wherein the step of distributing the respective behavior data to different execution queues, and the step of calculating the resource update data of each product by the work thread corresponding to the execution queue further comprises: comparing the resource types of the products, and calculating resource updating data through the same working site when the resource types of two or more products are the same.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 5 to 8 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any of claims 5 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110187032.7A CN112783633A (en) | 2021-02-08 | 2021-02-08 | Data updating system and method based on resource mutual exclusion scheduling model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110187032.7A CN112783633A (en) | 2021-02-08 | 2021-02-08 | Data updating system and method based on resource mutual exclusion scheduling model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112783633A true CN112783633A (en) | 2021-05-11 |
Family
ID=75761529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110187032.7A Pending CN112783633A (en) | 2021-02-08 | 2021-02-08 | Data updating system and method based on resource mutual exclusion scheduling model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783633A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023175413A1 (en) * | 2022-03-15 | 2023-09-21 | International Business Machines Corporation | Mutual exclusion data class analysis in data governance |
-
2021
- 2021-02-08 CN CN202110187032.7A patent/CN112783633A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023175413A1 (en) * | 2022-03-15 | 2023-09-21 | International Business Machines Corporation | Mutual exclusion data class analysis in data governance |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111782470B (en) | Distributed container log data processing method and device | |
| CN111367988A (en) | Data import method and device | |
| CN110472109B (en) | Dynamic data quality analysis method and platform system | |
| CN113656432A (en) | Data comparison method and device | |
| CN112783633A (en) | Data updating system and method based on resource mutual exclusion scheduling model | |
| CN111383104A (en) | Method and device for acquiring fund investment supervision indexes | |
| CN112052391B (en) | Data pushing method, device and system | |
| CN113741860A (en) | Data processing method and device based on RPA robot | |
| CN110297945B (en) | Data information processing method and system based on XBRL | |
| CN112116274A (en) | Automatic generation method and device of survey report | |
| CN111782204A (en) | Batch scheduling editing method and device | |
| CN113837870B (en) | Financial risk data approval method and device | |
| CN110022323A (en) | A kind of method and system of the cross-terminal real-time, interactive based on WebSocket and Redux | |
| CN113434423A (en) | Interface test method and device | |
| CN115495513A (en) | Event standardization method and device | |
| CN108984318B (en) | Message delivery method and device based on driving model and readable storage medium | |
| CN113515577A (en) | Data preprocessing method and device | |
| CN113515447A (en) | System automation test method and device | |
| CN112396511A (en) | Distributed wind control variable data processing method, device and system | |
| CN114648410A (en) | Stock staring method, apparatus, system, device and medium | |
| CN110674119A (en) | Data processing method, system and computer readable storage medium | |
| CN113485799B (en) | Method, device, equipment and storage medium for executing same timing task by multiple libraries | |
| CN112988603B (en) | Big data test case generation method and device | |
| CN114117209A (en) | 5G message recommendation method and device | |
| CN115564563A (en) | Service processing method, device and equipment based on 5G message |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |