CN110413696A - A kind of data processing method and device - Google Patents
A kind of data processing method and device Download PDFInfo
- Publication number
- CN110413696A CN110413696A CN201910710284.6A CN201910710284A CN110413696A CN 110413696 A CN110413696 A CN 110413696A CN 201910710284 A CN201910710284 A CN 201910710284A CN 110413696 A CN110413696 A CN 110413696A
- Authority
- CN
- China
- Prior art keywords
- data
- storage system
- propelling
- pending
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2219—Large Object storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This specification embodiment provides a kind of data processing method and device.The described method includes: the pending data received is stored in the first storage system；When data processing server is idle state, pending data is called to input the data processing server from first storage system；The propelling data that the data processing server is fed back is stored in the second storage system；After the propelling data acquisition request for receiving online client transmission, the propelling data obtained in second storage system is sent to the online client.Pass through the above method, it is interacted using the first storage system with pending data, and propelling data is stored using the second additional storage system, the architectural form that database interactive quantity is big, resource pressure is heavy is changed in the prior art, the rate of data processing is improved.
Description
Technical field
This specification embodiment is related to field of computer technology, in particular to a kind of data processing method and device.
Background technique
In real life, user can face many data in processing business, but many data therein generally require
The meaning in data could be intuitively given expression to after a series of calculating analysis, such as is implementing exploration and development, bill friendship
Numerous areas are easily waited, user needs first to store accessed intuitive data into corresponding database, recycles corresponding
Computing module call data to be calculated from database, and calculated result is stored into database, so that user
Corresponding calculated result can be called from database, obtain the information of itself needs.
But according to the above technical scheme, larger or when the calculative data volume of user is more in userbase, gesture
Database interactive quantity must be made to increase severely, database resource pressure is excessive, to affect the normal processing for data.If should
The real-time of data is stronger, fails normally to handle the validity that may lose the data, reduces the usage experience of user.
Summary of the invention
The purpose of this specification embodiment is to provide a kind of data processing method and device, to solve to handle in the prior art
The problem of interaction when data between database is excessive, causes database pressure excessive, influences working efficiency.
To solve the above-mentioned problems, a kind of data processing method and device that this specification embodiment proposes are realized in
:
A kind of data processing method, comprising:
The pending data received is stored in the first storage system；
When data processing server is idle state, pending data is called to input institute from first storage system
State data processing server；
The propelling data that the data processing server is fed back is stored in the second storage system；
After the propelling data acquisition request for receiving online client transmission, the push in second storage system is obtained
Data are sent to the online client.
A kind of data processing equipment, comprising:
Pending data memory module, for the pending data received to be stored in the first storage system；
Pending data calling module, for being from first storage when data processing server is idle state
Pending data is called to input the data processing server in system；
Propelling data memory module, the propelling data for being fed back the data processing server are stored in second and deposit
Storage system；
Data-pushing module after the propelling data acquisition request for receiving online client transmission, obtains described the
Propelling data in two storage systems is sent to the online client.
A kind of data processing method, comprising:
The pending data received is stored in the first storage system；
When data processing server is idle state, pending data is called to input institute from first storage system
State data processing server；
The propelling data that the data processing server is fed back is pushed into the visitor that active client state is presence
Family end.
A kind of data processing equipment, comprising:
Pending data memory module, for the pending data received to be stored in the first storage system；
Pending data calling module, for being from first storage when data processing server is idle state
Pending data is called to input the data processing server in system；
Data-pushing module, the propelling data for feeding back the data processing server push to active client shape
State is the client of presence.
A kind of data processing method, the executing subject of the method are data processing server, comprising:
If current state is idle state, pending data is received；The idle state is set according to following situations:
There is the thread for being not used for processing data in the data processing server；
According to the type of service of the pending data, pending data described in corresponding thread process is distributed, is pushed away
Send data；
Feed back the propelling data.
A kind of data processing equipment, described device are integrated in data processing server, comprising:
Pending data receiving module, for receiving pending data when current state is idle state；The free time
State is set according to following situations: there is the thread for being not used for processing data in the data processing server；
Propelling data computing module distributes corresponding thread process for the type of service according to the pending data
The pending data, obtains propelling data；
Feedback module, for feeding back the propelling data.
The technical solution provided by above this specification embodiment is as it can be seen that this specification embodiment passes through the first storage system
After storing pending data, data are called to be supplied to number from first storage system in the data processing server free time
It is handled according to processing server.After the data processing server returns to propelling data, the data are stored in the
Two servers, and after receiving propelling data acquisition request, the propelling data is sent in corresponding client.Pass through
The above method is utilized respectively the related calling that the first storage system carries out data to the second storage system, alleviates wherein specific
Storage system resource pressure, business is normally executed.
In another execution method, after the propelling data that can be exported in receiving data processing server, directly
Connect the client that the propelling data is pushed to presence, so as to avoid by the propelling data again with storage system
Between the step of interacting, reducing data interaction, equally also further improve server execution efficiency.
Detailed description of the invention
In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
The some embodiments recorded in this specification, for those of ordinary skill in the art, before not making the creative labor
It puts, is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of structure chart of data processing system of this specification embodiment；
Fig. 2 is a kind of flow chart of data processing method of this specification embodiment；
Fig. 3 is a kind of flow chart of data processing method of this specification embodiment；
Fig. 4 is a kind of flow chart of data processing method of this specification embodiment；
Fig. 5 is a kind of flow chart of data processing method of this specification embodiment；
Fig. 6 is a kind of module map of data processing equipment of this specification embodiment；
Fig. 7 is a kind of module map of data processing equipment of this specification embodiment；
Fig. 8 is a kind of module map of data processing equipment of this specification embodiment.
Specific embodiment
Below in conjunction with the attached drawing in this specification embodiment, the technical solution in this specification embodiment is carried out clear
Chu is fully described by, it is clear that described embodiment is only this specification a part of the embodiment, rather than whole implementation
Example.The embodiment of base in this manual, those of ordinary skill in the art are obtained without making creative work
Every other embodiment, all should belong to this specification protection range.
The technical concept of the data processing method in order to better understand, it is described to can be realized in conjunction with attached drawing 1 first
A kind of embodiment of data processing system of technical concept is introduced.
The data processing system may include the first storage system 110.First storage system includes but is not limited to
Server, database, cluster and distributed memory system.First storage system can be used for receiving and storing number to be processed
According to；First storage system can be also used for storage to propelling data；First storage system can also be stored with third
Data interaction is carried out between system.
The data processing system can also include data processing server 120.The data processing server can be used
In reception pending data；The data processing server can carry out calculating etc. for data to be received operating and being pushed
Data；Propelling data can be transmitted to other storage systems or client by the data processing server.
The data processing system can also include the second storage system 130.Second storage system includes but unlimited
In server, database, cluster and distributed memory system.Second storage system can be used for receiving propelling data；Institute
Stating the second storage system can be also used for pushing to propelling data in client.
The data processing system can also include third storage system 140.The third storage system includes but unlimited
In server, database, cluster and distributed memory system.The third storage system can be used for storing the first storage system
In data, the data stored can be sent in first storage system by the third storage system.
The data processing system can also include method execute server 150.The method execute server can be used
In send instructions to the first storage system, the second storage system and third storage system to designation date transmission.The side
Method execute server can be used for the state of detection data processing server；The method execute server can be also used for detecting
Client state.
In conjunction with the structure chart of above-mentioned data processing system, 2 a kind of data of this specification embodiment are introduced with reference to the accompanying drawing
The executing subject of the embodiment of processing method, the method is computer equipment, and the computer equipment includes server, industry control
Machine (industrial control computer), PC machine, all-in-one machine etc..Specific step is as follows for the method:
S210: the pending data received is stored in the first storage system.
Pending data is to need that the number that just can be pushed to client or be further processed is further processed
According to.The pending data can be the higher data of some requirement of real-time, be also possible to some numbers in face of a large amount of audients
According to.For example, the pending data can be the transaction details data in business processing.The pending data can come from visitor
Family end can be from other servers.
First storage system can be used for storing the pending data.Since the server of processing data may not be
It is provided with the ability of processing data at any time, it is therefore desirable in advance by pending data storage and the first storage system
In,
In one embodiment, first storage system can have multiple memory nodes, be using the first storage
System storing data can be through distributed caching technology the storage for realizing data.In the process of storing data and calling data
In, it can use multiple nodes and be respectively processed for data, to improve the interactivity between data, improve number
According to the ability of processing.
S220: when data processing server is idle state, pending data is called from first storage system
Input the data processing server.
Idle state can refer to the thread for existing in the data processing server and being not used for processing data.In idle shape
When state, there is the thread that can be used for handling data in the data processing server, it at this time can be for pending data point
Data are handled with corresponding thread.
It is idle state in the data processing server, can be called accordingly to from from first storage system
It manages data and inputs the data processing server, and performed corresponding processing by the data processing server.
S230: data processing server distributes described in thread process according to the type of service of the pending data wait locate
Data are managed, propelling data is obtained.
For pending data, the type of service corresponding to each pending data can be previously provided with.For example, corresponding
In the pending data of transaction details classification, the service class such as interbank discount, dialogue quotation, electric ticket, silver-colored ticket, 1M can be preset
Not.For having the pending data of single type of service, one is distributed when distributing pending data described in thread process
Pending data described in thread process.And it is corresponding with the pending data of multiple types of service, it can be with when distributing thread
According to the quantity of the corresponding type of service of the pending data, corresponding thread is used to handle the pending data respectively, thus
The high efficiency for guaranteeing the treatment process for such data, improves treatment effeciency.
By for the corresponding processing thread of pending data distribution, after being disposed, the push that can be obtained that treated
Data.Propelling data is that can push to the data that client is checked for user.For example, being business in the pending data
When transaction details data in treatment process, propelling data can be service condition delta data, for example, stock, bond etc. produce
The data that institute's real-time reception arrives in the market situation of change or exploration process of product.The requirement of real-time of such data is higher,
It needs technology to be handled using server and pushes to client.
S240: the propelling data is fed back to the first storage system by data processing server.
After processing obtains propelling data, obtained propelling data can be fed back to described by data processing server
In one storage system, for the data processing in subsequent process.It is to utilize distributed caching technology in first storage system
In the case where storing data, each node is able to achieve the interaction between data in the first storage system, so that for resource
Interactive ability to bear is stronger, can be preferably applied to handle all data.
The propelling data is sent to the second storage system by the S250: the first storage system.
Second storage system can be the storage system for propelling data described in temporary cache.Due to for some clients
End cannot in real time push data, need to cache propelling data in advance, when the client state is presence, then
Corresponding data are called to be sent in client from second storage system.
When the propelling data to be stored in the second storage system, Protobuf technology can use by the push
Data are converted into structural data, then will be in structural data storage and the second storage system.Utilize Protobuf technology
Data after conversion are smaller, storing and transmitting for data are more advantageous to, and the performance of coding and decoding is also very high, thus favorably
In the efficiency for improving processing.
In one embodiment, second storage system can be memory database.Since memory database will count
According to storage with memory, in terms of data exchange capability more remarkably, read relative to the database in traditional storage and disk
The rate for writing data is greatly improved.It, can using the memory database when needing to multiple client propelling data
More effectively to realize the push of data.
S280: it after the propelling data acquisition request for receiving online client transmission, obtains in second storage system
Propelling data be sent to the online client.
It, can not be directly to visitor in client and when being not maintained in linear state since client will not be always maintained at online
Family end pushes corresponding data.It needs to wait client just to send data to client when presence.
After client is online, when being presence, log-in events are triggered, sends and pushes to the method execute server
Data acquisition request.The propelling data acquisition request, can be used for obtaining the propelling data in the second storage system.
Method execute server obtains in second storage system after receiving the propelling data acquisition request
Propelling data is sent to the online client, to realize the push of data.
In order to adapt to the case where the first storage system breaks down, the method also includes step S260: by the push
Data are synchronized to third storage system.In one embodiment, in order to avoid in subsequent process due to the first storage system
Pressure is excessive, causes the loss of data wherein stored to which traffic affecting is normally carried out, can be in first storage system
After getting propelling data, the propelling data is synchronized to third storage system.
After propelling data is synchronized to third storage system, in the case where first storage system breaks down,
It executes step S270: the propelling data is sent to the second storage system.If detecting, the first storage system is malfunction,
Then illustrate that first storage system may be no longer able to continue to execute the data processing method at this time, it can be by third
Storage system replaces the first storage system to execute corresponding step, such as propelling data is stored in the second storage system.It is practical
In, pending data can also also be stored in the third storage system, be broken down in the first storage system
In the case of, therefrom pending data can also be called to be sent to data processing server, to ensure the data processing side
Method is normally carried out.
It can be seen that by the description of above method step and pending data and push received by distributed caching technology
Data, read-write efficiency is high, is capable of the write-in and reading of highly efficient rapidly progress data, passes through the overall process in data processing
In be utilized respectively three storage systems for the interaction in each step for data, alleviate each section data-storage system
Pressure improves working efficiency to ensure the high availability in data handling procedure, thus saved corresponding resource and
Time.
3 introduce a kind of embodiment of data processing method of this specification, the execution master of the method with reference to the accompanying drawing
Body is method execute server, and specific step is as follows for the method:
S310: the pending data received is stored in the first storage system.
The implementation procedure of the step can be with reference to the narration in step S210, and details are not described herein.
S320: when data processing server is idle state, pending data is called from first storage system
Input the data processing server.
The implementation procedure of the step can be with reference to the narration in step S220, and details are not described herein.
S330: the propelling data that the data processing server is fed back is stored in the second storage system.
The implementation procedure of the step can be with reference to the narration in step S250, and details are not described herein.
S340: it after the propelling data acquisition request for receiving online client transmission, obtains in second storage system
Propelling data be sent to the online client.
The implementation procedure of the step can be with reference to the narration in step S280, and details are not described herein.
4 introduce a kind of embodiment of data processing method of this specification, the execution master of the method with reference to the accompanying drawing
Body is method execute server, and specific step is as follows for the method:
S410: the pending data received is stored in the first storage system.
The implementation procedure of the step can be with reference to the narration in step S210, and details are not described herein.
S420: when data processing server is idle state, pending data is called from first storage system
Input the data processing server.
The implementation procedure of the step can be with reference to the narration in step S220, and details are not described herein.
S430: it is presence that the propelling data that the data processing server is fed back, which is pushed to active client state,
Client.
It in order to mitigate the pressure of data-storage system, while ensureing the real-time of data-pushing, is taken in the data processing
It is engaged in after device feedback propelling data, the propelling data of available feedback, and propelling data is directly pushed into current state and is
The client of presence alleviates storage pressure, simultaneously to eliminate the process using storage system storage propelling data
Improve the real-time of data-pushing.
5 introduce a kind of embodiment of data processing method of this specification, the executing subject of the method with reference to the accompanying drawing
For data processing server, specific step is as follows for the method:
S510: if current state is idle state, pending data is received.
Idle state can refer to the thread for existing in the data processing server and being not used for processing data.In the number
Multiple threads are assigned according to realization in processing server respectively to handle data.Thread is right in real time after data processing
Pending data is grabbed.
Detect in the data processing server there are current state and be not used for processing data thread when, explanation
Data processing server has the ability of processing data at this time, can receive pending data and performs corresponding processing.
S520: according to the type of service of the pending data, pending data described in corresponding thread process is distributed, is obtained
To propelling data.
For pending data, the type of service corresponding to each pending data can be previously provided with.For example, corresponding
In the pending data of transaction details classification, the service class such as interbank discount, dialogue quotation, electric ticket, silver-colored ticket, 1M can be preset
Not.For having the pending data of single type of service, one is distributed when distributing pending data described in thread process
Pending data described in thread process.And it is corresponding with the pending data of multiple types of service, it can be with when distributing thread
According to the quantity of the corresponding type of service of the pending data, corresponding thread is used to handle the pending data respectively, thus
The high efficiency for guaranteeing the treatment process for such data, improves treatment effeciency.
During using thread process data, for the thread of each processing data, beats can be regularly sent
According to thus treatment progress of the monitoring thread to data.If by the default beats for handling the time and receive thread transmission
According to, by the pending data distribute to other be not used for processing data process.So that process can not be normal right
When data are handled, other threads can be distributed to, it is ensured that data handling procedure is normally carried out.
S530: the propelling data is fed back.
After processing obtains the propelling data, the propelling data is fed back.For example, passing through the first storage
In the case that server sends pending data to the data processing server, the propelling data can be fed back to described
First storage server.
By the introduction of above-mentioned data processing method, illustrates the process of data processing server processing data, ensure that
Data processing server handles high availability when data, while being capable of thread in the server when can not normally handle data
It is effectively detected, enhances the ability of data processing server processing data.
In order to be better carried out the data processing method, a kind of data processing of 6 pairs of this specification with reference to the accompanying drawing is filled
The embodiment set is introduced, and described device is integrated in method execute server, and described device specifically includes:
Pending data memory module 610, for the pending data received to be stored in the first storage system；
Pending data calling module 620, for being stored from described first when data processing server is idle state
Pending data is called to input the data processing server in system；
Propelling data memory module 630, the propelling data for being fed back the data processing server are stored in
Two storage systems；
Data-pushing module 640, for when detecting client state is presence, obtaining second storage system
Propelling data in system pushes to the client.
In order to be better carried out the data processing method, a kind of data processing of 7 pairs of this specification with reference to the accompanying drawing is filled
The embodiment set is introduced, and described device is integrated in method execute server, and described device specifically includes:
Pending data memory module 710, for the pending data received to be stored in the first storage system；
Pending data calling module 720, for being stored from described first when data processing server is idle state
Pending data is called to input the data processing server in system；
Data-pushing module 730, the propelling data for feeding back the data processing server push to existing customer
End state is the client of presence.
In order to be better carried out the data processing method, a kind of data processing of 8 pairs of this specification with reference to the accompanying drawing is filled
The embodiment set is introduced, and described device is integrated in data processing server, and described device specifically includes:
Pending data receiving module 810, for receiving pending data when current state is idle state；It is described
Idle state is set according to following situations: there is the thread for being not used for processing data in the data processing server；
Propelling data computing module 820 distributes at corresponding thread for the type of service according to the pending data
The pending data is managed, propelling data is obtained；
Feedback module 830, for feeding back the propelling data.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " is patrolled
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed is most generally used at present
Integrated Circuit Hardware Description Language) and Verilog2.Those skilled in the art
It will be apparent to the skilled artisan that only needing method flow slightly programming in logic and being programmed into integrated circuit with above-mentioned several hardware description languages
In, so that it may it is readily available the hardware circuit for realizing the logical method process.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
As seen through the above description of the embodiments, those skilled in the art can be understood that this specification
It can realize by means of software and necessary general hardware platform.Based on this understanding, the technical solution of this specification
Substantially the part that contributes to existing technology can be embodied in the form of software products in other words, the computer software
Product can store in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) executes each embodiment of this specification or embodiment
Certain parts described in method.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method
Part explanation.
This specification can be used in numerous general or special purpose computing system environments or configuration.Such as: personal computer,
Server computer, handheld device or portable device, laptop device, multicomputer system, microprocessor-based system,
Set top box, programmable consumer-elcetronics devices, network PC, minicomputer, mainframe computer including any of the above system are set
Standby distributed computing environment etc..
This specification can describe in the general context of computer-executable instructions executed by a computer, such as journey
Sequence module.Generally, program module include routines performing specific tasks or implementing specific abstract data types, programs, objects,
Component, data structure etc..This specification can also be practiced in a distributed computing environment, in these distributed computing environment
In, by executing task by the connected remote processing devices of communication network.In a distributed computing environment, program module
It can be located in the local and remote computer storage media including storage equipment.
Although depicting this specification by embodiment, it will be appreciated by the skilled addressee that there are many become for this specification
Shape and the spirit changed without departing from this specification, it is desirable to which the attached claims include these deformations and change without departing from this
The spirit of specification.
Claims (11)
1. a kind of data processing method characterized by comprising
Received pending data is stored in the first storage system；
When data processing server is idle state, pending data is called to input the number from first storage system
According to processing server；
The propelling data that the data processing server is fed back is stored in the second storage system；
After the propelling data acquisition request for receiving online client transmission, the propelling data in second storage system is obtained
It is sent to the online client.
2. the method as described in claim 1, which is characterized in that the pending data includes: that the transaction in business processing is bright
Count evidence accurately；Correspondingly, the propelling data includes: service condition delta data.
3. the method as described in claim 1, which is characterized in that include multiple memory nodes in first storage system；Institute
It states the pending data that will be received and is stored in the first storage system, comprising:
The pending data received is stored in the first storage system using distributed caching technology.
4. the method as described in claim 1, which is characterized in that the method also includes:
The propelling data exported in the data processing server is received, is stored in the first storage system；
Propelling data in first storage system is synchronized to third storage system；
Correspondingly, the propelling data that the data processing server is fed back is stored in the second storage system, comprising:
When detecting first storage system is malfunction, the propelling data in the third storage system is obtained, is deposited
It is stored in the second storage system.
5. the method as described in claim 1, which is characterized in that the push number for being fed back the data processing server
According to being stored in the second storage system, comprising:
Receive the propelling data of the data processing server feedback；
Structural data is converted by the propelling data using Protobuf technology；
The structural data is stored in the second storage system.
6. a kind of data processing equipment characterized by comprising
Pending data memory module, for the pending data received to be stored in the first storage system；
Pending data calling module is used for when data processing server is idle state, from first storage system
Pending data is called to input the data processing server；
Propelling data memory module, the propelling data for being fed back the data processing server are stored in the second storage system
System；
Data-pushing module obtains described second and deposits after receiving the propelling data acquisition request that online client is sent
Propelling data in storage system is sent to the online client.
7. a kind of data processing method characterized by comprising
The pending data received is stored in the first storage system；
When data processing server is idle state, pending data is called to input the number from first storage system
According to processing server；
The propelling data that the data processing server is fed back is pushed into the client that active client state is presence.
8. a kind of data processing equipment characterized by comprising
Pending data memory module, for the pending data received to be stored in the first storage system；
Pending data calling module is used for when data processing server is idle state, from first storage system
Pending data is called to input the data processing server；
Data-pushing module, the propelling data for feeding back the data processing server push to active client state and are
The client of presence.
9. a kind of data processing method, the executing subject of the method is data processing server characterized by comprising
If current state is idle state, pending data is received；The idle state is set according to following situations: described
There is the thread for being not used for processing data in data processing server；
According to the type of service of the pending data, pending data described in corresponding thread process is distributed, obtains push number
According to；
Feed back the propelling data.
10. method as claimed in claim 9, which is characterized in that the type of service according to the pending data,
Pending data described in corresponding thread process is distributed, propelling data is obtained, comprising:
According to the type of service of the pending data, pending data described in corresponding thread process is distributed；
If the heartbeat data for handling the thread of the pending data and sending is not received by the default processing time, by described wait locate
Reason data distribute to other processes for being not used for processing data, obtain propelling data.
11. a kind of data processing equipment, described device are integrated in data processing server characterized by comprising
Pending data receiving module, for receiving pending data when current state is idle state；The idle state
Being set according to following situations: there is the thread for being not used for processing data in the data processing server；
Propelling data computing module distributes described in corresponding thread process for the type of service according to the pending data
Pending data obtains propelling data；
Feedback module, for feeding back the propelling data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910710284.6A CN110413696A (en) | 2019-08-02 | 2019-08-02 | A kind of data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910710284.6A CN110413696A (en) | 2019-08-02 | 2019-08-02 | A kind of data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110413696A true CN110413696A (en) | 2019-11-05 |
Family
ID=68365325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910710284.6A Pending CN110413696A (en) | 2019-08-02 | 2019-08-02 | A kind of data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110413696A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611251A (en) * | 2020-04-24 | 2020-09-01 | 华智众创(北京)投资管理有限责任公司 | Data processing system |
| CN113590590A (en) * | 2021-07-30 | 2021-11-02 | 中国工商银行股份有限公司 | Database calculation method and device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102186064A (en) * | 2011-05-30 | 2011-09-14 | 无锡中星微电子有限公司 | Distributed video monitoring system and monitoring method |
| CN102340545A (en) * | 2011-10-31 | 2012-02-01 | 深圳市五巨科技有限公司 | Server and data processing method thereof |
| CN102571992A (en) * | 2012-03-29 | 2012-07-11 | 哈尔滨工业大学 | Cloud service system for budget traveling system and data processing method for each cloud service node in cloud service system |
| CN105554059A (en) * | 2015-11-25 | 2016-05-04 | 北京华油信通科技有限公司 | Logistics transportation intelligent perception and position service system based on Beidou navigation technology |
| CN106294486A (en) * | 2015-06-04 | 2017-01-04 | 交通银行股份有限公司 | The processing method of finance market data and system |
| CN109298990A (en) * | 2018-10-17 | 2019-02-01 | 平安科技（深圳）有限公司 | Log storing method, device, computer equipment and storage medium |
| CN109388504A (en) * | 2018-09-26 | 2019-02-26 | 平安科技（深圳）有限公司 | Messageization docks processing method, device, computer equipment and storage medium |
-
2019
- 2019-08-02 CN CN201910710284.6A patent/CN110413696A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102186064A (en) * | 2011-05-30 | 2011-09-14 | 无锡中星微电子有限公司 | Distributed video monitoring system and monitoring method |
| CN102340545A (en) * | 2011-10-31 | 2012-02-01 | 深圳市五巨科技有限公司 | Server and data processing method thereof |
| CN102571992A (en) * | 2012-03-29 | 2012-07-11 | 哈尔滨工业大学 | Cloud service system for budget traveling system and data processing method for each cloud service node in cloud service system |
| CN106294486A (en) * | 2015-06-04 | 2017-01-04 | 交通银行股份有限公司 | The processing method of finance market data and system |
| CN105554059A (en) * | 2015-11-25 | 2016-05-04 | 北京华油信通科技有限公司 | Logistics transportation intelligent perception and position service system based on Beidou navigation technology |
| CN109388504A (en) * | 2018-09-26 | 2019-02-26 | 平安科技（深圳）有限公司 | Messageization docks processing method, device, computer equipment and storage medium |
| CN109298990A (en) * | 2018-10-17 | 2019-02-01 | 平安科技（深圳）有限公司 | Log storing method, device, computer equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611251A (en) * | 2020-04-24 | 2020-09-01 | 华智众创(北京)投资管理有限责任公司 | Data processing system |
| CN111611251B (en) * | 2020-04-24 | 2021-06-29 | 华智众创(北京)投资管理有限责任公司 | Data processing system |
| CN113590590A (en) * | 2021-07-30 | 2021-11-02 | 中国工商银行股份有限公司 | Database calculation method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111831420B (en) | Method for task scheduling, related device and computer program product | |
| CN108090731B (en) | Information processing method and device | |
| US12072828B2 (en) | Article, device, and techniques for serverless message stream processing | |
| US10693816B2 (en) | Communication methods and systems, electronic devices, and computer clusters | |
| CN107704269A (en) | A kind of method and system based on block chain generation block | |
| CN112988499A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN201993811U (en) | Cloud fusion business intelligence terminal | |
| CN105871959A (en) | Message delivery method, system and device | |
| CN111209310B (en) | Service data processing method and device based on stream computing and computer equipment | |
| CN104951358A (en) | Priority based on context preemption | |
| CN110377268A (en) | Serial number generation method, device and storage medium | |
| CN111782365A (en) | Timed task processing method, device, equipment and storage medium | |
| CN114564435A (en) | Inter-core communication method, device and medium for heterogeneous multi-core chip | |
| CN110221927A (en) | Asynchronous message processing method and device | |
| CN109033814A (en) | intelligent contract triggering method, device, equipment and storage medium | |
| CN110413696A (en) | A kind of data processing method and device | |
| CN110471787A (en) | Problem catching method, system, device, electronic equipment and storage medium on line | |
| CN109471893A (en) | Querying method, equipment and the computer readable storage medium of network data | |
| CN108228330A (en) | The multi-process method for scheduling task and device of a kind of serialization | |
| CN111683140B (en) | Method and apparatus for distributing messages | |
| CN111290842A (en) | Task execution method and device | |
| CN105991416B (en) | A kind of message delivery method and messaging middleware | |
| CN113723893A (en) | Method and apparatus for processing an order | |
| CN110955461B (en) | Computing task processing methods, devices, systems, servers and storage media | |
| CN110147984A (en) | A kind of business transmission target processing method and processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191105 |