CN105978968A - Real-time transmission processing method, server and system of mass data - Google Patents
Real-time transmission processing method, server and system of mass data Download PDFInfo
- Publication number
- CN105978968A CN105978968A CN201610309434.9A CN201610309434A CN105978968A CN 105978968 A CN105978968 A CN 105978968A CN 201610309434 A CN201610309434 A CN 201610309434A CN 105978968 A CN105978968 A CN 105978968A
- Authority
- CN
- China
- Prior art keywords
- data
- real
- consuming
- thread
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/289—Intermediate processing functionally located close to the data consumer application, e.g. in same machine, in same home or in same sub-network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a real-time transmission processing method, server and system of mass data. The method comprises that data, which is sent from a client on the basis of a data one-way transmission protocol, is received; the received data is written in message middleware for caching to ensure that the data is complete in the transmission process; a consumption thread group is started, and different consumption threads in the consumption thread group are in the idle task-waiting state after being started; and data is taken from the message middleware and converted into processing tasks, and the processing tasks are distributed to the different consumption threads in the consumption thread group for processing; a result data is stored in a caching queue after the processing tasks are completed by the consumption threads; and a work thread group is started to obtain the result data from the caching queue and stores the result data in a database in a batch manner.
Description
Technical field
The invention belongs to data processing field, particularly relate to real-time Transmission processing method, server and the process of a kind of mass data
System.
Background technology
Computer and network technologies and the high speed development of intelligent terminal, data grow at top speed with a kind of immeasurable magnitude.And such as
What accepts and completely, quickly, safely, effectively processes mass data to be particularly important.At present, domestic data transmission system
System or product all compatibility can not completely, quickly, safely, effectively process these features of mass data.Generally, protecting
On the premise of integrity of verified data, data but can not be transmitted quickly and safely and preserve to data base；If compatible speed,
The integrity of data can not be ensured the most again.
Therefore, the real-time biography of a kind of mass data that can completely, quickly, safely, effectively process mass data is needed at present badly
Defeated processing method, server and processing system solve the problems referred to above.
Summary of the invention
In order to solve the shortcoming of prior art, the present invention provides real-time Transmission processing method, server and the place of a kind of mass data
Reason system.The present invention, by data one-way transmission, message-oriented middleware, multithreading and queue mechanism technology, solves current magnanimity
The problem that the complete and efficiency generally existed in data transmission procedure cannot get both, also ensure that data are in whole transmitting procedure
Safety problem.
For achieving the above object, the present invention is by the following technical solutions:
The invention provides the real-time Transmission processing method of a kind of mass data, the method includes:
Receive client sends, based on data Unidirectional Transport protocol, the data of coming；
The data of reception are write in message-oriented middleware and caches, ensure data integrity in transmitting procedure；
Expanding consumption sets of threads, each consuming thread in consuming thread group is in idle waiting task status after starting；
From message-oriented middleware, take out data, and change into the distribution of process task and enter to each consuming thread in consuming thread group
Row processes；
Result data after each consuming thread process task being completed preserves to buffer queue；
Start worker thread group, from buffer queue, obtain result data and batch stores to data base.
After each consuming thread group process task completes, from message-oriented middleware, take out data, and change into the distribution of process task
Each consuming thread to consuming thread group processes.
In the real-time Transmission processing method of the mass data of the present invention, client uses data Unidirectional Transport protocol to be transmitted number
According to, when data are by the i.e. High Security Level end of client transmissions to Intranet, the data of Intranet will not be revealed, thus ensure the safety of data
Property.
The data of reception are also write in message-oriented middleware and cache by the present invention, this ensure that high concurrent data is in transmitting procedure
In integrity, and different business data, message-oriented middleware can also cache according to different titles, greatly improves
Traffic handing capacity in data transmission procedure.
When each consuming thread group process task complete after and message-oriented middleware in no data time, consuming thread group is in idle waiting
Task status.
In the real-time Transmission processing method of the mass data of the present invention, call the corresponding interface and batch data is stored to different pieces of information
Storehouse.
Owing to the final purpose of data transmission is data to be stored, therefore, by configuring and calling distinct interface, then can be by
Data are batch write in one or more data bases, improves the development efficiency of Data Transport Protocol.
Determine to start the number of worker thread in worker thread group by configuration parameter.
Starting each worker thread, each thread waits task in queue；In a factory, there is a lot of workmans, task is put
In a queue, those workmans will obtain task automatically, after each process task completes, again waits new in queue
Task.When having task, taking-up task processes；Time without task, idle waiting.Each task can only be taken by a thread
Go out and process.Queue mechanism is the means functions reliably and efficiently solving multi-threaded parallel collaborative work.
Present invention also offers the server that a kind of real-time Transmission for mass data processes, including:
Data reception module, its data come based on the transmission of data Unidirectional Transport protocol being configured to receive client；
Data write. module, it is configured to write in message-oriented middleware the data of reception cache, and ensures that data are passing
Integrity during defeated；
Consuming thread starts module, and it is configured to sets of threads of expanding consumption, after each consuming thread in consuming thread group starts
It is in idle waiting task status；
Data transfer distribution module, and it is configured to from message-oriented middleware take out data, and changes into the distribution of process task to disappearing
Each consuming thread in expense sets of threads processes；
Data line up cache module, and its result data being configured to after each consuming thread process task being completed preserves to caching
In queue；
Worker thread starts module, and it is configured to start worker thread group, obtains result data and batch is deposited from buffer queue
Store up to data base.
Start in module at consuming thread, when each consuming thread group processes after task completes and during no data in message-oriented middleware,
Consuming thread group is in idle waiting task status.
The server that the described real-time Transmission for mass data processes also includes interface interchange module, and it is configured to call accordingly
Batch data is stored to disparate databases by interface.
Described worker thread starts module and determines to start the number of worker thread in worker thread group by configuration parameter.
Described worker thread starts module and is adjusted starting according further to the scale of data and the disposal ability of described server
The number of worker thread in worker thread group.
Additionally, present invention also offers a kind of real-time Transmission processing system for mass data, this processing system includes above-mentioned use
In the server that the real-time Transmission of mass data processes.
The invention have the benefit that
(1) in the real-time Transmission processing method of the mass data of the present invention, client uses data Unidirectional Transport protocol to carry out sending out
Sending data, when data are by the i.e. High Security Level end of client transmissions to Intranet, the data of Intranet will not be revealed, thus ensure data
Safety；
(2) data of reception are also write in message-oriented middleware and cache by the present invention, this ensure that high concurrent data is in transmission
During integrity, and different business data, message-oriented middleware can also cache according to different titles, greatly
Improve the traffic handing capacity in data transmission procedure；
(3) present invention also uses multithread mode, and sets of threads of expanding consumption reception processes task and the process of each consuming thread appointed
Result data after being engaged in preserves to buffer queue, and starts worker thread group, obtains result data from buffer queue
And batch stores to data base；In conjunction with lining up mechanism, put into each task queue after data being processed when neutralizing order from queue
Reading data to carry out storing in batches, queue mechanism solves reliability and the high efficiency of multi-threaded parallel collaborative work；
(4) present invention replaces traditional working method in the way of multi-threaded parallel work, and can adjust maximum by configuration file
Active line number of passes limit；And allow user according to the scale of real data, tuning parameter, reach the optimal of actual environment
Performance.
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of the real-time Transmission processing method of the mass data of the present invention；
Fig. 2 is the structural representation of the server of the real-time Transmission process for mass data of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely retouched
State, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole embodiments.Based on the present invention
In embodiment, the every other embodiment that those of ordinary skill in the art are obtained under not making creative work premise,
Broadly fall into the scope of protection of the invention.
The client mentioned in the embodiment of the present invention can include PC, panel computer and smart mobile phone.The embodiment of the present invention
In server can be directly in communication with each other with client by network.
Fig. 1 is the schematic flow sheet of the real-time Transmission processing method of a kind of mass data in the embodiment of the present invention, and the present embodiment is main
It is to describe from server-side, the real-time Transmission processing method of the mass data in the present embodiment as shown in the figure, including:
S101, receive client sends, based on data Unidirectional Transport protocol, the data of coming；
In implementing, data Unidirectional Transport protocol (unidirectional file delivery of such as TFTP, the unidirectional file that client uses
Host-host protocol FLUTE transmits), carry out transmitting data.So make data when the i.e. High Security Level end of client transmissions to Intranet,
The data of Intranet will not be revealed, thus ensures the safety of data.
The data of reception are write in message-oriented middleware and cache, ensure data integrity in transmitting procedure by S102；
In implementing, message-oriented middleware is as a example by the distributed message system kafka developed by LinkedIn:
By kafka, can be cached by high concurrent data, client can progressively be consumed, it is ensured that data are in transmitting procedure
In integrity.Different business data, can be cached by TOPIC titles different in kafka again, greatly increase
Traffic handing capacity in this data transmission procedure.
S103, sets of threads of expanding consumption, each consuming thread in consuming thread group is in idle waiting task status after starting；
In implementing, after each consuming thread group process task completes, from message-oriented middleware, take out data, and change into
The distribution of process task processes to each consuming thread in consuming thread group.
S104, takes out data from message-oriented middleware, and changes into each consumption line that process task is distributed to consuming thread group
Journey processes；
At this in overload, receiving data form and there is inconsistent situation with form during storage, user can be according to business demand
Self-defining.
In implementing, when each consuming thread group process task complete after and message-oriented middleware in no data time, consuming thread group
It is in idle waiting task status.
S105, the result data after each consuming thread process task being completed preserves to buffer queue.
After each consuming thread group process task completes, from message-oriented middleware, take out data, and change into the distribution of process task
Each consuming thread to consuming thread group processes.
S106, starts worker thread group, obtains result data and batch stores to data base from buffer queue.
In implementing, also call the corresponding interface and batch data is stored to disparate databases.
Owing to the final purpose of data transmission is data to be stored, therefore, by configuring and calling distinct interface, then can be by
Data are batch write in one or more data bases, improves the development efficiency of Data Transport Protocol.
Additionally, the present embodiment determines to start the number of worker thread in worker thread group also by configuration parameter.
Starting each worker thread, each thread waits task in queue；In a factory, there is a lot of workmans, task is put
In a queue, those workmans will obtain task automatically, after each process task completes, again waits new in queue
Task.When having task, taking-up task processes；Time without task, idle waiting.Each task can only be taken by a thread
Go out and process.Queue mechanism is the means functions reliably and efficiently solving multi-threaded parallel collaborative work.
In the present embodiment, client uses data Unidirectional Transport protocol to be transmitted data, in data by client transmissions to Intranet
I.e. during High Security Level end, the data of Intranet will not be revealed, thus ensures the safety of data；
The present embodiment also uses multithread mode, and sets of threads of expanding consumption reception processes task and each consuming thread is processed task
Result data after completing preserves to buffer queue, and starts worker thread group, obtains result data also from buffer queue
Batch stores to data base；In conjunction with lining up mechanism, put into each task queue after data being processed when neutralizing order reading from queue
Fetching data and carry out storing in batches, queue mechanism solves reliability and the high efficiency of multi-threaded parallel collaborative work；
The present embodiment replaces traditional working method in the way of multi-threaded parallel work, and can adjust maximum by configuration file
Active line number of passes limits；And allow user according to the scale of real data, tuning parameter, reach the optimality of actual environment
Energy.
Fig. 2 is the structural representation of the server that a kind of real-time Transmission for mass data processes in the embodiment of the present invention, this
The client mentioned in bright embodiment can include PC, panel computer and smart mobile phone.Service in the embodiment of the present invention
Device can be directly in communication with each other with client by network.
As shown in Figure 2 in the present embodiment for mass data real-time Transmission process server, including data reception module,
Data write. module, consuming thread start module, data transfer distribution module, data line up cache module and worker thread starts
This six big module of module.
Wherein, data reception module, its data come based on the transmission of data Unidirectional Transport protocol being configured to receive client.
Data Unidirectional Transport protocol (unidirectional file delivery of such as TFTP, unidirectional file delivery agreement FLUTE that client uses
Transmission), carry out transmitting data.So making data when the i.e. High Security Level end of client transmissions to Intranet, the data of Intranet will not
Reveal, thus ensure the safety of data.
Data write. module, it is configured to write in message-oriented middleware the data of reception cache, and ensures that data are passing
Integrity during defeated.
In implementing, message-oriented middleware is as a example by the distributed message system kafka developed by LinkedIn:
By kafka, can be cached by high concurrent data, client can progressively be consumed, it is ensured that data are in transmitting procedure
In integrity.Different business data, can be cached by TOPIC titles different in kafka again, greatly increase
Traffic handing capacity in this data transmission procedure.
Consuming thread starts module, and it is configured to sets of threads of expanding consumption, after each consuming thread in consuming thread group starts
It is in idle waiting task status.
In implementing, when each consuming thread group process task complete after and message-oriented middleware in no data time, consuming thread group
It is in idle waiting task status.
Start in module at consuming thread, when each consuming thread group processes after task completes and during no data in message-oriented middleware,
Consuming thread group is in idle waiting task status.
Data transfer distribution module, and it is configured to from message-oriented middleware take out data, and changes into the distribution of process task to disappearing
Each consuming thread in expense sets of threads processes.
Data line up cache module, and its result data being configured to after each consuming thread process task being completed preserves to caching
In queue.
Worker thread starts module, and it is configured to start worker thread group, obtains result data and batch is deposited from buffer queue
Store up to data base.
The server that the real-time Transmission for mass data in the present embodiment processes also includes interface interchange module, and it is configured to
Call the corresponding interface to store batch data to disparate databases.
Wherein, worker thread starts module and determines to start the number of worker thread in worker thread group by configuration parameter.
Preferably, worker thread startup module is adjusted according further to the scale of data and the disposal ability of described server
Start the number of worker thread in worker thread group.
In the present embodiment, client uses data Unidirectional Transport protocol to be transmitted data, in data by client transmissions to Intranet
I.e. during High Security Level end, the data of Intranet will not be revealed, thus ensures the safety of data；
The present embodiment also uses multithread mode, and sets of threads of expanding consumption reception processes task and each consuming thread is processed task
Result data after completing preserves to buffer queue, and starts worker thread group, obtains result data also from buffer queue
Batch stores to data base；In conjunction with lining up mechanism, put into each task queue after data being processed when neutralizing order reading from queue
Fetching data and carry out storing in batches, queue mechanism solves reliability and the high efficiency of multi-threaded parallel collaborative work；
The present embodiment replaces traditional working method in the way of multi-threaded parallel work, and can adjust maximum by configuration file
Active line number of passes limits；And allow user according to the scale of real data, tuning parameter, reach the optimality of actual environment
Energy.
Additionally, present invention also offers a kind of real-time Transmission processing system for mass data, this processing system includes above-mentioned use
In the server that the real-time Transmission of mass data processes.Specifically the concrete structure of this server will be not repeated.
Present invention also offers a kind of real-time Transmission processing system for mass data, receive client and use data one-way transmission
Agreement is transmitted data, and when data are by the i.e. High Security Level end of client transmissions to Intranet, the data of Intranet will not be revealed, thus
Ensure the safety of data；
Also the data write message-oriented middleware of reception will cache, this ensure that complete in transmitting procedure of high concurrent data
Whole property, and different business data, message-oriented middleware can also cache according to different titles, greatly increases data
Traffic handing capacity in transmitting procedure；
The present invention also uses multithread mode, and sets of threads of expanding consumption reception processes task and by complete for each consuming thread process task
Result data after one-tenth preserves to buffer queue, and starts worker thread group, obtains result data and criticize from buffer queue
Amount stores to data base；In conjunction with lining up mechanism, put into each task queue after data being processed when neutralizing order reading from queue
Data carry out batch and store, and queue mechanism solves reliability and the high efficiency of multi-threaded parallel collaborative work；
The present invention replaces traditional working method in the way of multi-threaded parallel work, and can adjust maximum work by configuration file
Make Thread Count to limit；And allow user according to the scale of real data, tuning parameter, reach the optimum performance of actual environment.
One of ordinary skill in the art will appreciate that all or part of flow process realizing in above-described embodiment method, can be by meter
Calculation machine program instructs relevant hardware and completes, and described program can be stored in a computer read/write memory medium, this journey
Sequence is upon execution, it may include such as the flow process of the embodiment of above-mentioned each method.Wherein, described storage medium can be magnetic disc, CD,
Read-only store-memory body (Read-Only Memory, ROM) or at random store-memory body (Random AccessMemory,
RAM) etc..
Although the detailed description of the invention of the present invention is described by the above-mentioned accompanying drawing that combines, but not limit to scope
System, one of ordinary skill in the art should be understood that on the basis of technical scheme, and those skilled in the art need not pay
Go out various amendments or deformation that creative work can make still within protection scope of the present invention.
Claims (10)
1. the real-time Transmission processing method of a mass data, it is characterised in that including:
Receive client sends, based on data Unidirectional Transport protocol, the data of coming；
The data of reception are write in message-oriented middleware and caches, ensure data integrity in transmitting procedure；
Expanding consumption sets of threads, each consuming thread in consuming thread group is in idle waiting task status after starting；
From message-oriented middleware, take out data, and change into the distribution of process task and enter to each consuming thread in consuming thread group
Row processes；
Result data after each consuming thread process task being completed preserves to buffer queue；
Start worker thread group, from buffer queue, obtain result data and batch stores to data base.
2. a kind of mass data live transmission method as claimed in claim 1, it is characterised in that at each consuming thread group
After reason task completes and in message-oriented middleware during no data, consuming thread group is in idle waiting task status.
3. a kind of mass data live transmission method as claimed in claim 1, it is characterised in that call the corresponding interface by data
Batch stores to disparate databases.
4. a kind of mass data live transmission method as claimed in claim 1, it is characterised in that determine to open by configuration parameter
Start building to make the number of worker thread in sets of threads.
5. the server that the real-time Transmission for mass data processes, it is characterised in that including:
Data reception module, its data come based on the transmission of data Unidirectional Transport protocol being configured to receive client；
Data write. module, it is configured to write in message-oriented middleware the data of reception cache, and ensures that data are passing
Integrity during defeated；
Consuming thread starts module, and it is configured to sets of threads of expanding consumption, after each consuming thread in consuming thread group starts
It is in idle waiting task status；
Data transfer distribution module, and it is configured to from message-oriented middleware take out data, and changes into the distribution of process task to disappearing
Each consuming thread in expense sets of threads processes；
Data line up cache module, and its result data being configured to after each consuming thread process task being completed preserves to caching
In queue；
Worker thread starts module, and it is configured to start worker thread group, obtains result data and batch is deposited from buffer queue
Store up to data base.
The server that a kind of real-time Transmission for mass data processes, it is characterised in that disappearing
Take thread to start in module, when each consuming thread group processes after task completes and during no data in message-oriented middleware, consuming thread
Group is in idle waiting task status.
The server that a kind of real-time Transmission for mass data processes, it is characterised in that described
The server processed for the real-time Transmission of mass data also includes interface interchange module, and it is configured to call the corresponding interface by number
Store to disparate databases according to batch.
The server that a kind of real-time Transmission for mass data processes, it is characterised in that described
Worker thread starts module and determines to start the number of worker thread in worker thread group by configuration parameter.
The server that a kind of real-time Transmission for mass data processes, it is characterised in that described
Worker thread starts module and is adjusted starting worker thread according further to the scale of data and the disposal ability of described server
The number of worker thread in group.
10. the real-time Transmission processing system for mass data, it is characterised in that this processing system includes such as claim
The server that the arbitrary described real-time Transmission for mass data of 5-9 processes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610309434.9A CN105978968A (en) | 2016-05-11 | 2016-05-11 | Real-time transmission processing method, server and system of mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610309434.9A CN105978968A (en) | 2016-05-11 | 2016-05-11 | Real-time transmission processing method, server and system of mass data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105978968A true CN105978968A (en) | 2016-09-28 |
Family
ID=56991642
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610309434.9A Pending CN105978968A (en) | 2016-05-11 | 2016-05-11 | Real-time transmission processing method, server and system of mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105978968A (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025074A (en) * | 2017-04-25 | 2017-08-08 | 航天科技控股集团股份有限公司 | A kind of picture storage method based on recorder platform |
| CN107046471A (en) * | 2017-04-07 | 2017-08-15 | 苏州乐麟无线信息科技有限公司 | Processing method, device and the server system of metering data |
| CN107147635A (en) * | 2017-04-28 | 2017-09-08 | 上海斐讯数据通信技术有限公司 | The protocol analysis processing method and system for significantly improving the up file data of magnanimity based on multithreading |
| CN107172601A (en) * | 2017-04-20 | 2017-09-15 | 努比亚技术有限公司 | A kind of application message management platform and method |
| CN107239237A (en) * | 2017-06-28 | 2017-10-10 | 阿里巴巴集团控股有限公司 | Method for writing data and device and electronic equipment |
| CN107273542A (en) * | 2017-07-06 | 2017-10-20 | 华泰证券股份有限公司 | High concurrent method of data synchronization and system |
| CN107590210A (en) * | 2017-08-25 | 2018-01-16 | 咪咕互动娱乐有限公司 | A kind of data processing method, device, system and computer-readable recording medium |
| CN107633034A (en) * | 2017-09-08 | 2018-01-26 | 郑州金润高科电子有限公司 | A kind of parallel store method of more buffer queues |
| CN107886352A (en) * | 2017-10-27 | 2018-04-06 | 微梦创科网络科技（中国）有限公司 | A kind of method and system of advertisement clearing |
| CN108170758A (en) * | 2017-12-22 | 2018-06-15 | 福建天泉教育科技有限公司 | High concurrent date storage method and computer readable storage medium |
| CN108737292A (en) * | 2018-04-18 | 2018-11-02 | 千寻位置网络有限公司 | A kind of sending method and system, server of bulk messages |
| CN109117286A (en) * | 2018-07-30 | 2019-01-01 | 佛山市甜慕链客科技有限公司 | A kind of method of data collection and adjusting |
| CN109873904A (en) * | 2019-03-05 | 2019-06-11 | 平安科技（深圳）有限公司 | User of message status report process method, device and the storage medium of message push |
| CN110213317A (en) * | 2018-07-18 | 2019-09-06 | 腾讯科技（深圳）有限公司 | The method, apparatus and storage medium of message storage |
| CN111831446A (en) * | 2020-07-15 | 2020-10-27 | 北京思特奇信息技术股份有限公司 | Data read-write method, device and medium based on Disproptor |
| CN112104731A (en) * | 2020-09-11 | 2020-12-18 | 北京奇艺世纪科技有限公司 | Request processing method and device, electronic equipment and storage medium |
| CN112527844A (en) * | 2020-12-22 | 2021-03-19 | 北京明朝万达科技股份有限公司 | Data processing method and device and database architecture |
| CN112597234A (en) * | 2020-12-29 | 2021-04-02 | 拉卡拉支付股份有限公司 | Data transmission method, data transmission device, electronic device, storage medium, and program product |
| CN113110940A (en) * | 2021-03-02 | 2021-07-13 | 武汉木仓科技股份有限公司 | Method and system for storing problem-making data of students in driving schools |
| CN113572704A (en) * | 2020-04-29 | 2021-10-29 | 北京沃东天骏信息技术有限公司 | Information processing method, production end, consumption end and server |
| CN114172846A (en) * | 2021-12-03 | 2022-03-11 | 北京云迹科技有限公司 | Kafka platform-based data current limiting method and device, electronic equipment and storage medium |
| CN115905268A (en) * | 2023-01-03 | 2023-04-04 | 北京力控元通科技有限公司 | Queue-based ammeter data processing method and system and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102025653A (en) * | 2010-06-04 | 2011-04-20 | 西本新干线股份有限公司 | Enterprise service bus and message processing method thereof |
| CN103870337A (en) * | 2014-02-28 | 2014-06-18 | 浪潮集团山东通用软件有限公司 | ESB assembly realization method based on SEDA |
| CN104216899A (en) * | 2013-05-31 | 2014-12-17 | 济南观澜数据技术有限公司 | Mass-unstructured data distributed type processing structure for description information |
| CN104935658A (en) * | 2015-06-17 | 2015-09-23 | 交通银行股份有限公司 | Bank task allocation method and device |
| CN105373420A (en) * | 2014-08-28 | 2016-03-02 | 北京奇虎科技有限公司 | Data transmission method and apparatus |
| CN105468705A (en) * | 2015-11-18 | 2016-04-06 | 广东南方通信建设有限公司 | Mobile communication background data file importing method |
-
2016
- 2016-05-11 CN CN201610309434.9A patent/CN105978968A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102025653A (en) * | 2010-06-04 | 2011-04-20 | 西本新干线股份有限公司 | Enterprise service bus and message processing method thereof |
| CN104216899A (en) * | 2013-05-31 | 2014-12-17 | 济南观澜数据技术有限公司 | Mass-unstructured data distributed type processing structure for description information |
| CN103870337A (en) * | 2014-02-28 | 2014-06-18 | 浪潮集团山东通用软件有限公司 | ESB assembly realization method based on SEDA |
| CN105373420A (en) * | 2014-08-28 | 2016-03-02 | 北京奇虎科技有限公司 | Data transmission method and apparatus |
| CN104935658A (en) * | 2015-06-17 | 2015-09-23 | 交通银行股份有限公司 | Bank task allocation method and device |
| CN105468705A (en) * | 2015-11-18 | 2016-04-06 | 广东南方通信建设有限公司 | Mobile communication background data file importing method |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107046471A (en) * | 2017-04-07 | 2017-08-15 | 苏州乐麟无线信息科技有限公司 | Processing method, device and the server system of metering data |
| CN107172601A (en) * | 2017-04-20 | 2017-09-15 | 努比亚技术有限公司 | A kind of application message management platform and method |
| CN107025074A (en) * | 2017-04-25 | 2017-08-08 | 航天科技控股集团股份有限公司 | A kind of picture storage method based on recorder platform |
| CN107147635A (en) * | 2017-04-28 | 2017-09-08 | 上海斐讯数据通信技术有限公司 | The protocol analysis processing method and system for significantly improving the up file data of magnanimity based on multithreading |
| CN107239237A (en) * | 2017-06-28 | 2017-10-10 | 阿里巴巴集团控股有限公司 | Method for writing data and device and electronic equipment |
| CN107273542B (en) * | 2017-07-06 | 2020-11-27 | 华泰证券股份有限公司 | High-concurrency data synchronization method and system |
| CN107273542A (en) * | 2017-07-06 | 2017-10-20 | 华泰证券股份有限公司 | High concurrent method of data synchronization and system |
| CN107590210A (en) * | 2017-08-25 | 2018-01-16 | 咪咕互动娱乐有限公司 | A kind of data processing method, device, system and computer-readable recording medium |
| CN107633034A (en) * | 2017-09-08 | 2018-01-26 | 郑州金润高科电子有限公司 | A kind of parallel store method of more buffer queues |
| CN107633034B (en) * | 2017-09-08 | 2020-12-01 | 河南德朗智能科技有限公司 | Multi-buffer queue parallel storage method |
| CN107886352A (en) * | 2017-10-27 | 2018-04-06 | 微梦创科网络科技（中国）有限公司 | A kind of method and system of advertisement clearing |
| CN107886352B (en) * | 2017-10-27 | 2021-11-19 | 微梦创科网络科技（中国）有限公司 | Advertisement settlement method and system |
| CN108170758A (en) * | 2017-12-22 | 2018-06-15 | 福建天泉教育科技有限公司 | High concurrent date storage method and computer readable storage medium |
| CN108737292A (en) * | 2018-04-18 | 2018-11-02 | 千寻位置网络有限公司 | A kind of sending method and system, server of bulk messages |
| CN110213317A (en) * | 2018-07-18 | 2019-09-06 | 腾讯科技（深圳）有限公司 | The method, apparatus and storage medium of message storage |
| CN110213317B (en) * | 2018-07-18 | 2021-10-29 | 腾讯科技（深圳）有限公司 | Message storage method, device and storage medium |
| CN109117286A (en) * | 2018-07-30 | 2019-01-01 | 佛山市甜慕链客科技有限公司 | A kind of method of data collection and adjusting |
| CN109873904A (en) * | 2019-03-05 | 2019-06-11 | 平安科技（深圳）有限公司 | User of message status report process method, device and the storage medium of message push |
| CN113572704A (en) * | 2020-04-29 | 2021-10-29 | 北京沃东天骏信息技术有限公司 | Information processing method, production end, consumption end and server |
| CN111831446A (en) * | 2020-07-15 | 2020-10-27 | 北京思特奇信息技术股份有限公司 | Data read-write method, device and medium based on Disproptor |
| CN112104731A (en) * | 2020-09-11 | 2020-12-18 | 北京奇艺世纪科技有限公司 | Request processing method and device, electronic equipment and storage medium |
| CN112104731B (en) * | 2020-09-11 | 2022-05-20 | 北京奇艺世纪科技有限公司 | Request processing method and device, electronic equipment and storage medium |
| CN112527844A (en) * | 2020-12-22 | 2021-03-19 | 北京明朝万达科技股份有限公司 | Data processing method and device and database architecture |
| CN112597234A (en) * | 2020-12-29 | 2021-04-02 | 拉卡拉支付股份有限公司 | Data transmission method, data transmission device, electronic device, storage medium, and program product |
| CN113110940A (en) * | 2021-03-02 | 2021-07-13 | 武汉木仓科技股份有限公司 | Method and system for storing problem-making data of students in driving schools |
| CN114172846A (en) * | 2021-12-03 | 2022-03-11 | 北京云迹科技有限公司 | Kafka platform-based data current limiting method and device, electronic equipment and storage medium |
| CN115905268A (en) * | 2023-01-03 | 2023-04-04 | 北京力控元通科技有限公司 | Queue-based ammeter data processing method and system and electronic equipment |
| CN115905268B (en) * | 2023-01-03 | 2023-09-29 | 北京力控元通科技有限公司 | Ammeter data processing method and system based on queue and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105978968A (en) | Real-time transmission processing method, server and system of mass data | |
| CN102469064B (en) | Realization Method of Communication and communication equipment | |
| CN101719929A (en) | Method for realizing real-time data transmission under Web Service | |
| CN103207785B (en) | The processing method of data download request, Apparatus and system | |
| CN111580995A (en) | Synchronous communication method and system of distributed cloud platform and Internet of things intelligent terminal based on MQTT asynchronous communication scene | |
| CN110134534B (en) | System and method for optimizing message processing for big data distributed system based on NIO | |
| CN105516086B (en) | Method for processing business and device | |
| CN103036961A (en) | Distributed collection and storage method of journal | |
| CN113259415B (en) | Network message processing method and device and network server | |
| CN101917394B (en) | Middleware system for sharing data in mobile phone equipment and working method | |
| CN209297373U (en) | Online patrol system based on Cloud Server and bluetooth inspection point device | |
| CN106850698A (en) | A kind of User space RPC agreement multithreading optimization methods and system | |
| CN102394880A (en) | Method and device for processing jump response in content delivery network | |
| CN108631955A (en) | It is a kind of to ensure that message sends reachable mthods, systems and devices | |
| CN102447628A (en) | Data packet transmission processing method and system | |
| CN104683472A (en) | Data transmission method supporting large data volume | |
| CN105933380A (en) | Data transmission method, device and terminal | |
| CN108900627B (en) | Network request method, terminal device and storage medium | |
| CN110071968B (en) | Block chain based message storage method and device | |
| CN110400043A (en) | Order allocation method, device, equipment and storage medium | |
| CN103578027A (en) | Method and device for transferring accounts | |
| CN106131162A (en) | A kind of method realizing network service agent based on IOCP mechanism | |
| CN107666430B (en) | E-mail sending method, device and terminal | |
| CN106254448B (en) | A kind of information acquisition method and device | |
| CN101155370B (en) | Business serial processing method and system in multi-stream application of mobile communication system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160928 |