CN110659270A - Data processing and transmitting method and device - Google Patents
Data processing and transmitting method and device Download PDFInfo
- Publication number
- CN110659270A CN110659270A CN201910764376.2A CN201910764376A CN110659270A CN 110659270 A CN110659270 A CN 110659270A CN 201910764376 A CN201910764376 A CN 201910764376A CN 110659270 A CN110659270 A CN 110659270A
- Authority
- CN
- China
- Prior art keywords
- real
- time
- data
- source data
- time source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a data processing and transmitting method and device. The method comprises the following steps: real-time source data are collected in real time, and non-real-time source data are collected in fixed time; calculating the real-time source data in real time to generate real-time data, and calculating the non-real-time source data in an off-line manner to generate non-real-time data; and pushing the real-time data and the non-real-time data to a client through a bidirectional long connection. The invention can acquire real-time source data in time, and can synchronize the real-time data in real time and send the real-time data to the effective client after real-time calculation and backup are carried out on the real-time source data; the invention processes and calculates the non-real-time source data acquired from the public network, writes the non-real-time data into the database at regular time, synchronizes the incremental change data to the effective client in real time after the non-real-time data is prepared, can improve the timeliness of data service to the greatest extent, can reduce the data transmission amount due to the adoption of the incremental data, and can alleviate the system congestion caused by large data amount.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for data processing and transmission.
Background
At present, the demand for data in the application market is increasing day by day, and in order to ensure the timeliness quality of data service, a client usually needs to call the data from a third-party database. Data calling of the existing third-party database and the client is performed in the mode of the third-party database of the timing wheel comply with, but by adopting the calling mode, data acquisition is prone to be out of time, and if the transmitted data volume is too large, data transmission congestion is prone to being caused. Moreover, the existing third-party database lacks an exception handling mechanism of data, and once the calculation fails, the database is paralyzed, so that the use is influenced.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present invention provide a data processing and transmitting method and apparatus. The technical scheme is as follows:
in one aspect, a data processing and transmitting method is provided, where the method includes:
real-time source data are collected in real time, and non-real-time source data are collected in fixed time;
calculating the real-time source data in real time to generate real-time data, and calculating the non-real-time source data in an off-line manner to generate non-real-time data;
and pushing the real-time data and the non-real-time data to a client through a bidirectional long connection.
Further, the calculating the real-time source data in real time to generate real-time data includes:
the real-time source data is backed up in a distributed off-line manner;
calculating the real-time source data in a distributed manner in real time to generate the real-time data;
and carrying out distributed offline backup on the real-time data.
Further, after the offline backup of the real-time source data, the obtaining of the real-time data further includes:
calculating the real-time source data in an off-line manner to generate an off-line calculation result;
and when the real-time computation is abnormal, the offline computation result is pushed to the client in real time instead of the real-time data.
Further, the performing offline computation on the non-real-time source data to generate non-real-time data includes:
offline backing up the non-real-time source data;
calculating the non-real-time source data in an off-line manner to generate the non-real-time data;
and backing up and storing the non-real-time data offline.
Further, the pushing the real-time data and the non-real-time data to the client through the bidirectional long connection includes:
receiving an authentication request sent by the client;
and auditing the authentication request, and if the authentication request passes the audit, pushing the real-time data and the non-real-time data to the client through the bidirectional long connection.
Further, the offline backup of the real-time source data and the real-time data comprises:
and respectively writing the real-time source data and the real-time data into a first data table and a second data table and reserving the data to a preset time limit.
Further, the offline backup of the non-real-time source data and the non-real-time data includes:
and respectively writing the non-real-time source data and the non-real-time data into a third data table and a fourth data table.
In another aspect, a data processing and transmitting apparatus is provided, the apparatus including:
the data acquisition module is used for acquiring real-time source data in real time and acquiring non-real-time source data in a timing mode;
the real-time data acquisition module is used for calculating real-time source data in real time to generate real-time data;
the non-real-time data acquisition module is used for performing off-line calculation on non-real-time source data to generate non-real-time data;
and the communication module is used for pushing the real-time data and the non-real-time data to a client through a bidirectional long connection.
Further, the real-time data acquisition module includes:
the real-time computing module is used for computing the real-time source data in real time and generating the real-time data;
the first backup module is used for backing up the real-time source data and the real-time data in an off-line manner;
the device further comprises: and the distributed message system is used for distributively transmitting the real-time source data and the real-time data.
Further, the real-time data calculation module further includes:
and the source data calculation module is used for performing off-line calculation on the real-time source data to generate an off-line calculation result to replace the real-time data.
Further, the non-real-time data acquisition module includes:
the off-line computing module is used for off-line computing the non-real-time source data and generating the non-real-time data;
the second backup module is used for backing up the non-real-time source data and the non-real-time data in an off-line mode;
a database for storing the non-real time data.
Further, the communication module is configured to receive an authentication request sent by the client; and auditing the authentication request, and if the authentication request passes the audit, pushing the real-time data and the non-real-time data to the client in real time through bidirectional long connection.
Further, the first backup module includes:
the first data table storage module is used for writing the real-time source data into a first data table and reserving the real-time source data to a preset time limit;
and the second data table storage module is used for writing the real-time data into a second data table and reserving the real-time data until a preset time limit.
Further, the second backup module includes:
the third data table storage module is used for writing the non-real-time source data into a third data table;
and the fourth data table storage module is used for writing the non-real-time data into a fourth data table.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
1. the embodiment of the invention can acquire real-time source data in time, and can synchronize the real-time data to the client in real time after real-time calculation and backup are carried out on the real-time source data, so that the condition that the real-time data is not updated in time due to the client timing wheel comply with database in the prior art is avoided;
2. according to the embodiment of the invention, after the non-real-time source data acquired from the public network at regular time is processed and calculated, the non-real-time data is written into the database, and after the non-real-time data is prepared, incremental change data is synchronized to the registered effective client in real time, so that the problem that the transmission quantity of the non-real-time data is large, and the system is blocked can be avoided;
3. the embodiment of the invention is provided with an on-line mechanism when the real-time source data is abnormal in calculation, the backup real-time source data is adopted for off-line calculation, and the off-line calculation result is transmitted to the server instead of the real-time data calculated in real time, so that the normal operation of the system is ensured.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a data processing and transmitting method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a functional module of a data processing and transmitting device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to overcome the defects that data of an existing third-party database is not provided timely and congestion faults are prone to occurring, the embodiment of the invention discloses a data processing and transmitting method and device.
As shown in fig. 1, a data processing and transmitting method includes:
real-time source data are collected in real time, and non-real-time source data are collected in fixed time;
calculating the real-time source data in real time to generate real-time data, and calculating the non-real-time source data in an off-line manner to generate non-real-time data;
and pushing the real-time data and the non-real-time data to a client in real time through bidirectional long connection.
It should be noted that, in the above method, acquiring real-time source data in real time includes: the embodiment of the invention adopts a multi-source acquisition mode to acquire data on different data sources and mutually verify the data so as to ensure the accuracy and the authenticity of the data in order to ensure the stability and the reliability of the acquired data sources by utilizing the real-time changing data information acquired from a public network by using a crawler technology. The timing acquisition of the non-real-time source data comprises the following steps: and (3) adopting a multi-source data acquisition mode at preset time, crawling non-real-time source data from the public network by using a crawler technology, and storing multiple copies of source data acquired by the crawler in a MySQL database of the crawler system. Based on the difference between the real-time source data and the non-real-time source data, after a user makes a subscription request, the client establishes long connection with the server, real-time data prepared in real time is pushed to the client in real time, and the regularly prepared non-real-time data is also pushed to the client immediately after being prepared. The real-time source data refers to real-time information on a public network, and the real-time calculation of the real-time source data may include: and integrating real-time source data on the public network, and combining a preset calculation analysis method to form data information based on the real-time source data, namely the real-time data. For example, in the financial field, for the real-time fund valuation service, the real-time source data may include real-time market data of stocks from different sources, and the real-time calculation of the real-time source data may be multi-source integration (deduplication, cleaning, sorting) of the real-time market data, and the valuation of the current fund is calculated in real time through a specific formula in combination with valuation factors. The non-real-time source data refers to non-real-time information on a public network, and the offline calculation of the non-real-time source data may include: and (3) performing off-line integration (de-duplication, cleaning and sorting) on the non-real-time source data, and combining a preset calculation analysis method to form data information based on the non-real-time source data, namely the non-real-time data. For example, also in the financial field, for the construction of a fund database, the non-real-time information may be multi-source information that does not change in real-time, such as fund announcement information, fund funding, rates, fund associations, net worth, fund financial statements, fund manager information, fund company information, fund custodian information, and the like. The offline calculation of the non-real-time source data is mainly to analyze and evaluate the fund on the basis of the non-real-time source data, and the formed non-real-time data may include: fund diagnosis, fund multi-factor analysis, fund combination performance calculation, fund risk model and other strategic indicators.
The above method for calculating the real-time source data in real time to generate real-time data includes:
the real-time source data is backed up in a distributed off-line manner;
calculating the real-time source data in a distributed manner in real time to generate the real-time data;
and carrying out distributed offline backup on the real-time data.
In the method for acquiring real-time data, the distributed real-time calculation of real-time source data includes: and writing the real-time source data into the distributed message system, and then sequentially performing off-line backup and real-time calculation on the real-time source data according to the message queue. The distributed messaging system may be any distributed messaging system, such as: kafka. The real-time source data is input into the distributed message system, and the real-time source data can be sent in a distributed mode to guarantee that the subsequent steps are carried out in order. Real-time source data input to the distributed messaging system has two consumption directions, one for backup and one for real-time computation. The real-time calculation comprises the following steps: and performing real-time calculation processes such as cleaning, sorting, real-time processing calculation and the like on the real-time source data. Also distributed offline backup of real-time data includes: real-time data is generated after real-time calculation, and then the real-time data is input into the distributed message system, wherein the real-time data input into the distributed message system has two consumption directions, one is used for backup, and the other is directly synchronized to a registered effective client by means of a communication module. Further, if the system has an intranet user, the real-time data is synchronized to the intranet consumer in real time.
In the method, the offline backup real-time source data and the offline backup real-time data both utilize data table storage, specifically Hive table storage. The offline backup of the real-time source data comprises: a distributed log collection system is employed, for example: and flash, synchronizing the real-time source data in the distributed message system, and then storing the real-time source data in the Hive table form to form a first data table, and reserving the first data table for a preset number of days. The offline backup of real-time data is the same as the implementation source data: and synchronizing the real-time data in the distributed message system by using the distributed log collection system, and then storing the real-time data in the Hive table form to form a second data table and reserving the second data table for a preset number of days.
In order to solve the problem that real-time data cannot be normally provided due to the fact that real-time computing is abnormal, the embodiment of the invention utilizes the real-time source data of the off-line backup to perform off-line computing to generate an off-line computing result, and the off-line computing result can replace the real-time data to be synchronized to the client.
Performing offline calculation on the non-real-time source data to generate non-real-time data comprises:
offline backing up the non-real-time source data, and offline calculating the non-real-time source data to generate the non-real-time data;
and backing up and storing the non-real-time data offline.
The off-line calculation of the non-real-time source data comprises off-line calculation processes of cleaning, sorting, off-line processing calculation and the like of the non-real-time source data. Storing non-real-time data includes: the method comprises the steps of inputting non-real-time data into a database at regular time, preferably, storing the non-real-time data into a cache system after the non-real-time data is stored into the database, and issuing the non-real-time data to a client through the cache system. Here, the cache system may be: redis cache systems. It should be noted that, the cache module in the embodiment of the present invention is used to avoid repeatedly loading data from the database, reduce unnecessary database overhead, and improve data response efficiency, and only when the cache fails, relevant data is loaded from the database, and after the data is loaded, a broadcast signal is sent to the communication module, and with the assistance of the communication module, the relevant data is issued to the registered valid client.
In the above method, the process of offline backing up the non-real-time source data and the non-real-time data includes:
and respectively writing the non-real-time source data and the non-real-time data into a third data table and a fourth data table.
Specifically, writing the non-real-time source data into the third data table includes: and importing MySQL data into the Hive table by an ETL technology to form a third data table, and importing offline calculation results into the Hive table for storage after offline calculation to form a fourth data table.
It should be noted that, in the above method, when the client sends the subscription request and successfully receives the subscription request, the client establishes a bidirectional long link with the client. Specifically, a bidirectional long link is established between the client and the server in a WebSocket-based bidirectional long link manner. The server side adopts the token issuing mode, the white list mode and the like to ensure the safety of the access request of the client side.
On the other hand, the embodiment of the present invention discloses a data processing and transmitting apparatus based on the above data processing and transmitting method, including: the system comprises a data acquisition module, a real-time data acquisition module, a non-real-time data acquisition module, a communication module and a distributed message system.
And the data acquisition module is used for acquiring real-time source data in real time and acquiring non-real-time source data in a timing mode.
The real-time data acquisition module is used for calculating real-time source data in real time to generate real-time data, and comprises: the device comprises a real-time computing module, a source data computing module and a first backup module.
And the real-time computing module is used for computing the real-time source data in real time and generating the real-time data.
And the source data calculation module is used for performing off-line calculation on the real-time source data to generate an off-line calculation result to replace the real-time data.
And the first backup module is used for off-line backup of the real-time source data and the real-time data. The first backup module includes: the first data table storage module is used for writing the real-time source data into a first data table and reserving the real-time source data to a preset time limit; and the second data table storage module is used for writing the real-time data into a second data table and reserving the real-time data until a preset time limit.
And the distributed message system is used for distributively transmitting the real-time source data and the real-time data and related signals.
The non-real-time data acquisition module is used for acquiring non-real-time data according to the non-real-time source data offline calculation, and comprises: the device comprises an offline calculation module, a second backup module, a cache module and a database.
And the off-line calculation module is used for off-line calculating the non-real-time source data and generating the non-real-time data.
And the second backup module is used for off-line backup of the non-real-time source data and the non-real-time data. The offline calculation module comprises: the third data table module is used for writing the non-real-time source data into a third data table; and the fourth data table module is used for writing the non-real-time data into a fourth data table.
And the cache module is used for caching the non-real-time data.
And the database is used for storing the non-real-time data.
And the communication module is used for receiving the authentication request sent by the client and auditing the authentication request, and if the authentication request passes, the real-time data and the non-real-time data are pushed to the client in real time through the bidirectional long connection.
It should be noted that, in the above system, a bidirectional long link is established with a client, the real-time data and the non-real-time data are pushed to the client in real time, the client logs in and registers access requirements to obtain a pass certificate, and the pass certificate authentication is required to be implemented for obtaining data each time.
Fig. 2 shows data and signal transmission of each module of the data processing and transmitting device disclosed in the embodiment of the present invention. In the figure, solid arrows indicate data transfer, and dashed arrows indicate signal transfer. The first backup module, the second backup module, the source data calculation module and the off-line calculation module in the device all operate in an off-line calculation platform. The operation process of each module is as follows: the data acquisition module respectively acquires real-time source data and timing non-real-time source data from a public network in real time. The real-time source data respectively enter the first backup module and the distributed message system, the real-time source data entering the distributed message system are distributively transmitted to the real-time computing module to generate real-time data, the real-time data also respectively enter the first backup module and the distributed message system, and the real-time data in the distributed message system is issued to the registered effective client through the communication module. And if the real-time computing module operates abnormally, the real-time source data in the first backup module is sent to the source data computing module for off-line computing, and an off-line computing result is generated and used as real-time data which is synchronized to the client through the communication module by using the distributed message system and used as a compensation means. After the non-real-time source data are backed up in the second backup module, the non-real-time source data are generated through calculation of the offline calculation module, the non-real-time data are backed up through the second backup module and sent to the database for storage, the distributed message system calls the non-real-time data in the database, the non-real-time data are sent to the cache module to serve as a buffer, and finally the non-real-time data in the cache module are pushed to the client through the communication module.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
1. the embodiment of the invention can timely acquire the real-time source data, can synchronize the real-time data to the server in real time after the real-time source data is calculated and backed up in real time and send the real-time data to the client in real time, and avoids the situation that the real-time data is not updated timely caused by the server and the database of the client timing wheel comply with in the prior art;
2. according to the embodiment of the invention, after the non-real-time source data acquired from the public network at regular time is processed and calculated, the non-real-time data is cached and written into the database, and after the non-real-time data is prepared, incremental change data is synchronized to the client in real time, so that the problem that the transmission quantity of the non-real-time data is large, and the system is jammed can be avoided;
3. the embodiment of the invention is provided with an on-line mechanism when the real-time source data is abnormal in calculation, the backup real-time source data is adopted for off-line calculation, and the off-line calculation result is transmitted to the server instead of the real-time data calculated in real time, so that the normal operation of the system is ensured.
All the above-mentioned optional technical solutions can be combined arbitrarily to form the optional embodiments of the present invention, and are not described herein again.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A data processing and transmission method is characterized by comprising the following steps:
real-time source data are collected in real time, and non-real-time source data are collected in fixed time;
calculating the real-time source data in real time to generate real-time data, and calculating the non-real-time source data in an off-line manner to generate non-real-time data;
and pushing the real-time data and the non-real-time data to a client through a bidirectional long connection.
2. The data processing and transmission method of claim 1, wherein the real-time computing the real-time source data to generate real-time data comprises:
the real-time source data is backed up in a distributed off-line manner;
calculating the real-time source data in a distributed manner in real time to generate the real-time data;
and carrying out distributed offline backup on the real-time data.
3. The data processing and transmission method of claim 2, wherein after the offline backup of the real-time source data, the generating of the real-time data further comprises:
calculating the real-time source data in an off-line manner to generate an off-line calculation result;
and when the real-time computation is abnormal, the offline computation result is pushed to the client in real time instead of the real-time data.
4. The data processing and transmission method of claim 1, wherein the performing offline computation on the non-real-time source data to generate non-real-time data comprises:
offline backing up the non-real-time source data;
calculating the non-real-time source data in an off-line manner to generate the non-real-time data;
and backing up and storing the non-real-time data offline.
5. The data processing and transmission method according to claim 1, wherein the pushing the real-time data and the non-real-time data to the client in real time via a long bidirectional connection comprises:
receiving an authentication request sent by the client;
and auditing the authentication request, and if the authentication request passes the audit, pushing the real-time data and the non-real-time data to the client through the bidirectional long connection.
6. The data processing and transmission method of claim 2, wherein the offline backup of the real-time source data and the real-time data comprises:
and respectively writing the real-time source data and the real-time data into a first data table and a second data table and reserving the data to a preset time limit.
7. The data processing and transmission method of claim 4, wherein the offline backup of the non-real-time source data and the non-real-time data comprises:
and respectively writing the non-real-time source data and the non-real-time data into a third data table and a fourth data table.
8. A data processing and transmission device established based on the method of any one of claims 1 to 7, comprising:
the data acquisition module is used for acquiring real-time source data in real time and acquiring non-real-time source data in a timing mode;
the real-time data acquisition module is used for calculating real-time source data in real time to generate real-time data;
the non-real-time data acquisition module is used for performing off-line calculation on non-real-time source data to generate non-real-time data;
and the communication module is used for pushing the real-time data and the non-real-time data to a client through a bidirectional long connection.
9. The data processing and transmission device of claim 8, wherein the real-time data acquisition module comprises:
the real-time computing module is used for computing the real-time source data in real time and generating the real-time data;
the first backup module is used for backing up the real-time source data and the real-time data in an off-line manner;
the device further comprises: and the distributed message system is used for processing the real-time source data and the real-time data in a distributed mode.
10. The data processing and transmission apparatus of claim 9, wherein the non-real-time data acquisition module comprises:
the off-line computing module is used for off-line computing the non-real-time source data and generating the non-real-time data;
the second backup module is used for backing up the non-real-time source data and the non-real-time data in an off-line mode;
a database for storing the non-real time data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910764376.2A CN110659270A (en) | 2019-08-19 | 2019-08-19 | Data processing and transmitting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910764376.2A CN110659270A (en) | 2019-08-19 | 2019-08-19 | Data processing and transmitting method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110659270A true CN110659270A (en) | 2020-01-07 |
Family
ID=69036952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910764376.2A Pending CN110659270A (en) | 2019-08-19 | 2019-08-19 | Data processing and transmitting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110659270A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111211973A (en) * | 2020-01-15 | 2020-05-29 | 百望股份有限公司 | Information processing method and device in invoice field and storage medium |
| CN111291056A (en) * | 2020-02-25 | 2020-06-16 | 帆软软件有限公司 | Method for associating data table bottom layers of BI tool |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103440333A (en) * | 2013-09-05 | 2013-12-11 | 深圳好视网络科技有限公司 | Data processing method and device |
| CN103780681A (en) * | 2014-01-06 | 2014-05-07 | 南京莱斯信息技术股份有限公司 | High real-time Web application system and data real-time pushing method thereof |
| CN105303292A (en) * | 2015-09-17 | 2016-02-03 | 国网北京市电力公司 | Distribution data storage method and device |
| CN105677836A (en) * | 2016-01-05 | 2016-06-15 | 北京汇商融通信息技术有限公司 | Big data processing and solving system simultaneously supporting offline data and real-time online data |
| EP3213163A1 (en) * | 2014-10-31 | 2017-09-06 | Cloudbased Industry 4.0 Technologies AG | A client device for data acquisition and pre-processing of process-related mass data from at least one cnc machine or industrial robot |
| CN107908690A (en) * | 2017-11-01 | 2018-04-13 | 南京欣网互联网络科技有限公司 | A kind of data processing method based on big data OA operation analysis |
| CN108959445A (en) * | 2018-06-13 | 2018-12-07 | 云南电网有限责任公司信息中心 | Distributed information log processing method and processing device |
| CN109270565A (en) * | 2018-09-04 | 2019-01-25 | 广东翼卡车联网服务有限公司 | A kind of processing unit of vehicle GPS big data |
-
2019
- 2019-08-19 CN CN201910764376.2A patent/CN110659270A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103440333A (en) * | 2013-09-05 | 2013-12-11 | 深圳好视网络科技有限公司 | Data processing method and device |
| CN103780681A (en) * | 2014-01-06 | 2014-05-07 | 南京莱斯信息技术股份有限公司 | High real-time Web application system and data real-time pushing method thereof |
| EP3213163A1 (en) * | 2014-10-31 | 2017-09-06 | Cloudbased Industry 4.0 Technologies AG | A client device for data acquisition and pre-processing of process-related mass data from at least one cnc machine or industrial robot |
| CN105303292A (en) * | 2015-09-17 | 2016-02-03 | 国网北京市电力公司 | Distribution data storage method and device |
| CN105677836A (en) * | 2016-01-05 | 2016-06-15 | 北京汇商融通信息技术有限公司 | Big data processing and solving system simultaneously supporting offline data and real-time online data |
| CN107908690A (en) * | 2017-11-01 | 2018-04-13 | 南京欣网互联网络科技有限公司 | A kind of data processing method based on big data OA operation analysis |
| CN108959445A (en) * | 2018-06-13 | 2018-12-07 | 云南电网有限责任公司信息中心 | Distributed information log processing method and processing device |
| CN109270565A (en) * | 2018-09-04 | 2019-01-25 | 广东翼卡车联网服务有限公司 | A kind of processing unit of vehicle GPS big data |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111211973A (en) * | 2020-01-15 | 2020-05-29 | 百望股份有限公司 | Information processing method and device in invoice field and storage medium |
| CN111211973B (en) * | 2020-01-15 | 2022-06-21 | 百望股份有限公司 | Information processing method and device in invoice field and storage medium |
| CN111291056A (en) * | 2020-02-25 | 2020-06-16 | 帆软软件有限公司 | Method for associating data table bottom layers of BI tool |
| CN111291056B (en) * | 2020-02-25 | 2020-12-01 | 帆软软件有限公司 | Method for associating data table bottom layers of BI tool |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107908494B (en) | Abnormal event processing method and device, electronic equipment and storage medium | |
| CN105719182B (en) | Automatic account checking method and automatic account checking system | |
| US20130006949A1 (en) | Systems and methods for data integrity checking | |
| CN106815254B (en) | Data processing method and device | |
| CN105631026A (en) | Security data analysis system | |
| CN106940677A (en) | One kind application daily record data alarm method and device | |
| CN110659270A (en) | Data processing and transmitting method and device | |
| CN113570468A (en) | Enterprise payment wind control service platform | |
| CN114328132A (en) | Method, device, equipment and medium for monitoring state of external data source | |
| CN105867895A (en) | Method for realizing code management and need management information synchronization and device thereof | |
| CN107943958B (en) | Individual tax owner data sharing method and system | |
| CN111353763B (en) | Method, device, server and storage medium for processing data | |
| CN111711512B (en) | Message sending method and device | |
| CN111367934B (en) | Data consistency checking method, device, server and medium | |
| CN111738728A (en) | Transaction compensation method and device | |
| WO2023155437A1 (en) | Online-consultation-based data processing method, apparatus, and computer device | |
| CN108255472B (en) | Layered experiment frame system | |
| CN115174246B (en) | Information processing method and system | |
| CN111130882A (en) | Monitoring system and method of network equipment | |
| CN101004802A (en) | System and method for controlling lifecycle of patent case | |
| CN115658745A (en) | Data processing method, data processing device, computer equipment and computer readable storage medium | |
| CN113627963B (en) | Electric power refined operation rule base creation method | |
| CN109062920B (en) | Memory-based data fast collision subsystem for data mining system | |
| CN115271835A (en) | Invoice generation method and device, electronic equipment and storage medium | |
| CN101894119B (en) | Mass data storage system for monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200107 |