CN115048418A - Data processing method and system - Google Patents

Data processing method and system Download PDF

Info

Publication number
CN115048418A
CN115048418A CN202210969596.0A CN202210969596A CN115048418A CN 115048418 A CN115048418 A CN 115048418A CN 202210969596 A CN202210969596 A CN 202210969596A CN 115048418 A CN115048418 A CN 115048418A
Authority
CN
China
Prior art keywords
data
redis
json file
queue
batch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210969596.0A
Other languages
Chinese (zh)
Inventor
罗国帝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Bifan Entertainment Technology Co ltd
Original Assignee
Shenzhen Bifan Entertainment Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Bifan Entertainment Technology Co ltd filed Critical Shenzhen Bifan Entertainment Technology Co ltd
Priority to CN202210969596.0A priority Critical patent/CN115048418A/en
Publication of CN115048418A publication Critical patent/CN115048418A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24568Data stream processing; Continuous queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The application relates to a data processing method and a system, relating to the technical field of data processing, wherein the method comprises the steps of sending data to Redis queues in service scene types for scraping when the data are received according to the service scene types to which the data belong; at the end of the Redis queue scraping batch, taking out all data in the Redis queue and emptying all data of the current scraping batch of the Redis queue so that the Redis queue can be reused for scraping batch data; generating a JSON file based on all the taken data; uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully; and adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming. The method and the device have the effect of better achieving the streaming batch processing of the data.

Description

Data processing method and system
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method.
Background
With the continuous progress and development of society, the application of computers is more and more extensive, and the data volume processed is bigger and bigger.
Data batch processing, one is processing logic that represents the execution of a batch of data at a time. The stream processing of data refers to converting data into a data stream format for transmission and processing. At present, a lot of scenes are available for scraping up streaming data, and in the prior art, the stability is poor when scraping up streaming data, which is not beneficial to data processing. How to better realize the streaming batch processing of data is a technical difficulty which needs to be overcome.
Disclosure of Invention
In order to better realize the stream batch processing of data, the application provides a data processing method and a data processing system.
In a first aspect, the following technical solutions are adopted in a data processing method provided by the present application.
A method of data processing, comprising:
when data is received, sending the data to Redis queues in the business scene types for scraping according to the business scene types to which the data belongs;
at the end of the Redis queue scraping batch, taking out all data in the Redis queue and emptying all data of the current scraping batch of the Redis queue so that the Redis queue can be reused for scraping batch data;
generating a JSON file based on all the taken data;
uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully; and the number of the first and second groups,
adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming; and the client pulls the JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file.
By adopting the technical scheme, when the processor receives new data, the processor sends the data to Redis queues in the service scene types for scraping according to the service scene types to which the data belong; when the Redis queue scraping batch is finished, all data in the Redis queue are taken out; the processor generates a JSON file based on all the taken data; the processor uploads the generated JSON file to a cloud service platform, then synchronously obtains a result fed back by the cloud service platform, and obtains a network address corresponding to the JSON file when the JSON file is successfully uploaded; the processor adds the network address into a message queue and sends the message queue to a client which needs to consume the data for monitoring and consuming; after obtaining the message queue, the client pulls a JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file; through the arrangement, the technical scene of stream-to-batch processing is realized uniformly, and stream batch processing of data can be realized better.
Optionally, the method for judging whether the Redis queue scraping batch is finished comprises:
when the Redis queue receives data, accumulating the data number in the Redis queue to obtain the data amount sum; and the number of the first and second groups,
judging whether the sum of the data amount is larger than a preset number or not; if so, determining that the Redis queuing scraping batch has ended.
Optionally, the method for determining whether the Redis queue scraping batch is finished further includes:
when the Redis queue receives data, recording the time of the latest received data;
judging whether the next data is received within a preset time length; if not, determining that the Redis queuing scraping batch has ended.
Optionally, the pulling, by the client, the JSON file stored in the cloud service platform according to the network address in the message queue, analyzing and consuming the JSON file, including:
monitoring batch processing messages in the message queue based on an API provided by the datastream;
judging whether the network address of the batch processing message is the network address of the needed JSON file; if yes, downloading and analyzing a corresponding JSON file; and the number of the first and second groups,
and performing data consumption by a data consumption method.
Optionally, the configuration method of the data consumption method includes:
constructing at least one abstract decoration listener by means of a decorator schema; and the number of the first and second groups,
and configuring the abstract device listener for a client needing to consume data and rewriting a data consumption method in the abstract device listener.
Optionally, rocktmq is selected as the message queue.
Optionally, the cloud service platform includes at least one of an arilocos, a tengcong cloud, a huazhiyun, a google cloud, and a microsoft cloud.
In a second aspect, the following technical solution is adopted in the data processing method provided by the present application.
A data processing system comprising:
a first processing module to: when data is received, sending the data to Redis queues in the business scene types for scraping according to the business scene types to which the data belongs;
a second processing module to: at the end of the Redis queue scraping batch, taking out all data in the Redis queue and emptying all data of the current scraping batch by the Redis queue so that the Redis queue can be reused for scraping batch data;
a third processing module to: generating a JSON file based on all the taken data;
a fourth processing module to: uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully; and the number of the first and second groups,
a fifth processing module to: adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming; and the client pulls the JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file.
In a third aspect, the present application discloses a computer device comprising a memory and a server, the memory having stored thereon a computer program that is loaded by the server and that performs any of the methods described above.
In a fourth aspect, the present application discloses a computer readable storage medium storing a computer program that can be loaded by a server and execute any of the methods described above.
Drawings
FIG. 1 is a method flow diagram of a data processing method of an embodiment of the present application;
FIG. 2 is a system block diagram of a data processing system according to an embodiment of the present application;
in the figure, 201, a first processing module; 202. a second processing module; 203. a third processing module; 204. a fourth processing module; 205. and a fifth processing module.
Detailed Description
The present application is further described with reference to the following figures and specific examples:
first, it should be noted here that: in the description of the present application, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", and the like, which indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, are used for convenience of description only, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, should not be construed as limiting the present application; furthermore, the use of numerical terms such as the terms first, second, third, etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the present application, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly, and may be, for example, a fixed connection, a detachable connection, a limit connection such as an interference fit, a transition fit, or an integral connection; can be directly connected or indirectly connected through an intermediate medium; therefore, the specific meanings of the above terms in the present application can be understood according to specific situations by those of ordinary skill in the art.
The embodiment of the application discloses a data processing method. Referring to fig. 1, as an embodiment of a data processing method, the method includes the following steps:
step 101, when data is received, sending the data to Redis queues in service scene types for scraping according to the service scene types to which the data belongs.
Specifically, Redis (Remote Dictionary Server). Redis is currently the most popular kv class database. When the processor receives new data, the processor sends the data to Redis queues in the business scene types for batching according to the business scene types to which the data belong. It should be understood that one type of service scene or one scraping scene corresponds to one Redis queue, and each piece of data in the Redis queue is java object transformation. For example, if there are 50 pieces of approval data to be converted to approval, the 50 pieces of approval data are scraped and approved into a single Redis queue.
At the end of the Redis queue scraping batch, all data in the Redis queue is fetched and emptied of all data of the current scraping batch of the Redis queue so that the Redis queue can be reused for scraping batch data, step 102.
Specifically, at the end of the Redis queue scraping batch, i.e., after the Redis queue satisfies the end of the current scraping batch flow, the processor fetches all the data in the Redis queue and treats it as a single whole. The processor also empties all data of the current scraping batch so that the Redis queue can be reused for scraping batch data.
And 103, generating a JSON file based on all the taken data.
Specifically, the processor generates a JSON file based on all the fetched data, which can further reduce the number of data communications.
And 104, uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully.
Specifically, the cloud service platform comprises at least one of the Ali cloud, Tencent cloud, Huazhi cloud, Google cloud, and Microsoft cloud. The cloud service platform receives the corresponding file and then feeds back the result, and the content of the data feedback can be uploading failure or network address of the corresponding file when the uploading is successful. The processor uploads the generated JSON file to the cloud service platform, then synchronously obtains a result fed back by the cloud service platform, and obtains a network address corresponding to the JSON file when the JSON file is successfully uploaded, wherein each information resource has a uniform and unique address on the network, and the address is the network address.
105, adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming; and the client pulls the JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file.
Specifically, rocktmq may be selected as a message queue. The processor adds the network address into the message queue and sends the message queue to the client which needs to consume the data for monitoring and consumption. And after the client acquires the message queue, the JSON file stored in the cloud service platform is pulled according to the network address in the message queue, and is analyzed and consumed. Through the arrangement, the technical scene of stream-to-batch processing is realized uniformly, and stream batch processing of data can be realized better.
As one of the data processing methods, the method for determining whether the Redis queue scraping batch is finished includes:
when the Redis queue receives data, accumulating the data number in the Redis queue to obtain the sum of the data amount; and (c) a second step of,
judging whether the sum of the data amount is larger than a preset number or not; if so, it is determined that the Redis queuing scraping batch has ended.
Specifically, when the Redis queue receives data, the processor accumulates the data amount in the Redis queue to obtain a data amount sum of the current Redis queue, and the processor compares the obtained data amount sum with a preset number, where the preset number may be configured correspondingly as needed, for example, 500. When the processor judges that the sum of the data amount is larger than the preset value, the Redis queue scraping batch is judged to be finished.
As one embodiment of the data processing method, the method for determining whether the Redis queue scraping batch is finished further includes:
when the Redis queue receives data, recording the time of the latest received data;
judging whether the next data is received within a preset time length; if not, it is determined that the Redis queuing scraping batch has ended.
Specifically, the preset time period may be 3 seconds, and the processor records the time of the most recently received data when the data is received by the Redis queue, and if the next data is not received within the preset time period, the processor determines that the Redis queue saving batch has ended. It should be understood that the two determination methods are related, and only one of them needs to be satisfied to determine that the scraping batch of the Redis queue is finished.
As one implementation manner of the data processing method, a client pulls a JSON file stored in a cloud service platform according to a network address in a message queue, analyzes and consumes the JSON file, and the method comprises the following steps:
monitoring batch processing messages in a message queue based on an API provided by the dataslow;
judging whether the network address of the batch processing message is the network address of the required JSON file; if yes, downloading and analyzing a corresponding JSON file; and the number of the first and second groups,
and performing data consumption by a data consumption method.
Specifically, the dataFlow major capabilities include: stream data transfer batch processing, batch data asynchronous communication and message consumption idempotency checking. The client monitors batch processing messages in the message queue based on an API (application programming interface) provided by the dataFlow and judges whether the network address of the batch processing messages is the network address of the required JSON (Java Server object notation) file; and if so, downloading and analyzing the corresponding JSON file and performing data consumption through a data consumption method.
As one embodiment of a data processing method, a configuration method of a data consumption method includes:
constructing at least one abstract decor listener in a decorator schema; and the number of the first and second groups,
and configuring the abstract device listener for the client needing to consume the data and rewriting the data consumption method in the abstract device listener.
The present application also provides a data processing system, as one of the implementation modes of the data processing system, the system includes:
a first processing module 201, configured to: when data are received, the data are sent to Redis queues in the service scene types according to the service scene types to which the data belong to carry out scraping;
a second processing module 202 configured to: at the end of the Redis queue scraping batch, taking out all data in the Redis queue and emptying all data of the current scraping batch by the Redis queue so that the Redis queue can be reused for the scraping batch data;
a third processing module 203, configured to: generating a JSON file based on all the taken data;
a fourth processing module 204, configured to: uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully; and the number of the first and second groups,
a fifth processing module 205, configured to: adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming; and the client pulls the JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file.
As another embodiment of a data processing system, a method of determining whether a Redis queue scraping batch is complete comprises:
when the Redis queue receives data, accumulating the data number in the Redis queue to obtain the data amount sum; and the number of the first and second groups,
judging whether the sum of the data amount is larger than a preset number or not; if so, it is determined that the Redis queuing scraping batch has ended.
As another embodiment of the data processing system, the method of determining whether the Redis queue scraping batch is finished further comprises:
when the Redis queue receives data, recording the time of the latest received data;
judging whether the next data is received within a preset time length; if not, it is determined that the Redis queuing scraping batch has ended.
As another embodiment of the data processing system, the client pulls a JSON file stored in a cloud service platform according to a network address in a message queue, and analyzes and consumes the JSON file, including:
monitoring batch processing messages in a message queue based on an API provided by the dataslow;
judging whether the network address of the batch processing message is the network address of the required JSON file; if yes, downloading and analyzing a corresponding JSON file; and the number of the first and second groups,
and performing data consumption by a data consumption method.
As another embodiment of a data processing system, a method of configuring a data consumption method includes:
constructing at least one abstract decor listener in a decorator schema; and the number of the first and second groups,
and configuring an abstract device monitor for the client needing to consume the data and rewriting a data consumption method in the abstract device monitor.
The embodiment of the application also discloses the electronic equipment.
Specifically, the apparatus includes a memory and a server, the memory having stored thereon a computer program that can be loaded by the server and that executes any of the data processing methods described above.
The embodiment of the application also discloses a computer readable storage medium.
Specifically, the computer-readable storage medium stores a computer program that can be loaded by a server and executes any of the data processing methods described above, and includes, for example: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random access Memory (RaM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
It should be noted that: although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can make modifications and substitutions on the present application, and all technical solutions and modifications thereof without departing from the spirit and scope of the present application should be covered by the claims of the present application.

Claims (10)

1. A data processing method, comprising:
when data is received, sending the data to Redis queues in the business scene types for scraping according to the business scene types to which the data belongs;
at the end of the Redis queue scraping batch, taking out all data in the Redis queue and emptying all data of the current scraping batch of the Redis queue so that the Redis queue can be reused for scraping batch data;
generating a JSON file based on all the taken data;
uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully; and the number of the first and second groups,
adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming; and the client pulls the JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file.
2. The data processing method of claim 1, wherein the method of determining whether the Redis queue scraping batch is finished comprises:
when the Redis queue receives data, accumulating the data number in the Redis queue to obtain the data amount sum; and the number of the first and second groups,
judging whether the sum of the data amount is larger than a preset number or not; if so, determining that the Redis queuing scraping batch has ended.
3. The data processing method of claim 2, wherein the method of determining whether the Redis queue scraping batch is complete further comprises:
when the Redis queue receives data, recording the time of the latest received data;
judging whether the next data is received within a preset time length; if not, determining that the Redis queuing scraping batch has ended.
4. The data processing method according to claim 3, wherein the client pulls the JSON file saved in the cloud service platform according to the network address in the message queue, and parses and consumes the JSON file, and the method comprises the steps of:
monitoring batch processing messages in the message queue based on an API provided by the datastream;
judging whether the network address of the batch processing message is the network address of the required JSON file; if yes, downloading and analyzing a corresponding JSON file; and the number of the first and second groups,
and performing data consumption by a data consumption method.
5. The data processing method of claim 4, wherein the configuration method of the data consumption method comprises:
constructing at least one abstract decor listener in a decorator schema; and the number of the first and second groups,
and configuring the abstract device listener for a client needing to consume data and rewriting a data consumption method in the abstract device listener.
6. A data processing method as claimed in claim 5, characterized in that a rocktmq is selected as the message queue.
7. The data processing method of claim 1, wherein the cloud service platform comprises at least one of the cloud Ali, Tencent, Huashi, Google, and Microsoft clouds.
8. A data processing system, comprising:
a first processing module (201) for: when data is received, sending the data to Redis queues in the business scene types for scraping according to the business scene types to which the data belongs;
a second processing module (202) for: at the end of the Redis queue scraping batch, taking out all data in the Redis queue and emptying all data of the current scraping batch by the Redis queue so that the Redis queue can be reused for scraping batch data;
a third processing module (203) for: generating a JSON file based on all the taken data;
a fourth processing module (204) configured to: uploading the JSON file to a cloud service platform, acquiring an uploading result fed back by the cloud service platform, and acquiring a network address corresponding to the JSON file when the JSON file is uploaded successfully; and the number of the first and second groups,
a fifth processing module (205) configured to: adding the network address into a message queue and sending the message queue to a client needing to be consumed for monitoring and consuming; and the client pulls the JSON file stored in the cloud service platform according to the network address in the message queue, and analyzes and consumes the JSON file.
9. A computer device, characterized by: comprising a memory and a server, said memory having stored thereon a computer program for a method according to any one of claims 1 to 7, when loaded and executed by the server.
10. A computer-readable storage medium, in which a computer program is stored which can be loaded by a server and which executes the method according to any one of claims 1 to 7.
CN202210969596.0A 2022-08-12 2022-08-12 Data processing method and system Pending CN115048418A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210969596.0A CN115048418A (en) 2022-08-12 2022-08-12 Data processing method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210969596.0A CN115048418A (en) 2022-08-12 2022-08-12 Data processing method and system

Publications (1)

Publication Number Publication Date
CN115048418A true CN115048418A (en) 2022-09-13

Family

ID=83166715

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210969596.0A Pending CN115048418A (en) 2022-08-12 2022-08-12 Data processing method and system

Country Status (1)

Country Link
CN (1) CN115048418A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107943802A (en) * 2016-10-12 2018-04-20 北京京东尚科信息技术有限公司 A kind of log analysis method and system
CN113269547A (en) * 2021-05-31 2021-08-17 中国农业银行股份有限公司 Data processing method and device, electronic equipment and storage medium
WO2021180025A1 (en) * 2020-03-13 2021-09-16 北京金山云网络技术有限公司 Message processing method and apparatus, electronic device and medium
CN113590284A (en) * 2021-07-22 2021-11-02 济南浪潮数据技术有限公司 Cloud platform distributed component interaction task batch processing method, system and equipment
CN113687958A (en) * 2021-07-29 2021-11-23 上海浦东发展银行股份有限公司 Data processing method, system, computer device and storage medium
CN113961651A (en) * 2021-10-28 2022-01-21 中远海运科技股份有限公司 Global ship AIS big data processing method and system
CN114357068A (en) * 2021-12-22 2022-04-15 天津南大通用数据技术股份有限公司 Method for synchronizing data from kafka to database
CN114422496A (en) * 2021-12-10 2022-04-29 广东联合电子服务股份有限公司 Mass traffic data processing system and method

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107943802A (en) * 2016-10-12 2018-04-20 北京京东尚科信息技术有限公司 A kind of log analysis method and system
WO2021180025A1 (en) * 2020-03-13 2021-09-16 北京金山云网络技术有限公司 Message processing method and apparatus, electronic device and medium
CN113269547A (en) * 2021-05-31 2021-08-17 中国农业银行股份有限公司 Data processing method and device, electronic equipment and storage medium
CN113590284A (en) * 2021-07-22 2021-11-02 济南浪潮数据技术有限公司 Cloud platform distributed component interaction task batch processing method, system and equipment
CN113687958A (en) * 2021-07-29 2021-11-23 上海浦东发展银行股份有限公司 Data processing method, system, computer device and storage medium
CN113961651A (en) * 2021-10-28 2022-01-21 中远海运科技股份有限公司 Global ship AIS big data processing method and system
CN114422496A (en) * 2021-12-10 2022-04-29 广东联合电子服务股份有限公司 Mass traffic data processing system and method
CN114357068A (en) * 2021-12-22 2022-04-15 天津南大通用数据技术股份有限公司 Method for synchronizing data from kafka to database

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
孙良君等: "分布式实时抽取计算框架设计与应用", 《信息技术》 *
张英辉: "消息队列技术在短信通信中的应用", 《中小企业管理与科技(上旬刊)》 *
王煜骢等: "NTCI-Flow:一种可扩展的高速网络流量处理框架", 《工程科学与技术》 *
罗东锋等: "基于Docker的大规模日志采集与分析系统", 《计算机系统应用》 *

Similar Documents

Publication Publication Date Title
US8239578B2 (en) Method and device for transferring digital data with a progressive format
KR102415845B1 (en) Internet of Things Resource Subscription Methods, Devices, and Systems
US7990900B2 (en) Event notification control based on data about a user's communication device stored in a user notification profile
US9288538B2 (en) Methods and apparatus for conveying a delivery schedule to mobile terminals
EP2631820B1 (en) Computer-implemented method, mobile device, computer network system, and computer program product for optimized audio data provision
CN108781226B (en) Communication system
CN110213758B (en) Data communication method based on Bluetooth Mesh, storage medium and electronic equipment
US8051136B2 (en) Optimizing a presence enabled managed service
CN114401447A (en) Video stuck prediction method, device, equipment and medium
KR20130044373A (en) Systems and methods for optimizing the configuration of a set of performance scaling algorithms
WO2023174254A1 (en) Video posting method and apparatus, and device and storage medium
CN113891114B (en) Transcoding task scheduling method and device
CN115048418A (en) Data processing method and system
US7779115B2 (en) Method and apparatus for processing client capability information over a network
WO2011150968A1 (en) Communication method and device
CN117270790A (en) Method, electronic device and computer program product for processing data
CN113141403B (en) Log transmission method and device
CN109788508B (en) Data caching method and storage medium
CN110278265B (en) Processing method and device for file uploaded by user, storage medium and electronic equipment
CN113923142A (en) Method, system and medium for monitoring state of equipment of Internet of things
EP2184923A2 (en) Moving-picture processing device, moving-picture processing method, and program
CN116596065B (en) Gradient calculation method and device, storage medium, product and electronic equipment
WO2018100687A1 (en) EDGE DEVICE CONTROL METHOD, IoT HUB, AND STORAGE MEDIUM
CN115906809A (en) Text transmission method and electronic equipment
CN114710692B (en) Multimedia file processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20220913