CN109088933B - Large-batch list transmission method, large-batch list acquisition method, corresponding device and electronic equipment - Google Patents
Large-batch list transmission method, large-batch list acquisition method, corresponding device and electronic equipment Download PDFInfo
- Publication number
- CN109088933B CN109088933B CN201810951773.6A CN201810951773A CN109088933B CN 109088933 B CN109088933 B CN 109088933B CN 201810951773 A CN201810951773 A CN 201810951773A CN 109088933 B CN109088933 B CN 109088933B
- Authority
- CN
- China
- Prior art keywords
- data
- message queue
- queue middleware
- list
- sender
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Abstract
The application provides a method for transmitting and acquiring a large batch of lists, a corresponding device and electronic equipment. The method comprises the following steps: cutting and subdividing a large number of list files uploaded by a user, and analyzing each data fragment file after cutting and subdividing into a memory; determining a specified theme corresponding to a data receiver in a pre-built message queue middleware; and sending each data fragment file to a specified theme of the message queue middleware for the data receiver to acquire. The method and the device are based on a step-by-step message queue technology, and the interaction of the large-batch list data is carried out through the message queue middleware, so that the accumulation of a large number of interface requests is avoided, and the efficiency of the large-batch list data transmission is improved.
Description
Technical Field
The application relates to the technical field of data transmission, in particular to a method for transmitting and acquiring a large quantity of lists, a corresponding device and electronic equipment.
Background
The current age is the age of increasingly penetrating life of data, and the reasonable application of the data brings great innovation potential and innovation space for various industries. In practical applications of various services, a large amount of list data, such as a directory, a white list, a black list, etc., related to a certain organization or a certain product is often required. With the rapid increase in the volume of the list data, the conventional techniques for storing, transmitting, processing and managing the list data in large quantities are severely challenged.
The transmission of the large-batch list data by the system on the market at present is carried out through a preset interface, but for the project with high complexity of the architecture, the large-batch list data interaction among a plurality of systems is carried out through the preset interface, a large number of interface requests are generated for the systems, the interface requests are easily accumulated, and the production application performance is affected.
Disclosure of Invention
The application provides a method for transmitting and acquiring a large-batch list, a corresponding device and electronic equipment, which are used for improving the efficiency of large-batch list data transmission among multiple systems.
In a first aspect, the present application provides a method for bulk list delivery, the method comprising:
cutting and subdividing a large number of list files uploaded by a user, and analyzing each data fragment file after cutting and subdividing into a memory;
determining a specified theme corresponding to a data receiver in a pre-built message queue middleware;
and sending each data fragment file to a specified theme of the message queue middleware for the data receiver to acquire.
In a second aspect, the present application provides a method for obtaining a bulk list, where the method includes:
determining a specified theme corresponding to a data sender in a pre-built message queue middleware;
monitoring the appointed theme of the message queue middleware;
and acquiring the monitored data fragment files from the appointed theme of the message queue middleware to obtain a large number of list files.
In a third aspect, the present application provides a bulk list delivery device, the device comprising:
the cutting subdivision module is used for cutting and subdividing a large amount of list files uploaded by a user, and analyzing each data fragment file after cutting and subdivision into a memory;
the determining module is used for determining a specified theme corresponding to the data receiver in the pre-built message queue middleware;
and the sending module is used for sending each data fragment file to the appointed theme of the message queue middleware so as to be acquired by a data receiver.
In a fourth aspect, the present application provides a device for obtaining a bulk list, the device comprising:
a determining module, configured to determine a specified topic corresponding to a data sender in a pre-built message queue middleware
The monitoring module is used for monitoring the appointed theme of the message queue middleware;
and the acquisition module is used for acquiring the monitored data fragment files from the appointed theme of the message queue middleware so as to obtain a large number of list files.
In a fifth aspect, the present application provides an electronic device, including: a processor and a memory storing at least one instruction, at least one program, code set, or instruction set that is loaded and executed by the processor to implement the method of bulk list delivery as shown in the first aspect of the application or to implement the method of bulk list retrieval as shown in the second aspect of the application.
In a sixth aspect, the present application provides a computer readable storage medium for storing computer instructions that, when run on a computer, enable the computer to perform a method of bulk list delivery as shown in the first aspect of the present application or a method of bulk list retrieval as shown in the second aspect of the present application.
The method for transmitting and acquiring the large-batch list, the corresponding device and the electronic equipment adopt the method for cutting and subdividing the large-batch list files uploaded by the user, and analyze each data fragment file after cutting and subdivision into a memory; determining a specified theme corresponding to a data receiver in a pre-built message queue middleware; and sending each data fragment file to the appointed theme of the message queue middleware for the data receiver to acquire, and carrying out interaction of a large amount of list data through the message queue middleware, so that accumulation of a large number of interface requests is avoided, and the efficiency of transmitting the large amount of list data is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a flow chart of a method for transmitting a bulk list according to an embodiment of the present application;
fig. 2 is a flow chart of a method for obtaining a bulk list according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a bulk list conveying apparatus according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a device for obtaining a bulk list according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of illustrating the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
Example 1
The embodiment of the application provides a method for transmitting a large-batch list, as shown in fig. 1, the method comprises the following steps:
step S101: cutting and subdividing a large number of list files uploaded by a user, and analyzing each data fragment file after cutting and subdividing into a memory.
In the embodiment of the application, the execution subject of the method is a data sender.
Specifically, when a user needs to transmit a large amount of list data, the user can upload the large amount of list files on a terminal system of a data sender. After the user uploading is completed, the data sender cuts and subdivides the large-batch list files into a large number of data fragment files.
In practical application, the cutting subdivision mode can be any one of the following: horizontal slicing, vertical slicing, derived slicing, mixed slicing, etc. Wherein, the liquid crystal display device comprises a liquid crystal display device,
(1) Horizontal slicing: and dividing all tuples of the large-batch list file into a plurality of disjoint subsets according to preset slicing conditions. As an example, the slicing is performed by the number of people.
(2) Vertical slicing: the large-batch list file is divided into a plurality of subsets according to the attribute set.
(3) Export sharding: in effect, a horizontal slice is derived, but the conditions of the horizontal slice are not attributes in the bulk list file, but other preset attribute relationships.
(4) Mixing and slicing: and mixing the above three methods. The method can be performed by firstly performing horizontal slicing and then performing vertical slicing, or by firstly performing vertical slicing and then performing horizontal slicing, or in other forms, and it is to be noted that the results of different mixed slicing modes are different, but the effects of the embodiments of the present application are not affected.
Those skilled in the art may set the slicing manner and the slicing conditions according to actual situations, which are not limited in the embodiment of the present application.
In the embodiment of the application, after a large amount of list files uploaded by a user are cut and subdivided, the list files are analyzed into a memory.
Wherein the storage includes, but is not limited to, memory, persistent storage, and the like.
In the step, the large-batch list files are cut and subdivided into a large number of data fragment files, and then the subsequent storage and transmission processes are carried out, namely the cut and subdivided large-batch list files are stored, so that the data fragment files are read and transmitted in the subsequent process, and the storage, reading and transmission efficiency is effectively improved.
Step S102: and determining the appointed theme corresponding to the data receiver in the pre-built message queue middleware.
The message queue middleware is a system component built in advance in the embodiment of the application, can be internally provided with an efficient communication mechanism, is connected with the data sender and the data receiver at the same time, can receive and store the data slicing files sent by the data sender, and sends the data slicing files to the data receiver when needed.
It can be understood that when one data sender needs to send a large number of list files to a plurality of data receivers at the same time, through the message queue middleware, the data interaction between the data sender and the plurality of data receivers can be avoided, and the decoupling of the data sender is realized.
Or when a plurality of data transmitters need to send a large amount of list files to the same data receiver, the data interaction requested by the plurality of data transmitters at the same time can be avoided through the message queue middleware, so that the decoupling of the data receivers is realized.
In practical application, in order to distinguish data interaction between different data senders and data receivers, in the embodiment of the application, a specified TOPIC corresponding to the data receiver in the pre-built message queue middleware needs to be determined.
Specifically, the specified TOPIC is pre-agreed by the data sender and the data receiver, that is, the message queue middleware sets corresponding TOPIC TOPICs according to different combinations of the data sender and the data receiver, and each pair of data sender and data receiver can be identified by the corresponding TOPIC TOPICs through the message queue middleware.
When a data transmitting party transmits a large amount of list files to a preset data receiving party, a designated TOPIC TOPIC corresponding to the data receiving party in the message queue middleware needs to be determined.
Step S103: and sending each data fragment file to a specified theme of the message queue middleware for the data receiver to acquire.
Because of the disorder of the large-batch list files, the batch sending mode can adopt modes of parallel sending and/or sequential sending and the like according to network conditions, and the embodiment of the application is not limited herein.
And the designated TOPIC TOPIC of the message queue middleware is used for carrying out the interaction of a large amount of list data, so that the accuracy and the reliability of the data are ensured.
In this embodiment, in consideration of the need to ensure the integrity of the data slicing file, after cutting and subdividing the bulk list file uploaded by the user in step S101, the method may further include the steps of: and determining the batch numbers and the corresponding data amounts of the data fragment files after cutting and subdividing, and notifying a data receiver.
The data sender can inform the data receiver of each batch number to be pushed and the data quantity of each batch number through the interface so that the data receiver can monitor in time. The only notification message transmitted through the interface does not affect the performance of the production application.
The integrity and consistency of the large-batch list data transmission between the data sender and the data receiver can be ensured through the message notification of the batch number and the corresponding data quantity.
According to the method for transmitting the large-batch list, interaction of the large-batch list data is performed through the message queue middleware, accumulation of a large number of interface requests is avoided, and efficiency of transmitting the large-batch list data is improved. And the interaction of a large amount of list data is carried out through the appointed theme of the message queue middleware, so that the accuracy and the reliability of the data are ensured. And the integrity and consistency of the mass list data transmission between the data sender and the data receiver can be ensured through the message notification of the batch number and the corresponding data quantity.
Example two
The embodiment of the application provides a method for acquiring a large-batch list, as shown in fig. 2, the method comprises the following steps:
step S201: and determining the appointed theme corresponding to the data sender in the pre-built message queue middleware.
In the embodiment of the application, the execution subject of the method is a data receiver.
The message queue middleware is a system component built in advance in the embodiment of the application, can be internally provided with an efficient communication mechanism, is connected with the data sender and the data receiver at the same time, can receive and store the data slicing files sent by the data sender, and sends the data slicing files to the data receiver when needed.
It can be understood that when one data sender needs to send a large number of list files to a plurality of data receivers at the same time, through the message queue middleware, the data interaction between the data sender and the plurality of data receivers can be avoided, and the decoupling of the data sender is realized.
Or when a plurality of data transmitters need to send a large amount of list files to the same data receiver, the data interaction requested by the plurality of data transmitters at the same time can be avoided through the message queue middleware, so that the decoupling of the data receivers is realized.
In practical application, in order to distinguish data interaction between different data senders and data receivers, a specified TOPIC corresponding to the data sender in a pre-built message queue middleware needs to be determined.
Specifically, the specified TOPIC is pre-agreed by the data sender and the data receiver, that is, the message queue middleware sets corresponding TOPIC TOPICs according to different combinations of the data sender and the data receiver, and each pair of data sender and data receiver can be identified by the corresponding TOPIC TOPICs through the message queue middleware.
Step S202: and monitoring the appointed theme of the message queue middleware.
After determining the specified TOPIC TOPIC corresponding to the data sender, monitoring can be performed so as to acquire a large number of list files in time.
Step S203: and acquiring the monitored data fragment files from the appointed theme of the message queue middleware to obtain a large number of list files.
The data receiver acquires the monitored data fragment files from the designated TOPIC TOPIC of the message queue middleware.
And the designated TOPIC TOPIC of the message queue middleware is used for carrying out the interaction of a large amount of list data, so that the accuracy and the reliability of the data are ensured.
In practical applications, the data receiver may immediately request to receive when it is monitored that the designated TOPIC of the message queue middleware includes any data fragment file. That is, the data receiver receives the data fragment file in the designated TOPIC that has been transmitted to the message queue middleware in real time while the data transmission is transmitting data to the designated TOPIC of the message queue middleware.
Or, according to the batch number and the corresponding data volume of each data fragment file, when the designated TOPIC of the message queue middleware comprises all the data fragment files, the receiving is requested. That is, after the data transmission direction sends the full list data to the designated TOPIC of the message queue middleware, the data receiver receives all the data fragment files in the designated TOPIC that have been transmitted to the message queue middleware. In this case, parallel transmission and/or sequential transmission may be adopted according to the network situation, which is not limited in the embodiments of the present application.
The message queue middleware is used for carrying out interaction of a large number of list data, so that accumulation of a large number of interface requests is avoided, and the efficiency of transmitting the large number of list data is improved.
In addition, as can be seen from the first embodiment, since the large-batch list file is transmitted in the form of the data slicing file after being cut and subdivided, in order to ensure the integrity of the data slicing file, the data sender needs to determine the batch number and the corresponding data amount of each data slicing file, and notify the data receiver.
Namely, the method for acquiring the large-batch lists provided by the embodiment of the application further comprises the following steps: and receiving the batch numbers and the corresponding data volumes of the data fragment files notified by the data sender.
For the data receiver, the batch numbers and the corresponding data amounts of the data fragment files notified by the data sender can be received, so that a large number of list files can be definitely required to be acquired, and the subsequent acquisition step can be executed. At this time, only the notification message is received, and even if a plurality of data senders simultaneously notify to acquire a large number of list files, the production application performance is not affected.
The integrity and consistency of the large-batch list data transmission between the data sender and the data receiver can be ensured through the message notification of the batch number and the corresponding data quantity.
Then, in order to improve the monitoring efficiency and avoid wasting resources, step S202 may monitor the specified theme of the message queue middleware after receiving the batch number and the corresponding data amount of each data fragment file notified by the data sender.
In this embodiment, in consideration of the need to ensure the integrity of the data slicing file, after step S203, the method may further include the steps of: and judging whether the large-batch list files are accepted to be completed or not according to the batch numbers of the data fragment files and the corresponding data quantity.
And until the fact that the large-batch list file is accepted to be completed is judged, namely the large-batch list data interaction between the data sender and the data receiver is completed, and the integrity and consistency of data transmission are ensured.
In practical application, after receiving the complete data slicing files, the data receiver can store the data according to the slicing mode, so that the reading and writing efficiency of the data is guaranteed, and the complete large-batch list files can be directly synthesized for use.
According to the method for acquiring the large-batch list, the integrity and consistency of large-batch list data transmission between the data sender and the data receiver can be ensured through the batch number and the message notification of the corresponding data quantity. The message queue middleware is used for carrying out interaction of a large number of list data, so that accumulation of a large number of interface requests is avoided, and the efficiency of transmitting the large number of list data is improved. And the interaction of a large amount of list data is carried out through the appointed theme of the message queue middleware, so that the accuracy and the reliability of the data are ensured.
Example III
The embodiment of the present application provides a bulk list transmitting apparatus, as shown in fig. 3, the bulk list transmitting apparatus 30 may include: a cut subdivision module 301, a determination module 302, and a transmission module 303, wherein,
the cutting subdivision module 301 is configured to cut and subdivide a large amount of list files uploaded by a user, and parse each cut and subdivided data fragment file into a memory;
a determining module 302, configured to determine a specified topic corresponding to the data receiver in the pre-built message queue middleware;
and the sending module 303 is configured to send each data fragment file to a specified theme of the message queue middleware, so that the data receiver obtains the specified theme.
Further, the sending module 303 is further specifically configured to determine a batch number and a corresponding data amount of each data fragment file after cutting and subdividing, and notify the data receiver.
Further, the cut subdivision module 301 is specifically configured to either: horizontal slicing, vertical slicing, derived slicing, mixed slicing.
The method for transmitting a large batch of lists provided in the embodiments of the present application may be specific hardware on a server or software or firmware installed on a server, where the principle and the technical effects of implementation are the same as those of the foregoing method embodiments, and for brevity description, the corresponding content in the foregoing method embodiments may be referred to for brevity description, which is not repeated here.
According to the method for transmitting the large-batch list, interaction of the large-batch list data is performed through the message queue middleware, accumulation of a large number of interface requests is avoided, and efficiency of transmitting the large-batch list data is improved. And the interaction of a large amount of list data is carried out through the appointed theme of the message queue middleware, so that the accuracy and the reliability of the data are ensured. And the integrity and consistency of the mass list data transmission between the data sender and the data receiver can be ensured through the message notification of the batch number and the corresponding data quantity.
Example IV
The embodiment of the present application provides a device for obtaining a bulk list, as shown in fig. 4, the device 40 for obtaining a bulk list may include: a determination module 401, a listening module 402, and an acquisition module 403, wherein,
the determining module 401 is configured to determine a specified topic corresponding to a data sender in a pre-built message queue middleware;
the monitoring module 402 is configured to monitor a specified topic of the message queue middleware;
the obtaining module 403 is configured to obtain monitored data fragment files from a specified topic of the message queue middleware, so as to obtain a large number of list files.
Further, the monitoring module 402 is further configured to receive the batch number and the corresponding data amount of each data fragment file notified by the data sender;
and the monitoring module 402 is specifically configured to monitor a specified topic of the message queue middleware after receiving the batch number and the corresponding data amount of each data fragment file notified by the data sender.
Further, the obtaining module 403 is further specifically configured to determine whether the large-batch list file is accepted to be completed according to the batch number and the corresponding data amount of each data fragment file.
The method for obtaining a large batch of lists provided in the embodiments of the present application may be specific hardware on a server or software or firmware installed on a server, where the principle and the technical effects of implementation are the same as those of the foregoing method embodiments, and for brevity description, the corresponding content in the foregoing method embodiments may be referred to for brevity description, and will not be repeated here.
According to the large-batch list acquisition device, interaction of large-batch list data is carried out through the message queue middleware, accumulation of a large number of interface requests is avoided, and efficiency of large-batch list data transmission is improved. And the interaction of a large amount of list data is carried out through the appointed theme of the message queue middleware, so that the accuracy and the reliability of the data are ensured. And the integrity and consistency of the mass list data transmission between the data sender and the data receiver can be ensured through the message notification of the batch number and the corresponding data quantity.
Example five
The embodiment of the application further provides an electronic device, as shown in fig. 5, where the electronic device 50 shown in fig. 5 includes: a processor 501 and a memory 502, the memory 502 storing at least one instruction, at least one program, a set of codes or a set of instructions, which are loaded and executed by the processor 501 to implement the method shown in embodiment one or embodiment two.
Wherein the electronic device 50 is the data sender when the processor 501 is loaded and executed to implement the method as shown in embodiment one. When loaded and executed by the processor 501 to implement the method of embodiment two, the electronic device 50 is a data sink.
In practical application, the structure of the electronic device is not limited to the embodiments of the present application.
The processor 501 may be a CPU, general purpose processor, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 501 may also be a combination that implements computing functionality, such as a combination comprising one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
Memory 502 may be, but is not limited to, a ROM or other type of static storage device, a RAM or other type of dynamic storage device, which may store static information and instructions, an EEPROM, CD-ROM or other optical disk storage, optical disk storage (including compact, laser, optical, digital versatile, blu-ray, etc.), magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The present application also provides a computer-readable storage medium for storing computer instructions that, when executed on a computer, cause the computer to perform the method of embodiment one or embodiment two.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for a person skilled in the art, several improvements and modifications can be made without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (10)
1. A method for bulk list transfer, comprising:
cutting and subdividing a large number of list files uploaded by a user, and analyzing each data fragment file after cutting and subdividing into a memory;
determining appointed themes respectively corresponding to at least one data receiver in a pre-built message queue middleware, wherein the at least one data receiver is in communication connection with the message queue middleware, the message queue middleware is respectively provided with corresponding themes according to different data sender and data receiver combinations, and each pair of data sender and data receiver are identified by the corresponding themes through data transmission of the message queue middleware;
and sending each data fragment file to the appointed theme of the message queue middleware so as to be obtained after the corresponding data receiver monitors the data fragment file.
2. The method for transmitting a large batch list according to claim 1, wherein after cutting and subdividing the large batch list file uploaded by the user, the method further comprises:
and determining the batch numbers and the corresponding data amounts of the data fragment files after cutting and subdividing, and notifying a data receiver.
3. The method for transmitting the bulk list according to claim 1 or 2, wherein the cutting and subdividing the bulk list file uploaded by the user comprises any one of the following modes:
horizontal slicing, vertical slicing, derived slicing, mixed slicing.
4. The method for acquiring the large-batch list is characterized by comprising the following steps of:
determining appointed themes respectively corresponding to at least one data sender in a pre-built message queue middleware, wherein the at least one data sender is in communication connection with the message queue middleware, corresponding themes are respectively set in the message queue middleware according to different data sender and data receiver combinations, and each pair of data sender and data receiver are identified by the corresponding themes through data transmission of the message queue middleware;
monitoring the data fragment files uploaded by the corresponding data transmitters in the appointed theme of the message queue middleware;
and acquiring the monitored data fragment files from the appointed theme of the message queue middleware to obtain a large number of list files.
5. The method of bulk roster acquisition of claim 4, further comprising:
receiving batch numbers and corresponding data amounts of all data fragment files notified by a data sender;
the monitoring of the data fragment files uploaded by the corresponding data sender in the appointed theme of the message queue middleware comprises the following steps:
and after receiving the batch numbers and the corresponding data amounts of the data fragment files notified by the data sender, monitoring the data fragment files uploaded by the corresponding data sender in the appointed theme of the message queue middleware.
6. The method of bulk roster acquisition of claim 5, further comprising:
and judging whether the large-batch list files are accepted to be completed or not according to the batch numbers of the data fragment files and the corresponding data quantity.
7. A bulk roster delivery device, comprising:
the cutting subdivision module is used for cutting and subdividing a large amount of list files uploaded by a user, and analyzing each data fragment file after cutting and subdivision into a memory;
the system comprises a determining module, a message queue middleware and a data receiving module, wherein the determining module is used for determining specified topics corresponding to at least one data receiving party in a pre-built message queue middleware respectively, the at least one data receiving party is in communication connection with the message queue middleware, corresponding topics are respectively set in the message queue middleware according to different data sending party and data receiving party combinations, and each pair of data sending party and data receiving party are identified by the corresponding topics through data transmission of the message queue middleware;
and the sending module is used for sending each data fragment file to the appointed theme of the message queue middleware so as to enable the corresponding data receiver to acquire the data fragment file after monitoring the data fragment file.
8. A bulk list retrieval apparatus comprising:
the system comprises a determining module, a message queue middleware and a message receiving module, wherein the determining module is used for determining specified topics corresponding to at least one data sender in a pre-built message queue middleware respectively, the at least one data sender is in communication connection with the message queue middleware, corresponding topics are respectively set in the message queue middleware according to different data sender and data receiver combinations, and each pair of data sender and data receiver are identified by the corresponding topics through data transmission of the message queue middleware;
the monitoring module is used for monitoring the data fragment files uploaded by the corresponding data transmitters in the appointed theme of the message queue middleware;
and the acquisition module is used for acquiring the monitored data fragment files from the appointed theme of the message queue middleware so as to obtain a large number of list files.
9. An electronic device, comprising:
a processor and a memory storing at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the bulk list delivery method of any one of claims 1-3, or the bulk list retrieval method of any one of claims 4-6.
10. A computer readable storage medium storing computer instructions which, when run on a computer, cause the computer to perform the method of bulk list delivery of any of claims 1-3 or the method of bulk list retrieval of any of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810951773.6A CN109088933B (en) | 2018-08-21 | 2018-08-21 | Large-batch list transmission method, large-batch list acquisition method, corresponding device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810951773.6A CN109088933B (en) | 2018-08-21 | 2018-08-21 | Large-batch list transmission method, large-batch list acquisition method, corresponding device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109088933A CN109088933A (en) | 2018-12-25 |
| CN109088933B true CN109088933B (en) | 2023-07-21 |
Family
ID=64793848
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810951773.6A Active CN109088933B (en) | 2018-08-21 | 2018-08-21 | Large-batch list transmission method, large-batch list acquisition method, corresponding device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109088933B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489246B (en) * | 2019-08-12 | 2022-06-07 | 中科寒武纪科技股份有限公司 | Data transmission and memory management method, device and related product |
| CN111182047B (en) * | 2019-12-24 | 2023-04-07 | 中国银联股份有限公司 | Method and system for transferring files between large data platforms across a network |
| CN111680248A (en) * | 2020-04-28 | 2020-09-18 | 五八有限公司 | Method and device for generating batch number of message pushed |
| CN112822260B (en) * | 2020-12-31 | 2023-06-09 | 北京天融信网络安全技术有限公司 | File transmission method and device, electronic equipment and storage medium |
| CN113220470A (en) * | 2021-03-31 | 2021-08-06 | 浙江太美医疗科技股份有限公司 | Method and system for transparent transmission of parameters between applications by using message queue |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104869174A (en) * | 2015-06-15 | 2015-08-26 | 北京邮电大学 | File transmission method based on third party message-oriented middle-ware |
| CN107332787A (en) * | 2017-06-29 | 2017-11-07 | 北京奇艺世纪科技有限公司 | A kind of message distributing method and device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9727396B2 (en) * | 2010-11-30 | 2017-08-08 | Red Hat, Inc. | Message queue deployment |
| US9544356B2 (en) * | 2014-01-14 | 2017-01-10 | International Business Machines Corporation | Message switch file sharing |
| CN106293968B (en) * | 2016-08-04 | 2019-05-10 | 华中科技大学 | A kind of intercommunication system and method based on Kafka message-oriented middleware |
| CN106354434B (en) * | 2016-08-31 | 2019-07-23 | 中国人民大学 | The storage method and system of daily record data |
-
2018
- 2018-08-21 CN CN201810951773.6A patent/CN109088933B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104869174A (en) * | 2015-06-15 | 2015-08-26 | 北京邮电大学 | File transmission method based on third party message-oriented middle-ware |
| CN107332787A (en) * | 2017-06-29 | 2017-11-07 | 北京奇艺世纪科技有限公司 | A kind of message distributing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109088933A (en) | 2018-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109088933B (en) | Large-batch list transmission method, large-batch list acquisition method, corresponding device and electronic equipment | |
| CN110347651B (en) | Cloud storage-based data synchronization method, device, equipment and storage medium | |
| CN109788026B (en) | Message processing method and device | |
| CN113485962B (en) | Log file storage method, device, equipment and storage medium | |
| CN113779339A (en) | Automatic monitoring and alarming method and system | |
| CN104468222A (en) | Method, device and system for reporting log information | |
| CN101977361A (en) | Method for preprocessing messages in batch | |
| CN113645287B (en) | Automobile message storage method and device and automobile message storage system | |
| CN112835885B (en) | Processing method, device and system for distributed form storage | |
| CN102882960A (en) | Method and device for transmitting resource files | |
| CN105740294B (en) | Information pushing method and equipment | |
| CN106156258B (en) | Method, device and system for counting data in distributed storage system | |
| CN112969198A (en) | Data transmission method, terminal and storage medium | |
| CN101572764B (en) | Method and system for sending pictures between electronic picture frames and electronic picture frame | |
| CN112637357A (en) | File transmission method and device and terminal equipment | |
| CN104065684A (en) | Information processing method, electronic device and terminal device | |
| CN109739972A (en) | A kind of novel recommended method and equipment | |
| CN110061902B (en) | Method, device and storage medium for sending messages regularly | |
| EP4092961A1 (en) | Group updating method, message sending method, and apparatuses | |
| CN112749204B (en) | Method and device for reading data | |
| CN114500036A (en) | Method, device and medium for transmitting data | |
| CN114079909A (en) | Network slice information processing method and device and network equipment | |
| CN108959302B (en) | Method for data processing and apparatus therefor | |
| CN113986833A (en) | File merging method, system, computer system and storage medium | |
| CN111209071A (en) | Information configuration method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |