CN111225046A - Method, device, medium and electronic equipment for internal and external network data transmission - Google Patents
Method, device, medium and electronic equipment for internal and external network data transmission Download PDFInfo
- Publication number
- CN111225046A CN111225046A CN201911423421.4A CN201911423421A CN111225046A CN 111225046 A CN111225046 A CN 111225046A CN 201911423421 A CN201911423421 A CN 201911423421A CN 111225046 A CN111225046 A CN 111225046A
- Authority
- CN
- China
- Prior art keywords
- messages
- partitions
- server
- data transmission
- intranet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000005192 partition Methods 0.000 claims abstract description 91
- 238000012544 monitoring process Methods 0.000 claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 6
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 238000012546 transfer Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 14
- 230000003993 interaction Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
Abstract
The embodiment of the invention relates to a method, a device, a medium and electronic equipment for internal and external network data transmission, wherein the method comprises the following steps: establishing a plurality of partitions in a message queue; acquiring a plurality of messages actively pushed from a first monitoring server; distributing the plurality of messages into the plurality of partitions according to processing speed; sending the messages in the plurality of partitions to a second listening server in response to a pull instruction from the second listening server. According to the invention, a data transmission platform is established between the internal network and the external network, the message queues distribute messages to different partitions, message data of different partitions are not repeated, the throughput of the messages can be increased, and more messages can be received; on the other hand, the messages are distributed to the partitions according to the processing speed, backlogs of the messages of different partitions caused by inconsistent processing speeds can be prevented, and the second monitoring server actively pulls the data, so that the server is prevented from being crashed due to too large data volume.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a method, a device, a medium and electronic equipment for data transmission of an internal network and an external network.
Background
The self-service is an informationized and specialized citizen service means, can solve the problems of heavy work burden and low efficiency of clerks and front desk staffs, becomes an important informationized means for department optimization service, creation of service type governments and creation of good work relations, even becomes a main work way in partial services, and plays an important role in meeting diversified tax-handling requirements of clerks.
In the implementation process of self-service, the interconnection and deep fusion between the internet and an intranet handling specific services (such as tax handling) are realized by relying on an internet platform. The database for handling specific services is in the intranet, and for the consideration of information security, the external internet (called extranet for short) cannot directly access and perform data interaction inside and outside the specific services, so that support is provided for self-service tax handling with safety, convenience and wide range of construction, illegal access is prevented, and data leakage is prevented. And constructing an internal and external network interconnection platform. The self-service terminal can be deployed to a place outside a hall through the platform, so that the self-service terminal in the certified public place can use various self-service office services of the intranet tax terminal. However, the data transmission and processing efficiency of the existing internal and external network interconnection platform cannot meet the requirements of more and more services and more extensive service ranges, and the working efficiency needs to be further improved.
Based on the above, the internal and external network interconnection platform in the prior art has the problems of unstable data interaction and low transmission efficiency.
The above drawbacks are expected to be overcome by those skilled in the art.
Disclosure of Invention
Technical problem to be solved
In order to solve the above problems in the prior art, the invention provides a method, a device, a medium and an electronic device for internal and external network data transmission, which solve the problems of unstable data interaction and low transmission efficiency of an internal and external network interconnection platform in the prior art.
(II) technical scheme
In order to achieve the purpose, the invention adopts the main technical scheme that:
according to a first aspect of the embodiments of the present invention, there is provided a method for internal and external network data transmission, including:
establishing a plurality of partitions in a message queue;
acquiring a plurality of messages actively pushed from a first monitoring server;
distributing the plurality of messages into the plurality of partitions according to processing speed;
sending messages in the plurality of partitions to a second listening server by responding to a pull instruction from the second listening server;
the first monitoring server and the second monitoring server are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
In an exemplary embodiment of the present invention, the establishing the plurality of partitions in the message queue includes:
and dividing according to the service scene, the number of logs in the log list and the number of consumers at the consumption end to obtain a plurality of partitions.
In an exemplary embodiment of the invention, the partitioning employs RocktMQ, Kafka, or Pulsar.
In an exemplary embodiment of the present invention, the plurality of messages are pushed by the first listening server at a time, and the first listening server pushes the messages at intervals of a preset time.
In an exemplary embodiment of the present invention, before the distributing the plurality of messages to the plurality of partitions according to the processing speed, the method further includes:
generating a corresponding monitoring signal once the message enters the message queue;
and calculating to obtain the processing speed according to the number of the monitoring signals in the preset time period.
In an exemplary embodiment of the present invention, the allocating the plurality of messages to the plurality of partitions according to the processing speed comprises:
obtaining the size of the current available storage space of the plurality of partitions;
and performing balanced distribution on the plurality of messages according to the size of the current available storage space, so that the number difference of the messages in different partitions is within a preset range.
In an exemplary embodiment of the present invention, sending the messages in the plurality of partitions to a second listening server in response to a pull instruction from the second listening server comprises:
detecting a timing refreshing result of the second monitoring server, if the refreshing result is idle, establishing connection with the browser, and acquiring a pull instruction through the connection;
matching the corresponding partition according to the semantic information of the second monitoring server corresponding to the pull instruction;
and sending the matched message in the partition to the second monitoring server.
According to a second aspect of the embodiments of the present invention, there is provided an apparatus for internal and external network data transmission, including:
a partition module configured to establish a plurality of partitions in a message queue;
the pushing module is configured to acquire a plurality of messages actively pushed from the first monitoring server;
an allocation module configured to allocate the plurality of messages into the plurality of partitions according to processing speed;
a pull module configured to send messages in the plurality of partitions to a second listening server by responding to a pull instruction from the second listening server;
the first monitoring server and the second monitoring server are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
According to a third aspect of embodiments of the present invention, there is provided a computer readable medium, having stored thereon a computer program, which when executed by a processor, performs the steps of the method for intranet and extranet data transmission described above.
According to a fourth aspect of embodiments of the present invention, there is provided an electronic apparatus, including:
one or more processors;
a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the above-described method for intranet and extranet data transmission.
(III) advantageous effects
The invention has the beneficial effects that: according to the method, the device, the medium and the electronic equipment for data transmission of the internal network and the external network, on one hand, a data transmission platform is established between the internal network and the external network, the message queue distributes messages to different partitions, message data of different partitions are not repeated, message throughput can be increased, and more messages can be received; on the other hand, the messages are distributed to the partitions according to the processing speed, backlogs of the messages of different partitions caused by inconsistent processing speeds can be prevented, and the second monitoring server actively pulls the data, so that the server is prevented from being crashed due to too large data volume.
Drawings
Fig. 1 is a system scenario block diagram of a method and an apparatus for intranet and extranet data transmission provided in the present invention;
fig. 2 is a schematic flowchart of a method for internal and external network data transmission according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a system architecture for implementing Intranet and Intranet data transfer according to an embodiment of the present invention;
FIG. 4 is a process diagram of data interaction according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an apparatus for data transmission based on intranet and extranet according to another embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer system of an electronic device according to still another embodiment of the present invention.
Detailed Description
For the purpose of better explaining the present invention and to facilitate understanding, the present invention will be described in detail by way of specific embodiments with reference to the accompanying drawings.
All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Fig. 1 is a system scenario block diagram illustrating a method and apparatus for intranet and extranet data transmission according to an exemplary embodiment.
As shown in fig. 1, the system architecture 100 may include user terminal devices 101, 102, 103, a network 104, a server 105, and business service terminal devices 106, 107, 108. The network 104 serves to provide a medium for communication links between the terminal devices 101, 102, 103, the server 105 and the terminal devices 106, 107, 108. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal equipment comprises user terminal equipment and business service terminal equipment, wherein the user terminal equipment is tax application provided for a user and comprises a mobile phone APP, a self-service terminal or a web terminal and the like, and the business service terminal equipment is an intranet for handling specific business (such as tax) or government affairs. A user may interact with a server 105 via a network 104 using user terminal devices 101, 102, 103 to receive or send messages or the like. Business service personnel (government service personnel such as tax clerks) may also interact with the server 105 via the network 104 using business service terminals to receive or send messages and the like. Various communication client applications, such as tax handling software, etc., may be installed on the user terminal devices 101, 102, 103 and the business service terminal devices 106, 107, 108.
The user terminal devices 101, 102, 103 and the business service terminal devices 106, 107, 108 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various data services, and the server 105 may be provided with a tax administration platform (mainly a background) for acquiring messages input by the user through the extranet through the user terminal devices 101, 102, 103. The server 105 may also be a background server for performing data storage processing on messages provided by the service terminal devices 106, 107, and 108.
The server 105 can also obtain the information of the tax clerk through the business service terminal, and the server 105 can also establish a buffer area between the internal network and the external network for establishing a plurality of partitions in the information queue; acquiring a plurality of messages actively pushed from a first monitoring server; distributing the plurality of messages into the plurality of partitions according to processing speed; sending the messages in the plurality of partitions to a second listening server in response to a pull instruction from the second listening server.
The server 105 may be a server of one entity, or may be composed of a plurality of servers, for example, and a part of the server 105 may store a message queue.
It should be noted that the method for intranet and extranet data transmission provided by the embodiment of the present disclosure may be executed by the server 105, and accordingly, an apparatus for intranet and extranet data transmission may be disposed in the server 105.
The technical solution of the present disclosure is specifically described below.
Fig. 2 is a schematic flow chart of a method for internal and external network data transmission according to an embodiment of the present invention, as shown in fig. 2, the method includes the following steps:
as shown in fig. 2, in step S210, a plurality of partitions are established in the message queue;
as shown in fig. 2, in step S220, a plurality of messages actively pushed from the first listening server are obtained;
as shown in fig. 2, in step S230, the plurality of messages are allocated to the plurality of partitions according to the processing speed;
as shown in fig. 2, in step S240, the messages in the plurality of partitions are sent to the second listening server in response to a pull instruction from the second listening server.
The first monitoring server and the second monitoring server are respectively arranged in the outer network and the inner network or respectively arranged in the inner network and the outer network, namely, the data transmission method in the steps is suitable for data transmission from the inner network to the outer network and is also suitable for data transmission from the outer network to the inner network.
In the technical solution provided in the embodiment shown in fig. 2, on one hand, a data transmission platform is established between an intranet and an extranet, and a message queue allocates messages to different partitions, so that message data of different partitions are not repeated, message throughput can be increased, and more messages can be received; on the other hand, the messages are distributed to the partitions according to the processing speed, backlogs of the messages of different partitions caused by inconsistent processing speeds can be prevented, and the second monitoring server actively pulls the data, so that the server is prevented from being crashed due to too large data volume.
Fig. 3 is a system structure diagram for implementing data transmission between the internal network and the external network in an embodiment of the present invention, as shown in fig. 3, the system structure includes an external network 310, a DMZ Zone (isolated Zone) 320, and an internal network 330, where the external network 310 faces a user, and applications of the external network include a mobile phone APP, a self-service terminal APP, a WEB APP, and the like. The DMZ zone 320 prevents the intranet from directly communicating with the extranet to ensure the security of the intranet, and includes an extranet access service and a data interaction zone, where data interaction is performed between the extranet access service and the data interaction zone, and data interaction is also performed between the data interaction zone and the intranet 330. A request configuration service is built into intranet 330 and data requests are distributed to a tax intranet service, a government intranet service, or other services based on the configuration service.
In fig. 3, the intranet 330 can freely access the extranet 310, the firewall needs to perform source address conversion, the intranet 330 can access the DMZ zone 320, and can facilitate the intranet user to use and manage the servers in the DMZ, the extranet 310 can also access the DMZ zone 320, but the extranet 310 cannot access the intranet 330, and the DMZ zone 320 cannot access both the intranet 330 and the extranet 310.
The following describes in detail the specific implementation of each step of the embodiment shown in fig. 2 by taking data transmission between the self-service terminal and the intranet for tax handling as an example and combining with the system structure shown in fig. 3:
in step S210, a plurality of partitions are established in the message queue.
In an exemplary embodiment of the present invention, the establishing of the partition in this step may specifically be: and dividing according to the service scene, the number of logs in the log list and the number of consumers at the consumption end to obtain a plurality of partitions.
In an exemplary embodiment of the invention, the partitioning employs RocktMQ, Kafka, or Pulsar.
In step S220, a plurality of messages actively pushed from the first listening server are obtained.
In an exemplary embodiment of the present invention, the plurality of messages are pushed by the first listening server at a time, and the first listening server pushes the messages at intervals of a preset time.
In the step, the first monitoring server actively pushes the data, so that the data are pushed every time, and the resource consumption of null polling is avoided.
In step S230, the plurality of messages are distributed into the plurality of partitions according to the processing speed.
In an exemplary embodiment of the present invention, before the distributing the plurality of messages to the plurality of partitions according to the processing speed, the method further includes:
generating a corresponding monitoring signal once the message enters the message queue;
and calculating to obtain the processing speed according to the number of the monitoring signals in the preset time period.
In an exemplary embodiment of the present invention, the allocating the plurality of messages to the plurality of partitions according to the processing speed comprises:
obtaining the size of the current available storage space of the plurality of partitions;
and performing balanced distribution on the plurality of messages according to the size of the current available storage space, so that the number difference of the messages in different partitions is within a preset range.
In the step, a load balancing means is adopted for partitioning to realize balanced distribution, and the realization mode can be that a plurality of messages are directly and evenly distributed to different partitions, and can also be that the messages are distributed according to the stored condition of the partitions, so that the different partitions after distribution have little difference. In other embodiments of the present invention, other load balancing mechanisms may be used to avoid data backlog caused by excessive partition messages.
In step S240, the messages in the plurality of partitions are sent to a second listening server by responding to a pull instruction from the second listening server.
The first monitoring server and the second monitoring server in the step are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
In an exemplary embodiment of the present invention, sending the messages in the plurality of partitions to a second listening server in response to a pull instruction from the second listening server comprises:
firstly, detecting a timing refreshing result of the second monitoring server, if the refreshing result is idle, establishing connection with a browser, and acquiring a pull instruction through the connection;
secondly, matching a corresponding partition according to semantic information of a second monitoring server corresponding to the pull instruction;
and finally, sending the matched message in the partition to the second monitoring server.
Based on the above, in the method in this embodiment, the producer actively pushes the message, and the consumer actively pulls the message to complete data transmission, so that resource waste caused by continuous polling is avoided, the message queue is partitioned in the middle, data backlog is avoided, the capability of processing data is improved, and the stability of the system is ensured.
Fig. 4 is a schematic diagram of a process of data interaction in an embodiment of the present invention, as shown in fig. 4, taking as an example that a first monitoring server is disposed in an external network (i.e., external network monitoring), and a second monitoring server is disposed in an internal network (i.e., internal network monitoring), the process is described as follows:
4.1) the extranet listens for actively pushing messages to the extranet message queue, which distributes the messages evenly to the different partitions, namely receiving partition 1, receiving partition 2 and receiving partition 3.
The starting process of the outer network monitoring and the message queue comprises the following steps: firstly, establishing connection with a browser, then acquiring all log lists (topoic lists), and finally sending messages to a message queue according to a specified routing strategy.
The message data of different partitions are not repeated, so that the throughput of the message can be increased, and the data receiving speed is improved.
4.2) the intranet monitors and subscribes the queue information of the extranet, and the intranet monitors and actively pulls the message to the message queue of the extranet and processes the message.
The starting process of the intranet monitoring and the message queue comprises the following steps: firstly, establishing connection with a browser, then acquiring metadata of all log lists (topic lists), connecting to different partitions according to different semantics, and finally pulling messages from a message queue.
4.3) the message queue of the external network distributes the data of different subareas to the internal network monitoring and pulling according to the processing speed of the internal network, thereby preventing the data backlog of different subareas caused by the inconsistency of the processing speed. Because the intranet monitoring is actively pulling the data, the crash caused by overlarge data volume can be avoided.
As shown in fig. 4, the process of the outer network monitoring and acquiring data from the inner network is similar, and the simple description is as follows: the inner network monitors the push message to the message queue, distributes the push message to different subareas, namely a return subarea, a return subarea 2 and a return subarea 3, and then monitors the active pull message by the outer network.
In summary, based on the method for transmitting data between the internal network and the external network provided by the present invention, on one hand, a data transmission platform is established between the internal network and the external network, and the message queues allocate messages to different partitions, so that message data of different partitions are not repeated, message throughput can be increased, more messages can be received, and processing efficiency can be increased; on the other hand, the messages are distributed to the partitions according to the processing speed, backlogs of the messages of different partitions caused by inconsistent processing speed can be prevented, and the second monitoring server actively pulls the data, so that the server is not crashed due to too large data volume, and efficient data transmission is continuously maintained. The method is applied to the self-service terminal, so that the application range of the self-service terminal can be enlarged, the work efficiency is improved, and the pressure of a foreground is reduced. By means of a large Internet platform, user experience of the self-service terminal is enhanced, functions of the Internet self-service terminal are enriched, the space and time range of the self-service terminal are expanded, and deep fusion of Internet and self-service business is achieved.
Corresponding to the method based on intranet and extranet data transmission, fig. 5 is a schematic diagram of an apparatus based on intranet and extranet data transmission according to another embodiment of the present invention, and referring to fig. 5, the apparatus 500 includes: a partition module 510, a push module 520, a distribution module 530, and a pull module 540.
The partition module 510 is configured to establish a plurality of partitions in a message queue; the pushing module 520 is configured to obtain a plurality of actively pushed messages from the first listening server; the distribution module 530 is configured to distribute the plurality of messages into the plurality of partitions according to processing speed; the pull module 540 is configured to send the messages in the plurality of partitions to a second listening server by responding to a pull instruction from the second listening server; the first monitoring server and the second monitoring server are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
For details that are not disclosed in the embodiment of the apparatus of the present disclosure, please refer to the embodiment of the method for internal and external network data transmission described above in the present disclosure for details that are not disclosed in the embodiment of the apparatus of the present disclosure, because each functional module of the apparatus for internal and external network data transmission of the present disclosure corresponds to the step of the above-described example embodiment of the method for internal and external network data transmission described above in fig. 2.
In summary, with the device for data transmission between the internal network and the external network provided by the embodiment of the present disclosure, on one hand, a data transmission platform is established between the internal network and the external network, and a message queue allocates messages to different partitions, and message data of different partitions are not repeated, so that throughput of messages can be increased, more messages can be received, and processing efficiency can be increased; on the other hand, the messages are distributed to the partitions according to the processing speed, backlogs of the messages of different partitions caused by inconsistent processing speed can be prevented, and the second monitoring server actively pulls the data, so that the server is not crashed due to too large data volume, and efficient data transmission is continuously maintained. The method is applied to the self-service terminal, so that the application range of the self-service terminal can be enlarged, the work efficiency is improved, and the pressure of a foreground is reduced.
Referring now to FIG. 6, shown is a block diagram of a computer system suitable for use with the electronic device implementing an embodiment of the present invention. The computer system 600 of the electronic device shown in fig. 6 is only an example, and should not bring any limitation to the function and the scope of the use of the embodiments of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for system operation are also stored. The CPU601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present application are executed when the computer program is executed by the Central Processing Unit (CPU) 601.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is enabled to implement the intranet and extranet data transmission method as described in the above embodiments.
For example, the electronic device may implement the following as shown in fig. 2: step S210, establishing a plurality of partitions in a message queue; step S220, a plurality of messages actively pushed by the first monitoring server are obtained; step S230, distributing the plurality of messages to the plurality of partitions according to the processing speed; step S240, sending the messages in the plurality of partitions to a second monitoring server by responding to a pull instruction from the second monitoring server; the first monitoring server and the second monitoring server are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. A method for intranet and extranet data transmission, comprising:
establishing a plurality of partitions in a message queue;
acquiring a plurality of messages actively pushed from a first monitoring server;
distributing the plurality of messages into the plurality of partitions according to processing speed;
sending messages in the plurality of partitions to a second listening server by responding to a pull instruction from the second listening server;
the first monitoring server and the second monitoring server are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
2. The method for intranet and extranet data transmission according to claim 1, wherein the establishing the plurality of partitions in the message queue comprises:
and dividing according to the service scene, the number of logs in the log list and the number of consumers at the consumption end to obtain a plurality of partitions.
3. The method for intranet and extranet data transmission according to claim 2, wherein the partitioning is performed by using rockmq, Kafka or Pulsar.
4. The method for intranet and extranet data transmission according to claim 1, wherein the plurality of messages are pushed by the first listening server at a time, and the first listening server is pushed at intervals of a preset time.
5. The method for intranet and extranet data transmission according to claim 1, wherein before distributing the plurality of messages to the plurality of partitions according to the processing speed, further comprising:
generating a corresponding monitoring signal once the message enters the message queue;
and calculating to obtain the processing speed according to the number of the monitoring signals in the preset time period.
6. The method for intranet and extranet data transmission according to claim 1, wherein distributing the plurality of messages to the plurality of partitions according to processing speed comprises:
obtaining the size of the current available storage space of the plurality of partitions;
and performing balanced distribution on the plurality of messages according to the size of the current available storage space, so that the number difference of the messages in different partitions is within a preset range.
7. The method for intranet and extranet data transfer of claim 1 wherein sending messages in the plurality of partitions to a second listening server in response to a pull instruction from the second listening server comprises:
detecting a timing refreshing result of the second monitoring server, if the refreshing result is idle, establishing connection with the browser, and acquiring a pull instruction through the connection;
matching the corresponding partition according to the semantic information of the second monitoring server corresponding to the pull instruction;
and sending the matched message in the partition to the second monitoring server.
8. An apparatus for intranet and extranet data transmission, comprising:
a partition module configured to establish a plurality of partitions in a message queue;
the pushing module is configured to acquire a plurality of messages actively pushed from the first monitoring server;
an allocation module configured to allocate the plurality of messages into the plurality of partitions according to processing speed;
a pull module configured to send messages in the plurality of partitions to a second listening server by responding to a pull instruction from the second listening server;
the first monitoring server and the second monitoring server are respectively arranged in an outer network and an inner network or respectively arranged in the inner network and the outer network.
9. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the method for intranet and extranet data transmission according to any one of claims 1 to 7.
10. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method for intranet and extranet data transmission according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911423421.4A CN111225046A (en) | 2019-12-31 | 2019-12-31 | Method, device, medium and electronic equipment for internal and external network data transmission |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911423421.4A CN111225046A (en) | 2019-12-31 | 2019-12-31 | Method, device, medium and electronic equipment for internal and external network data transmission |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111225046A true CN111225046A (en) | 2020-06-02 |
Family
ID=70829216
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911423421.4A Pending CN111225046A (en) | 2019-12-31 | 2019-12-31 | Method, device, medium and electronic equipment for internal and external network data transmission |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111225046A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113242244A (en) * | 2021-05-12 | 2021-08-10 | 北京百度网讯科技有限公司 | Data transmission method, device and system |
| CN113630611A (en) * | 2021-07-09 | 2021-11-09 | 河北迈道信息技术有限公司 | Method for realizing court trial live broadcast without networking internal and external networks |
| CN113660338A (en) * | 2021-08-17 | 2021-11-16 | 中国农业银行股份有限公司 | Data interaction method, device and system and server |
| CN114827064A (en) * | 2022-04-18 | 2022-07-29 | 京东科技信息技术有限公司 | Message pushing method and device, electronic equipment and readable medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912412A (en) * | 2016-03-31 | 2016-08-31 | 北京奇虎科技有限公司 | Message distribution method, message distribution device and message distribution system |
| CN106060011A (en) * | 2016-05-12 | 2016-10-26 | 浪潮软件集团有限公司 | Intranet and extranet interaction method, self-service tax handling system, extranet device and intranet device |
| CN108365971A (en) * | 2018-01-10 | 2018-08-03 | 深圳市金立通信设备有限公司 | Daily record analytic method, equipment and computer-readable medium |
| CN108737570A (en) * | 2018-06-22 | 2018-11-02 | 北京奇艺世纪科技有限公司 | A kind of information-pushing method, apparatus and system |
-
2019
- 2019-12-31 CN CN201911423421.4A patent/CN111225046A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912412A (en) * | 2016-03-31 | 2016-08-31 | 北京奇虎科技有限公司 | Message distribution method, message distribution device and message distribution system |
| CN106060011A (en) * | 2016-05-12 | 2016-10-26 | 浪潮软件集团有限公司 | Intranet and extranet interaction method, self-service tax handling system, extranet device and intranet device |
| CN108365971A (en) * | 2018-01-10 | 2018-08-03 | 深圳市金立通信设备有限公司 | Daily record analytic method, equipment and computer-readable medium |
| CN108737570A (en) * | 2018-06-22 | 2018-11-02 | 北京奇艺世纪科技有限公司 | A kind of information-pushing method, apparatus and system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113242244A (en) * | 2021-05-12 | 2021-08-10 | 北京百度网讯科技有限公司 | Data transmission method, device and system |
| CN113242244B (en) * | 2021-05-12 | 2022-11-15 | 北京百度网讯科技有限公司 | Data transmission method, device and system |
| CN113630611A (en) * | 2021-07-09 | 2021-11-09 | 河北迈道信息技术有限公司 | Method for realizing court trial live broadcast without networking internal and external networks |
| CN113660338A (en) * | 2021-08-17 | 2021-11-16 | 中国农业银行股份有限公司 | Data interaction method, device and system and server |
| CN114827064A (en) * | 2022-04-18 | 2022-07-29 | 京东科技信息技术有限公司 | Message pushing method and device, electronic equipment and readable medium |
| CN114827064B (en) * | 2022-04-18 | 2023-08-04 | 京东科技信息技术有限公司 | Message pushing method, device, electronic equipment and readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111225046A (en) | Method, device, medium and electronic equipment for internal and external network data transmission | |
| CN112860451A (en) | Multi-tenant data processing method and device based on SaaS | |
| CN111427706A (en) | Data processing method, multi-server system, database, electronic device, and storage medium | |
| CN111478781B (en) | Message broadcasting method and device | |
| CN111917687A (en) | Method and device for circularly pushing reminding message | |
| CN110120961A (en) | A kind of method of Distributed Services cluster and its route synchronization | |
| CN112379982A (en) | Task processing method and device, electronic equipment and computer readable storage medium | |
| CN112084042B (en) | Message processing method and device | |
| CN115237589A (en) | SR-IOV-based virtualization method, device and equipment | |
| CN111596992B (en) | Navigation bar display method and device and electronic equipment | |
| CN112686528A (en) | Method, apparatus, server and medium for allocating customer service resources | |
| CN112306685A (en) | Task isolation method and device, electronic equipment and computer readable medium | |
| US10893015B2 (en) | Priority topic messaging | |
| CN113778499A (en) | Method, device, equipment and computer readable medium for publishing service | |
| CN106408793B (en) | A kind of Service Component sharing method and system suitable for ATM business | |
| CN111797160B (en) | Method, system and electronic device for sharing intelligent contract | |
| CN110661857B (en) | Data synchronization method and device | |
| CN114374703A (en) | Method, device and equipment for acquiring cloud mobile phone information and storage medium | |
| CN111538721A (en) | Account processing method and device, electronic equipment and computer readable storage medium | |
| CN110099122B (en) | Method and apparatus for sending network request | |
| CN112468479B (en) | Remote assistance method of contact center and CTI (computer telephony integration) assembly | |
| CN112966008B (en) | Data caching method, loading method, updating method and related devices | |
| CN116755889B (en) | Data acceleration method, device and equipment applied to server cluster data interaction | |
| US20230214766A1 (en) | Product management method and related device | |
| CN113452739A (en) | Message processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200602 |
|
| RJ01 | Rejection of invention patent application after publication |