CN111930542A - Data acquisition method and device, electronic equipment and computer readable storage medium - Google Patents
Data acquisition method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111930542A CN111930542A CN202010834490.0A CN202010834490A CN111930542A CN 111930542 A CN111930542 A CN 111930542A CN 202010834490 A CN202010834490 A CN 202010834490A CN 111930542 A CN111930542 A CN 111930542A
- Authority
- CN
- China
- Prior art keywords
- server
- data acquisition
- target client
- available
- connection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 53
- 238000004891 communication Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 8
- 238000013480 data collection Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/541—Client-server
Abstract
When a control server acquires a data acquisition instruction, an available server is selected from a server cluster, and the data acquisition instruction is sent to the available server, so that the available server finds a corresponding connection channel from a cache server according to a target client identifier pointed by the data acquisition instruction and establishes long connection with a target client, and data acquisition is performed on the target client. Because the connection channel is not stored in the server which establishes connection with the target client, the connection channel still does not disappear even if the server which establishes long connection with the target client is down. After the control server determines a new available server again, long connection can be established with the target client based on the original connection channel, so that compared with the prior art, the time required for reestablishing the connection pipeline is saved, and the data acquisition efficiency can be improved.
Description
Technical Field
The application belongs to the field of data acquisition, and particularly relates to a data acquisition method and device, electronic equipment and a computer-readable storage medium.
Background
In some application scenarios, the server collects data provided by the client by establishing a long TCP connection with the client. The long connection is a class object occupying thread resources and can be understood as a pipe.
In the prior art, after a long connection pipeline is established between a client and a server, the pipeline is stored in the corresponding server, that is, the pipeline and the server are bound together. Once a server is down, the pipe is unavailable and, correspondingly, the long connection between the server and the client is interrupted. At this time, in order to ensure successful acquisition of data, the client needs to search for a new server to initiate a request for establishing a long connection pipeline again, and perform data interaction through the new pipeline after the pipeline is established.
However, the process of establishing the pipeline requires additional time and memory resources, which affects the data acquisition efficiency.
Disclosure of Invention
In view of this, an object of the present application is to provide a data acquisition method, an apparatus, an electronic device and a computer-readable storage medium, which can improve the efficiency of data acquisition.
The embodiment of the application is realized as follows:
in a first aspect, an embodiment of the present application provides a data acquisition method, which is applied to a control server, where the control server is in communication connection with a server cluster, the server cluster is in communication connection with a cache server, and a connection pipeline corresponding to a client identifier is pre-stored in the cache server, where the method includes: acquiring a data acquisition instruction, wherein the data acquisition instruction comprises a target client identification corresponding to a target client; determining an available server from the server cluster; and sending the data acquisition instruction to the available server so that the available server searches for and calls a connecting pipeline corresponding to the target client identifier from the cache server to establish long connection with the target client. Because the connection channel is not stored in the server which establishes connection with the target client, the connection channel corresponding to the target client identifier still does not disappear even if the server which establishes long connection with the target client is down. After the control server determines a new available server again, the new available server can still establish a long connection with the target client based on the original connection channel without establishing a connection pipeline again, so that compared with the prior art, the time required for establishing the connection pipeline again can be saved, and the data acquisition efficiency is improved.
With reference to the embodiment of the first aspect, in a possible implementation manner, the determining an available server from the server cluster includes: sending state query information to each server in the server cluster; acquiring state information fed back by each server in the server cluster, wherein the state information is used for representing whether the server cluster is in an idle state or not; according to the state information, determining a server in an idle state from the servers feeding back the state information; and determining one server from the servers in the idle state as the available server. Namely, the state information fed back by each server can be obtained by sending the state query information to the server.
With reference to the embodiment of the first aspect, in a possible implementation manner, the determining, from the servers in the idle state, that one server is determined to be the available server includes: randomly determining one server from the servers in the idle state to be determined as the available server; or, according to a priority order stored in advance, determining the server with the highest priority among the servers in the idle state as the available server.
With reference to the embodiment of the first aspect, in a possible implementation manner, the data acquisition instruction includes a data acquisition time period, and before the determining an available server from the server cluster, the method further includes: and determining that no server which establishes effective connection with the target client exists in the server cluster in the time period represented by the data acquisition time period.
With reference to the embodiment of the first aspect, in a possible implementation manner, storing a plurality of connection pipes corresponding to the target client identifier in the cache server, where determining an available server from the server cluster includes: and determining a plurality of available servers from the server cluster, wherein the number of the available servers is consistent with the number of the connecting pipelines corresponding to the target client identification.
In a second aspect, an embodiment of the present application provides a data acquisition method, which is applied to a server, where the server establishes a communication connection with a cache server, and a connection pipe that establishes a correspondence with a client identifier is pre-stored in the cache server, and the method includes: acquiring a data acquisition instruction, wherein the data acquisition instruction comprises a target client identification corresponding to a target client; searching and calling a connecting pipeline corresponding to the target client identifier from the cache server; and establishing long connection with the target client through the connecting pipeline corresponding to the target client identifier.
In a third aspect, an embodiment of the present application provides a data acquisition device, which is applied to a control server, where the control server is in communication connection with a server cluster, the server cluster is in communication connection with a cache server, and a connection pipeline corresponding to a client identifier is pre-stored in the cache server, where the data acquisition device includes: the device comprises an acquisition module, a determination module and a sending module. The system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a data acquisition instruction, and the data acquisition instruction comprises a target client identification corresponding to a target client; a determining module, configured to determine an available server from the server cluster; and the sending module is used for sending the data acquisition instruction to the available server so that the available server searches for and calls a connecting pipeline corresponding to the target client identifier from the cache server to establish long connection with the target client.
With reference to the third aspect of the embodiments, in a possible implementation manner, the determining module is configured to send status query information to each server in the server cluster; acquiring state information fed back by each server in the server cluster, wherein the state information is used for representing whether the server cluster is in an idle state or not; according to the state information, determining a server in an idle state from the servers feeding back the state information; and determining one server from the servers in the idle state as the available server.
With reference to the third aspect of the embodiments, in a possible implementation manner, the determining module is configured to randomly determine, from the servers in the idle state, one server as the available server;
or, the server determining unit is configured to determine, according to a priority order stored in advance, a server with a highest priority among the servers in the idle state as the available server.
With reference to the embodiment of the third aspect, in a possible implementation manner, the data acquisition instruction includes a data acquisition time period, and the determining module is further configured to determine, in a time period represented by the data acquisition time period, that a server that establishes an effective connection with the target client does not exist in the server cluster.
With reference to the third aspect embodiment, in a possible implementation manner, a plurality of connection pipes corresponding to the target client identifier are stored in the cache server, and the determining module is configured to determine a plurality of available servers from the server cluster, where the number of the available servers is consistent with the number of the connection pipes corresponding to the target client identifier.
In a fourth aspect, an embodiment of the present application provides a data acquisition device, which is applied to a server, where the server establishes a communication connection with a cache server, and a connection pipe that establishes a correspondence with a client identifier is pre-stored in the cache server, and the device includes: the device comprises an acquisition module, a search calling module and an establishment module. The system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a data acquisition instruction, and the data acquisition instruction comprises a target client identification corresponding to a target client; the searching and calling module is used for searching and calling the connecting pipeline corresponding to the target client identifier from the cache server; and the establishing module is used for establishing long connection with the target client through the connecting pipeline corresponding to the target client identifier.
A fifth embodiment of the present application further provides an electronic device, including: a memory and a processor, the memory and the processor connected; the memory is used for storing programs; the processor calls a program stored in the memory to perform the method of the first aspect embodiment and/or any possible implementation manner of the first aspect embodiment.
In a sixth aspect, the present application further provides a non-transitory computer-readable storage medium (hereinafter, referred to as a computer-readable storage medium), on which a computer program is stored, where the computer program is executed by a computer to perform the method in the foregoing first aspect and/or any possible implementation manner of the first aspect.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts. The foregoing and other objects, features and advantages of the application will be apparent from the accompanying drawings. Like reference numerals refer to like parts throughout the drawings. The drawings are not intended to be to scale as practical, emphasis instead being placed upon illustrating the subject matter of the present application.
Fig. 1 shows a schematic structural diagram of a data acquisition system provided in an embodiment of the present application.
Fig. 2 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Fig. 3 shows one of flowcharts of a data acquisition method provided in an embodiment of the present application.
Fig. 4 shows a second flowchart of a data acquisition method according to an embodiment of the present application.
Fig. 5 shows one of the block diagrams of the data acquisition device according to the embodiment of the present application.
Fig. 6 shows a second block diagram of a data acquisition device according to an embodiment of the present application.
Icon: 10-a data acquisition system; 11-a control server; 12-a cluster of servers; 13-a cache server; 14-a client; 100-an electronic device; 110-a processor; 120-a memory; 400-a data acquisition device; 410-an obtaining module; 420-a determination module; 430-a sending module; 500-a data acquisition device; 510-an obtaining module; 520-find call module; 530-building block.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, relational terms such as "first," "second," and the like may be used solely in the description herein to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Further, the term "and/or" in the present application is only one kind of association relationship describing the associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone.
In addition, the defects (affecting the data acquisition efficiency) existing in the data acquisition schemes in the prior art are the results obtained after the applicant has practiced and studied carefully, and therefore, the discovery process of the defects and the solutions proposed in the following embodiments of the present application for the defects should be considered as contributions of the applicant to the present application.
In order to solve the above problem, embodiments of the present application provide a data acquisition method, an apparatus, an electronic device, and a computer-readable storage medium, which can improve the efficiency of data acquisition.
The technology can be realized by adopting corresponding software, hardware and a combination of software and hardware. The following describes embodiments of the present application in detail.
First, a data acquisition system 10 for implementing the data acquisition method of the embodiment of the present application is described with reference to fig. 1. The data acquisition system 10 includes a control server 11, a server cluster 12, and a cache server 13.
The control server 11 is connected with the server cluster 12 in a communication way, and the server cluster 12 is connected with the cache server 13 in a communication way.
The control server 11 may perform data interaction with each server included in the server cluster 12. Each server included in the server cluster 12 may have the same data acquisition function on the premise that the server is not down, and of course, each server may also execute different acquisition functions according to the configuration of the user.
The cache server 13 is configured to store a connection pipe having a correspondence relationship with the client identifier of each client 14. Of course, the cache server 13 here may also be a cache server cluster including a plurality of cache servers 13, for example, a redis cluster.
It is worth noting that the connection pipe pre-creates class objects for the corresponding clients 14, which occupy thread resources. In the large context of mass interconnections, there must be a large number of connecting conduits. In the embodiment of the present application, the connection pipeline is separately stored in the cache server 13 instead of the server cluster 12, so that consumption of the memory resource of the server cluster 12 can be reduced, and on the premise that the total memory resource of the server cluster 12 is not changed, the server cluster 12 can use more memory resources for data acquisition, thereby improving acquisition performance of the server cluster 12. In addition, since the content to be stored by the server cluster 12 is reduced, the difficulty of maintaining the server cluster 12 can be reduced.
If the connection pipeline has a corresponding relationship with a certain client identifier, correspondingly, one of the ports of the connection pipeline is in communication with the client 14 corresponding to the client identifier, so that the device in communication with the other port of the connection pipeline can acquire data provided by the client 14 corresponding to the client identifier.
For example, if there is a correspondence between the connection pipe a and the client identifier a, one port of the connection pipe a communicates with the client a. At this time, after the server a is communicated with another port of the connection pipeline a by calling the connection pipeline a, the server a may acquire the data provided by the client a through the connection pipeline a.
In addition, please refer to fig. 2, an embodiment of the present application further provides an electronic device 100, which can be used as any server in the control server 11 or the server cluster 12.
Among them, the electronic device 100 may include: a processor 110, a memory 120.
It should be noted that the components and structure of electronic device 100 shown in FIG. 2 are exemplary only, and not limiting, and electronic device 100 may have other components and structures as desired.
The processor 110, memory 120, and other components that may be present in the electronic device 100 are electrically connected to each other, directly or indirectly, to enable the transfer or interaction of data. For example, the processor 110, the memory 120, and other components that may be present may be electrically coupled to each other via one or more communication buses or signal lines.
The memory 120 is used to store a program, for example, a program corresponding to a data acquisition method appearing later or a data acquisition device appearing later. Optionally, when the data acquisition device is stored in the memory 120, the data acquisition device includes at least one software functional module that can be stored in the memory 120 in the form of software or firmware (firmware).
Alternatively, the software function module included in the data acquisition device may also be solidified in an Operating System (OS) of the electronic device 100.
The processor 110 is used to execute executable modules stored in the memory 120, such as software functional modules or computer programs included in the data acquisition device. When the processor 110 receives the execution instruction, it may execute a computer program, for example, to perform the data collection method included in any embodiment.
Of course, the method disclosed in any of the embodiments of the present application can be applied to the processor 110, or implemented by the processor 110.
The following description will be directed to the data acquisition method provided in the present application.
Referring to fig. 3, an embodiment of the present application provides a data acquisition method applied to the control server 11. The steps involved will be described below in conjunction with fig. 3.
Step S110: and acquiring a data acquisition instruction, wherein the data acquisition instruction comprises a target client identification corresponding to the target client.
Wherein, the data acquisition instruction is triggered by the operation of a user. For example, when a user needs to acquire data of the client a, a data acquisition instruction for the client a is triggered.
The data acquisition instruction comprises a target client identification, and represents which client the data required to be acquired by the user comes from. For example, the control server obtains a data acquisition instruction for the client a, and the client a is a target client.
Step S120: an available server is determined from the cluster of servers.
After the control server obtains the data acquisition instruction, an available server needs to be determined from the server cluster, so that the available server can execute a data acquisition action corresponding to the data acquisition instruction.
The procedure for determining available servers is as follows.
The control server may send the status query information to each server in the server cluster at preset intervals, or may send the status query information to each server in the server cluster after acquiring the data acquisition instruction.
After each normally working server in the server cluster obtains the state query information, state information can be generated based on the state query information, and the state information is fed back to the control server.
The server-generated state information may be used to characterize whether itself is in an idle state. If the server is executing a data acquisition task for a client at the moment, the server is in a busy state at the moment, and if the server is not executing the data acquisition task for any client at the moment, the server is in an idle state at the moment.
Because the server does not perform data processing after being down, and accordingly does not feed back the state information to the control server, when the control server acquires the state information fed back by a certain server, the control server can firstly determine that the server is not down at the moment.
In addition, the control server can also determine the server in an idle state from the servers feeding back the state information according to the state information fed back by each server in the server cluster.
If only one server in the idle state is determined by the control server, the control server may directly determine the server in the idle state as an available server.
If there are a plurality of servers in the idle state determined by the control server, at this time, there is a possibility that all the plurality of servers in the idle state become available servers.
In an alternative embodiment, the control server may randomly determine a server from a plurality of servers in an idle state and determine the server as an available server.
In another alternative embodiment, the priority order may be set in advance for each server in the server cluster. When the control server needs to determine an available server from the plurality of servers in the idle state, the server with the highest priority may be determined as the available server according to the priority order corresponding to the plurality of servers in the idle state.
Furthermore, in some embodiments, multiple connection pipes stored within the cache server may correspond to one client identification at a time. When a plurality of connecting pipelines exist in the cache server for the target client identifier included in the data acquisition instruction acquired by the control server, in order to improve the data acquisition efficiency, the control server can determine a plurality of available servers from the server cluster, so that the plurality of available servers can be respectively connected with the target client by a long connection pipeline corresponding to the target client identifier, and parallel data acquisition of the target client is realized.
And the number of the plurality of available servers is consistent with the number of the connecting pipelines corresponding to the target client identification.
Of course, in this embodiment, the control server may establish a communication connection with the cache server to obtain a correspondence between a connection pipe in the cache server and the client identifier; the corresponding relation between the connection pipeline in the cache server and the client identifier can be stored in the local control server in advance.
In an optional implementation manner, the data acquisition instruction includes a data acquisition time period, which is used to characterize that the server needs to perform data acquisition on the target client at a time corresponding to the acquisition time period. In this embodiment, before determining the available server, the control server may first determine whether a server that establishes an effective connection with the target client exists in the server cluster within a time period represented by the data acquisition time period. If not, the control server determines an available server from the server cluster.
Optionally, the control server may continuously poll each server in the server cluster, so as to determine a connection object of each server in the server cluster, and thus determine whether a server that establishes an effective connection with the target client exists in the server cluster within the time period represented by the data acquisition time period.
In the case that there is no server that establishes an effective connection with the target client, it may be that the data acquisition instruction for the target client is initiated for the first time, or that the data acquisition instruction for the target client is initiated for the nth time (N > 1), but the server that establishes a long connection with the target client for the first N-1 times is down and is in an unavailable state, and a new available server needs to be newly selected to perform data acquisition on the target client.
Step S130: and sending the data acquisition instruction to the available server so that the available server searches for and calls a connecting pipeline corresponding to the target client identifier from the cache server to establish long connection with the target client.
After the available server is determined, the control server sends the data acquisition instruction to the determined available server, so that the available server can search the connecting pipeline corresponding to the target client identifier from the cache server based on the target client identifier in the data acquisition instruction.
And after the available server finds the corresponding connecting pipeline, calling the corresponding connecting pipeline, so that long connection with the target client is established, and data provided by the target client is collected.
According to the data acquisition method provided by the embodiment of the application, a connection channel used for establishing long connection between the client and the server is stored in a cache server independent of the client and the server. When the control server obtains the data acquisition instruction, the available server is selected from the server cluster, and the data acquisition instruction is sent to the available server, so that the available server finds the corresponding connection channel from the cache server according to the target client identification pointed by the data acquisition instruction and establishes long connection with the target client, and data acquisition is carried out on the target client. Because the connection channel is not stored in the server which establishes connection with the target client, the connection channel corresponding to the target client identifier still does not disappear even if the server which establishes long connection with the target client is down. After the control server determines a new available server again, the new available server can still establish a long connection with the target client based on the original connection channel without establishing a connection pipeline again, so that compared with the prior art, the time required for establishing the connection pipeline again can be saved, and the data acquisition efficiency is improved.
Referring to fig. 4, an embodiment of the present application provides a data acquisition method applied to any server in the server cluster 12. The steps involved will be described below in conjunction with fig. 4.
Step S210: and acquiring a data acquisition instruction, wherein the data acquisition instruction comprises a target client identification corresponding to the target client.
Step S220: and searching and calling a connecting pipeline corresponding to the target client identification from the cache server.
Step S230: and establishing long connection with the target client through the connecting pipeline corresponding to the target client identifier.
After the server establishes long connection with the target client, the target client can be polled at preset time intervals, so that data provided by the target client can be acquired.
In addition, after the server collects the data, the data can be analyzed according to the predetermined existing message analysis rule, and then the analyzed data is converted into the format required by the server and stored in the database.
In some embodiments, after the server collects the data, the server may further store the data in a specific database according to a target client from which the data is sourced. In this embodiment, the server needs to store a path table in advance, and the path table specifies which database the data from which client needs to be stored in.
For example, in an application scenario, the clients located in area 1 correspond to database 1, and the clients located in area 2 correspond to database 2.
For example, in an application scenario, a client of data type a of the provided data corresponds to the database 3, and a client of data type B of the provided data corresponds to the database 4. Where A may be temperature data and B may be humidity data.
In addition, in the embodiment of the application, the server can analyze the data after acquiring the data. In this embodiment, the data threshold is previously stored in the server. After the server acquires the data, the data is compared with the data threshold value, and when the data is larger than the data threshold value, an alarm instruction can be sent to alarm equipment in communication connection.
Corresponding to fig. 3, please refer to fig. 5, an embodiment of the present application further provides a data acquisition apparatus 400, which is applied to a control server, where the control server is in communication connection with a server cluster, the server cluster is in communication connection with a cache server, and a connection pipeline corresponding to a client identifier is pre-stored in the cache server. The data acquisition device 400 may include: an acquisition module 410, a determination module 420, and a sending module 430.
An obtaining module 410, configured to obtain a data acquisition instruction, where the data acquisition instruction includes a target client identifier corresponding to a target client;
a determining module 420, configured to determine an available server from the server cluster;
a sending module 430, configured to send the data acquisition instruction to the available server, so that the available server searches for and calls a connection pipe corresponding to the target client identifier from the cache server to establish a long connection with the target client.
In a possible implementation manner, the determining module 420 is configured to send status query information to each server in the server cluster; acquiring state information fed back by each server in the server cluster, wherein the state information is used for representing whether the server cluster is in an idle state or not; according to the state information, determining a server in an idle state from the servers feeding back the state information; and determining one server from the servers in the idle state as the available server.
In a possible implementation manner, the determining module 420 is configured to randomly determine, from the servers in the idle state, one server to be determined as the available server;
or, the server determining unit is configured to determine, according to a priority order stored in advance, a server with a highest priority among the servers in the idle state as the available server.
In a possible implementation manner, the data collection instruction includes a data collection time period, and the determining module 420 is further configured to determine that no server in the server cluster exists to establish an effective connection with the target client within a time period represented by the data collection time period.
In a possible implementation manner, a plurality of connection pipes corresponding to the target client identifier are stored in the cache server, and the determining module 420 is configured to determine a plurality of available servers from the server cluster, where the number of the available servers is consistent with the number of the connection pipes corresponding to the target client identifier.
The data acquisition apparatus 400 provided in the embodiment of the present application has the same implementation principle and the same technical effect as those of the foregoing method embodiments, and for the sake of brief description, reference may be made to corresponding contents in the foregoing method embodiments for parts of embodiments that are not mentioned in the present application.
Corresponding to fig. 4, please refer to fig. 6, an embodiment of the present application further provides a data acquisition apparatus 500, which is applied to a server, where the server establishes a communication connection with a cache server, and a connection pipeline establishing a corresponding relationship with a client identifier is pre-stored in the cache server. The data collection device 500 may include an acquisition module 510, a lookup invocation module 520, and a setup module 530.
An obtaining module 510, configured to obtain a data acquisition instruction, where the data acquisition instruction includes a target client identifier corresponding to a target client;
a searching and calling module 520, configured to search and call a connection pipe corresponding to the target client identifier from the cache server;
an establishing module 530, configured to establish a long connection with the target client through the connection pipe corresponding to the target client identifier.
The data acquisition device 500 provided in the embodiment of the present application has the same implementation principle and the same technical effect as those of the foregoing method embodiments, and for the sake of brief description, no mention is made in the apparatus embodiment, and reference may be made to the corresponding contents in the foregoing method embodiments.
In addition, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computer, the steps included in the data acquisition method are executed.
In summary, the data acquisition method, the data acquisition device, the electronic device, and the computer-readable storage medium according to the embodiments of the present invention store a connection channel for establishing a long connection between a client and a server in a cache server independent from the client and the server in advance. When the control server obtains the data acquisition instruction, the available server is selected from the server cluster, and the data acquisition instruction is sent to the available server, so that the available server finds the corresponding connection channel from the cache server according to the target client identification pointed by the data acquisition instruction and establishes long connection with the target client, and data acquisition is carried out on the target client. Because the connection channel is not stored in the server which establishes connection with the target client, the connection channel corresponding to the target client identifier still does not disappear even if the server which establishes long connection with the target client is down. After the control server determines a new available server again, the new available server can still establish a long connection with the target client based on the original connection channel without establishing a connection pipeline again, so that compared with the prior art, the time required for establishing the connection pipeline again can be saved, and the data acquisition efficiency is improved.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a notebook computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.
Claims (10)
1. A data acquisition method is characterized in that the method is applied to a control server, the control server is in communication connection with a server cluster, the server cluster is in communication connection with a cache server, and a connection pipeline establishing a corresponding relation with a client identifier is pre-stored in the cache server, and the method comprises the following steps:
acquiring a data acquisition instruction, wherein the data acquisition instruction comprises a target client identification corresponding to a target client;
determining an available server from the server cluster;
and sending the data acquisition instruction to the available server so that the available server searches for and calls a connecting pipeline corresponding to the target client identifier from the cache server to establish long connection with the target client.
2. The method of claim 1, wherein determining available servers from the cluster of servers comprises:
sending state query information to each server in the server cluster;
acquiring state information fed back by each server in the server cluster, wherein the state information is used for representing whether the server cluster is in an idle state or not;
according to the state information, determining a server in an idle state from the servers feeding back the state information;
and determining one server from the servers in the idle state as the available server.
3. The method of claim 2, wherein said determining one of the servers in the idle state as the available server comprises:
randomly determining one server from the servers in the idle state to be determined as the available server; alternatively, the first and second electrodes may be,
and determining the server with the highest priority in the servers in the idle state as the available server according to a pre-stored priority sequence.
4. The method of claim 1, wherein the data collection instruction comprises a data collection time period, and wherein prior to said determining available servers from the cluster of servers, the method further comprises:
and determining that no server which establishes effective connection with the target client exists in the server cluster in the time period represented by the data acquisition time period.
5. The method of claim 1, wherein a plurality of connection pipes corresponding to the target client id are stored in the cache server, and wherein determining available servers from the server cluster comprises:
and determining a plurality of available servers from the server cluster, wherein the number of the available servers is consistent with the number of the connecting pipelines corresponding to the target client identification.
6. A data acquisition method is characterized in that the method is applied to a server, the server establishes communication connection with a cache server, and a connection pipeline establishing a corresponding relation with a client identifier is pre-stored in the cache server, and the method comprises the following steps:
acquiring a data acquisition instruction, wherein the data acquisition instruction comprises a target client identification corresponding to a target client;
searching and calling a connecting pipeline corresponding to the target client identifier from the cache server;
and establishing long connection with the target client through the connecting pipeline corresponding to the target client identifier.
7. A data acquisition device is characterized in that the data acquisition device is applied to a control server, the control server is in communication connection with a server cluster, the server cluster is in communication connection with a cache server, and a connection pipeline which establishes a corresponding relation with a client identifier is pre-stored in the cache server, and the device comprises:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a data acquisition instruction, and the data acquisition instruction comprises a target client identification corresponding to a target client;
a determining module, configured to determine an available server from the server cluster;
and the sending module is used for sending the data acquisition instruction to the available server so that the available server searches for and calls a connecting pipeline corresponding to the target client identifier from the cache server to establish long connection with the target client.
8. A data acquisition device is characterized in that the data acquisition device is applied to a server, the server establishes communication connection with a cache server, and a connection pipeline establishing a corresponding relation with a client identifier is pre-stored in the cache server, and the data acquisition device comprises:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a data acquisition instruction, and the data acquisition instruction comprises a target client identification corresponding to a target client;
the searching and calling module is used for searching and calling the connecting pipeline corresponding to the target client identifier from the cache server;
and the establishing module is used for establishing long connection with the target client through the connecting pipeline corresponding to the target client identifier.
9. An electronic device, comprising: a memory and a processor, the memory and the processor connected;
the memory is used for storing programs;
the processor calls a program stored in the memory to perform the method of any of claims 1-6.
10. A computer-readable storage medium, on which a computer program is stored which, when executed by a computer, performs the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010834490.0A CN111930542A (en) | 2020-08-18 | 2020-08-18 | Data acquisition method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010834490.0A CN111930542A (en) | 2020-08-18 | 2020-08-18 | Data acquisition method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111930542A true CN111930542A (en) | 2020-11-13 |
Family
ID=73304645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010834490.0A Pending CN111930542A (en) | 2020-08-18 | 2020-08-18 | Data acquisition method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111930542A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113452765A (en) * | 2021-06-21 | 2021-09-28 | 北京达佳互联信息技术有限公司 | Long-connection communication method, system, device, storage medium and program product |
| CN114584607A (en) * | 2022-05-05 | 2022-06-03 | 施维智能计量系统服务(长沙)有限公司 | Communication connection establishing method and device, electronic equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016119343A1 (en) * | 2015-01-30 | 2016-08-04 | 北京佰才邦技术有限公司 | Data acquisition method, apparatus and system |
| US20180278611A1 (en) * | 2017-03-21 | 2018-09-27 | Dell Products, Lp | System and method for securing an inter-process communication via a named pipe |
| CN108881200A (en) * | 2018-06-08 | 2018-11-23 | 江苏中威科技软件系统有限公司 | Multiple terminals based on TCP method and system online simultaneously |
| CN109933432A (en) * | 2019-03-19 | 2019-06-25 | 北京百度网讯科技有限公司 | Method and apparatus for sending data |
| CN110351246A (en) * | 2019-06-12 | 2019-10-18 | 深圳市奥克多普科技有限公司 | Server cluster system Socket management method and device |
| CN111131188A (en) * | 2019-12-09 | 2020-05-08 | 北京海益同展信息科技有限公司 | Communication connection method, server, client, and storage medium |
-
2020
- 2020-08-18 CN CN202010834490.0A patent/CN111930542A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016119343A1 (en) * | 2015-01-30 | 2016-08-04 | 北京佰才邦技术有限公司 | Data acquisition method, apparatus and system |
| US20180278611A1 (en) * | 2017-03-21 | 2018-09-27 | Dell Products, Lp | System and method for securing an inter-process communication via a named pipe |
| CN108881200A (en) * | 2018-06-08 | 2018-11-23 | 江苏中威科技软件系统有限公司 | Multiple terminals based on TCP method and system online simultaneously |
| CN109933432A (en) * | 2019-03-19 | 2019-06-25 | 北京百度网讯科技有限公司 | Method and apparatus for sending data |
| CN110351246A (en) * | 2019-06-12 | 2019-10-18 | 深圳市奥克多普科技有限公司 | Server cluster system Socket management method and device |
| CN111131188A (en) * | 2019-12-09 | 2020-05-08 | 北京海益同展信息科技有限公司 | Communication connection method, server, client, and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113452765A (en) * | 2021-06-21 | 2021-09-28 | 北京达佳互联信息技术有限公司 | Long-connection communication method, system, device, storage medium and program product |
| CN113452765B (en) * | 2021-06-21 | 2023-02-17 | 北京达佳互联信息技术有限公司 | Long-connection communication method, system, device, storage medium and program product |
| CN114584607A (en) * | 2022-05-05 | 2022-06-03 | 施维智能计量系统服务(长沙)有限公司 | Communication connection establishing method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106059825A (en) | Distributed system and configuration method | |
| CN103237060B (en) | A kind of data object acquisition, Apparatus and system | |
| CN110543512B (en) | Information synchronization method, device and system | |
| CN111930542A (en) | Data acquisition method and device, electronic equipment and computer readable storage medium | |
| CN113434249A (en) | Mirror image synchronization method and device, docker host and storage medium | |
| CN114024972A (en) | Long connection communication method, system, device, equipment and storage medium | |
| CN112511580A (en) | Message pushing method, device, storage medium and equipment | |
| CN105373563B (en) | Database switching method and device | |
| CN113645287B (en) | Automobile message storage method and device and automobile message storage system | |
| CN111352716A (en) | Task request method, device and system based on big data and storage medium | |
| CN112631756A (en) | Distributed regulation and control method and device applied to space flight measurement and control software | |
| CN110798358B (en) | Distributed service identification method and device, computer readable medium and electronic equipment | |
| CN114356456A (en) | Service processing method, device, storage medium and electronic equipment | |
| CN114356713A (en) | Thread pool monitoring method and device, electronic equipment and storage medium | |
| CN112099864A (en) | Asynchronous data processing method and device | |
| CN112131263A (en) | Software package obtaining method, system, device, electronic equipment and storage medium | |
| CN114553944A (en) | Early warning message pushing method and system | |
| CN109327529B (en) | Distributed scanning method and system | |
| CN112565025A (en) | Data transmission method and device | |
| CN111427703A (en) | Industrial data real-time display method and system | |
| US20140280347A1 (en) | Managing Digital Files with Shared Locks | |
| WO2017096791A1 (en) | Diagnosis test message processing method and device | |
| CN111771191A (en) | Cross-domain inline event handler | |
| CN110855785B (en) | Data processing method and device | |
| CN111431951A (en) | Data processing method, node equipment, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |