CN115396494A - Real-time monitoring method and system based on stream computing - Google Patents
Real-time monitoring method and system based on stream computing Download PDFInfo
- Publication number
- CN115396494A CN115396494A CN202210999988.1A CN202210999988A CN115396494A CN 115396494 A CN115396494 A CN 115396494A CN 202210999988 A CN202210999988 A CN 202210999988A CN 115396494 A CN115396494 A CN 115396494A
- Authority
- CN
- China
- Prior art keywords
- service request
- message queue
- queue partition
- identifier
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The disclosure relates to a real-time monitoring method and a real-time monitoring system based on stream computing. The method comprises the steps of receiving a service request sent by a client, allocating a message queue partition to the service request, adding an identifier of the message queue partition into the service request, sending the service request to the message queue partition, processing the service request added with the identifier of the message queue partition by the message queue partition to obtain a response result, receiving the response result returned by the message queue partition, and returning the response result to the client. Because one message queue partition is distributed to one service request, each message queue partition processes the service request of the service request, and the service requests are not influenced mutually, a plurality of service requests or a plurality of service requests can be processed simultaneously, and a plurality of service requirements of different clients can be met, thereby improving the flexibility and the practicability of the real-time monitoring method based on the streaming computation.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a real-time monitoring method and system based on stream-oriented computing.
Background
With the development of technologies such as internet, mobile internet, digital device, internet of things, etc., global data production is increasing at a high rate, and information has become strategic assets of enterprises for user behavior analysis, market research, etc. Data processing systems can be divided into batch data and streaming data according to the timeliness of data processing.
In the prior art, a chimney-type architecture, that is, a vertical architecture, is usually adopted for a streaming type mass data processing system, data is directly captured from each data source to a final destination, data processing of specific services is performed, collection, mirroring and calculation are tightly coupled, no gap exists in the middle, and requirements of the specific services can be quickly responded.
However, the data processing system in the prior art cannot be in butt joint with other information systems, is difficult to realize an asynchronous real-time service system, and cannot simultaneously meet various service requirements.
Disclosure of Invention
In order to solve the technical problem, the present disclosure provides a real-time monitoring method and system based on stream-oriented computing to meet various service requirements.
In a first aspect, an embodiment of the present disclosure provides a real-time monitoring method based on streaming computing, including:
receiving a service request sent by a client;
distributing a message queue partition for the service request, and adding the identifier of the message queue partition into the service request;
sending the service request to the message queue partition, wherein the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result;
receiving the response result returned by the message queue partition;
and returning the response result to the client.
In a second aspect, an embodiment of the present disclosure provides a real-time monitoring apparatus based on streaming computing, including:
the first receiving module is used for receiving a service request sent by a client;
the distribution module is used for distributing a message queue partition for the service request and adding the identifier of the message queue partition into the service request;
the first sending module is used for sending the service request to the message queue partition, and the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result;
the second receiving module is used for receiving the response result returned by the message queue partition;
and the return module is used for returning the response result to the client.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of the first aspect.
In a fourth aspect, the present disclosure provides a computer-readable storage medium, on which a computer program is stored, the computer program being executed by a processor to implement the method of the first aspect.
In a fifth aspect, the disclosed embodiments provide a streaming computing based real-time monitoring system, which includes at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform the streaming computing based real-time monitoring method according to the first aspect.
In a sixth aspect, the disclosed embodiments also provide a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the real-time monitoring method based on streaming computing as described above is implemented.
The real-time monitoring method and system based on the stream computing provided by the embodiment of the disclosure allocate a message queue partition to a service request by receiving the service request sent by a client, add an identifier of the message queue partition to the service request, and send the service request to the message queue partition, where the message queue partition is used to process the service request added with the identifier of the message queue partition to obtain a response result, receive the response result returned by the message queue partition, and return the response result to the client. Because one message queue partition is distributed to one service request, each message queue partition processes the service request of the service request, and the service requests are not influenced mutually, a plurality of service requests or a plurality of service requests can be processed simultaneously, and a plurality of service requirements of different clients can be met, thereby improving the flexibility and the practicability of the real-time monitoring method based on the streaming computation.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the embodiments or technical solutions in the prior art description will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
Fig. 1 is a flowchart of a real-time monitoring method based on stream computing according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of an access gateway architecture provided by an embodiment of the present disclosure;
fig. 3 is a flowchart of a real-time monitoring method based on streaming computing according to another embodiment of the disclosure;
fig. 4 is a flowchart of a real-time monitoring method based on streaming computing according to another embodiment of the disclosure;
fig. 5 is a schematic diagram of a real-time monitoring system architecture based on streaming computing according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a real-time monitoring apparatus based on streaming computing according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments. The specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the invention, are within the scope of the invention.
With the development of technologies such as internet, mobile internet, digital device, internet of things, etc., global data production is increasing at a high rate, and information has become strategic assets of enterprises for user behavior analysis, market research, etc. Data processing systems can be divided into batch data and streaming data according to the timeliness of data processing.
In the prior art, a chimney-type architecture, that is, a vertical architecture, is generally adopted for a streaming mass data processing system, data is directly captured from each data source to a final destination, data processing of a specific service is performed, collection, mirroring and calculation are tightly coupled, no gap exists in the middle, and the requirement of the specific service can be quickly responded.
However, the data processing system in the prior art cannot be in butt joint with other information systems, is difficult to realize an asynchronous real-time service system, and cannot simultaneously meet various service requirements. To solve this problem, embodiments of the present disclosure provide a real-time monitoring method based on streaming computing, and the method is described below with reference to specific embodiments.
Fig. 1 is a flowchart of a real-time monitoring method based on streaming computing according to an embodiment of the present disclosure. The method can be applied to the scene of processing the streaming data and can also be applied to the scene of processing various services simultaneously. It can be understood that the real-time monitoring method based on streaming computing provided by the embodiment of the present disclosure can also be applied in other scenarios.
The following describes a real-time monitoring method based on streaming computing shown in fig. 1 with reference to a schematic diagram of an access gateway architecture shown in fig. 2 and a schematic diagram of an asynchronous real-time service system architecture shown in fig. 5, where the method includes the following specific steps:
s101, receiving a service request sent by a client.
As shown in fig. 2, the client sends a service request to the gateway, and the gateway receives the service request sent by the client.
Alternatively, as shown in fig. 5, a plurality of access gateways form an access gateway cluster. The gateway has the function of accessing a service request sent by a client and forwarding the service request of the client to the message queue partition. The Protocol followed by the gateway is HyperText Transfer Protocol (HTTP), and the user can directly access the gateway and can also realize connection to the access gateway through the load balancing service.
S102, distributing a message queue partition for the service request, and adding the identifier of the message queue partition into the service request.
As shown in fig. 2, the gateway randomly assigns a message queue partition to the service request, and adds the identifier of the message queue partition to the service request, so that the response result of the service request can be returned to the message queue partition according to the identifier of the message queue partition. The message queue is allocated to one service request, so that a plurality of service requests or a plurality of service requests can be processed simultaneously.
S103, sending the service request to the message queue partition, wherein the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result.
And after the gateway randomly assigns a message queue partition for the service request, sending the service request to the message queue partition. As shown in fig. 5, the specific processing procedure is as follows: the gateway sends the service request sent by the client to the request message queue, the request message queue forwards the service request sent by the client to the service processing module, after the service processing module obtains the service request information in the request message queue and performs service processing, a response result is obtained and sent to the response message queue, the response message queue returns the response result to the gateway, and the whole cycle that the service request goes from the client to the gateway and then to the message queue and the response result goes from the message queue to the gateway and then to the client is realized.
S104, receiving the response result returned by the message queue partition.
For example, the message queue partition returns the response result to the gateway, and the gateway receives the response result returned by the message queue partition.
And S105, returning the response result to the client.
And after receiving the response result returned by the message queue partition, the gateway returns the response result to the client, so that the circulation of the response result from the message queue to the gateway and then to the client is realized.
Optionally, as shown in fig. 5, the asynchronous real-time service system further includes a service request processing module, a monitoring and timing database, an event trigger, a service configuration and management site, and the like.
The service processing module is used for receiving various service request data in the request message queue, realizing various service processing according to a predefined service flow and sending a service processing result to the response message queue. Meanwhile, the service request processing module can receive the task setting information in the task configuration updating queue to realize the real-time updating of the task setting.
The monitoring and time sequence database is used for receiving various operation index data sent by the service request processing module and is responsible for storing the data, and meanwhile, various index events of the operation index data production system in operation can be generated according to the operation index data sent by the service request processing module by utilizing a predefined event definition rule. The generated various index events may be sent to an event trigger for further processing.
The event trigger is used for receiving the index event sent from the monitoring and timing database, and generating a specific trigger task according to a predefined rule by using the event, for example, sending a mail reminder to a system administrator when the occupancy rate of a Central Processing Unit (CPU) of a certain node of the service request Processing module is too high or the node has no response for a long time.
The service configuration and management site is a human-computer interface of a system administrator, and the system administrator can check and retrieve the running state and various index information of each node of the current system in the task configuration and management site and complete the setting management of various running parameters of the task.
The method and the device for processing the service request comprise the steps of receiving the service request sent by a client, distributing a message queue partition for the service request, adding an identifier of the message queue partition into the service request, and sending the service request to the message queue partition, wherein the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result, receiving the response result returned by the message queue partition, and returning the response result to the client. Because one message queue is allocated to one service request, each message queue processes the service request of the service request, and the message queues are not influenced mutually, a plurality of service requests or a plurality of service requests can be processed simultaneously, and a plurality of service requirements of different clients can be met, thereby improving the flexibility and the practicability of the real-time monitoring method based on the streaming computation.
Fig. 3 is a flowchart of a real-time monitoring method based on streaming computing according to another embodiment of the present disclosure, as shown in fig. 3, the method includes the following steps:
s301, receiving a service request sent by a client.
Specifically, the implementation process and principle of S301 and S101 are the same, and are not described herein again
And S302, judging whether the service request meets a preset condition, if so, executing S303, and if not, executing S308.
After receiving a service request sent by a client, a gateway determines whether the service request meets preset conditions, such as whether the preset conditions have permission, whether an access protocol is correct, and the like. If the preset condition is met, executing the following steps S303 and S303, if the preset condition is not met, executing S308, and ending the connection between the current client and the gateway.
S303, generating a request identifier, and adding the request identifier into the service request.
The gateway is provided with one or more hash tables, one message queue partition corresponds to one hash table, and the hash table is used for recording the connection information of the service request and the corresponding client. When a service request is sent by a client and meets a preset condition, a gateway generates a request identifier for the service request. The request identification is used as a key value of the hash table, and information such as a network address, a port and the like of the service request sending client is used as a numerical value corresponding to the key value to be stored. Therefore, the gateway can find the information such as the network address, the port and the like of the client corresponding to the service request according to the request identifier and return a response result to the corresponding client.
S304, distributing a message queue partition for the service request, and adding the identifier of the message queue partition into the service request.
Specifically, the implementation process and principle of S304 and S102 are consistent, and are not described herein again.
S305, sending the service request to the message queue partition, where the message queue partition is used to process the service request added with the identifier of the message queue partition to obtain a response result.
Specifically, the implementation process and principle of S305 and S103 are consistent, and are not described herein again.
S306, receiving the response result returned by the message queue partition.
Specifically, the implementation process and principle of S306 and S104 are consistent, and are not described herein again.
And S307, returning the response result to the client.
Specifically, the implementation process and principle of S307 and S105 are the same, and are not described herein again.
And S308, ending.
The embodiment of the disclosure judges whether the service request meets a preset condition by receiving the service request sent by a client, generates a request identifier if the service request meets the preset condition, and adds the request identifier to the service request. Further, a message queue partition is allocated to the service request, the identifier of the message queue partition is added to the service request, the service request is sent to the message queue partition, the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result, the response result returned by the message queue partition is received, and the response result is returned to the client. After receiving a service request sent by a client, the service request is firstly judged whether to meet a preset condition, if so, a request identifier is generated, a response result can be returned to a port of the corresponding client according to the request identifier, a response can be made to different requests of each client, and various service requirements can be met. If not, the connection is disconnected, the safety of the service request is improved, and information leakage is prevented.
On the basis of the above embodiment, the service request includes the relevant information of the user and the user instruction.
Optionally, the service request includes related information of the user and a user instruction. The related information of the user comprises the name, birth date, account and the like of the user, and the user instruction comprises inquiry, calculation, screening and the like.
Correspondingly, the step of judging whether the service request meets the preset conditions comprises the following steps: and judging whether the user is a blacklist user or not according to the related information of the user, and judging whether the instruction has the authority to execute according to the user instruction.
For example, the preset condition may be to determine whether the user is a blacklist user according to the related information of the user, determine whether the instruction has an authority to execute the instruction according to the user instruction, determine whether the access protocol is correct, and the like.
The embodiment of the present disclosure receives a service request sent by a client, where the service request includes user-related information and a user instruction, and further, determines whether the service request satisfies a preset condition, including: and judging whether the user is a blacklist user or not according to the related information of the user, and judging whether the instruction has the authority to execute according to the user instruction. After receiving a service request sent by a client, the method firstly judges whether the service request meets a preset condition, specifically judges whether a user is a blacklist user according to the relevant information of the user, and judges whether the instruction has an authority to execute according to the user instruction, thereby further improving the safety of the real-time monitoring method based on the streaming calculation.
Fig. 4 is a flowchart of a real-time monitoring method based on streaming computing according to another embodiment of the present disclosure, and as shown in fig. 4, the method includes the following steps:
s401, receiving a service request sent by a client.
Specifically, the implementation process and principle of S401 and S101 are consistent, and are not described herein again.
S402, distributing a message queue partition for the service request, and adding the identifier of the message queue partition into the service request.
Specifically, the implementation process and principle of S402 and S102 are consistent, and are not described herein again.
S403, caching the state information of the service request into the message queue partition, and setting the service timeout time of the service request.
After distributing a message queue partition for the service request, caching the state information of the service request into the message queue partition, setting the service overtime time of the service request, and if no response result is returned within the service overtime time, sending the overtime information to the client so that the client can obtain a response in time, wherein the state information comprises the current service state, the network address information of the client and the port.
S404, the service request is sent to the message queue partition, and the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result.
Specifically, the implementation process and principle of S404 and S103 are consistent, and are not described herein again.
Optionally, the response result includes a processing result of the service request, the request identifier, and an identifier of the message queue partition.
S405, receiving the response result returned by the message queue partition.
Specifically, the implementation process and principle of S405 and S104 are consistent, and are not described herein again.
S406, judging whether the response result returned by the message queue partition is received within the service timeout time, if so, executing S407, otherwise, executing S409.
Optionally, it is determined whether the response result returned by the message queue partition is received within the service timeout period, if so, the following steps S407 and steps after S407 are executed, otherwise, S409 is executed.
S407, obtaining the state information cached in the message queue partition according to the identifier of the message queue partition.
And finding a corresponding message queue partition according to the identifier of the message queue partition, and then acquiring the state information from the message queue partition, wherein the state information comprises the current service state, the network address information of the client and the port.
S408, returning the response result to the client according to the state information.
In this step, after acquiring the state information, the gateway returns the response result to the corresponding port of the client according to the state information.
And S409, sending response timeout information to the client.
If no response result is returned within the service timeout time, the timeout information is sent to the client so that the client can obtain a response in time.
And S410, ending.
The method comprises the steps of receiving a service request sent by a client, distributing a message queue partition for the service request, adding an identifier of the message queue partition into the service request, caching state information of the service request into the message queue partition, and setting service timeout time of the service request. Further, the service request is sent to the message queue partition, the message queue partition is configured to process the service request added with the identifier of the message queue partition to obtain a response result, and the response result includes a processing result of the service request, the request identifier, and the identifier of the message queue partition, and receives the response result returned by the message queue partition. Then, judging whether the response result returned by the message queue partition is received within the service timeout time, if so, acquiring the state information cached by the message queue partition according to the identifier of the message queue partition, and returning the response result to the client according to the state information; otherwise, response timeout information is sent to the client. Because a service request is distributed with a message queue partition, each message queue partition processes the service request of the service request, and the service requests are not influenced mutually, a plurality of service requests or a plurality of service requests can be processed simultaneously, and a plurality of service requirements of different clients can be met, so that the flexibility and the practicability of the real-time monitoring method based on stream computing are improved, the service timeout time is set, and if no response result is returned within the service timeout time, timeout information is sent to the clients, so that the requests of the clients can be responded in time.
Fig. 6 is a schematic structural diagram of a real-time monitoring apparatus based on streaming computing according to an embodiment of the present disclosure. The streaming computing based real-time monitoring apparatus may be a gateway as described in the above embodiments, or the streaming computing based real-time monitoring apparatus may be a component or assembly in the gateway. The real-time monitoring apparatus based on stream-oriented computing according to the embodiment of the present disclosure may execute the processing procedure provided by the embodiment of the real-time monitoring method based on stream-oriented computing, as shown in fig. 6, the real-time monitoring apparatus 60 based on stream-oriented computing includes: a first receiving module 61, a distributing module 62, a first sending module 63, a second receiving module 64, and a returning module 65; the first receiving module 61 is configured to receive a service request sent by a client; the allocating module 62 is configured to allocate a message queue partition to the service request, and add an identifier of the message queue partition to the service request; the first sending module 63 is configured to send the service request to the message queue partition, where the message queue partition is configured to process the service request added with the identifier of the message queue partition to obtain a response result; the second receiving module 64 is configured to receive the response result returned by the message queue partition; the returning module 65 is configured to return the response result to the client.
Optionally, the apparatus further includes a determining module 66 and a generating module 67; the judging module 66 is configured to judge whether the service request meets a preset condition; the generating module 67 is configured to generate a request identifier when the service request meets a preset condition, and add the request identifier to the service request.
Optionally, the service request includes related information of the user and a user instruction; correspondingly, when the determining module determines whether the service request meets the preset condition, the determining module is specifically configured to: and judging whether the user is a blacklist user or not according to the relevant information of the user, and judging whether the instruction has the authority to execute the instruction or not according to the instruction of the user.
Optionally, the real-time monitoring apparatus 60 based on streaming calculation further includes: a setting module 68, a second sending module 69; the setting module 68 is configured to cache the state information of the service request in the message queue partition, and set a service timeout time of the service request; the second sending module 69 is configured to send response timeout information to the client when the response result returned by the message queue partition is not received within the service timeout period.
Optionally, the state information includes a current service state, network address information of the client, and a port; the response result comprises a processing result of the service request, the request identifier and the identifier of the message queue partition; accordingly, the return module 65 includes an acquisition unit 651 and a return unit 652; the obtaining unit 651 is configured to obtain the status information cached in the message queue partition according to the identifier of the message queue partition; the returning unit 652 is configured to return the response result to the client according to the status information.
The real-time monitoring apparatus based on stream-oriented computing in the embodiment shown in fig. 6 can be used to implement the technical solution of the above method embodiment, and the implementation principle and technical effect are similar, and are not described herein again.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device may be a gateway as described in the above embodiments. The electronic device provided in the embodiment of the present disclosure may execute the processing procedure provided in the embodiment of the real-time monitoring method based on stream-based computing, as shown in fig. 7, the electronic device 70 includes: memory 71, processor 72, computer programs and communication interface 73; wherein a computer program is stored in the memory 71 and is configured to be executed by the processor 72 for the streaming computation based real-time monitoring method as described above.
In addition, the embodiment of the present disclosure further provides a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the real-time monitoring method based on streaming computing described in the foregoing embodiment.
The disclosed embodiments also provide a streaming computing based real-time monitoring system, which includes at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to execute the streaming computing based real-time monitoring method as described above.
Furthermore, the embodiment of the present disclosure also provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the real-time monitoring method based on streaming computing as described above is implemented.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may be separate and not incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
receiving a service request sent by a client;
distributing a message queue partition for the service request, and adding the identifier of the message queue partition into the service request;
sending the service request to the message queue partition, wherein the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result;
receiving the response result returned by the message queue partition;
and returning the response result to the client.
In addition, the electronic device can also execute other steps in the real-time monitoring method based on the streaming calculation.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the name of an element does not in some cases constitute a limitation on the element itself.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The previous description is only for the purpose of describing particular embodiments of the present disclosure, so as to enable those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A real-time monitoring method based on streaming computing, the method comprising:
receiving a service request sent by a client;
distributing a message queue partition for the service request, and adding the identifier of the message queue partition into the service request;
sending the service request to the message queue partition, wherein the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result;
receiving the response result returned by the message queue partition;
and returning the response result to the client.
2. The method of claim 1, wherein before assigning a message queue partition to the service request, further comprising:
judging whether the service request meets a preset condition or not;
and if the service request meets the preset condition, generating a request identifier, and adding the request identifier into the service request.
3. The method of claim 2, wherein the service request comprises user related information, user instructions;
correspondingly, the step of judging whether the service request meets the preset conditions comprises the following steps:
and judging whether the user is a blacklist user or not according to the related information of the user, and judging whether the instruction has the authority to execute according to the user instruction.
4. The method of claim 1, wherein after assigning a message queue partition to the service request and adding an identification of the message queue partition to the service request, the method further comprises:
caching the state information of the service request to the message queue partition, and setting the service timeout time of the service request;
and in the service timeout time, if the response result returned by the message queue partition is not received, response timeout information is sent to the client.
5. The method of claim 4, wherein the state information comprises a current traffic state, network address information of a client, a port;
the response result comprises a processing result of the service request, the request identifier and the identifier of the message queue partition;
correspondingly, returning the response result to the client side comprises:
acquiring the state information cached in the message queue partition according to the identifier of the message queue partition;
and returning the response result to the client according to the state information.
6. A real-time monitoring apparatus based on streaming computing, comprising:
the first receiving module is used for receiving a service request sent by a client;
the distribution module is used for distributing a message queue partition for the service request and adding the identifier of the message queue partition into the service request;
the first sending module is used for sending the service request to the message queue partition, and the message queue partition is used for processing the service request added with the identifier of the message queue partition to obtain a response result;
the second receiving module is used for receiving the response result returned by the message queue partition;
and the return module is used for returning the response result to the client.
7. The apparatus of claim 6, further comprising:
the judging module is used for judging whether the service request meets a preset condition or not;
and the generating module is used for generating a request identifier when the service request meets the preset condition, and adding the request identifier into the service request.
8. An electronic device, comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any one of claims 1-5.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-5.
10. A real-time monitoring system based on streaming computing, the system comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform the real-time monitoring method based on streaming computing according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210999988.1A CN115396494A (en) | 2022-08-19 | 2022-08-19 | Real-time monitoring method and system based on stream computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210999988.1A CN115396494A (en) | 2022-08-19 | 2022-08-19 | Real-time monitoring method and system based on stream computing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115396494A true CN115396494A (en) | 2022-11-25 |
Family
ID=84120472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210999988.1A Pending CN115396494A (en) | 2022-08-19 | 2022-08-19 | Real-time monitoring method and system based on stream computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115396494A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116401117A (en) * | 2023-03-09 | 2023-07-07 | 北京海致星图科技有限公司 | Data processing method combining stream computing system and traditional software application system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106656863A (en) * | 2016-12-31 | 2017-05-10 | 广东欧珀移动通信有限公司 | Business monitoring method and apparatus, and computer device |
| CN108365971A (en) * | 2018-01-10 | 2018-08-03 | 深圳市金立通信设备有限公司 | Daily record analytic method, equipment and computer-readable medium |
| CN109819021A (en) * | 2019-01-09 | 2019-05-28 | 网宿科技股份有限公司 | The method of business backstage and its asynchronous process service request |
| CN110312283A (en) * | 2018-03-20 | 2019-10-08 | 华为技术有限公司 | A kind of information processing method and device |
| CN110661849A (en) * | 2019-08-30 | 2020-01-07 | 中国人民财产保险股份有限公司 | Request processing method and device, electronic equipment and storage medium |
| CN111221793A (en) * | 2019-12-31 | 2020-06-02 | 新华网股份有限公司 | Data mining method, platform, computer equipment and storage medium |
| CN112073398A (en) * | 2020-08-27 | 2020-12-11 | 北京金山云网络技术有限公司 | Message queue processing method, device and system, storage medium and electronic device |
| CN113111129A (en) * | 2021-04-16 | 2021-07-13 | 挂号网（杭州）科技有限公司 | Data synchronization method, device, equipment and storage medium |
-
2022
- 2022-08-19 CN CN202210999988.1A patent/CN115396494A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106656863A (en) * | 2016-12-31 | 2017-05-10 | 广东欧珀移动通信有限公司 | Business monitoring method and apparatus, and computer device |
| CN108365971A (en) * | 2018-01-10 | 2018-08-03 | 深圳市金立通信设备有限公司 | Daily record analytic method, equipment and computer-readable medium |
| CN110312283A (en) * | 2018-03-20 | 2019-10-08 | 华为技术有限公司 | A kind of information processing method and device |
| CN109819021A (en) * | 2019-01-09 | 2019-05-28 | 网宿科技股份有限公司 | The method of business backstage and its asynchronous process service request |
| CN110661849A (en) * | 2019-08-30 | 2020-01-07 | 中国人民财产保险股份有限公司 | Request processing method and device, electronic equipment and storage medium |
| CN111221793A (en) * | 2019-12-31 | 2020-06-02 | 新华网股份有限公司 | Data mining method, platform, computer equipment and storage medium |
| CN112073398A (en) * | 2020-08-27 | 2020-12-11 | 北京金山云网络技术有限公司 | Message queue processing method, device and system, storage medium and electronic device |
| CN113111129A (en) * | 2021-04-16 | 2021-07-13 | 挂号网（杭州）科技有限公司 | Data synchronization method, device, equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116401117A (en) * | 2023-03-09 | 2023-07-07 | 北京海致星图科技有限公司 | Data processing method combining stream computing system and traditional software application system |
| CN116401117B (en) * | 2023-03-09 | 2024-04-09 | 北京海致星图科技有限公司 | Data processing method combining stream computing system and traditional software application system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107819829B (en) | Method and system for accessing block chain, block chain node point equipment and user terminal | |
| CN111488595B (en) | Method for realizing authority control and related equipment | |
| CN108737473B (en) | Data processing method, device and system | |
| CN110311983B (en) | Service request processing method, device and system, electronic equipment and storage medium | |
| US8719427B2 (en) | Efficiency for network group communication | |
| CN106603394B (en) | Method and device for realizing subscription notification | |
| EP3170091B1 (en) | Method and server of remote information query | |
| CN108134764B (en) | Distributed data sharing and exchanging method and system | |
| US8918528B2 (en) | Operating group resources in sub-groups and nested groups | |
| WO2020052605A1 (en) | Network slice selection method and device | |
| US7844707B2 (en) | Web service multi-key rate limiting method and system | |
| CN109936571B (en) | Mass data sharing method, open sharing platform and electronic equipment | |
| CN108062243B (en) | Execution plan generation method, task execution method and device | |
| CN110554927A (en) | Micro-service calling method based on block chain | |
| CN108512672B (en) | Service arranging method, service management method and device | |
| US10425475B2 (en) | Distributed data management | |
| CN107306247B (en) | Resource access control method and device | |
| CN114567650A (en) | Data processing method and Internet of things platform system | |
| CN115396494A (en) | Real-time monitoring method and system based on stream computing | |
| CN109743357B (en) | Method and device for realizing service access continuity | |
| CN109862134B (en) | Lease time configuration method and system of IP address and DHCP client | |
| CN116743785A (en) | Cloud network data storage method, device, equipment and medium based on fog calculation | |
| CN114338133B (en) | Application access system, method, communication device and storage medium | |
| CN113553206B (en) | Data event execution method and device, electronic equipment and computer readable medium | |
| CN109547552B (en) | API request processing method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |