CN112422412A - Information processing method, apparatus, device and medium - Google Patents
Information processing method, apparatus, device and medium Download PDFInfo
- Publication number
- CN112422412A CN112422412A CN202011240270.1A CN202011240270A CN112422412A CN 112422412 A CN112422412 A CN 112422412A CN 202011240270 A CN202011240270 A CN 202011240270A CN 112422412 A CN112422412 A CN 112422412A
- Authority
- CN
- China
- Prior art keywords
- information
- streaming
- generate
- data
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/18—Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The disclosure relates to an information processing method, an information processing device, information processing equipment and a medium, relates to the field of cloud computing, and can be used for a cloud platform. The method comprises the following steps: processing acquired data obtained from at least one information source to generate first information usable for streaming; generating second information through streaming calculation based on the first information, wherein the second information can embody indexes of the first information in a preset time period; and sending the second information to a cloud server.
Description
Technical Field
The present disclosure relates to the field of cloud computing, and in particular, to a method and an apparatus for information processing, a server, and a computer-readable storage medium.
Background
In a conventional data processing flow, data is always collected and then placed in a database. When people need to do access to the data through the database, the answer is obtained or the relevant processing is carried out.
The database is often located in a cloud computing data center, and when the cloud computing data center is far away, a certain delay is generated in data transmission, and especially when the data is massive, rapid and variable, the data is limited by a network bottleneck in the process of being transmitted to the cloud computing data center.
Therefore, edge computing is developed to solve the problem of network bottleneck, and information processing on the lower end side in an edge computing scenario is based on a function mode to perform simple message filtering and conversion on data at a certain moment, for example, if a condition is satisfied, the message is forwarded to a destination, and if the condition is not satisfied, the message is discarded.
However, when processing data, edge calculation cannot analyze large-scale streaming data in real time while the data is changing, and captures messages that may be useful, and therefore it is difficult to process large-scale data with low delay and high timeliness.
The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, the problems mentioned in this section should not be considered as having been acknowledged in any prior art.
Disclosure of Invention
According to an aspect of the present disclosure, there is provided an information processing method, the method including: processing acquired data obtained from at least one information source to generate first information usable for streaming; generating second information through streaming calculation based on the first information, wherein the second information can embody indexes of the first information in a preset time period; and sending the second information to a cloud server.
According to another aspect of the present disclosure, there is provided an information processing apparatus including: a generation module configured to process acquisition data obtained from at least one information source to generate first information usable for streaming; a calculation module configured to generate second information through streaming calculation based on the first information, wherein the second information can embody an index of the first information within a predetermined time period; and the sending module is configured to send the second information to a cloud server.
According to another aspect of the present disclosure, there is provided an electronic device including: a processor; and a memory storing a program comprising instructions that, when executed by the processor, cause the processor to perform the method described in this disclosure.
According to another aspect of the present disclosure, there is provided a computer-readable storage medium, the program comprising instructions, which when executed by a processor of a server, cause the server to perform the method of the present disclosure.
The information processing method, the device, the equipment and the medium can process large-scale data with low delay and high aging.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of illustration only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.
Fig. 1 shows a flow chart of an information processing method according to an embodiment of the present disclosure;
FIG. 2 shows a flow diagram of a method of generating first information, according to an embodiment of the disclosure;
FIG. 3 illustrates a topology diagram of a system that performs an information processing method according to an embodiment of the present disclosure;
fig. 4 shows a schematic configuration diagram of an information processing apparatus according to an embodiment of the present disclosure;
fig. 5 shows a block diagram of an electronic device to which an information processing method according to an embodiment of the present disclosure is applied.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
FIG. 1 shows a flow diagram of an information processing method 100 according to one embodiment of the present disclosure. As shown in fig. 1, an information processing method 100 includes:
illustratively, the information source may include a first device, a second device. The information source obtains the collection and transmits the first information through a southbound protocol, wherein the southbound protocol comprises OPC, MODBUS, TCP and the like.
102, generating second information through streaming calculation based on the first information, wherein the second information can embody indexes of the first information in a preset time period;
illustratively, the streaming computation can count the metrics of the first information over a period of time to richly process the first information.
And 103, sending the second information to a cloud server.
Illustratively, the second information obtained by the edge calculation is sent to the cloud server.
As can be seen from this, by merging stream calculations into edge calculations based on the information processing method 100 shown in fig. 1, it is possible to realize data processing with low latency and high time efficiency.
According to some embodiments, before step 101, further comprising: periodically collecting information of the plurality of information sources to obtain the collected data.
Illustratively, the step of collecting data is performed by a collection module that periodically collects information sources according to a configuration and then sends the messages to a local message center. For example, acquisition protocols include, but are not limited to, Modbus, OPC protocols. The information transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol.
The time continuity of the data can be ensured by acquiring the acquired data by periodically acquiring the information of a plurality of information sources.
Fig. 2 shows a flow diagram of a method 201 of generating first information according to an embodiment of the disclosure.
According to some embodiments, screening and parsing acquired data obtained from at least one information source to generate first information having predetermined criteria usable for streaming processing includes: step 2011, acquisition data is obtained from multiple information sources via compatible protocol interfaces.
For example, the collected data collected from the multiple information sources are all sent to the local message center, and the local message center can process the collected data from the multiple information sources, for example, a software-layer interface is provided in the local message center for interfacing with different hardware, and the interface can be compatible with multiple protocols.
Therefore, the information processing method disclosed by the invention can be compatible with different types of information sources through interfaces compatible with multiple protocols.
For example, a software interface is arranged in the local message center to communicate with collected data of a plurality of information sources through southbound protocols (such as OPC, MODBUS, TCP, and the like), information of various industries (such as agriculture, electric power, transportation, and the like) can be processed, when data is collected for the information sources in the agricultural field, data transmission is performed by using a first protocol, when data is collected for the information sources in the electric power field, data transmission is performed by using a second protocol, when data is collected for the information sources in the transportation field, data transmission is performed by using a third protocol, and therefore, the local message center can be in butt joint with different hardware, and therefore, interfaces of a plurality of protocols can be compatible.
According to some embodiments, the plurality of information sources comprises a plurality of sensors applied in different fields, said fields being selected from the fields of agriculture, electricity and transportation. More particularly, the method can be applied to data acquisition and information processing of electric meter boxes, agricultural machinery and muck trucks.
For example, the sensors in different fields may be the same or different hardware, even in the same field, due to the difference of the collected objects, the adopted sensors may be different, and the sensors formed by different hardware often adopt different protocols in the data transmission process, so that the information processing method disclosed by the present disclosure is compatible with different protocols and then with different sensor hardware, thereby being capable of being compatible with information in various industries and processing the information.
With continued reference to fig. 2, according to some embodiments, processing acquired data obtained from at least one information source to generate first information usable for streaming processing includes: step 2012, subscribing to the contracted topics in the collected data to generate screening information.
Illustratively, the rule engine subscribes to the appointed topic of the local message center to generate the information flow and forwards the information flow. The protocol for information transmission in the process includes, but is not limited to, standard MQTT protocol, standard COAP protocol.
And screening the acquired data through a rule engine to obtain screened data, and using the screened data for the next processing.
With continued reference to fig. 2, according to some embodiments, processing acquired data obtained from at least one information source to generate first information usable for streaming processing includes: step 2013, performing syntax parsing on the screening information to generate the first information.
Illustratively, the information flow from the rules engine is received by the SQL syntax parsing module and the SQL statement is data-populated with a piece of information data. For example, the data types supported during data filling by SQL statements include, but are not limited to, Int, String, Float, book.
And the information flow is parsed semantically so as to obtain first information capable of being processed in a streaming manner, and the fluency and the performability of information processing are ensured.
At step 102, according to some embodiments, said streaming based on said first information to generate second information comprises: and counting the indexes of the first information in a preset time period determined by the preset time window by taking the preset time window as a unit.
Illustratively, the streaming computation engine performs streaming computation based on the first information generated by the SQL syntax parsing module, wherein the streaming computation engine intercepts information in a certain time window from the first information through a predetermined time window, and analyzes data in the time window to obtain an index of the first information in the time window. For example, the first information is continuously intercepted through the preset time window, and the first information can be analyzed in time, so that the first information can be analyzed in real time through streaming calculation of the first information, the delay of data processing is reduced, and the timeliness of the data processing is improved.
Illustratively, the first information is continuously intercepted through a predetermined time window, the obtained continuous time period is a predetermined time period, and the first information in the appointed time period is subjected to attrition calculation.
Wherein, according to some embodiments, the indicator referred to in step 102 is at least one of an average value and a maximum value.
By calculating the average value and/or the maximum value of the first information, the influence of error data in the first information can be reduced.
According to some embodiments, the time window involved in step 102 is a rolling window, a sliding window, or a counting window.
Illustratively, the time Window may be selected as a scrolling Window (scrolling Window), a Sliding Window (Sliding Window), or a Counting Window (Counting Window). For example, when the time window is a rolling window, the rolling window has a fixed size, and the adjacent rolling windows do not overlap with each other; when the time window is a sliding window, the sliding window also has a fixed size, the sliding window slides to one direction based on a preset step length, and the sliding step length is often smaller than the size of the sliding window, so that adjacent sliding windows are often overlapped; when the time window is a counting window, the size of the window is determined based on the amount of data accumulated within the window.
Therefore, the first information can be quickly obtained by using the rolling window and the streaming processing can be carried out, the processing efficiency can be improved, the flow control can be carried out by using the sliding window, the flow in the streaming processing process can be controlled, and the stability of the streaming processing can be ensured by using the counting window. The accurate processing of the data can be realized through the selection of the time window.
At step 103, according to some embodiments, sending the second information to a cloud server, includes: and synchronizing the second information and then sending the second information to the cloud server.
Illustratively, the end cloud synchronization module sends the second information to the cloud end or receives a message of the cloud end. The message transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol.
It is understood that although the present disclosure describes the flow of the collected data as being along the direction of processing the collected data and finally sending the collected data to the cloud server, based on the same architecture, the cloud server can also send the data to the device side generating the collected data. In some embodiments, the second information obtained by the streaming computation engine may be sent to the device side to control the device.
According to some embodiments, the information processing method further comprises: a configuration step for configuring the at least one information source, the commitment topic or the rules of the streaming computation.
Configuring at least one information source to be acquired to ensure the accuracy of acquired data, configuring an appointed theme to ensure that the required theme can be acquired, and configuring a rule of stream-oriented computation to perform corresponding stream-oriented computation on the acquired data, so that the configuration step can ensure that the information processing method can be normally executed.
Fig. 3 illustrates a topology diagram of a system that performs an information processing method according to an embodiment of the present disclosure. An embodiment of the present disclosure is specifically described below with reference to fig. 3.
As shown in fig. 3, in this embodiment, first, a configuration step is described, where, in the first step, information sources (e.g., device a and device B) to be acquired by an acquisition module and a contract topic sent to a local message center are configured; secondly, configuring a rule of a rule engine, subscribing the appointed theme in the local message center in the first step, and forwarding the appointed theme to a syntax analysis module based on an SQL statement; thirdly, configuring a stream type calculation rule for processing the first information forwarded to the grammar parsing module in the second step; fourthly, configuring a rule of the end cloud synchronization module, wherein the rule is used for forwarding second information (for example, a statistical result) of the streaming computation engine module to the cloud server; fifthly, respectively starting an acquisition module, a local message center, a rule engine, an SQL syntax analysis module, a stream type calculation engine, an end cloud synchronization module and the like; sixthly, the acquisition module continuously transmits the acquired data to the local message center; seventhly, forwarding the information flow to a syntax analysis module based on the SQL statement by the rule engine according to the rule defined in the second step; eighthly, the grammar parsing module parses the grammar according to the stream type calculation rule (SQL statement) configured in the third step and the message stream in the seventh step; ninth, the stream type calculation engine carries out stream type calculation according to the first information parsed by the grammar in the eighth step to generate second information; tenth, the end cloud synchronization module forwards the second information to a destination (e.g., a cloud server) according to the rules defined in the fourth step.
Illustratively, in a first step, the collection module periodically collects information sources according to configuration and then sends the messages to a contracted topic at a local message center (Broker). The information transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol. Information collection protocols include, but are not limited to, Modbus, OPC protocols.
Illustratively, in a second step, the rules engine is used to subscribe to topics at the local message center and forward to the destination. The message transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol. Destinations include, but are not limited to, local message centers, SQL syntax parsing modules. The rule engine may subscribe to a topic in the local message center or a topic in the SQL syntax parsing module according to the information flowing direction, which is not limited in this disclosure.
Illustratively, in the third step, the streaming rules are configured using SQL statements. The SQL statement uses the SQL92 standard and extends streaming computing specific keywords including, but not limited to, rolling Window (scrolling Window), Sliding Window (Sliding Window), Counting Window (Counting Window), and the like.
Illustratively, in the fourth step, the end cloud synchronization module is configured to send the end-side message to the cloud end or receive a message from the cloud end. The message transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol.
Illustratively, in the fifth step, the collection module, the local message center, the rule engine, the SQL syntax parsing module, the streaming computation engine, and the end cloud synchronization module are respectively started, for example, it is ensured that the collection module is started last for the starting sequence, so as to avoid missing the collected data.
Illustratively, in the sixth step, the collection module adopts a polling strategy to collect the information source periodically to ensure the time continuity of the data.
Illustratively, in a seventh step, the rules engine is configured to subscribe to topics at the local message center and forward to the destination. The message transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol. Destinations include, but are not limited to, local message centers, SQL syntax parsing modules.
Illustratively, in the eighth step, the SQL syntax parsing module receives the first information from the rules engine and performs data filling on the SQL statement using a piece of message data. The types of data currently supported include, but are not limited to, Int, String, Float, boot, and the like.
Illustratively, in the ninth step, the streaming engine will count the index data (e.g., average and/or maximum) of the data over a period of time in basic units of a time window of a certain size. Time windows include, but are not limited to, a scrolling Window (scrolling Window), a Sliding Window (Sliding Window), a Counting Window (Counting Window);
illustratively, in the tenth step, the end cloud synchronization module is configured to send the end-side message to the cloud end or receive a message from the cloud end. The message transmission protocol includes, but is not limited to, standard MQTT protocol, standard COAP protocol.
Fig. 4 shows a schematic configuration diagram of an information processing apparatus 400 according to an embodiment of the present disclosure.
As shown in fig. 4, there is provided an information processing apparatus 400 including:
a generation module 410 configured for processing acquisition data obtained from at least one information source to generate first information usable for streaming processing;
a calculating module 420 configured to generate second information through streaming calculation based on the first information, wherein the second information can embody an index of the first information within a predetermined time period;
a sending module 430 configured to send the second information to a cloud server.
Based on the information processing apparatus 400 shown in fig. 4, by merging stream calculations into edge calculations, it is possible to realize data processing with low delay and high time efficiency.
The present disclosure also provides an electronic device 500 and a readable storage medium according to an embodiment of the present disclosure. Fig. 5 shows a block diagram of an electronic device to which an information processing method according to an embodiment of the present disclosure is applied.
As shown in fig. 5, a block diagram of an exemplary electronic device to which embodiments of the present disclosure can be applied is shown.
Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the electronic device 500 includes: one or more processors 501, memory 502, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may be directed to a display device coupled to the interface within the electronic device) to display graphical information of the GUI. In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 5, one processor 501 is taken as an example.
The memory 502, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the methods of information processing (e.g., the obtaining module 410, the determining module 420, and the recommending module 430 shown in fig. 4) in the embodiments of the present disclosure. The processor 501 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 502, that is, implements the information processing method in the above-described method embodiments.
The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of an electronic device to implement the method of information processing, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 502 optionally includes memory located remotely from processor 501, which may be connected via a network to an electronic device implementing the method of information processing. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device 500 to the information processing method may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected by a bus or other means, and fig. 5 illustrates the connection by a bus as an example.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of an electronic apparatus to implement the method of information processing, such as an input device of a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or the like. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, special purpose ASI5 (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a server of a distributed system or a server incorporating a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (14)
1. An information processing method, the method comprising:
processing acquired data obtained from at least one information source to generate first information usable for streaming;
generating second information through streaming calculation based on the first information, wherein the second information can embody indexes of the first information in a preset time period; and
and sending the second information to a cloud server.
2. The method of claim 1, wherein the processing acquisition data obtained from at least one information source to generate first information usable for streaming comprises:
acquisition data is obtained from a plurality of information sources through compatible protocol interfaces.
3. The method of claim 2, wherein,
the plurality of information sources comprises a plurality of sensors applied in different fields, the fields being selected from the fields of agriculture, electricity and transportation.
4. The method of claim 2 or 3, further comprising:
periodically collecting information of the plurality of information sources to obtain the collected data.
5. The method of claim 4, wherein the processing acquisition data obtained from at least one information source to generate first information usable for streaming comprises:
and subscribing the appointed subject in the collected data to generate screening information.
6. The method of claim 5, the processing acquisition data obtained from at least one information source to generate first information usable for streaming processing, comprising:
and parsing the syntax of the screening information to generate the first information.
7. The method of claim 6, wherein,
the streaming calculation based on the first information to generate second information comprises:
and counting the indexes of the first information in a preset time period determined by the preset time window by taking the preset time window as a unit.
8. The method of claim 7, wherein,
the time window is a rolling window, a sliding window or a counting window.
9. The method of claim 7 or 8,
the index is at least one of an average value and a maximum value.
10. The method of claim 1, wherein,
sending the second information to a cloud server, including:
and synchronizing the second information and then sending the second information to the cloud server.
11. The method of claim 5, further comprising:
a configuration step for configuring the at least one information source, the commitment topic or the rules of the streaming computation.
12. An information processing apparatus comprising:
a generation module configured to process acquisition data obtained from at least one information source to generate first information usable for streaming;
a calculation module configured to generate second information through streaming calculation based on the first information, wherein the second information can embody an index of the first information within a predetermined time period; and
a sending module configured to send the second information to a cloud server.
13. An electronic device, comprising:
a processor; and
a memory storing a program comprising instructions that, when executed by the processor, cause the processor to perform the method of any of claims 1 to 11.
14. A computer readable storage medium storing a program, the program comprising instructions that when executed by a processor of a server cause the server to perform the method of any of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011240270.1A CN112422412B (en) | 2020-11-09 | 2020-11-09 | Information processing method, apparatus, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011240270.1A CN112422412B (en) | 2020-11-09 | 2020-11-09 | Information processing method, apparatus, device and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112422412A true CN112422412A (en) | 2021-02-26 |
| CN112422412B CN112422412B (en) | 2023-03-24 |
Family
ID=74780793
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011240270.1A Active CN112422412B (en) | 2020-11-09 | 2020-11-09 | Information processing method, apparatus, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112422412B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114938353A (en) * | 2022-05-27 | 2022-08-23 | 中国银行股份有限公司 | Asynchronous notification current limiting method and system based on stream computing |
| CN115017222A (en) * | 2022-08-01 | 2022-09-06 | 深圳市其域创新科技有限公司 | Information processing system and method based on multiple information sources |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8924581B1 (en) * | 2012-03-14 | 2014-12-30 | Amazon Technologies, Inc. | Managing data transfer using streaming protocols |
| CN106815338A (en) * | 2016-12-25 | 2017-06-09 | 北京中海投资管理有限公司 | A kind of real-time storage of big data, treatment and inquiry system |
| CN109144014A (en) * | 2018-10-10 | 2019-01-04 | 北京交通大学 | The detection system and method for industrial equipment operation conditions |
| CN109522341A (en) * | 2018-11-27 | 2019-03-26 | 北京京东金融科技控股有限公司 | Realize method, apparatus, the equipment of the stream data processing engine based on SQL |
| CN110232054A (en) * | 2019-06-19 | 2019-09-13 | 北京百度网讯科技有限公司 | Log transmission system and streaming log transmission method |
| CN110245120A (en) * | 2019-06-19 | 2019-09-17 | 北京百度网讯科技有限公司 | The daily record data processing method of streaming computing system and streaming computing system |
| CN110460521A (en) * | 2019-09-19 | 2019-11-15 | 北京中电普华信息技术有限公司 | A kind of edge calculations AnyRouter |
| CN110597057A (en) * | 2019-08-22 | 2019-12-20 | 浙江工业大学 | Data processing system in industrial application scene |
| CN110688399A (en) * | 2019-08-26 | 2020-01-14 | 远光软件股份有限公司 | Stream type calculation real-time report system and method |
-
2020
- 2020-11-09 CN CN202011240270.1A patent/CN112422412B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8924581B1 (en) * | 2012-03-14 | 2014-12-30 | Amazon Technologies, Inc. | Managing data transfer using streaming protocols |
| CN106815338A (en) * | 2016-12-25 | 2017-06-09 | 北京中海投资管理有限公司 | A kind of real-time storage of big data, treatment and inquiry system |
| CN109144014A (en) * | 2018-10-10 | 2019-01-04 | 北京交通大学 | The detection system and method for industrial equipment operation conditions |
| CN109522341A (en) * | 2018-11-27 | 2019-03-26 | 北京京东金融科技控股有限公司 | Realize method, apparatus, the equipment of the stream data processing engine based on SQL |
| CN110232054A (en) * | 2019-06-19 | 2019-09-13 | 北京百度网讯科技有限公司 | Log transmission system and streaming log transmission method |
| CN110245120A (en) * | 2019-06-19 | 2019-09-17 | 北京百度网讯科技有限公司 | The daily record data processing method of streaming computing system and streaming computing system |
| CN110597057A (en) * | 2019-08-22 | 2019-12-20 | 浙江工业大学 | Data processing system in industrial application scene |
| CN110688399A (en) * | 2019-08-26 | 2020-01-14 | 远光软件股份有限公司 | Stream type calculation real-time report system and method |
| CN110460521A (en) * | 2019-09-19 | 2019-11-15 | 北京中电普华信息技术有限公司 | A kind of edge calculations AnyRouter |
Non-Patent Citations (1)
| Title |
|---|
| 周煜敏等: "基于Storm的实时大规模传感器监控平台的开发和实现", 《计算机应用与软件》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114938353A (en) * | 2022-05-27 | 2022-08-23 | 中国银行股份有限公司 | Asynchronous notification current limiting method and system based on stream computing |
| CN114938353B (en) * | 2022-05-27 | 2024-04-16 | 中国银行股份有限公司 | Asynchronous notification current limiting method and system based on stream computing |
| CN115017222A (en) * | 2022-08-01 | 2022-09-06 | 深圳市其域创新科技有限公司 | Information processing system and method based on multiple information sources |
| CN115017222B (en) * | 2022-08-01 | 2022-11-08 | 深圳市其域创新科技有限公司 | Information processing system and method based on multiple information sources |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112422412B (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11310313B2 (en) | Multi-threaded processing of search responses returned by search peers | |
| CN111753997B (en) | Distributed training method, system, device and storage medium | |
| CN110543944B (en) | Neural network structure searching method, apparatus, electronic device, and medium | |
| CN111858248B (en) | Application monitoring method, device, equipment and storage medium | |
| CN112422412B (en) | Information processing method, apparatus, device and medium | |
| CN112069201A (en) | Target data acquisition method and device | |
| CN112532972B (en) | Fault detection method and device for live broadcast service, electronic equipment and readable storage medium | |
| CN111639753B (en) | Method, apparatus, device and storage medium for training image processing super network | |
| CN106537347B (en) | System and method for distributing and processing streams | |
| CN111461343A (en) | Model parameter updating method and related equipment thereof | |
| CN110796191B (en) | Trajectory classification method and device | |
| CN112667795A (en) | Dialog tree construction method and device, dialog tree operation method, device and system | |
| CN110795456B (en) | Map query method and device, computer equipment and storage medium | |
| CN112559808B (en) | Data processing method and device and electronic equipment | |
| CN111680599A (en) | Face recognition model processing method, device, equipment and storage medium | |
| CN111339344B (en) | Indoor image retrieval method and device and electronic equipment | |
| CN114648642A (en) | Model training method, image detection method, image classification method and device | |
| CN111581049B (en) | Distributed system running state monitoring method, device, equipment and storage medium | |
| CN114020741A (en) | Vehicle mileage information storage method and device, vehicle electronic system and vehicle | |
| CN114756301A (en) | Log processing method, device and system | |
| CN102184693B (en) | Web-based high-efficiency method capable of customizing epidemiological questionnaire | |
| KR20210132719A (en) | Adaptation methods, devices and electronics of deep learning models | |
| CN111782794A (en) | Question-answer response method and device | |
| CN111783872A (en) | Method and device for training model, electronic equipment and computer readable storage medium | |
| CN111683086B (en) | Network data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |