CN115793961A - Data concurrency processing method and device - Google Patents
Data concurrency processing method and device Download PDFInfo
- Publication number
- CN115793961A CN115793961A CN202211334490.XA CN202211334490A CN115793961A CN 115793961 A CN115793961 A CN 115793961A CN 202211334490 A CN202211334490 A CN 202211334490A CN 115793961 A CN115793961 A CN 115793961A
- Authority
- CN
- China
- Prior art keywords
- data
- storage area
- priority
- bus
- steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data concurrency processing method and device, and belongs to the technical field of electric power data processing. The method comprises the following steps: s10, extracting data which cannot be sent, adding a timestamp, and storing the data into an external flash in a classified mode; s20, judging whether the CAN bus is busy every 2 seconds, and if the CAN bus is busy, operating a data selection algorithm; and S30, if the CAN bus is not busy, judging whether no new data is stored in the external flash for 30 seconds continuously or the time from the last data sending of the external flash reaches 10 minutes, and if so, sending the data of the special storage area to the gateway. The invention CAN distinguish the priority of the data and reserve historical data as much as possible according to different priorities under the condition that the data cannot be sent out due to the busy CAN bus, thereby solving the problem of data concurrency when a plurality of devices report the data simultaneously.
Description
Technical Field
The invention belongs to the technical field of electric power data processing, and particularly relates to a data concurrency processing method and device.
Background
With the increasing requirements of power data analysis and control, data transmission has become an indispensable key step. The CAN bus is a serial communication protocol standardized by ISO international standards, and is one of field buses which are applied internationally most widely. The method has the advantages of strong real-time performance of data communication among nodes of the network, bidirectional operation without setting a master station and a slave station, high transmission rate and the like. Moreover, a plurality of CAN bus transceivers CAN be connected to one CAN bus to perform communication at the same time, which is called data concurrence. In order to effectively process data during data concurrency, a method and an apparatus for processing data concurrency are needed.
However, if too many CAN bus transceivers are connected to the CAN bus, the communication on the CAN bus is busy due to limited communication resources, and some data cannot be sent out, thereby causing data loss. Therefore, a method and an apparatus for processing data efficiently when the CAN bus is busy during data concurrency are needed. Under the condition that the CAN bus is busy and data cannot be sent out, historical data are kept as far as possible, and therefore the problem of data concurrency when multiple devices report data simultaneously is solved.
Disclosure of Invention
The invention provides a data concurrency processing method and device, which CAN be used for keeping historical data as much as possible under the condition that data cannot be sent out due to busy CAN buses, so that the problem of data concurrency when multiple devices report data simultaneously is solved.
In order to solve at least one of the above technical problems, according to an aspect of the present invention, there is provided a data concurrent processing method, including the steps of:
s10, extracting data which cannot be sent, adding a timestamp, and storing the data into an external flash in a classified mode;
s20, judging whether the CAN bus is busy every 2 seconds, and if the CAN bus is busy, operating a data selection algorithm;
and S30, if the CAN bus is not busy, judging whether no new data is stored in the external flash for 30 seconds continuously or the time for sending data from the last external flash reaches 10 minutes, and if the new data is consistent, sending the data in the special storage area to the gateway.
Further, in step S10, the external flash is divided into 20 small blocks, and each small block stores data of one type of collection object.
Furthermore, each small block in the external flash is divided into 30 data areas, wherein the data areas 1-20 are temporary storage areas, and the data areas 21-30 are special storage areas;
further, the temporary storage area is used for storing original data; the special storage area is used for storing the data processed by the data selection algorithm.
Further, in step S20, the data selecting algorithm includes determining data priority, data blocking, data comparison selecting, and data selecting of a special storage area;
determining the data priority: aiming at data of different collected objects on the same CAN bus, when the data volume of a certain collected object temporary storage area reaches 20 at first, checking the data volume of all the collected objects, and determining the priority of each collected object according to the data volume; if the data volume is 10-20, the acquisition object has high priority, and if the data volume is 0-9, the acquisition object has low priority;
data blocking: when the data volume of a certain acquisition object reaches 20, partitioning the data according to the time stamp of the data; the data blocking method comprises the following steps: if the time difference of the data of two adjacent times is more than 2 seconds, partitioning the data between the two data; high priority data is divided into at most 2 blocks, and low priority data is divided into at most 4 blocks; if the number of the divided blocks is larger than the upper limit, the high-priority block dividing method is changed into the following steps: partitioning the data into 2 blocks in the middle of two adjacent data with the longest interval time; the low-priority blocking method is changed into the following steps: three groups of adjacent data with the interval time ranking of the first three are found out, the three groups of data are divided into 4 blocks;
and (3) data comparison and selection: after data is partitioned, data comparison and selection are carried out on each block of data; the method for comparing and selecting the high-priority data comprises the following steps: keeping the maximum value and the minimum value, calculating the average value of the time of the maximum value and the minimum value, and keeping the value of which the timestamp is closest to the average value; the method for comparing and selecting the low-priority data comprises the following steps: keeping the maximum value and the minimum value; all reserved data is put into a special storage area, and then all data in the temporary storage area are emptied;
data selection of the special storage area: if n data still need to be stored in the special storage area after the special storage area is full, calculating the average value of all data in the special storage area, deleting the n data closest to the average value, supplementing the vacated data area by the data behind the time, and then storing the new data in the last data.
Furthermore, the history data selected by the data selection algorithm is specifically the data stored in the special storage area.
According to another aspect of the present invention, a data concurrent processing device is provided, which includes an MCU, an external flash and a CAN bus transceiver;
the MCU is used for operating a data selection algorithm;
the external flash is used for storing data which cannot be normally sent to the gateway;
the CAN bus transceiver is used for receiving and transmitting data.
Further, the number of the CAN bus transceiving devices is at most 10.
According to another aspect of the present invention, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps in the data concurrency processing method and apparatus of the present invention.
According to yet another aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the data concurrent processing method and apparatus of the present invention when executing the program.
Compared with the prior art, the invention has at least the following beneficial effects:
the invention CAN distinguish the priority of the data and reserve historical data as much as possible according to different priorities under the condition that the data cannot be sent out due to the busy CAN bus, thereby solving the problem of data concurrency when a plurality of devices report the data simultaneously.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments will be briefly described below, and it is apparent that the drawings in the following description only relate to some embodiments of the present invention and are not limiting on the present invention.
FIG. 1 is a schematic diagram of a data concurrent processing method according to the present invention;
FIG. 2 is a schematic diagram of a data selection algorithm according to the present invention;
fig. 3 is a schematic diagram of the application structure of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings of the embodiments of the present invention. It should be apparent that the described embodiments are only some of the embodiments of the present invention, and not all of them.
Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs.
As shown in the figures 1-3 of the drawings,
example 1:
the invention provides a data concurrent processing method, which comprises the following steps:
step 1: and extracting data which cannot be transmitted, adding a time stamp, and storing the data in an external flash in a classified manner.
Step 2: and judging whether the CAN bus is busy every 2 seconds, and running a data selection algorithm if the CAN bus is busy.
And step 3: and if the CAN bus is not busy, judging whether new data are continuously stored in the external flash for 30 seconds or the time from the last data sending of the external flash reaches 10 minutes, and if so, sending the data of the special storage area to the gateway.
Fig. 2 is a schematic diagram of the data selection algorithm of the present invention, which comprises the following steps:
step 1: determining the data priority: aiming at data of different collected objects on the same CAN bus, when the data volume of a certain collected object temporary storage area reaches 20 at first, the data volume of all the collected objects is checked, and the priority of each collected object is determined according to the data volume. And if the data volume is 10-20, the acquisition object has high priority, and if the data volume is 0-9, the acquisition object has low priority.
Step 2: data blocking: when the data amount of a certain acquisition object reaches 20, the data is partitioned according to the time stamp of the data. The data blocking method comprises the following steps:
if the time difference between two adjacent time data is more than 2 seconds, the data is partitioned between the two data.
Wherein high priority data is divided into at most 2 blocks and low priority data is divided into at most 4 blocks. If the number of the divided blocks is larger than the upper limit, the high-priority block dividing method is changed into the following steps: partitioning the data into 2 blocks in the middle of two adjacent data with the longest interval time; the low-priority blocking method is changed into the following steps: three groups of adjacent data with the first three time intervals are found, the three groups of adjacent data are partitioned, and the data are partitioned into 4 blocks.
And step 3: and (3) data comparison and selection: after the data are divided into blocks, data comparison and selection are carried out on each block of data.
The method for comparing and selecting the high-priority data comprises the following steps: the maximum value, the minimum value are retained, and the average of the times at which the maximum value and the minimum value are found is calculated, and the value at which the timestamp is closest to the average is retained.
The method for comparing and selecting the low-priority data comprises the following steps: the maximum value and the minimum value are retained.
All the reserved data will be put into a special storage area and then the temporary storage area is emptied of all data.
And 4, step 4: data selection of the special storage area: if n data still need to be stored in the special storage area after the special storage area is full, calculating the average value of all data in the special storage area, deleting n data closest to the average value, supplementing the vacated data area by data behind time, and then storing new data in the last data area.
FIG. 3 is a schematic diagram of the application structure of the present invention, which includes MCUs 21A-21J, external flashes 22A-22J, CAN transceivers 23A-23J, and a gateway 30.
Fig. 3 shows a case where 10 CAN transceivers are connected to a CAN bus, and the operation steps are as follows:
step 1: one or more CAN transceivers detect that there is the data that CAN't send to the gateway, will CAN't send the data that CAN't send to the gateway and add the time stamp, transmit for MCU, by the different collection object of MCU classification, store in outside flash according to the collection object of difference.
Step 2: every 2 seconds, MCU checks the network state of CAN bus, when CAN bus is busy, the algorithm is chosen to the operation data, its step as follows:
step 2-1: determining the data priority: aiming at data of different collected objects on the same CAN bus, when the data volume of a certain collected object temporary storage area reaches 20 at first, the data volume of all the collected objects is checked, and the priority of each collected object is determined according to the data volume. And if the data volume is 10-20, the acquisition object has high priority, and if the data volume is 0-9, the acquisition object has low priority.
Step 2-2: data blocking: when the data amount of a certain acquisition object reaches 20, the data is partitioned according to the time stamp of the data. The data blocking method comprises the following steps:
if the time difference between two adjacent time data is more than 2 seconds, the data is partitioned between the two data.
Wherein high priority data is divided into at most 2 blocks and low priority data is divided into at most 4 blocks. If the number of the divided blocks is larger than the upper limit, the high-priority blocking method is changed into the following steps: partitioning the data into 2 blocks in the middle of two adjacent data with the longest interval time; the low-priority blocking method is changed into the following steps: three groups of adjacent data with the first three time intervals are found, the three groups of adjacent data are partitioned, and the data are partitioned into 4 blocks.
Step 2-3: and (3) data comparison and selection: after the data are divided into blocks, data comparison and selection are carried out on each block of data.
The method for comparing and selecting the high-priority data comprises the following steps: the maximum value, the minimum value are retained, and the average of the times at which the maximum value and the minimum value are found is calculated, and the value at which the timestamp is closest to the average is retained.
The method for comparing and selecting the low-priority data comprises the following steps: the maximum value and the minimum value are retained.
All the reserved data will be put into a special storage area and then the temporary storage area is emptied of all data.
Step 2-4: data selection of the special storage area: if n data still need to be stored in the special storage area after the special storage area is full, calculating the average value of all data in the special storage area, deleting n data closest to the average value, supplementing the vacated data area by data behind time, and then storing new data in the last data area.
And step 3: when the CAN bus is idle, if no new data is stored in the external flash for 30 seconds continuously or the time for sending data from the last external flash reaches 10 minutes, all data in the special storage area in the external flash are sent to the gateway through the CAN transceiver, and then all data in the special storage area corresponding to the external flash are deleted.
Example 2:
the computer-readable storage medium of the present embodiment has stored thereon a computer program that, when executed by a processor, implements the steps in the data concurrency processing method of embodiment 1.
The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, and the like, provided on the terminal; further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the terminal.
The computer-readable storage medium of the present embodiment is used to store a computer program and other programs and data required by the terminal, and may also be used to temporarily store data that has been output or is to be output.
Example 3:
the computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps in the data concurrent processing method of embodiment 1.
In this embodiment, the processor may be a central processing unit, or may also be other general-purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where a general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.; the memory may include both read-only memory and random access memory and provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory, e.g., the memory may also store device type information.
It will be appreciated by one skilled in the art that the present disclosure of embodiments may be provided as a method, system, or computer program product. Accordingly, the present solution may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present solution may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
While the present solution has been described with reference to flowchart illustrations and/or block diagrams of methods and computer program products according to embodiments of the solution, it should be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions; these computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-only memory (ROM), a Random Access Memory (RAM), or the like.
The examples described herein are merely illustrative of the preferred embodiments of the present invention and do not limit the spirit and scope of the invention, and various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the design concept of the present invention should fall within the protection scope of the present invention.
Claims (10)
1. A data concurrent processing method is characterized by comprising the following steps:
s10, extracting data which cannot be sent, adding a timestamp, and storing the data into an external flash in a classified mode;
s20, judging whether the CAN bus is busy every 2 seconds, and if the CAN bus is busy, operating a data selection algorithm;
and S30, if the CAN bus is not busy, judging whether no new data is stored in the external flash for 30 seconds continuously or the time from the last data sending of the external flash reaches 10 minutes, and if so, sending the data of the special storage area to the gateway.
2. The method according to claim 1, wherein in step S10, the external flash is divided into 20 small blocks, and each small block stores data of one type of the collected object.
3. The method of claim 2, wherein each of the small blocks in the external flash is divided into 30 data areas, wherein the data areas 1-20 are temporary storage areas and 21-30 are special storage areas.
4. The method of claim 3, wherein the temporary storage area is used to store raw data; the special storage area is used for storing the data processed by the data selection algorithm.
5. The method according to claim 4, wherein in step S20, the data selection algorithm comprises determining data priority, data blocking, data comparison selection and data selection of a special storage area;
determining the data priority: aiming at data of different collected objects on the same CAN bus, when the data volume of a certain collected object temporary storage area reaches 20 at first, checking the data volume of all the collected objects, and determining the priority of each collected object according to the data volume; if the data volume is 10-20, the acquisition object has high priority, and if the data volume is 0-9, the acquisition object has low priority;
data blocking: when the data volume of a certain acquisition object reaches 20, partitioning the data according to the time stamp of the data; the data blocking method comprises the following steps: if the time difference of the data of two adjacent times is more than 2 seconds, partitioning the data between the two data; high priority data is divided into at most 2 blocks, and low priority data is divided into at most 4 blocks; if the number of the divided blocks is larger than the upper limit, the high-priority block dividing method is changed into the following steps: partitioning the data into 2 blocks in the middle of two adjacent data with the longest interval time; the low-priority blocking method is changed into the following steps: three groups of adjacent data with the first three time intervals are found, the three groups of adjacent data are partitioned, and the data are partitioned into 4 blocks;
and (3) data comparison and selection: after the data are partitioned, data comparison and selection are carried out on each block of data; the method for comparing and selecting the high-priority data comprises the following steps: keeping the maximum value and the minimum value, calculating the average value of the time of the maximum value and the minimum value, and keeping the value of which the timestamp is closest to the average value; the method for comparing and selecting the low-priority data comprises the following steps: keeping the maximum value and the minimum value; all reserved data is put into a special storage area, and then all data in the temporary storage area are emptied;
data selection of the special storage area: if n data still need to be stored in the special storage area after the special storage area is full, calculating the average value of all data in the special storage area, deleting the n data closest to the average value, supplementing the vacated data area by the data behind the time, and then storing the new data in the last data.
6. The method of claim 5, wherein the historical data selected by the data selection algorithm is specifically data stored in a special storage area.
7. A data concurrent processing device is characterized by comprising an MCU, an external flash and a CAN bus transceiver;
the MCU is used for operating a data selection algorithm;
the external flash is used for storing data which cannot be normally sent to the gateway;
the CAN bus transceiver is used for receiving and transmitting data.
8. The apparatus of claim 7 wherein the number of CAN bus transceivers is at most 10.
9. A computer-readable storage medium having stored thereon a computer program, characterized in that: the program, when executed by a processor, implements the steps in the method of processing data concurrently as claimed in any one of claims 1 to 6.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps in the data concurrent processing method according to any one of claims 1 to 6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211334490.XA CN115793961A (en) | 2022-10-28 | 2022-10-28 | Data concurrency processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211334490.XA CN115793961A (en) | 2022-10-28 | 2022-10-28 | Data concurrency processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115793961A true CN115793961A (en) | 2023-03-14 |
Family
ID=85434239
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211334490.XA Pending CN115793961A (en) | 2022-10-28 | 2022-10-28 | Data concurrency processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115793961A (en) |
-
2022
- 2022-10-28 CN CN202211334490.XA patent/CN115793961A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109918141B (en) | Thread execution method, thread execution device, terminal and storage medium | |
| CN111045810B (en) | Task scheduling processing method and device | |
| CN110874265A (en) | Method, device and equipment for rapidly acquiring data and storage medium | |
| CN110300040A (en) | A kind of communication means and relevant device | |
| CN113127314B (en) | Method and device for detecting program performance bottleneck and computer equipment | |
| CN111651595A (en) | Abnormal log processing method and device | |
| CN111061570A (en) | Image calculation request processing method and device and terminal equipment | |
| CN116112507A (en) | Hierarchical blockchain-based multitasking asynchronous execution method and device | |
| CN110221914B (en) | File processing method and device | |
| CN108287760A (en) | Terminal equipment control method and device, terminal device and computer readable storage medium | |
| CN112000019B (en) | Method, device and equipment for sharing equipment instruction control | |
| CN107707395B (en) | Data transmission method, device and system | |
| CN115793961A (en) | Data concurrency processing method and device | |
| CN108777810A (en) | Video data storage method, device, equipment and storage medium | |
| CN112929251A (en) | CAN-based slave station data refreshing method, device and system | |
| CN106028292B (en) | A kind of channel distribution method and system sending short message | |
| CN115408173A (en) | Event processing method and device, electronic equipment and storage medium | |
| CN110825528B (en) | Resource management method, device and equipment | |
| CN114553727A (en) | Data processing method and device based on content distribution network | |
| CN111625358A (en) | Resource allocation method and device, electronic equipment and storage medium | |
| CN112231090A (en) | Application process management method and device and terminal equipment | |
| CN112818055B (en) | Block chain performance optimization method, device and equipment | |
| CN112015699A (en) | File data feature extraction method and device and computing equipment | |
| CN114401232A (en) | Data transmission method and device, electronic equipment and storage medium | |
| CN109992357B (en) | Method and system for avoiding conflict when executing change to virtual machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |