CN113064888B - Data proofreading method, device and system, server and equipment - Google Patents
Data proofreading method, device and system, server and equipment Download PDFInfo
- Publication number
- CN113064888B CN113064888B CN202110320349.3A CN202110320349A CN113064888B CN 113064888 B CN113064888 B CN 113064888B CN 202110320349 A CN202110320349 A CN 202110320349A CN 113064888 B CN113064888 B CN 113064888B
- Authority
- CN
- China
- Prior art keywords
- data
- server
- data transmission
- frame
- transmission device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Abstract
The disclosure relates to a data proofreading method, a device and a system, a server and equipment. The data checking method comprises the following steps: the server receives the equipment data reported by the data transmission device of the equipment; the server loads a duplicate removal table which is the same as the data transmission device; the server checks whether the frame serial numbers of the equipment data are continuous or not, wherein the data transmission device adds the continuous frame serial numbers to the collected changed equipment data frames and sends the frame serial numbers; when the frame number of the device data is discontinuous, the server requests the data transmission apparatus to perform data retransmission from the frame next to the frame with the discontinuous number. The present disclosure can find out local data loss in time under the condition that the data transmission device is used for retransmission.
Description
Technical Field
The present disclosure relates to the field of intelligent devices, and in particular, to a data proofreading method, apparatus and system, a server, and a device.
Background
With the advent of the big data era, smart home is gradually pursued in people's life. Products such as air conditioners, which are indispensable to life, are being developed toward intelligent home appliances. The intellectualization of the household electrical appliances makes the operation parameters more and more abundant and the data transmission frequency more and more high. A DTU (Data Transfer Unit) is responsible for collecting operation Data of the air conditioner and then transmitting the operation Data to the server.
When the related art DTU transmits data, it is only responsible for forwarding the received unit data to the remote server, and does not perform data processing, and this data transmission mode is called a transparent transmission mode (transparent transmission mode for short). In order to reduce the higher and higher flow generated in the data transmission process, the DTU has implemented a transmission mode for reducing the flow, and the method determines whether to send data by comparing the changes of the unit data, which is called deduplication transmission (abbreviated as deduplication mode).
Disclosure of Invention
When the related art DTU transmits data in the deduplication mode, the following problems occur: 1. the remote service cache data completion is needed during deduplication, and if local data loss exists, the completed data may have an error value.
In view of at least one of the above technical problems, the present disclosure provides a data checking method, device and system, a server and a device, which can timely find that local data is lost in a situation that a data transmission device is used for retransmission.
According to an aspect of the present disclosure, there is provided a data collation method including:
the server receives the equipment data reported by the data transmission device of the equipment;
the server loads a duplicate removal table which is the same as the data transmission device;
the server checks whether the frame serial numbers of the equipment data are continuous or not, wherein the data transmission device adds the continuous frame serial numbers to the collected changed equipment data frames and sends the frame serial numbers;
when the frame number of the device data is discontinuous, the server requests the data transmission apparatus to perform data retransmission from the frame next to the frame with the discontinuous number.
In some embodiments of the present disclosure, the data collation method further includes:
and under the condition that the frame sequence numbers of the equipment data are continuous, the server completes the received equipment data according to the duplication elimination table and updates the data cache.
In some embodiments of the present disclosure, the data collation method further includes:
the server judges whether the data transmission device is switched from a non-duplication elimination mode to a duplication elimination mode;
under the condition that the data transmission device is switched from a non-duplication elimination mode to a duplication elimination mode, the server loads a duplication elimination table which is the same as that of the data transmission device;
the server clears the data cache outside the deduplication table.
In some embodiments of the present disclosure, the server clearing the data cache outside the deduplication table comprises:
the server screens the parameters in the cache according to the duplication elimination table;
the server deletes the parameter in the data cache if the parameter is not in the deduplication table.
In some embodiments of the present disclosure, the data collation method further includes:
the server judges whether the current data transmission of the data transmission device is duplicate removal transmission;
under the condition that the current data transmission of the data transmission device is the duplicate removal transmission, executing the step of judging whether the data transmission device is switched from the non-duplicate removal mode to the duplicate removal mode;
and under the condition that the data transmission device is not switched from the non-duplication removal mode to the duplication removal mode, the server loads the duplication removal table which is the same as the data transmission device, and then, the step of checking whether the frame sequence numbers of the equipment data are continuous is executed.
According to another aspect of the present disclosure, there is provided a data collation method including:
the data transmission device maintains a duplicate removal table which is the same as the server;
the data transmission device adds continuous frame serial numbers to the device data frames to be reported;
the data transmission device reports the equipment data added with the frame number to the server so that the server can check whether the frame number of the equipment data is continuous or not;
the data transmission device retransmits the data from the frame next to the frame with the disconnected sequence number according to a retransmission request sent by the server when the frame sequence number of the device data is discontinuous.
In some embodiments of the present disclosure, the data collation method further includes:
the data transmission device receives the parameters reported by the bus;
the data transmission device judges whether the reported parameters are repeated with the parameters in the duplication elimination table;
and under the condition that the reported parameters are not repeated with the parameters in the deduplication table, updating the parameters in the deduplication table, and then executing the step of adding continuous frame numbers to the acquired changed equipment data frames.
In some embodiments of the present disclosure, the data collation method further includes:
under the condition that the data transmission device is connected with the server and receives a server data request, judging whether the current data report is the first report;
under the condition that the current data report is the first report, all data in the deduplication table are used as equipment data frames to be reported, and the step of adding continuous frame numbers to the equipment data frames to be reported is executed;
and under the condition that the current data report is not the first report, taking the changed data in the deduplication table as the equipment data frame to be reported, and executing the step of adding continuous frame sequence numbers to the equipment data frame to be reported.
According to another aspect of the present disclosure, there is provided a server including:
the data receiving module is configured to receive the device data reported by the data transmission device of the device;
a deduplication table loading module configured to load a deduplication table identical to the data transmission apparatus;
the data transmission device comprises a serial number checking module, a data transmission module and a data transmission module, wherein the serial number checking module is configured to check whether the frame serial numbers of the equipment data are continuous or not, and the data transmission device adds the continuous frame serial numbers to the collected changed equipment data frames and sends the frame serial numbers;
and the retransmission request module is configured to request the data transmission device to perform data retransmission from the next frame of the frame with the disconnected sequence number when the frame sequence number of the equipment data is discontinuous.
In some embodiments of the present disclosure, the server is configured to perform operations for implementing the data collation method according to any one of the above embodiments.
According to another aspect of the present disclosure, there is provided a data transmission apparatus including:
a deduplication table maintenance module configured to maintain a deduplication table identical to the server;
a sequence number adding module configured to add continuous frame sequence numbers to the device data frames to be reported;
the data reporting module is configured to report the equipment data with the added frame number to the server so that the server can check whether the frame numbers of the equipment data are continuous or not;
a data retransmission module configured to perform data retransmission from a frame next to the frame with the disconnected sequence number according to a retransmission request transmitted by the server in a case where the frame sequence number of the device data is discontinuous;
in some embodiments of the present disclosure, the data transmission apparatus is configured to perform operations for implementing the data collation method according to any one of the above embodiments.
According to another aspect of the present disclosure, there is provided an apparatus comprising a data transmission device as described in any of the above embodiments.
According to another aspect of the present disclosure, there is provided a computer apparatus comprising:
a memory configured to store instructions;
a processor configured to execute the instructions to cause the computer apparatus to perform operations to implement the data collation method according to any one of the above embodiments.
According to another aspect of the present disclosure, there is provided a data collation system comprising a server as described in any one of the above embodiments and a data transmission apparatus as described in any one of the above embodiments, or comprising a server as described in any one of the above embodiments and a device as described in any one of the above embodiments.
According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, in which computer instructions are stored, and when executed by a processor, implement the data collation method according to any one of the above embodiments.
The present disclosure can find out local data loss in time under the condition that the data transmission device is used for retransmission.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic diagram of some embodiments of the disclosed data collation method.
FIG. 2 is a schematic diagram of other embodiments of the data collation method of the present disclosure.
Fig. 3 is a diagram illustrating all parameters reported on a bus according to some embodiments of the disclosure.
FIG. 4 is a schematic diagram of further embodiments of the disclosed data collation method.
FIG. 5 is a schematic diagram of further embodiments of the disclosed data collation method.
Fig. 6 is a schematic diagram of some embodiments of a server of the present disclosure.
Fig. 7 is a schematic diagram of some embodiments of a data transmission device according to the present disclosure.
FIG. 8 is a schematic block diagram of some embodiments of a computer apparatus according to the present disclosure.
FIG. 9 is a schematic diagram of some embodiments of the data collation system of the present disclosure.
FIG. 10 is a schematic diagram of further embodiments of the data collation system of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
In the related art, when the DTU has mode switching (for example, the transparent transmission mode is switched to the deduplication mode), the possibility of mistaken completion of the non-deduplication parameters exists.
In view of at least one of the above technical problems, the present disclosure provides a data collation method, device and system, a server and a device, and the present disclosure is explained by specific embodiments below.
FIG. 1 is a schematic diagram of some embodiments of the disclosed data collation method. Preferably, the present embodiment may be performed by the data transmission apparatus or the device of the present disclosure. The method comprises the following steps:
And step 12, the data transmission device adds continuous frame serial numbers to the device data frames to be reported.
In some embodiments of the present disclosure, the device may be a home appliance such as an air conditioner. The device data may be air conditioning group data.
In some embodiments of the present disclosure, step 12 may comprise: the data transmission device adds a sequence number to each data frame sent to the server, such as: the sequence number of the first frame is 0, followed by 1, 2, 3, … …, and the sequence of data transmission in this round is determined by the number.
And step 13, the data transmission device reports the equipment data with the added frame number to the server so that the server can check whether the frame number of the equipment data is continuous or not.
In step 14, the data transmission apparatus retransmits the data from the frame next to the frame with the disconnected sequence number according to a retransmission request transmitted by the server when the frame sequence number of the device data is discontinuous.
For example: the currently received frame number is 100, but the previous frame number is 98, and the data frame number is not consecutive. And the server judges that data loss exists in the transmission process, immediately feeds back a request to the DTU, and requests the DTU to retransmit the data from the frame with the sequence number of 99.
According to the embodiment of the disclosure, by adding the continuous frame numbers to the device data frames to be reported, the server can complete the de-duplicated data conveniently, and meanwhile, the server can check whether the data is lost in the transmission process conveniently.
The embodiment of the disclosure solves the problem that the data transmission device reports the lost data and corrects the data in time, and ensures the integrity of the data.
FIG. 2 is a schematic diagram of other embodiments of the data collation method of the present disclosure. Preferably, the present embodiment may be performed by the data transmission apparatus or the device of the present disclosure. The method comprises the following steps:
and step 20, powering on the data transmission device.
Step 21, the data transmission device maintains the same deduplication table as the server.
In some embodiments of the present disclosure, step 21 may comprise: after the data transmission device is powered on, the data transmission device receives the unit data reported by the bus, and then an extensible deduplication parameter table is established in the memory to store the required unit data.
And step 23, the data transmission device judges whether the reported parameters are repeated with the parameters in the deduplication table. Under the condition that the reported parameters are repeated with the parameters in the deduplication table, continuing to execute the step 22; otherwise, if the reported parameter is not identical to the parameter in the deduplication table, step 24 is executed.
And step 24, updating the parameters of the deduplication table.
Step 26, after the data transmission device establishes connection with the server, determining whether a data transmission request or a data retransmission request of the server is received. When a data transmission request or a data retransmission request from the server is received, step 27 is executed.
And 27, judging whether the current data report is the first report. When the current data report is the first report, executing step 28; otherwise, if the current data report is not the first report, step 29 is executed.
And step 28, reporting (counting is 0) for the first time, and sending all information of the list to the server in a mode of acquiring all data (simply referred to as photographing) in the list by taking the deduplication list as a basis.
In some embodiments of the present disclosure, step 28 may comprise: taking all data in the deduplication table as equipment data frames to be reported, and adding continuous frame serial numbers to the equipment data frames to be reported; the data transmission device reports the equipment data added with the frame number to the server so that the server can check whether the frame number of the equipment data is continuous or not; the data transmission device retransmits the data from the frame next to the frame with the disconnected sequence number according to a retransmission request sent by the server when the frame sequence number of the device data is discontinuous.
And step 29, reporting only the parameters which are changed at the current moment in the deduplication table, and increasing the count.
In some embodiments of the present disclosure, the count in steps 28 and 29 is a count of data frame sequence numbers.
In some embodiments of the present disclosure, step 29 may comprise: taking the changed data in the deduplication table as the equipment data frames to be reported, and adding continuous frame serial numbers to the equipment data frames to be reported; the data transmission device reports the equipment data added with the frame number to the server so that the server can check whether the frame number of the equipment data is continuous or not; the data transmission device retransmits the data from the frame next to the frame with the disconnected sequence number according to a retransmission request sent by the server when the frame sequence number of the device data is discontinuous.
The disclosure of the above embodiments is illustrated by the following specific examples.
Fig. 3 is a diagram illustrating all parameters reported on a bus according to some embodiments of the disclosure. In the case of a client request (with configuration parameters) via the server, if the data transmission device uploads all the collected data frames, this is very traffic consuming.
According to the data transmission and correction method of the above embodiment of the present disclosure, taking table 2 as an example, assuming that a line of data at each time is a data frame sent to the server, and assuming that the line of data at 10:00 is reported for the first time, a frame number is added to each data frame, so that the data sent to the server is shown in table 1.
TABLE 1
In some embodiments of the present disclosure, the deduplication table is defined as: the DTU needs to detect and determine the set of parameters with changes, i.e. the set of parameter 1, parameter 2, …, and parameter n, and not all the parameter changes are detected and reported, and only the parameters in the deduplication table are detected and changed.
The frame sequence number in the above embodiments of the present disclosure mainly serves to determine whether there is a frame loss in the DTU reporting process.
After receiving the server retransmission request, the DTU of the above-mentioned embodiment of the present disclosure interrupts the current transmission frame, finds the target sequence number frame from the cache, and then sequentially transmits the changed parameter frames. In this way, the accuracy of remote caching of data in the retransmission-removing mode can be ensured.
The data correction method provided based on the above embodiment of the present disclosure is a data correction method for an air conditioner and other devices based on deduplication transmission. The data transmission device of the above embodiment of the present disclosure can find out that the local data is lost in time under the condition of retransmission elimination.
FIG. 4 is a schematic diagram of further embodiments of the disclosed data collation method. Preferably, the present embodiment may be performed by a server of the present disclosure. The method comprises the following steps:
and step 41, the server receives the device data reported by the data transmission device of the device.
The server loads the same deduplication table as the data transfer device, step 42.
In step 44, when the frame number of the device data is not continuous, the server requests the data transmission apparatus to retransmit the data from the frame next to the frame with the disconnected number.
FIG. 5 is a schematic diagram of further embodiments of the disclosed data collation method. Preferably, the present embodiment may be performed by a server of the present disclosure. The method comprises the following steps:
at step 500, the server starts.
In step 502, the server determines whether the current data transmission of the data transmission device is a deduplication transmission. In case that the current data transmission of the data transmission apparatus is a deduplication transmission, execute step 504; otherwise, in case the current data transmission of the data transmission device is not a deduplication transmission, step 504 is performed.
In some embodiments of the present disclosure, the data transmission apparatus not only has one data transmission mode, but also includes a transparent transmission reporting mode of the parameter of the forwarding unit in addition to the re-reporting.
In step 504, it is detected whether there is a mode switch, i.e. it is determined whether the data transmission apparatus is switched from the non-deduplication mode to the deduplication mode. If the data transmission device is switched from the non-deduplication mode to the deduplication mode, go to step 506; otherwise, if the data transmission apparatus is not switched from the non-deduplication mode to the deduplication mode, step 506 is executed.
In step 505, the server loads the deduplication table identical to that of the data transmission apparatus, and then executes step 508.
In step 506, the server loads the deduplication table identical to that of the data transmission apparatus, and then performs step 507.
In step 507, the server clears the data cache outside the deduplication table, and then performs step 508.
In some embodiments of the present disclosure, step 507 may comprise: the server screens the parameters in the cache according to the duplication elimination table; the server deletes the parameter in the data cache if the parameter is not in the deduplication table.
Step 510-step 512, the server analyzes the received device data according to the duplication elimination table, updates the data cache, removes the data cache, and completes the data.
The embodiment of the disclosure not only solves the problem of abnormal data bit filling caused by mode switching in the deduplication reporting process of the data transmission device, but also solves the problem of timely correction when the data transmission device reports lost data, and ensures the integrity of the data.
In the above embodiment of the present disclosure, when the DTU is switched from the non-deduplication mode to the deduplication mode, the parameters processed by the server are also changed correspondingly, the parameters in the cache are screened according to the deduplication table, and if the parameters are not in the deduplication table, the parameters are deleted in the cache, so as to eliminate a situation that a false complement value occurs when data is not reported during deduplication transmission.
The above embodiments of the present disclosure can find out that the local data is lost in time when the data transmission apparatus is in retransmission.
The above-mentioned embodiments of the present disclosure can filter the non-duplication removal parameter by double filtering when the data transmission apparatus switches the transmission mode. The double filtration of the above embodiments of the present disclosure refers to: firstly, screening and collecting parameters in a cache by a DTU according to a duplication elimination table; second, when the DTU is switched to the deduplication mode, the server segment may filter the parameters of the non-deduplication table in the server-side cache according to the deduplication table, because the parameters of the server cache in the transparent transmission mode may include parameters that are not in the deduplication table.
Fig. 6 is a schematic diagram of some embodiments of a server of the present disclosure. As shown in fig. 6, the server of the present disclosure may include a data receiving module 61, a deduplication table loading module 62, a sequence number checking module 63, and a retransmission request module 64, where:
and the data receiving module 61 is configured to receive the device data reported by the data transmission device of the device.
A deduplication table loading module 62 configured to load the same deduplication table as the data transmission apparatus.
And a sequence number checking module 63 configured to check whether the frame sequence numbers of the device data are continuous, wherein the data transmission device adds the continuous frame sequence numbers to the collected changed device data frames and transmits the frame sequence numbers.
And a retransmission request module 64 configured to request the data transmission apparatus to perform data retransmission from the frame next to the frame with the disconnected sequence number if there is a discontinuity in the frame sequence number of the device data.
In some embodiments of the present disclosure, the server may be configured to, in a case where frame sequence numbers of the device data are consecutive, complement the received device data according to the deduplication table, and update the data cache.
In some embodiments of the present disclosure, the server may be configured to determine whether the data transmission apparatus is switched from the non-deduplication mode to the deduplication mode; under the condition that the data transmission device is switched from a non-deduplication mode to a deduplication mode, loading the deduplication table which is the same as the data transmission device; and clearing the data cache outside the deduplication table.
In some embodiments of the present disclosure, the server, in the case of clearing the data cache other than the deduplication table, may be configured to filter the parameters in the cache according to the deduplication table; in the event that a parameter is not in the deduplication table, the parameter is deleted in the data cache.
In some embodiments of the present disclosure, the server may be further configured to determine whether a current data transmission of the data transmission apparatus is a deduplication transmission; under the condition that the current data transmission of the data transmission device is the duplicate removal transmission, executing the operation of judging whether the data transmission device is switched from the non-duplicate removal mode to the duplicate removal mode; and loading the same deduplication table as the data transmission device under the condition that the data transmission device is not switched from the non-deduplication mode to the deduplication mode, and then performing operation of checking whether the frame sequence numbers of the equipment data are continuous or not.
In some embodiments of the present disclosure, the server may be configured to perform operations for implementing the data checking method according to any one of the embodiments (for example, fig. 4 or fig. 5) described above.
Based on the server provided by the embodiment of the disclosure, the problem of abnormal data bit filling caused by mode switching in the duplicate removal reporting process of the data transmission device is solved, and the problem of timely correction when the data transmission device reports lost data is solved, so that the data integrity is ensured.
In the above embodiment of the present disclosure, when the DTU is switched from the non-deduplication mode to the deduplication mode, the parameters processed by the server are also changed correspondingly, the parameters in the cache are screened according to the deduplication table, and if the parameters are not in the deduplication table, the parameters are deleted in the cache, so as to eliminate a situation that a false complement value occurs when data is not reported during deduplication transmission.
The above embodiments of the present disclosure can find out that the local data is lost in time when the data transmission apparatus is in retransmission.
The above-mentioned embodiments of the present disclosure can filter the non-duplication removal parameter by double filtering when the data transmission apparatus switches the transmission mode. The double filtration of the above embodiments of the present disclosure refers to: firstly, screening and collecting parameters in a cache by a DTU according to a duplication elimination table; second, when the DTU is switched to the deduplication mode, the server segment may filter the parameters of the non-deduplication table in the server-side cache according to the deduplication table, because the parameters of the server cache in the transparent transmission mode may include parameters that are not in the deduplication table.
Fig. 7 is a schematic diagram of some embodiments of a data transmission device according to the present disclosure. As shown in fig. 7, the data transmission apparatus of the present disclosure may include a deduplication table maintaining module 71, a sequence number adding module 72, a data reporting module 73, and a data retransmitting module 74, where:
a deduplication table maintenance module 71 configured to maintain the same deduplication table as the server.
A sequence number adding module 72 configured to add consecutive frame sequence numbers to the device data frames to be reported.
And a data reporting module 73 configured to report the device data with the added frame number to the server, so that the server checks whether the frame numbers of the device data are consecutive.
And a data retransmission module 74 configured to perform data retransmission from a frame next to the frame with the disconnection sequence number according to a retransmission request transmitted by the server when there is discontinuity in the frame sequence number of the device data.
In some embodiments of the present disclosure, the data transmission device may be further configured to receive parameters reported by a bus; judging whether the reported parameters are repeated with the parameters in the deduplication table; and under the condition that the reported parameters are not repeated with the parameters in the deduplication table, updating the parameters in the deduplication table, and then executing the operation of adding continuous frame serial numbers to the acquired changed equipment data frames.
In some embodiments of the present disclosure, the data transmission apparatus may be further configured to determine whether the current data report is a first report when the data transmission apparatus is connected to the server and receives a server data request; under the condition that the current data report is the first report, all data in the deduplication table are used as equipment data frames to be reported, and the operation of adding continuous frame serial numbers to the equipment data frames to be reported is executed; and under the condition that the current data report is not the first report, taking the changed data in the deduplication table as the equipment data frame to be reported, and executing the operation of adding continuous frame serial numbers to the equipment data frame to be reported.
In some embodiments of the present disclosure, the data transmission apparatus may be configured to perform operations for implementing the data checking method according to any of the embodiments (for example, the embodiments of fig. 1 or fig. 2).
According to another aspect of the present disclosure, there is provided an apparatus including a data transmission device as described in any of the above embodiments (e.g., the embodiment of fig. 7).
Based on the data transmission device and the equipment provided by the embodiment of the disclosure, by adding continuous frame numbers to the equipment data frames to be reported, the server can complete the de-duplicated data conveniently, and meanwhile, the server can check whether the data is lost in the transmission process conveniently.
The embodiment of the disclosure solves the problem that the data transmission device reports the lost data and corrects the data in time, and ensures the integrity of the data.
FIG. 8 is a schematic block diagram of some embodiments of a computer apparatus according to the present disclosure. As shown in fig. 8, the computer apparatus includes a memory 81 and a processor 82.
The memory 81 is used for storing instructions, the processor 82 is coupled to the memory 81, and the processor 82 is configured to execute a data checking method according to any of the embodiments (for example, any of the embodiments of fig. 1-2 and 4-5) based on the instructions stored in the memory.
In some embodiments of the present disclosure, in a case where the processor 82 implements the data collation method according to any of the above embodiments (e.g., fig. 1-2), the computer apparatus of the present disclosure may be implemented as a server of the present disclosure.
In other embodiments of the present disclosure, in the case that the processor 82 implements the data checking method according to any of the above embodiments (for example, fig. 4 to fig. 5), the computer apparatus of the present disclosure may be implemented as the data transmission apparatus of the present disclosure.
As shown in fig. 8, the computer apparatus further comprises a communication interface 83 for information interaction with other devices. The computer device also includes a bus 84, and the processor 82, the communication interface 83, and the memory 81 communicate with each other via the bus 84.
The memory 81 may include a high-speed RAM memory, and may further include a non-volatile memory (e.g., at least one disk memory). The memory 81 may also be a memory array. The storage 81 may also be partitioned and the blocks may be combined into virtual volumes according to certain rules.
Further, the processor 82 may be a central processing unit CPU, or may be an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the present disclosure.
FIG. 9 is a schematic diagram of some embodiments of the data collation system of the present disclosure. As shown in fig. 9, the data collation system of the present disclosure may include a server 91 and a data transmission device 92, wherein:
the server 91 may be implemented as the server described in any of the above embodiments (e.g., the embodiment of fig. 6).
The Data Transmission Unit (DTU)92 may be implemented as the data transmission unit described in any of the above embodiments (e.g., the embodiment of fig. 7).
The foregoing embodiments of the present disclosure are directed to a data transmission apparatus sending data to a server in a deduplication mode, where a same parameter table, that is, a deduplication table, is maintained by a DTU and the server together in a memory. In the duplicate removal reporting mode, after the data transmission device 92 is connected with the server 91, all the unit operation parameters are sent in a photographing transmission mode when data occurs for the first time, and then only the parameters with changes at the current moment in the duplicate removal table are reported (the transparent transmission mode neither filters nor judges the changes, and directly forwards the unit parameters). After the server 91 is connected with the data transmission device, a duplicate removal cache is established, the data sent by the data transmission device is received, the data is analyzed, the cache is updated, and then the latest cache data is stored in the database.
FIG. 10 is a schematic diagram of further embodiments of the data collation system of the present disclosure. As shown in fig. 10, the data collation system of the present disclosure may include a server 91 and a device 93, in which:
the server 91 may be implemented as the server described in any of the above embodiments (e.g., the embodiment of fig. 6).
The device 93 may be a home appliance such as an air conditioner.
The device 93 may comprise a data transmission apparatus as described in any of the above embodiments (e.g., the embodiment of fig. 7 or the embodiment of fig. 9).
Based on the data checking system provided by the above embodiment of the present disclosure, when the DTU is switched from the non-deduplication mode to the deduplication mode, the parameters processed by the server are also changed correspondingly, the parameters in the cache are screened according to the deduplication table, and if the parameters are not in the deduplication table, the parameters are deleted in the cache, so as to eliminate the situation that the data is not reported but the false complement value occurs during deduplication.
The above embodiments of the present disclosure can find out that the local data is lost in time when the data transmission apparatus is in retransmission.
The above-mentioned embodiments of the present disclosure can filter the non-duplication removal parameter by double filtering when the data transmission apparatus switches the transmission mode. The double filtration of the above embodiments of the present disclosure refers to: firstly, screening and collecting parameters in a cache by a DTU according to a duplication elimination table; second, when the DTU is switched to the deduplication mode, the server segment may filter the parameters of the non-deduplication table in the server-side cache according to the deduplication table, because the parameters of the server cache in the transparent transmission mode may include parameters that are not in the deduplication table.
According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, and the instructions when executed by a processor implement the data checking method according to any one of the embodiments (for example, any one of fig. 1-2 and fig. 4-5).
Based on the non-transitory computer readable storage medium provided by the above embodiment of the present disclosure, the problem of abnormal data bit padding caused by mode switching in the deduplication reporting process of the data transmission device is solved, and the data transmission device reports lost data and corrects the data in time to ensure the integrity of the data.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The servers and data transfer devices described above may be implemented as a general purpose processor, a Programmable Logic Controller (PLC), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof, for performing the functions described herein.
Thus far, the present disclosure has been described in detail. Some details that are well known in the art have not been described in order to avoid obscuring the concepts of the present disclosure. It will be fully apparent to those skilled in the art from the foregoing description how to practice the presently disclosed embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware to implement the above embodiments, where the program may be stored in a non-transitory computer readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic or optical disk, and the like.
The description of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.
Claims (12)
1. A method of data collation, comprising:
the server receives the equipment data reported by the data transmission device of the equipment;
the server loads a duplicate removal table which is the same as the data transmission device;
the server checks whether the frame serial numbers of the equipment data are continuous or not, wherein the data transmission device adds the continuous frame serial numbers to the collected changed equipment data frames and sends the frame serial numbers;
under the condition that the frame sequence number of the equipment data is discontinuous, the server requests the data transmission device to retransmit the data from the next frame of the frame with the disconnected sequence number;
the data proofreading method further comprises the following steps:
the server judges whether the data transmission device is switched from a non-duplication elimination mode to a duplication elimination mode;
under the condition that the data transmission device is switched from a non-duplication elimination mode to a duplication elimination mode, the server loads a duplication elimination table which is the same as that of the data transmission device;
the server clears the data cache outside the deduplication table.
2. The data collation method according to claim 1, further comprising:
and under the condition that the frame sequence numbers of the equipment data are continuous, the server completes the received equipment data according to the duplication elimination table and updates the data cache.
3. The data collation method according to claim 1 or 2, wherein the server clearing the data cache other than the deduplication table comprises:
the server screens the parameters in the cache according to the duplication elimination table;
the server deletes the parameter in the data cache if the parameter is not in the deduplication table.
4. The data collation method according to claim 1 or 2, further comprising:
the server judges whether the current data transmission of the data transmission device is duplicate removal transmission;
under the condition that the current data transmission of the data transmission device is the duplicate removal transmission, executing the step of judging whether the data transmission device is switched from the non-duplicate removal mode to the duplicate removal mode;
and under the condition that the data transmission device is not switched from the non-duplication removal mode to the duplication removal mode, the server loads the duplication removal table which is the same as the data transmission device, and then, the step of checking whether the frame sequence numbers of the equipment data are continuous is executed.
5. A method of data collation, comprising:
the data transmission device maintains a duplicate removal table which is the same as the server;
the data transmission device adds continuous frame serial numbers to the device data frames to be reported;
the data transmission device reports the equipment data added with the frame number to the server so that the server can check whether the frame number of the equipment data is continuous or not;
the data transmission device retransmits the data from the next frame of the frame with the disconnected sequence number according to the retransmission request sent by the server under the condition that the frame sequence number of the equipment data is discontinuous;
wherein, the data checking method further comprises:
the data transmission device receives the parameters reported by the bus;
the data transmission device judges whether the reported parameters are repeated with the parameters in the duplication elimination table;
and under the condition that the reported parameters are not repeated with the parameters in the deduplication table, updating the parameters in the deduplication table, and then executing the step of adding continuous frame numbers to the acquired changed equipment data frames.
6. The data collation method according to claim 5, further comprising:
under the condition that the data transmission device is connected with the server and receives a server data request, judging whether the current data report is the first report;
under the condition that the current data report is the first report, all data in the deduplication table are used as equipment data frames to be reported, and the step of adding continuous frame numbers to the equipment data frames to be reported is executed;
and under the condition that the current data report is not the first report, taking the changed data in the deduplication table as the equipment data frame to be reported, and executing the step of adding continuous frame sequence numbers to the equipment data frame to be reported.
7. A server, comprising:
the data receiving module is configured to receive the device data reported by the data transmission device of the device;
a deduplication table loading module configured to load a deduplication table identical to the data transmission apparatus;
the data transmission device comprises a serial number checking module, a data transmission module and a data transmission module, wherein the serial number checking module is configured to check whether the frame serial numbers of the equipment data are continuous or not, and the data transmission device adds the continuous frame serial numbers to the collected changed equipment data frames and sends the frame serial numbers;
a retransmission request module configured to request the data transmission apparatus to perform data retransmission from a frame next to the frame with the disconnected sequence number, in a case where the frame sequence number of the device data is discontinuous;
wherein the server is configured to execute operations for implementing the data collation method according to any one of claims 1 to 4.
8. A data transmission apparatus, comprising:
a deduplication table maintenance module configured to maintain a deduplication table identical to the server;
a sequence number adding module configured to add continuous frame sequence numbers to the device data frames to be reported;
the data reporting module is configured to report the equipment data with the added frame number to the server so that the server can check whether the frame numbers of the equipment data are continuous or not;
a data retransmission module configured to perform data retransmission from a frame next to the frame with the disconnected sequence number according to a retransmission request transmitted by the server in a case where the frame sequence number of the device data is discontinuous;
wherein the data transmission apparatus is configured to perform an operation of implementing the data collation method according to claim 5 or 6.
9. An electric household appliance comprising a data transmission device according to claim 8.
10. A computer device, comprising:
a memory configured to store instructions;
a processor configured to execute the instructions to cause the computer apparatus to perform operations to implement the data collation method according to any one of claims 1 to 6.
11. A data collation system comprising a server according to claim 7 and a data transmission apparatus according to claim 8, or comprising a server according to claim 7 and a home appliance according to claim 9.
12. A non-transitory computer readable storage medium storing computer instructions which, when executed by a processor, implement the data collation method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110320349.3A CN113064888B (en) | 2021-03-25 | 2021-03-25 | Data proofreading method, device and system, server and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110320349.3A CN113064888B (en) | 2021-03-25 | 2021-03-25 | Data proofreading method, device and system, server and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113064888A CN113064888A (en) | 2021-07-02 |
| CN113064888B true CN113064888B (en) | 2021-12-07 |
Family
ID=76563491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110320349.3A Active CN113064888B (en) | 2021-03-25 | 2021-03-25 | Data proofreading method, device and system, server and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113064888B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110019169A (en) * | 2017-12-29 | 2019-07-16 | 中国移动通信集团陕西有限公司 | A kind of method and device of data processing |
| CN110716928A (en) * | 2019-09-09 | 2020-01-21 | 上海凯京信达科技集团有限公司 | Data processing method, device, equipment and storage medium |
| CN112084179A (en) * | 2020-09-02 | 2020-12-15 | 北京锐安科技有限公司 | Data processing method, device, equipment and storage medium |
| CN112463737A (en) * | 2020-11-17 | 2021-03-09 | 中科金审(北京)科技有限公司 | System and method for rapidly acquiring data aiming at multi-format data intelligent matching template |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106792262A (en) * | 2016-12-05 | 2017-05-31 | 乐视控股（北京）有限公司 | Method of transmitting video data and device |
| CN107276728B (en) * | 2017-06-12 | 2019-05-31 | 北京三快在线科技有限公司 | A kind of data transmission method and device |
| CN107395396A (en) * | 2017-06-22 | 2017-11-24 | 中国科学院西安光学精密机械研究所 | The double network interfaces of redundancy based on FPGA can configure Ethernet IP kernel |
| CN109101394B (en) * | 2018-07-09 | 2023-05-09 | 珠海格力电器股份有限公司 | Data processing method and device |
| CN111200624B (en) * | 2018-11-19 | 2023-05-23 | 浙江宇视科技有限公司 | Cross-gatekeeper data transmission method and system |
| CN111654523A (en) * | 2020-04-28 | 2020-09-11 | 珠海格力电器股份有限公司 | Data processing method and device, storage medium and server |
| CN111770023B (en) * | 2020-06-28 | 2022-04-15 | 湖南有马信息技术有限公司 | Message duplicate removal method and device based on FPGA and FPGA chip |
| CN112350912B (en) * | 2020-10-29 | 2021-07-27 | 山东八五信息技术有限公司 | Data acquisition method, system and device based on Modbus protocol |
-
2021
- 2021-03-25 CN CN202110320349.3A patent/CN113064888B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110019169A (en) * | 2017-12-29 | 2019-07-16 | 中国移动通信集团陕西有限公司 | A kind of method and device of data processing |
| CN110716928A (en) * | 2019-09-09 | 2020-01-21 | 上海凯京信达科技集团有限公司 | Data processing method, device, equipment and storage medium |
| CN112084179A (en) * | 2020-09-02 | 2020-12-15 | 北京锐安科技有限公司 | Data processing method, device, equipment and storage medium |
| CN112463737A (en) * | 2020-11-17 | 2021-03-09 | 中科金审(北京)科技有限公司 | System and method for rapidly acquiring data aiming at multi-format data intelligent matching template |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113064888A (en) | 2021-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111464635B (en) | Dictionary index transmission method and device | |
| CN108513268B (en) | Short message abnormity processing method and short message platform | |
| CN109981450B (en) | Path communication maintenance method, device and system | |
| US9313241B2 (en) | Communication system and relay device, computer-readable recording medium storing relay program, as well as communication method and relay method | |
| CN111106944A (en) | Fault alarm information processing method and equipment | |
| CN113064888B (en) | Data proofreading method, device and system, server and equipment | |
| CN110889143A (en) | File verification method and device | |
| CN113282587B (en) | Method and device for transmitting state data, storage medium and electronic device | |
| JP2003162430A (en) | Apparatus and method for trouble information management | |
| CN110460486B (en) | Service node monitoring method and system | |
| CN113794622B (en) | Message processing method and device, electronic equipment and storage medium | |
| CN114356625A (en) | Distributed system redundancy diagnosis method, device, electronic device and storage medium | |
| CN108874530B (en) | Method and device for expanding and shrinking service board of message forwarding equipment | |
| CN106681954A (en) | Method for parameter setting based on bus and controller for robot | |
| CN105827606A (en) | Terminal corrupted data processing method and device, and terminal | |
| CN114567536B (en) | Abnormal data processing method, device, electronic equipment and storage medium | |
| CN115712618B (en) | DCS system data processing and uploading method, system and storage medium | |
| JP2856074B2 (en) | Communication trace mechanism | |
| CN112702236B (en) | Method and processor for realizing packet loss detection | |
| CN108512769B (en) | Method and related equipment for sending and receiving flow table items in software defined network | |
| JP6301750B2 (en) | Relay device | |
| CN105100272B (en) | electrocardiograph intelligent file transmission method and system | |
| CN117768508A (en) | Data processing method and device based on Internet of things | |
| CN117336304A (en) | Data retransmission method, device, electronic equipment and storage medium | |
| CN115865623A (en) | Multi-platform-oriented alarm data processing method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |