CN115314553A - Data processing method, device, equipment, system and readable storage medium - Google Patents

Data processing method, device, equipment, system and readable storage medium Download PDF

Info

Publication number
CN115314553A
CN115314553A CN202210887031.8A CN202210887031A CN115314553A CN 115314553 A CN115314553 A CN 115314553A CN 202210887031 A CN202210887031 A CN 202210887031A CN 115314553 A CN115314553 A CN 115314553A
Authority
CN
China
Prior art keywords
data
uploaded
target
preprocessing
rule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210887031.8A
Other languages
Chinese (zh)
Other versions
CN115314553B (en
Inventor
唐天旭
黎清顾
李爽
郭强
林浩生
张锴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gree Electric Appliances Inc of Zhuhai
Zhuhai Lianyun Technology Co Ltd
Original Assignee
Gree Electric Appliances Inc of Zhuhai
Zhuhai Lianyun Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Gree Electric Appliances Inc of Zhuhai, Zhuhai Lianyun Technology Co Ltd filed Critical Gree Electric Appliances Inc of Zhuhai
Priority to CN202210887031.8A priority Critical patent/CN115314553B/en
Publication of CN115314553A publication Critical patent/CN115314553A/en
Application granted granted Critical
Publication of CN115314553B publication Critical patent/CN115314553B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/08Protocols for interworking; Protocol conversion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22Parsing or analysis of headers
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)

Abstract

According to the data processing method, the data processing device, the data processing equipment, the data processing system and the readable storage medium, the electronic equipment determines a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule by acquiring the data to be uploaded of the target equipment and the data type of the data to be uploaded, preprocesses the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded, and sends the preprocessed data to be uploaded to the server. Therefore, on a channel for sending data to be uploaded to the server by the target device, the data to be uploaded is preprocessed by the electronic device, and is preprocessed by the upper computer, so that the data processing time of the server is shortened, the preprocessing pressure of the server is reduced, and the efficiency of the server is improved.

Description

Data processing method, device, equipment, system and readable storage medium
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a method, an apparatus, a device, a system, and a readable storage medium for data processing.
Background
In production environments such as factories, various devices are often used in a factory or on a production line, and data collected by the devices are uploaded to a cloud server for unified management and use.
After receiving the data, the cloud server generally cannot be used directly, and needs to process the data, for example, different parsing processes are performed on the original data sent by the device according to different protocols.
Because the data format that different equipment gathered is different, and data type is many, and the high in the clouds server needs carry out the preliminary treatment to different types of data, and high in the clouds server data processing's efficiency is not high.
Disclosure of Invention
In view of the above problems in the related art, the present application provides a method, an apparatus, a device, a system and a readable storage medium for data processing.
In a first aspect, an embodiment of the present application provides a data processing method, where the method includes:
acquiring data to be uploaded of target equipment and a data type of the data to be uploaded;
determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule;
preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded;
and sending the preprocessed data to be uploaded to a server.
In some embodiments, the determining the target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule includes:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode;
the preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded comprises the following steps:
and analyzing the data to be uploaded based on the target analysis mode to obtain preprocessed data to be uploaded.
In some embodiments, the determining, according to the correspondence between the data type and the pre-processing rule, a target pre-processing rule corresponding to the data to be uploaded includes:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode;
determining a target calculation rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the calculation rule;
the preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded comprises the following steps:
analyzing the data to be uploaded based on the target analysis mode to obtain analyzed data to be uploaded;
and processing the analyzed data to be uploaded based on the target calculation rule to obtain preprocessed data to be uploaded.
In some embodiments, before determining the target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule, the method further includes:
and receiving the data type and the preprocessing rule sent by the server.
In some embodiments, before determining the target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule, the method further includes:
receiving the corresponding relation between the data type and the preprocessing rule input by the user;
the method further comprises the following steps:
and sending the corresponding relation between the data type and the preprocessing rule to the server so that the server sends the corresponding relation between the data type and the preprocessing rule to other equipment except the target equipment.
In some embodiments, the obtaining of the data to be uploaded of the target device and the data type of the data to be uploaded includes:
receiving an upload message sent by the target device, where the upload message includes data to be uploaded and parameter information of the data to be uploaded, and the parameter information includes: identification information of the target device or type identification of the data to be uploaded;
and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
In some embodiments, the obtaining of the data to be uploaded of the target device and the data type of the data to be uploaded includes:
receiving data to be uploaded of a target device and parameter information of the data to be uploaded, wherein the data to be uploaded are input by a user, and the parameter information comprises: identification information of the target device or type identification of the data to be uploaded;
and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
In some embodiments, the preprocessing the data to be uploaded based on the target preprocessing rule, and before obtaining the preprocessed data to be uploaded, further includes:
carrying out format verification on the data to be uploaded of the target equipment input by the user to obtain a format verification result, wherein the format verification result comprises data accuracy and data errors;
the preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded comprises the following steps:
and if the result of the format verification is that the data is accurate, preprocessing the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded.
In a second aspect, an embodiment of the present application provides an apparatus for data processing, including:
the acquisition module is used for acquiring data to be uploaded of target equipment and the data type of the data to be uploaded;
the processing module is used for determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule; preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded;
and the sending module is used for sending the preprocessed data to be uploaded to a server.
In some embodiments, the preprocessing rule includes an analytic style, and the processing module is specifically configured to:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode; and analyzing the data to be uploaded based on the target analysis mode to obtain preprocessed data to be uploaded.
In some embodiments, the preprocessing rule includes an analysis method and a calculation rule, and the processing module is specifically configured to:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode; determining a target calculation rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the calculation rule; analyzing the data to be uploaded based on the target analysis mode to obtain analyzed data to be uploaded; and processing the analyzed data to be uploaded based on the target calculation rule to obtain preprocessed data to be uploaded.
In some embodiments, the apparatus further comprises:
and the receiving module is used for receiving the data type and the preprocessing rule sent by the server.
In some embodiments, the apparatus further comprises:
the receiving module is used for receiving the corresponding relation between the data type and the preprocessing rule input by the user;
the sending module is further configured to:
and sending the corresponding relation between the data type and the preprocessing rule to the server so that the server sends the corresponding relation between the data type and the preprocessing rule to other equipment except the target equipment.
In some embodiments, the obtaining module is specifically configured to:
receiving an upload message sent by the target device, where the upload message includes data to be uploaded and parameter information of the data to be uploaded, and the parameter information includes: the identification information of the target equipment or the type identification of the data to be uploaded; and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
In some embodiments, the obtaining module is specifically configured to:
receiving data to be uploaded of a target device and parameter information of the data to be uploaded, wherein the data to be uploaded are input by a user, and the parameter information comprises: identification information of the target device or type identification of the data to be uploaded; and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
In some embodiments, the apparatus further comprises:
the verification module is used for carrying out format verification on the data to be uploaded of the target equipment input by the user to obtain a format verification result, and the format verification result comprises data accuracy and data errors;
the processing module is specifically configured to:
and if the result of the format verification is that the data is accurate, preprocessing the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded.
In a third aspect, an embodiment of the present application provides an electronic device, including:
a memory having stored thereon a computer program for performing the method of data processing as described in the above first aspect when executed by the processor, and a processor.
In a fourth aspect, an embodiment of the present application provides a data processing system, including: target equipment, an upper computer and a server; the target equipment is in communication connection with an upper computer, and the upper computer is in communication connection with the server; the upper computer acquires data to be uploaded of the target equipment and the data type of the data to be uploaded; determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule; preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded; and sending the preprocessed data to be uploaded to the server.
In a fifth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program, which can be executed by one or more processors, and can be used to implement the data processing method according to the first aspect.
According to the data processing method, the data processing device, the data processing equipment, the data processing system and the readable storage medium, the electronic equipment determines a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule by acquiring the data to be uploaded of the target equipment and the data type of the data to be uploaded, preprocesses the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded, and sends the preprocessed data to be uploaded to the server. Therefore, on a channel for sending data to be uploaded to the server by the target device, the data to be uploaded is preprocessed by the electronic device, and is preprocessed by the upper computer, so that the data processing time of the server is shortened, the preprocessing pressure of the server is reduced, and the efficiency of the server is improved.
Drawings
The present application will be described in more detail hereinafter on the basis of embodiments and with reference to the accompanying drawings.
Fig. 1 is a schematic structural diagram of a data processing system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
The following description will be added if a similar description of "first \ second \ third" appears in the application file, and in the following description, the terms "first \ second \ third" merely distinguish similar objects and do not represent a specific ordering for the objects, and it should be understood that "first \ second \ third" may be interchanged under certain circumstances in a specific order or sequence, so that the embodiments of the application described herein can be implemented in an order other than that shown or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.
In production environments such as factories, various devices are often used in a factory or on a production line, and data collected by the devices are uploaded to a cloud server for unified management and use. There are many ways in which data may be collected by a device, including, for example, collection via manual recording, uploading by a gateway, etc. And the data formats collected by different devices may be different, so that the data received by the cloud server are various.
After receiving the data, the cloud server generally cannot be used directly, and needs to process the data, for example, different parsing processes are performed on the original data sent by the device according to different protocols. Therefore, the efficiency of data processing of the cloud server is not high, and the cloud data processing pressure is high.
Based on the problems in the related art, the embodiment of the application provides a data processing system, which comprises a target device, an electronic device and a server; the target device is in communication connection with the electronic device, and the electronic device is in communication connection with the server. The number of the target devices can be one or more, the number of the electronic devices can also be one or more, one or more target devices are respectively connected with one upper computer, and all the upper computers are respectively connected with the server.
The target device may be a device in a production environment such as a factory, for example, a cardboard bursting instrument, a thermometer, a multimeter, and the like.
The electronic device can also be an upper computer, a mobile terminal, a computer, a server, a 3D screen device and the like. The mobile terminal can be a smart phone, a tablet computer, a notebook computer, a wearable device, a smart mobile device and the like, and the smart mobile device can include a smart phone, a personal digital assistant, a game device, a navigation device, a handheld terminal and the like.
Optionally, many existing production devices need to be connected to an upper computer, so that the existing upper computer can be used as electronic equipment to realize data preprocessing.
The server may be one server or a server cluster composed of a plurality of servers.
Optionally, the server may be disposed in a cloud, and the server may be referred to as a cloud server.
According to the system provided by the embodiment of the application, the electronic equipment is additionally arranged on the channel for transmitting data between the server and the equipment of the original system, so that the data to be uploaded of the equipment are preprocessed through the electronic equipment in the data transmission and collection process of the equipment, the preprocessed data to be uploaded are obtained, the preprocessed data to be uploaded are sent to the server, the preprocessing pressure of the server is reduced through the electronic equipment, and the efficiency of the server is improved. The upper computer preprocesses the device data and uploads the device data to the cloud server, so that the types of data uploaded to the cloud server can be reduced, and the processing pressure of the cloud data is reduced.
In some scenes, a plurality of target devices can be connected, one part of the target devices can be connected with one electronic device, the other part of the target devices can be connected with the other electronic device, and the like, so that the processing pressure of the server on data to be uploaded is distributed to the electronic devices, the preprocessing efficiency is improved while the preprocessing pressure of the server is reduced, the efficiency of the server is improved, and the data processing efficiency of the cloud server is improved.
Taking the system shown in fig. 1 as an example, before the device data is uploaded to the cloud, the device data is preprocessed by the upper computers, and the multiple upper computers are equivalent to distributed data processing, so that the data processing time is reduced, the cloud data processing pressure is reduced, and the data processing speed is increased.
In the following, a target device is taken as a device, an electronic device is taken as an upper computer, and a server is taken as a cloud server.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a data processing system according to an embodiment of the present disclosure. The upper computer and the device can be connected through a serial port or a network port, the upper computer can be connected with the cloud server in a wired or wireless mode, for example, the upper computer can be connected with the cloud server through a hypertext Transfer Protocol (HTTP) interface, and the data transmission mode of the upper computer and the cloud server can be transmitted in a message subscribing and publishing mode through a message queue. The equipment can be grouped according to the placed area or the affiliated production line, each group of equipment is respectively connected with the same upper computer, for example, one vehicle or a plurality of equipment of one production line are connected with one upper computer according to the actual condition of a production field, and the equipment uploads data to the upper computer in a network port or serial port mode. In this embodiment, the data may be referred to as device raw data or raw data. The upper computer comprises input and output equipment, and also allows manual acquisition of equipment data, and manual entry into the upper computer. Data processing programs can be deployed in the upper computer, staff can define analysis protocols in related upper computers according to the actual equipment condition of a production line in the upper computer according to the condition of connected equipment, the cloud server issues related latest analysis protocols, and analysis rules of related equipment data can also be configured in a user-defined mode. Furthermore, independent calculation rules can be formulated for each analysis protocol or customized analysis rules, or a general calculation rule of related data is configured in a customized manner, or customized analysis rules and calculation rules in other upper computers issued by the cloud are used, so that the upper computers preprocess the data of the equipment, and then send the preprocessed data to the cloud server.
Therefore, the processing pressure of the cloud server on the original data of the equipment is shared by the plurality of upper computers, so that the preprocessing efficiency is improved while the preprocessing pressure of the cloud server is reduced, and the efficiency of the server is improved.
The data processing method provided by the embodiment of the application can be applied to the data processing system. The following describes a method for processing data provided by the present application with specific examples.
Referring to fig. 2, fig. 2 is a schematic flow chart of a data processing method according to an embodiment of the present disclosure, and as shown in fig. 2, the present embodiment is executed by an electronic device, which may be the electronic device in the foregoing embodiment. The method of the embodiment comprises the following steps:
step S21: and acquiring the data to be uploaded of the target equipment and the data type of the data to be uploaded.
The data to be uploaded of the target device refers to data which is acquired or generated by the target device and needs to be uploaded to the server through the electronic device.
The data type of the data to be uploaded is a variety of attributes of the data to be uploaded.
Optionally, the data type may be marked by a data type identifier. For example, the data type identifier may be a model identifier or a data type number of the target device.
Step S22: and determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule.
The corresponding relation between the data types and the preprocessing rules is pre-stored, and the preprocessing rules corresponding to different data types are stored.
The preprocessing rule refers to a method of preprocessing data, and may be, for example, an analysis protocol or an analysis rule for analyzing data, or a rule for counting or calculating data, and the like, which is not limited in the present application.
The electronic equipment acquires a corresponding preprocessing rule in the corresponding relation between the data type and the preprocessing rule according to the data type of the data to be uploaded, namely the target preprocessing rule corresponding to the data to be uploaded.
Step S23: and preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded.
Step S24: and sending the preprocessed data to be uploaded to a server.
In this embodiment, the electronic device determines a target preprocessing rule corresponding to the data to be uploaded according to a correspondence between a data type and a preprocessing rule by acquiring the data to be uploaded of the target device and the data type of the data to be uploaded, preprocesses the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded, and sends the preprocessed data to be uploaded to the server. Therefore, on a channel for sending data to be uploaded to the server by the target device, the data to be uploaded is preprocessed by the electronic device, and is preprocessed by the upper computer, so that the data processing time of the server is shortened, the preprocessing pressure of the server is reduced, and the efficiency of the server is improved.
In some scenarios, there are various ways of uploading the data of the target device acquired in step 21, and two ways are exemplarily described below in specific embodiments.
In a possible implementation manner, the acquired data to be uploaded of the target device may be original data sent by the target device to the electronic device.
On the basis of any of the above embodiments, further, step 21 may be implemented by the following steps 211 and 212.
Step 211: and receiving an upload message sent by the target equipment.
The uploading message comprises data to be uploaded and parameter information of the data to be uploaded, and the uploading message is used for indicating that the data to be uploaded is sent to the server after being preprocessed.
Wherein the parameter information includes: identification information of the target device or type identification of data to be uploaded. The identification information of the target device is used to indicate a device type of the target device, for example, identification number (ID) information of the target device or model identification information of the target device. The type identifier of the data to be uploaded is used for indicating the type of the data to be uploaded, for example, the type identifier of the data to be uploaded may be a type ID of the data to be uploaded.
Step 212: and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
Determining the data type of the data to be uploaded of the corresponding target equipment according to the identification information of the target equipment; or, according to the type identification of the data to be uploaded, determining the data type of the corresponding data to be uploaded.
In another possible implementation manner, the acquired data to be uploaded of the target device may also be data of the target device acquired manually and recorded into the electronic device.
On the basis of any of the above embodiments, further, step 21 may be implemented by the following steps 21a and 21 b.
Step 21a: and receiving data to be uploaded of the target equipment and parameter information of the data to be uploaded, which are input by a user.
Wherein the parameter information includes: identification information of the target device or type identification of data to be uploaded. The parameter information in this embodiment is similar to the parameter information mentioned in step 211, and is not described herein again.
Step 21b: and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
Step 21b is similar to step 212 described above and will not be described herein.
In this embodiment, under the condition that the target device cannot directly send the data to be uploaded, the data to be uploaded can be manually input into the electronic device, so that the type of the data to be uploaded processed by the electronic device is widened.
In some embodiments, there may be some data errors in the data to be uploaded input by the user in step 21 a. Therefore, when the data is analyzed, in order to prevent errors of the data analyzed by the custom analysis rule, the data to be uploaded can be verified before the data to be uploaded is preprocessed. The following will explain details of the present invention by specific examples.
On the basis of the above embodiment, further, the following steps can be included before step 23
Step 25: and carrying out format verification on the data to be uploaded of the target equipment input by the user to obtain a format verification result.
The format verification result comprises data accuracy and data errors.
Step 26: and judging whether the format verification result is accurate data.
If yes, go on to step 23; if not, continue to step 27.
The data acquired and entered manually needs advanced format verification, data errors are prevented, data accuracy is guaranteed, the verified data can be analyzed and processed, simple calculation is carried out, and special marking is carried out on the data with problems in format verification.
Step 27: and marking the data to be uploaded.
And labeling can be performed under the condition that the format verification result is data error, so that special labeling can be performed on abnormal data. And the label is used for indicating the format error of the data to be uploaded.
In this embodiment, the electronic device performs format verification on the manually acquired data to be uploaded, and can perform a preprocessing process when the verification is passed. Therefore, the readability and the accuracy of the uploaded data to the cloud end are improved.
In some embodiments, the general pre-treatment may be in various ways, and the specific way of pre-treatment is exemplified in the specific examples below.
In one possible implementation, the preprocessing may be parsing the data.
On the basis of any of the above embodiments, the preprocessing rule in step 22 includes a parsing manner, and step 22 can be implemented by the following step 22 a.
Step 22a: and determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode.
Optionally, the parsing protocol may be at least one of a decoding manner, a data dictionary, and a calculation formula.
Accordingly, step 23 can be implemented by step 23a as follows.
Step 23a: and analyzing the data to be uploaded based on the target analysis mode to obtain the preprocessed data to be uploaded.
Exemplary, hexadecimal format numbers for cardboard bursting resistance instruments (e.g., 4E5043533A for burst resistance. 2C6C62662F696E322C for lbf/in2.2C6B67662F636D322C for kgf/cm 2). The thermometer and the multimeter can respectively calculate the temperature value or the voltage value by decimal format numbers (16010000026000001961) according to the calculation rule of the analytic protocol.
In another possible implementation manner, the data volume of the data to be uploaded of some target devices is large, and the analyzed data needs to be counted; the data to be uploaded of some target devices is not the final data required by the service, and needs to be processed by some calculation rules, so the preprocessing may be calculation processing such as statistics on the data. On the basis of any of the above embodiments, the preprocessing rule in step 22 includes a calculation rule, and step 22 can be implemented by:
and determining a target calculation rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the calculation rule.
The calculation rule may be a statistical rule, a unit conversion rule, or the like for data.
Accordingly, step 23 may be implemented by:
and processing the data to be uploaded based on the target calculation rule to obtain the preprocessed data to be uploaded.
In another possible implementation manner, after being analyzed, data to be uploaded of some target devices is not final data required by the service, and needs to be processed by some calculation rules, so that preprocessing may be performed after analyzing the data and performing calculation processing such as statistics.
On the basis of any of the above embodiments, the preprocessing rule in step 22 includes a parsing method and a calculation rule, and step 22 can be implemented by steps 221 and 222 as follows.
Step 221: and determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode.
Step 222: and determining a target calculation rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the calculation rule.
It should be noted that, the execution of step 221 and step 222 is not in a sequential order. Step 221 may be executed first, and then step 222 may be executed; step 222 may be executed first, and then step 221 may be executed; step 221 and step 222 may also be performed simultaneously.
Accordingly, step 23 can be implemented by steps 231 and 232 as follows.
Step 231: and analyzing the data to be uploaded based on the target analysis mode to obtain the analyzed data to be uploaded.
Step 232: and processing the analyzed data to be uploaded based on the target calculation rule to obtain preprocessed data to be uploaded.
In some embodiments, the corresponding relationship between the data type and the preprocessing rule may be obtained in a variety of different manners, and the following describes an exemplary manner in which the corresponding relationship between the data type and the preprocessing rule is obtained in a specific embodiment.
In one possible implementation, step 22 may be sent by the server. For example, when the server has a corresponding relationship between the latest data type and the preprocessing rule, the latest data type and the preprocessing rule are uniformly issued to all the electronic devices.
On the basis of any of the above embodiments, further, the following step 220 may be further included before the step 22.
Step 220: and receiving the data type and the preprocessing rule sent by the server.
The system shown in fig. 1 is exemplified below with the preprocessing rule as the parsing protocol. The method comprises the steps of adding an upper computer between equipment and a cloud server, preprocessing equipment data in the upper computer, uniformly managing an equipment protocol through the cloud server, uniformly issuing all the upper computers after adding and changing the protocol, and preprocessing and analyzing the data according to an equipment data analysis protocol. When the newly added preprocessing rules exist on the upper computer, the data can be uploaded to the cloud server, and the cloud server shares the corresponding relation between the data type and the preprocessing rules to all the upper computers, so that the sharing and the unification of the preprocessing rule data are realized.
In this embodiment, the server uniformly manages the analysis protocol, and uniformly issues new and changed preprocessing rules to all the electronic devices, so that the electronic devices perform preprocessing analysis on data according to the device data analysis protocol. The server uniformly manages the corresponding relation between the data type and the preprocessing rule, so that the workload of the electronic equipment for managing the corresponding relation between the data type and the preprocessing rule can be reduced, and the sharing and the unification of the rule data are realized. Thereby improving the readability and accuracy of the equipment data.
In another possible implementation manner, the correspondence between the data type in step 22 and the preprocessing rule may also be customized. For example, if there is no pre-processing rule corresponding to a known data type in the target device connected to the electronic device, the user may define the pre-processing rule corresponding to the data type and store the pre-processing rule in the current electronic device.
On the basis of any of the above embodiments, further, the following step 220' may be included before the step 22.
Step 220': and receiving the corresponding relation between the data type input by the user and the preprocessing rule.
In this embodiment, the corresponding relationship between the data type and the preprocessing rule can also be user-defined, and the user-defined preprocessing rule added in the upper computer is shared by the server to other electronic devices, so that the preprocessing rule is ensured to be updated in time, the data type of the data to be uploaded for preprocessing is widened, and the preprocessing application range of the electronic devices is wider.
In some scenarios, in step 220', the user defines the data type and the pre-processing rule, and in order to avoid performing the same definition on other data of the same data type, the user-defined data type and the pre-processing rule may be issued to other electronic devices through the server, which is described in detail below with specific embodiments.
Further, step 220' may be followed by step 28 as follows.
Step 28: and sending the corresponding relation between the data type and the preprocessing rule to the server so that the server sends the corresponding relation between the data type and the preprocessing rule to other equipment except the target equipment.
The following description will be given with specific examples. The upper computer can be connected with the cloud server through a network, data analyzed and calculated by the upper computer are uploaded to the cloud server through an open http interface mode of the cloud server, the upper computer can also upload a custom rule to the cloud server through the http interface mode, the custom rule can be shared by other upper computers through the cloud server, the upper computer can actively acquire an analysis protocol and a custom rule in the cloud server through an open http interface mode of the cloud server through connected equipment, and analysis data are provided for other platforms or personnel to use and analyze.
In this embodiment, after the customized preprocessing rule is added to the electronic device, the corresponding relationship between the data type and the preprocessing rule may also be uploaded to the server, so that the server is shared with all other electronic devices, and the sharing and unification of the rule data are realized. Therefore, the readability and the accuracy of the data to be uploaded are improved.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure, and as shown in fig. 3, the apparatus according to the embodiment includes:
the acquisition module is used for acquiring data to be uploaded of the target equipment and the data type of the data to be uploaded;
the processing module is used for determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule; preprocessing data to be uploaded based on a target preprocessing rule to obtain preprocessed data to be uploaded;
and the sending module is used for sending the preprocessed data to be uploaded to the server.
In some embodiments, the pre-processing rule includes an analytic style, and the processing module is specifically configured to:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode; and analyzing the data to be uploaded based on a target analysis mode to obtain preprocessed data to be uploaded.
In some embodiments, the preprocessing rule includes an analysis method and a calculation rule, and the processing module is specifically configured to:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode; determining a target calculation rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the calculation rule; analyzing the data to be uploaded based on a target analysis mode to obtain analyzed data to be uploaded; and processing the analyzed data to be uploaded based on the target calculation rule to obtain the preprocessed data to be uploaded.
In some embodiments, the apparatus further comprises:
and the receiving module is used for receiving the data type and the preprocessing rule sent by the server.
In some embodiments, the apparatus further comprises:
the receiving module is used for receiving the corresponding relation between the data type input by the user and the preprocessing rule;
the sending module is further configured to:
and sending the corresponding relation between the data type and the preprocessing rule to the server so that the server sends the corresponding relation between the data type and the preprocessing rule to other equipment except the target equipment.
In some embodiments, the obtaining module is specifically configured to:
receiving an upload message sent by target equipment, wherein the upload message comprises data to be uploaded and parameter information of the data to be uploaded, and the parameter information comprises: identification information of the target equipment or type identification of data to be uploaded; and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
In some embodiments, the obtaining module is specifically configured to:
receiving data to be uploaded of a target device and parameter information of the data to be uploaded, which are input by a user, wherein the parameter information comprises: identification information of the target equipment or type identification of data to be uploaded; and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
In some embodiments, the apparatus further comprises:
the verification module is used for carrying out format verification on the data to be uploaded of the target equipment input by the user to obtain a format verification result, and the format verification result comprises data accuracy and data errors;
the processing module is specifically configured to:
and if the result of the format verification is that the data is accurate, preprocessing the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded.
The implementation principle and effect of the apparatus provided in this embodiment are similar to those of the method embodiments described above, and are not described herein again.
An embodiment of the present application provides an electronic device, including:
a memory having stored thereon a computer program which, when executed by the processor, performs the method of data processing as provided in any of the method embodiments described above.
In a fourth aspect, an embodiment of the present application provides a data processing system, including: target equipment, an upper computer and a server; the target equipment is in communication connection with the upper computer, and the upper computer is in communication connection with the server; the method comprises the steps that an upper computer obtains data to be uploaded of target equipment and the data type of the data to be uploaded; determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule; preprocessing data to be uploaded based on a target preprocessing rule to obtain preprocessed data to be uploaded; and sending the preprocessed data to be uploaded to a server.
It should be noted that, in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software functional module and is sold or used as a standalone product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof that contribute to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
Accordingly, embodiments of the present application provide a computer storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the method of data processing provided in the above embodiments.
The embodiment of the application provides an electronic device; fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 4, the electronic device 800 includes: a processor 801, at least one communication bus 802, a user interface 803, at least one external communication interface 804, memory 805. Wherein the communication bus 802 is configured to enable connective communication between these components. The user interface 803 may include a display screen, and the external communication interface 804 may include a standard wired interface and a wireless interface, among others. The processor 801 is configured to execute a program of a method of data processing stored in the memory to implement the steps in the method of data processing provided with the above-described embodiments.
It is to be noted here that: the above description of the storage medium and the electronic device embodiments is similar to the description of the method embodiments described above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of another identical element in a process, method, object, or apparatus that comprises the element.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.
Alternatively, the integrated unit described above may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially implemented in the form of a software product, which is stored in a storage medium and includes several instructions for causing a controller to execute all or part of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
The above embodiments are only examples of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. A method of data processing, comprising:
acquiring data to be uploaded of target equipment and a data type of the data to be uploaded;
determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule;
preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded;
and sending the preprocessed data to be uploaded to a server.
2. The method according to claim 1, wherein the preprocessing rule includes an analytic style, and the determining a target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule includes:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode;
the preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded comprises the following steps:
and analyzing the data to be uploaded based on the target analysis mode to obtain preprocessed data to be uploaded.
3. The method according to claim 1, wherein the preprocessing rule includes an analysis method and a calculation rule, and the determining a target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule includes:
determining a target analysis mode corresponding to the data to be uploaded according to the corresponding relation between the data type and the analysis mode;
determining a target calculation rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the calculation rule;
the preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded comprises the following steps:
analyzing the data to be uploaded based on the target analysis mode to obtain analyzed data to be uploaded;
and processing the analyzed data to be uploaded based on the target calculation rule to obtain preprocessed data to be uploaded.
4. The method according to claim 1, wherein before determining the target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule, the method further comprises:
and receiving the data type and the preprocessing rule sent by the server.
5. The method according to claim 1, wherein before determining the target preprocessing rule corresponding to the data to be uploaded according to the correspondence between the data type and the preprocessing rule, the method further comprises:
receiving the corresponding relation between the data type and the preprocessing rule input by the user;
the method further comprises the following steps:
and sending the corresponding relation between the data type and the preprocessing rule to the server so that the server sends the corresponding relation between the data type and the preprocessing rule to other equipment except the target equipment.
6. The method according to claim 1, wherein the acquiring data to be uploaded of a target device and a data type of the data to be uploaded comprises:
receiving an upload message sent by the target device, where the upload message includes data to be uploaded and parameter information of the data to be uploaded, and the parameter information includes: identification information of the target device or type identification of the data to be uploaded;
and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
7. The method according to claim 1, wherein the acquiring data to be uploaded of a target device and a data type of the data to be uploaded comprises:
receiving data to be uploaded of target equipment input by a user and parameter information of the data to be uploaded, wherein the parameter information comprises: identification information of the target device or type identification of the data to be uploaded;
and determining the data type of the data to be uploaded according to the parameter information of the data to be uploaded.
8. The method according to claim 7, wherein before preprocessing the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded, the method further comprises:
carrying out format verification on the data to be uploaded of the target equipment input by the user to obtain a format verification result, wherein the format verification result comprises data accuracy and data errors;
the preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded comprises the following steps:
and if the result of the format verification is that the data is accurate, preprocessing the data to be uploaded based on the target preprocessing rule to obtain the preprocessed data to be uploaded.
9. An apparatus for data processing, comprising:
the acquisition module is used for acquiring data to be uploaded of target equipment and the data type of the data to be uploaded;
the processing module is used for determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule; preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded;
and the sending module is used for sending the preprocessed data to be uploaded to a server.
10. An electronic device, comprising:
a memory and a processor, the memory having stored thereon a computer program which, when executed by the processor, performs a method of data processing according to any of claims 1 to 8.
11. A data processing system, comprising: target equipment, an upper computer and a server; the target equipment is in communication connection with an upper computer, and the upper computer is in communication connection with the server;
the upper computer acquires data to be uploaded of the target equipment and the data type of the data to be uploaded; determining a target preprocessing rule corresponding to the data to be uploaded according to the corresponding relation between the data type and the preprocessing rule; preprocessing the data to be uploaded based on the target preprocessing rule to obtain preprocessed data to be uploaded; and sending the preprocessed data to be uploaded to the server.
12. A computer-readable storage medium storing a computer program executable by one or more processors and operable to perform a method of data processing as claimed in any one of claims 1 to 8.
CN202210887031.8A 2022-07-26 2022-07-26 Method, device, equipment, system and readable storage medium for data processing Active CN115314553B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210887031.8A CN115314553B (en) 2022-07-26 2022-07-26 Method, device, equipment, system and readable storage medium for data processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210887031.8A CN115314553B (en) 2022-07-26 2022-07-26 Method, device, equipment, system and readable storage medium for data processing

Publications (2)

Publication Number Publication Date
CN115314553A true CN115314553A (en) 2022-11-08
CN115314553B CN115314553B (en) 2024-05-03

Family

ID=83859610

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210887031.8A Active CN115314553B (en) 2022-07-26 2022-07-26 Method, device, equipment, system and readable storage medium for data processing

Country Status (1)

Country Link
CN (1) CN115314553B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112015747A (en) * 2020-09-14 2020-12-01 支付宝(杭州)信息技术有限公司 Data uploading method and device
CN113301111A (en) * 2021-04-09 2021-08-24 厦门攸信信息技术有限公司 Digital twinning method, edge computing device, mobile terminal and storage medium
WO2021217624A1 (en) * 2020-04-30 2021-11-04 上海华东汽车信息技术有限公司 Vehicle data processing method and system
CN113760565A (en) * 2020-11-04 2021-12-07 苏州工品汇信息科技有限公司 Data processing platform, data processing method, storage medium and electronic equipment
CN113971172A (en) * 2020-07-24 2022-01-25 京东方科技集团股份有限公司 Data processing method and data processing device
CN114036216A (en) * 2021-10-29 2022-02-11 中国联合网络通信集团有限公司 Data processing method, device and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021217624A1 (en) * 2020-04-30 2021-11-04 上海华东汽车信息技术有限公司 Vehicle data processing method and system
CN113971172A (en) * 2020-07-24 2022-01-25 京东方科技集团股份有限公司 Data processing method and data processing device
CN112015747A (en) * 2020-09-14 2020-12-01 支付宝(杭州)信息技术有限公司 Data uploading method and device
CN113760565A (en) * 2020-11-04 2021-12-07 苏州工品汇信息科技有限公司 Data processing platform, data processing method, storage medium and electronic equipment
CN113301111A (en) * 2021-04-09 2021-08-24 厦门攸信信息技术有限公司 Digital twinning method, edge computing device, mobile terminal and storage medium
CN114036216A (en) * 2021-10-29 2022-02-11 中国联合网络通信集团有限公司 Data processing method, device and storage medium

Also Published As

Publication number Publication date
CN115314553B (en) 2024-05-03

Similar Documents

Publication Publication Date Title
CN108427725B (en) Data processing method, device and system
CN110445860B (en) Message sending method, device, terminal equipment and storage medium
CN109614327B (en) Method and apparatus for outputting information
CN109309736A (en) The generation method and generating means of globally unique ID
CN111683066A (en) Heterogeneous system integration method and device, computer equipment and storage medium
CN112702228B (en) Service flow limit response method, device, electronic equipment and readable storage medium
CN111064716B (en) Message conversion method, device, storage medium and server
CN110930176B (en) Prediction method, prediction device, electronic equipment and storage medium
CN112349428A (en) Family health code query method and equipment
CN112579910A (en) Information processing method, information processing apparatus, storage medium, and electronic device
CN114928574A (en) Information sending method, information sending device, electronic equipment and computer readable medium
CN113722369A (en) Method, device, equipment and storage medium for predicting field monitoring data
CN115314553A (en) Data processing method, device, equipment, system and readable storage medium
CN110737691B (en) Method and apparatus for processing access behavior data
CN107025294B (en) Information acquisition method and device
CN111259213B (en) Data visualization processing method and device
KR20210000041A (en) Method and apparatus for analyzing log data in real time
CN109491986B (en) Fire-fighting parameter configuration method and device and terminal equipment
JP2021033688A (en) Date generation apparatus, control method, and program
CN113676840B (en) Data processing method, apparatus, electronic device, storage medium, and program product
CN113836151B (en) Data processing method, device, electronic equipment and computer readable medium
CN115988100B (en) Gateway management method for intelligent perception of Internet of things of equipment based on multi-protocol self-adaption
CN109787851A (en) A kind of traffic statistics acquisition methods and device
EP4075735A1 (en) Data filter, measurement instrument, method
CN111738536B (en) Device operation method, device, electronic device and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant