CN116401239A - Data extraction method and system of carbon emission data storage analysis platform - Google Patents
Data extraction method and system of carbon emission data storage analysis platform Download PDFInfo
- Publication number
- CN116401239A CN116401239A CN202310389827.5A CN202310389827A CN116401239A CN 116401239 A CN116401239 A CN 116401239A CN 202310389827 A CN202310389827 A CN 202310389827A CN 116401239 A CN116401239 A CN 116401239A
- Authority
- CN
- China
- Prior art keywords
- data
- carbon emission
- repeated
- information
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 title claims abstract description 226
- 229910052799 carbon Inorganic materials 0.000 title claims abstract description 226
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000013500 data storage Methods 0.000 title claims abstract description 38
- 238000013075 data extraction Methods 0.000 title claims abstract description 37
- 230000008676 import Effects 0.000 claims abstract description 20
- 238000013507 mapping Methods 0.000 claims abstract description 20
- 230000007246 mechanism Effects 0.000 claims abstract description 14
- 238000012795 verification Methods 0.000 claims description 59
- 238000004364 calculation method Methods 0.000 claims description 47
- 238000013524 data verification Methods 0.000 claims description 21
- 230000002159 abnormal effect Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012216 screening Methods 0.000 claims description 13
- 239000000284 extract Substances 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims description 9
- 239000013589 supplement Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 10
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010438 heat treatment Methods 0.000 description 3
- 230000003647 oxidation Effects 0.000 description 3
- 238000007254 oxidation reaction Methods 0.000 description 3
- 230000001502 supplementing effect Effects 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000002485 combustion reaction Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000000975 dye Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- TWNQGVIAIRXVLR-UHFFFAOYSA-N oxo(oxoalumanyloxy)alumane Chemical compound O=[Al]O[Al]=O TWNQGVIAIRXVLR-UHFFFAOYSA-N 0.000 description 1
- 230000001915 proofreading effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/80—Management or planning
- Y02P90/84—Greenhouse gas [GHG] management systems
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a data extraction method and a system of a carbon emission data storage analysis platform, wherein the method comprises the steps that when a data import request sent by a user terminal representing a checking mechanism is received, a carbon emission data table of an enterprise to be imported to a platform terminal is obtained; transmitting the carbon emission data table to a data marking model, and identifying and extracting repeated data information in the carbon emission data table when the data marking model receives the carbon emission data table, and marking the repeated data; replacing the marked repeated data with a preset mark, transmitting a carbon emission data table with the replaced preset mark and the repeated data to a platform terminal, and identifying the format of the carbon emission data table by the platform terminal to determine the mapping relation between data information and the storage position of the platform terminal; and replacing the repeated data to a preset mark at the corresponding storage position. The method and the device have the effects of reducing the flow consumed by the carbon emission data import platform and improving the efficiency of carbon emission data import.
Description
Technical Field
The application relates to the technical field of intelligent analysis, in particular to a data extraction method and system of a carbon emission data storage analysis platform.
Background
The carbon emission data storage analysis platform has the main functions of storing and comparing carbon emission data of enterprises in various industries checked by a checking mechanism to form some analysis and prediction of algorithms based on big data so as to guide the enterprises in various industries and authorities to make energy-saving carbon reduction related decisions.
At present, in the process of importing carbon emission data of enterprises by a checking mechanism, part of repeated data exists, the repeated importing of the data can cause the increase of flow consumption, and the efficiency of the data importing process is low, so improvement is needed.
Disclosure of Invention
In order to reduce the flow consumed by the carbon emission data import platform and improve the efficiency of carbon emission data import; the application provides a data extraction method and system of a carbon emission data storage analysis platform.
The first object of the present invention is achieved by the following technical solutions:
a data extraction method of a carbon emission data storage analysis platform comprises the following steps:
when a data import request sent by a user terminal representing a checking mechanism is received, acquiring a carbon emission data table of an enterprise to be imported to a platform terminal;
transmitting the carbon emission data table to a data marking model, and identifying and extracting repeated data information in the carbon emission data table when the data marking model receives the carbon emission data table, and marking the repeated data;
Replacing the marked repeated data with preset marks, wherein the preset marks replaced by different repeated data are inconsistent;
transmitting the carbon emission data table with the preset identification replaced and the repeated data to a platform terminal, and identifying the format of the carbon emission data table by the platform terminal to determine the mapping relation between the data information and the storage position of the platform terminal;
and when the mapping relation between the storage position of the platform terminal and the data information is determined, replacing the repeated data to a preset mark at the corresponding storage position.
By adopting the technical scheme, after the checking mechanism completes the carbon emission detection of the enterprise, the carbon emission data and the enterprise main body information of the enterprise are recorded into the carbon emission data table of the user side, until a data import request is sent out by the user side to import the carbon emission data into the platform, before the data is sent to the platform terminal, the repeated data information in the carbon emission data table is replaced by a preset mark through the data mark model, namely, the repeated data is replaced by a single simple mark and is sent to the platform terminal, and after the data information is filled into a preset storage position according to a mapping relation, the preset mark is replaced by the corresponding repeated data, so that multiple importation of the repeated data is avoided in the process of importing the data information into the platform terminal, the consumption of flow in importing is also saved by adopting the simple preset mark, meanwhile, the transmission efficiency of the data information is improved after the redundant repeated data is replaced, the purposes of reducing the flow consumed by the carbon emission data importing the platform and improving the efficiency of the carbon emission data are achieved, and the safety of the data is partially replaced by the mark in the process of data transmission, and the safety of the data information is improved to a certain extent.
In a preferred example, the present application: the data marking model further comprises a preset operation rule, the carbon emission data table is sent to the data marking model, when the data marking model receives the carbon emission data table, repeated data information in the carbon emission data table is identified and extracted, and after the step of marking the repeated data, the following steps are further executed:
when the numerical value marking sub-model receives the carbon emission data table with the repeated data marking, identifying data information which does not belong to the repeated data in the carbon emission data table;
judging whether any marked repeated data can be obtained after the data information which does not belong to the repeated data is calculated through an operation rule;
if yes, replacing the data information with the preset identification of the repeated data obtained by calculation and the operation parameters of the operation rule;
when the mapping relation between the storage position of the platform terminal and the data information is determined, the platform terminal identifies the preset identifier with the operation parameter, and the repeated data corresponding to the preset identifier is calculated based on the operation parameter to obtain the data information.
By adopting the technical scheme, in order to further save the flow consumption of data information import and shorten the data information conveying time, the data information which can obtain repeated data is also replaced by the preset mark after being calculated by an operation rule, and the data information is added after the replaced preset mark to calculate the operation parameter of the repeated data, for example, the repeated data is 0.8, the data mark sub-model recognizes the value 80 which is not the repeated data, the value 80 is calculated to obtain 0.8, and the value is divided by 100, so that the operation parameter is '100', and when the platform terminal recognizes the operation parameter with '100' after the preset mark, the original data information 80 is obtained by multiplying 0.8, so that the data of the unified carbon emission data table can be further obtained by a simple operation rule, such as the multiplication and division rule of forward and backward displacement of decimal points, the flow of the data import consumption is reduced, and the data is safer during import.
In a preferred example, the present application: the carbon emission data table is sent to a data marking model, when the data marking model receives the carbon emission data table, repeated data information in the carbon emission data table is identified and extracted, and before the step of marking the repeated data, the following steps are executed:
identifying a text box to be filled in a carbon emission data table;
acquiring a prestored calculation formula associated with the text box to be filled in and a parameter text box participating in the calculation formula;
extracting data information in the parameter text box to corresponding parameter positions in a calculation formula, and calculating and outputting data to be filled if all the parameter positions of the calculation formula extract the data information;
and inputting the data to be filled into the text box to be filled.
By adopting the technical scheme, in order to more intuitively and widely display the carbon emission information of enterprises, after the carbon emission data are investigated and recorded, the statistical values such as the carbon oxidation rate and the like are calculated secondarily by utilizing the data of the investigated and recorded data, in order to facilitate the calculation of the statistical values, the parameter text boxes used for calculation are automatically acquired by identifying whether the text boxes to be filled are associated with calculation formulas, if so, the data information in the parameter text boxes is extracted as parameters participating in the calculation, and filling data are automatically output and filled into the text boxes to be filled, so that the labor calculation cost is reduced, and the data importing efficiency of the carbon emission data table is improved.
In a preferred example, the present application: the step of extracting the data information in the parameter text box to the corresponding parameter position in the calculation formula, and if all the parameter positions of the calculation formula extract the data information, calculating and outputting the data to be filled in comprises the following steps:
if the parameter position of the data information which is not extracted exists in the calculation formula, the data type of the parameter text box corresponding to the parameter position of the unfilled data information is identified;
judging whether the data information in the parameter text box is a filling item or not based on the data type pre-associated with the parameter text box;
if the item is the necessary item, sending a data supplement message to the user side;
if the data is not necessary to be filled, the step of calculating the data to be filled is skipped, and the carbon emission data table is sent to the data marking model.
By adopting the technical scheme, because the carbon emission data table needs to be periodically imported, namely updated, when the carbon emission data table is imported each time, if the data information in the parameter text box cannot be acquired, judging whether the parameter text box is a filling-in-demand item currently, for example, the text box to be filled is a annual statistical value, but the current year is not yet ended, and the unfilled parameter text box exists, so that the parameter text box is identified as not being the filling-in-demand item, the step of calculating the data to be filled is skipped, and the current parameter text box is the data to be filled, and a data supplementing message is sent to a user side to remind the user of the supplementing data, so that the calculation of the data to be filled in the carbon data emission table in the process of importing the platform terminal is more intelligent.
In a preferred example, the present application: the carbon emission data table is sent to a data marking model, when the data marking model receives the carbon emission data table, repeated data information in the carbon emission data table is identified and extracted, and before the step of marking the repeated data, the following steps are executed:
acquiring target data of accuracy to be checked in a carbon emission data table;
based on a data type preset by the target data, sending the target data to a data verification model associated with the data type, and outputting a verification result;
if the verification result is that the target data is abnormal, generating an error report based on the abnormal target data and sending the error report to the user side;
and if the verification results of the data verification model are all that the target data are normal, transmitting the carbon emission data table to the data marking model.
By adopting the technical scheme, before the carbon emission data table is input into the data marking model, by checking the data information in the carbon emission data table, the obvious error data in the carbon emission data table can be identified and the user side is prompted to modify the error data in a conventional form, so that the accuracy of the data information of the carbon emission data table is improved, the importing efficiency of the data information is improved, and for different data types, such as characters, numerical values and the like, different data checking models are adopted for checking simultaneously, and the checking efficiency is improved, therefore, the obvious form error or error data which does not accord with the conventional form after the carbon emission data is imported into the platform terminal is reduced, and the quality and the accuracy of the carbon emission data stored by the platform terminal are improved.
In a preferred example, the present application: the step of sending the target data to a data verification model associated with the data type based on the data type preset by the target data and outputting a verification result comprises the steps of:
when the character verification model receives character data, screening preset keyword groups based on the data types, judging whether the character data contains necessary keywords in the keyword groups, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal;
when the numerical value verification model receives numerical value data, screening a preset numerical value interval based on the data type, judging whether the numerical value data are located in the numerical value interval, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal.
By adopting the technical scheme, when the text data is verified, the text data is verified through the keyword group, so that the basic semantics of the text data and the accuracy of a text frame can be ensured, for example, the main body information is verified, the keyword can be selected to be a 'limited company' four word, the enterprise name is verified, and the keywords such as 'province, city, district, county, town' and the like can be adopted to verify, so that whether the filling of the information accords with the specification can be identified, and obvious form errors are reduced.
And the numerical value data is checked in a numerical value interval mode, and because the numerical value data is from the same data type, such as heating value, carbon content of unit heat value, carbon oxidation rate and the like, different numerical value types are checked in different numerical value intervals, when the numerical value data exceeds the range of the numerical value interval, the numerical value data is judged not to be in accordance with the conventional data of the numerical value type, so that obvious errors are confirmed to be required to be modified, such as the aluminum oxide passing percentage value is confirmed to be present, but the numerical value exceeds the percentage value, and the obvious errors are still confirmed to be present to be required to be modified.
By adopting the two verification modes, obvious errors of the text data and the numerical data can be accurately judged.
The second object of the present invention is achieved by the following technical solutions:
a data extraction system for a carbon emissions data storage analysis platform, comprising:
the data table acquisition module is used for acquiring a carbon emission data table of an enterprise to be imported to the platform terminal when a data import request sent by a user terminal representing the checking mechanism is received;
the data marking module is used for sending the carbon emission data table to the data marking model, and when the data marking model receives the carbon emission data table, the data marking model identifies and extracts repeated data information in the carbon emission data table and marks the repeated data;
The replacing module is used for replacing the marked repeated data with preset marks, and the preset marks replaced by different repeated data are inconsistent;
the data transmission module is used for transmitting the carbon emission data table with the preset identification replaced and the repeated data to the platform terminal, and the platform terminal identifies the format of the carbon emission data table to determine the mapping relation between the data information and the storage position of the platform terminal;
and the platform replacement module is used for replacing the repeated data to a preset identifier at the corresponding storage position when the mapping relation between the storage position of the platform terminal and the data information is determined.
By adopting the technical scheme, after the checking mechanism completes the carbon emission detection of the enterprise, the carbon emission data and the enterprise main body information of the enterprise are recorded into the carbon emission data table of the user side, until a data import request is sent out by the user side to import the carbon emission data into the platform, before the data is sent to the platform terminal, the repeated data information in the carbon emission data table is replaced by a preset mark through the data mark model, namely, the repeated data is replaced by a single simple mark and is sent to the platform terminal, and after the data information is filled into a preset storage position according to a mapping relation, the preset mark is replaced by the corresponding repeated data, so that multiple importation of the repeated data is avoided in the process of importing the data information into the platform terminal, the consumption of flow in importing is also saved by adopting the simple preset mark, meanwhile, the transmission efficiency of the data information is improved after the redundant repeated data is replaced, the purposes of reducing the flow consumed by the carbon emission data importing the platform and improving the efficiency of the carbon emission data are achieved, and the safety of the data is partially replaced by the mark in the process of data transmission, and the safety of the data information is improved to a certain extent.
Optionally, the data marking module includes:
the identification sub-module is used for identifying the data information one by one according to the format of the carbon emission data table when the data marking model receives the carbon emission data table;
the association sub-module is used for associating the data information appearing for the first time with preset identifiers in the identification process, wherein the preset identifiers associated with all the data information appearing for the first time are different;
the association sub-module is used for associating the data information with a preset identifier associated with the last occurrence when the data information appears again;
and the marking sub-module is used for marking all the data information associated with the same preset identifier as a group of repeated data.
The third object of the present application is achieved by the following technical solutions:
a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of a data extraction method of a carbon emission data storage analysis platform as described above when the computer program is executed.
The fourth object of the present application is achieved by the following technical solutions:
a computer readable storage medium storing a computer program which when executed by a processor performs the steps of a data extraction method of a carbon emission data storage analysis platform as described above.
In summary, the present application includes at least one of the following beneficial technical effects:
1. in the process of importing data information into a platform terminal, repeated data are prevented from being imported for multiple times, simple preset marks are adopted to save the consumption of flow during importing, and meanwhile, the transmission efficiency of the data information is improved after lengthy repeated data are replaced, so that the purposes of reducing the flow consumed by a carbon emission data importing platform and improving the efficiency of importing the carbon emission data are achieved, and part of the data is replaced with the marks during data transmission, so that the safety of the data information is improved to a certain extent;
2. the identification of the information is carried out according to the format of the carbon emission data table, the target data is immediately associated with the preset identification in the identification process, and the data information with the consistent preset identification is classified into a group of repeated data after the identification is finished, so that different repeated data can be extracted more efficiently;
3. the data of the unified carbon emission data table can be further unified through simple operation rules, such as multiplication and division rules of forward and backward displacement of decimal points, so that the flow consumed by data importing is reduced, and the data is safer during importing;
4. by identifying the position information of the target data in the carbon emission data table, namely classifying the data information through the table head of the carbon emission data table, the data type of the target data can be determined, and aiming at the data of the character type and the numerical value type, whether obvious form errors or errors which do not accord with the normal are judged by adopting a targeted check rule, so that abnormal information data can be identified more efficiently and accurately.
Drawings
FIG. 1 is a flow chart of an embodiment of a data extraction method for a carbon emission data storage analysis platform of the present application;
FIG. 2 is a schematic diagram of an interface of an embodiment of a data extraction method of a carbon emission data storage analysis platform according to the present application;
FIG. 3 is another interface diagram of an embodiment of a data extraction method of a carbon emission data storage analysis platform according to the present application;
FIG. 4 is a flowchart showing an implementation of step S20 in an embodiment of a data extraction method of a carbon emission data storage analysis platform according to the present application;
FIG. 5 is a flowchart of an implementation of the data extraction method of the carbon emission data storage analysis platform according to an embodiment of the present invention after step S20;
FIG. 6 is a flowchart of an implementation of the data extraction method of the carbon emission data storage analysis platform according to an embodiment of the present application before step S20;
FIG. 7 is a flowchart of an implementation of the data extraction method of the carbon emission data storage analysis platform according to an embodiment of the present application before step S20;
fig. 8 is a schematic block diagram of a computer device of the present application.
Detailed Description
The present application is described in further detail below in conjunction with figures 1-8.
In an embodiment, as shown in fig. 1, the application discloses a data extraction method of a carbon emission data storage analysis platform, which specifically includes the following steps:
S10: when a data import request sent by a user terminal representing a checking mechanism is received, acquiring a carbon emission data table of an enterprise to be imported to a platform terminal;
in one embodiment, the user side of the checking mechanism is a PC side or a mobile terminal for use by a checker, and the user side is in communication connection with a platform terminal for storing the carbon emission data of the enterprise;
the carbon emission data of enterprises is monitored and recorded by the checking staff of the checking mechanism, and as shown in fig. 2 and 3, the carbon emission data table comprises enterprise main body information, checked carbon emission projects and carbon emission data information of various element materials in the checking projects, including but not limited to carbon consumption, low-grade heating value, combustion heat, carbon content of unit heating value, carbon oxidation rate, combustion emission of fossil dyes and the like.
The platform terminal is used for storing and analyzing the carbon emission data table of all enterprises imported by each checking organization.
The data import request refers to: and importing the data of the carbon emission data table into a request of a storage analysis platform for storing the carbon emission data of each enterprise by a request sent by a checker.
Specifically, when information of a data import request sent by a checking person of a checking mechanism through a user side is received, a carbon emission data table of an enterprise to be imported to a platform terminal, which is attached to the request information, is obtained.
S20: transmitting the carbon emission data table to a data marking model, and identifying and extracting repeated data information in the carbon emission data table when the data marking model receives the carbon emission data table, and marking the repeated data;
in one embodiment, the data marking model is a trained model for identifying duplicate fields according to the format of a carbon emission data table, for example, the data information in each text box is an identification field, and if the data information in two text boxes in the carbon emission data table is partially the same, the two data information are still considered to be non-duplicate. I.e. the data information refers to text, numerical content within a single text box in the carbon emission data sheet.
The repeated data refers to a text or numerical value that is repeated twice or more in the carbon emission data table, for example, 5 times in the carbon emission data table by the xxxxxx company in guangzhou, and the repeated data extracted and marked for the text or numerical value is "xxxxxx company in guangzhou".
Specifically, the carbon emission data is sent to a data marking model, when the data marking model receives the carbon emission data table, the data marking model identifies data information in the text box according to the text box used for filling text or numerical values in the carbon emission data table as a unit, and marks repeated data information in the carbon emission data table as repeated data.
S30: replacing the marked repeated data with preset marks, wherein the preset marks replaced by different repeated data are inconsistent;
in an embodiment, the preset mark includes, but is not limited to, an arabic number, an english letter, or a combination thereof, and the preset mark preferably adopts a character to save the traffic of the data information leading into the platform terminal.
The repeated data marked in the carbon emission data table comprises a single or a plurality of repeated data, and when two or more repeated data exist in one carbon emission data table, the preset identification for replacing different repeated data is different.
Specifically, the marked repeated data is replaced by a preset identifier, and the preset identifiers replaced by a plurality of repeated data in a single carbon emission data table are different.
S40: transmitting the carbon emission data table with the preset identification replaced and the repeated data to a platform terminal, and identifying the format of the carbon emission data table by the platform terminal to determine the mapping relation between the data information and the storage position of the platform terminal;
in an embodiment, since different enterprises have different emission types, there are differences in detected projects, and thus, there are partial differences in formats of the carbon emission data table, so that the platform terminal can simultaneously identify a plurality of preset formats of different carbon emission data tables and establish a mapping relationship of storage locations.
Specifically, the carbon emission data table with the oak mark replaced and all repeated data appearing in the carbon emission data table are sent to the platform terminal, the platform terminal recognizes the format of the carbon emission data table, and the text or the numerical value of each text box of the carbon emission data table is mapped to the corresponding storage position of the platform terminal according to the format of the table, so that the data in the carbon emission data table presents original arrangement on the platform terminal.
S50: and when the mapping relation between the storage position of the platform terminal and the data information is determined, replacing the repeated data to a preset mark at the corresponding storage position.
Specifically, after all data information in the carbon emission data table with the preset identifier is mapped to a storage position corresponding to the platform terminal, the preset identifier of the corresponding storage position in the platform terminal is replaced back to original repeated data.
In one embodiment, referring to fig. 4, step S20 includes the steps of:
s201: when the data marking model receives the carbon emission data table, the data information is identified one by one according to the format of the carbon emission data table;
s202: in the identification process, associating the data information appearing for the first time with preset identifiers, wherein the preset identifiers associated with all the data information appearing for the first time are different;
S203: when the data information appears again, associating the data information with the preset identifier associated with the last occurrence;
s204: and marking all data information associated with the same preset identifier as a group of repeated data.
In one embodiment, the data tagging model is capable of identifying the formats of different preset types of carbon emission data sheets simultaneously and identifying the extracted data information.
And after the marking of all the data information in the carbon emission data table is completed, releasing the preset marks associated with all the data information which only appears once.
Two or more sets of duplicate data may be present in a single carbon emission data table.
Specifically, when the data marking model receives the carbon emission data table, the format of the carbon emission data table is identified, and the data information is identified one by one according to the format of the carbon emission data table, in the identification process, the data information which appears for the first time in the carbon emission data table is associated with preset marks, the preset marks which are associated with all the data information which appears for the first time are different, when the data information appears again, the preset marks which are associated with each other when the data information appears again are traced back and associated, all the data information which is associated with the same preset mark are marked as a group of repeated data, and when all the data information is identified, the data information which appears only once is released from the preset marks.
In an embodiment, the data marking model further includes a preset operation rule, referring to fig. 5, after step S20, the following steps are performed:
s21: when the numerical value marking sub-model receives the carbon emission data table with the repeated data marking, identifying data information which does not belong to the repeated data in the carbon emission data table;
s22: judging whether any marked repeated data can be obtained after the data information which does not belong to the repeated data is calculated through an operation rule;
s23: if yes, replacing the data information with the preset identification of the repeated data obtained by calculation and the operation parameters of the operation rule;
s24: when the mapping relation between the storage position of the platform terminal and the data information is determined, the platform terminal identifies the preset identifier with the operation parameter, and the repeated data corresponding to the preset identifier is calculated based on the operation parameter to obtain the data information.
In an embodiment, the numerical labeling submodel is trained data information for identifying a conversion relation with any set of repeated data, for example, the repeated data is 0.81, the numerical labeling submodel can identify a numerical value obtained by shifting a decimal point by a numerical value of 0.81, such as 8.1, 81, 810, 0.081, 0.0081, etc., that is, the operation rule refers to the data information with the same numerical value obtained by shifting the decimal point back and forth on the repeated data, the operation parameter refers to an operation formula and a manner required by the data information operation to obtain the repeated data, for example, the data information is 81, the repeated data is 0.81, and the corresponding operation parameter is "+.100". When the preset identifier with the operation parameters is mapped to the storage position of the platform terminal, the repeated data is converted back based on the preset identifier, and then the original data information is obtained based on the operation parameters.
Specifically, after the repeated data are replaced by the preset identifier by the data marking model, the numerical marking sub-model further identifies the data information which is not replaced in the carbon emission data table, judges that the data information passes through a preset operation rule, namely, the numerical value moves back and forth through decimal points to obtain the existing repeated data, if yes, the data information is replaced by the preset identifier and operation parameters of the repeated data obtained after calculation, namely, the platform terminal is shown as the preset identifier and the operation parameters, and further, after the data information is mapped to a storage position corresponding to the platform terminal, the platform terminal identifies the preset identifier with the operation parameters, replaces the preset identifier with the repeated data, and obtains the original data information through operation of the operation parameters.
In one embodiment, referring to fig. 6, prior to step S20, the following steps are performed,
S201A: identifying a text box to be filled in a carbon emission data table;
S202A: acquiring a prestored calculation formula associated with the text box to be filled in and a parameter text box participating in the calculation formula;
S203A: extracting data information in the parameter text box to corresponding parameter positions in a calculation formula, and calculating and outputting data to be filled if all the parameter positions of the calculation formula extract the data information;
S204A: and inputting the data to be filled into the text box to be filled.
In this embodiment, the text boxes to be filled are used for filling data information, and the calculation formulas corresponding to different text boxes to be filled are different, for example, different text boxes to be filled such as a ratio, an average value, a occupation ratio, and the like.
The parameter text box is also used for filling data information, and the data information in the parameter text box is a parameter of a calculation formula used for calculating the data information in the text box to be filled.
Specifically, before the carbon emission data table is input into the data marking model, identifying a text box to be filled in, which needs to be filled with a secondary statistical value, in the carbon emission data table, acquiring a prestored calculation formula associated with the text box to be filled in, acquiring a parameter text box of the calculation formula based on the calculation formula, further acquiring data information in the parameter text box to serve as a calculation parameter of the calculation formula, and extracting the data information in the parameter text box to a parameter position corresponding to the calculation formula.
Further, if all the parameter positions of the calculation formula extract the data information, calculating the data information in the text box to be filled, namely, the data to be filled.
In one embodiment, step S203A includes the steps of:
S203A1: if the parameter position of the data information which is not extracted exists in the calculation formula, the data type of the parameter text box corresponding to the parameter position of the unfilled data information is identified;
s203A2: judging whether the data information in the parameter text box is a filling item or not based on the data type pre-associated with the parameter text box;
s203A3: if the item is the necessary item, sending a data supplement message to the user side;
s203A4: if the data is not necessary to be filled, the step of calculating the data to be filled is skipped, and the carbon emission data table is sent to the data marking model.
The necessary filling item refers to data information which is required to be filled in currently when the user side fills in the carbon data emission table;
the unnecessary filling item refers to data information which cannot be filled in at present by the user side.
In this embodiment, the data types include types to which the data belongs, such as text and numerical values, and time information corresponding to the data, such as a certain numerical value of a certain month.
Specifically, if the parameter position of the data information which is not extracted exists in the calculation formula, that is, the data information is not filled in the parameter text box currently, whether the data information in the parameter text box is the data information which needs to be filled in currently is further judged, and if so, a data supplementing message user side is sent; if not, the step of calculating the data to be filled is skipped, and the carbon emission data is sent to the data marking model.
In one embodiment, referring to fig. 7, prior to step S20, the following steps are performed:
s11: acquiring target data of accuracy to be checked in a carbon emission data table;
s12: based on a data type preset by the target data, sending the target data to a data verification model associated with the data type, and outputting a verification result;
s13: if the verification result is that the target data is abnormal, generating an error report based on the abnormal target data and sending the error report to the user side;
s14: and if the verification results of the data verification model are all that the target data are normal, transmitting the carbon emission data table to the data marking model.
In one embodiment, the target data refers to the partial text content and the numerical content in the carbon emission data table that need to be checked.
The data types include the type of the data itself, such as a text type, a numerical type, a symbol type, and the classification of the data information in the carbon emission data table, such as the data information obtained by the same test item.
The data verification model comprises two types, one is used for verifying characters and the other is used for verifying numerical values.
The verification result comprises target data abnormality and target data normal, wherein the target data abnormality means that the text content has obvious form errors or the numerical content obviously does not accord with the theories.
The error prompt report is text information, and the error report comprises data information of errors and the position of the carbon emission data table.
Specifically, a text with obvious form errors or a numerical value which does not accord with normal conditions is obtained from a carbon emission data table, the text and the numerical value are sent to a corresponding data checking model, the data checking model checks target data and outputs a corresponding checking result, and if the checking result shows that the target data has obvious form errors or the data does not accord with normal conditions obviously, an error report is generated and sent to a user side to prompt the user side to carry out corresponding modification. If the checking result does not find that the data information has obvious form errors and is not normal, the numerical value information is further sent to the data marking model for marking.
In one embodiment, step S12 includes the steps of:
s121: determining the data type of the target data based on the position information of the target data in the carbon emission data table;
s122: identifying a data verification model with a data type pre-associated, and sending target data to the corresponding associated data verification model;
s123: the data verification model screens corresponding verification rules based on the data type of the target data to verify, and outputs a verification result.
In an embodiment, the target data belonging to the same test item can be determined by the location information.
The verification rule comprises two verification modes for characters and numerical values.
Specifically, based on the position of the target data in the carbon emission data table, determining data information belonging to the same carbon emission detection item and the same type, identifying a data verification model with the data type pre-associated, sending the target data to a corresponding data verification model, and after the corresponding data verification model receives the data information, screening a corresponding verification rule to verify based on the detection item and the text type to which the data information belongs.
In one embodiment, the data verification model includes a text verification model and a numerical verification model, the target data includes text data and numerical data, and the step S123 includes the steps of:
s1231: when the character verification model receives character data, screening preset keyword groups based on the data types, judging whether the character data contains necessary keywords in the keyword groups, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal;
s1233: when the numerical value verification model receives numerical value data, screening a preset numerical value interval based on the data type, judging whether the numerical value data are located in the numerical value interval, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal.
In one embodiment, the text data includes data information such as body information, detection items, detection results, etc. of the enterprise; the numerical data includes detection result data and data information calculated based on the detection result.
The keywords include checking the name of the enterprise by using the 'limited company' four words, and checking the address of the enterprise by using the 'province, city, district, county, town' keywords and the like.
The necessary keywords refer to a single or a plurality of keywords that the text data must have.
The numerical value interval is a normal numerical value interval of each detection numerical value obtained based on historical multiple experiments.
Specifically, when the text verification model receives text data, based on the information type of the text data, screening out corresponding keyword groups for verification, if the text data contains necessary keywords in the keyword groups, determining that the text data is normal, and if the necessary keywords are absent, determining that the text data is abnormal and needs to be sent back to a user side for modification.
When the numerical verification model receives numerical data, a corresponding numerical interval is screened out for verification based on the detection item type to which the numerical data belongs, when the numerical value is not in the numerical interval, the numerical data is still determined to be inconsistent with normal, and the user terminal is returned for modification.
And when the text data or the numerical data are normal, sending the text data or the numerical data to the data marking model to extract repeated data.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.
In an embodiment, a data extraction system of a carbon emission data storage analysis platform is provided, where the data extraction system of the carbon emission data storage analysis platform corresponds to the data extraction method of the carbon emission data storage analysis platform in the above embodiment. The data extraction system of the carbon emission data storage analysis platform comprises:
the data table acquisition module is used for acquiring a carbon emission data table of an enterprise to be imported to the platform terminal when a data import request sent by a user terminal representing the checking mechanism is received;
the data marking module is used for sending the carbon emission data table to the data marking model, and when the data marking model receives the carbon emission data table, the data marking model identifies and extracts repeated data information in the carbon emission data table and marks the repeated data;
The replacing module is used for replacing the marked repeated data with preset marks, and the preset marks replaced by different repeated data are inconsistent;
the data transmission module is used for transmitting the carbon emission data table with the preset identification replaced and the repeated data to the platform terminal, and the platform terminal identifies the format of the carbon emission data table to determine the mapping relation between the data information and the storage position of the platform terminal;
and the platform replacement module is used for replacing the repeated data to a preset identifier at the corresponding storage position when the mapping relation between the storage position of the platform terminal and the data information is determined.
Optionally, the data marking module includes:
the identification sub-module is used for identifying the data information one by one according to the format of the carbon emission data table when the data marking model receives the carbon emission data table;
the association sub-module is used for associating the data information appearing for the first time with preset identifiers in the identification process, wherein the preset identifiers associated with all the data information appearing for the first time are different;
the association sub-module is used for associating the data information with a preset identifier associated with the last occurrence when the data information appears again;
and the marking sub-module is used for marking all the data information associated with the same preset identifier as a group of repeated data.
Optionally, the data marking model further includes a preset operation rule, and further includes:
the numerical value marking module is used for identifying data information which does not belong to the repeated data in the carbon emission data table when the numerical value marking sub-model receives the carbon emission data table with the repeated data marking completed;
the operation judging module is used for judging whether any marked repeated data can be obtained after the data information which does not belong to the repeated data is calculated through an operation rule;
the first operation marking module is used for replacing the data information with the preset identification of the repeated data obtained by calculation and the operation parameters of the operation rule if the data information is available;
and the platform operation replacement module is used for identifying the preset identifier with the operation parameter when determining the mapping relation between the storage position of the platform terminal and the data information, and calculating repeated data corresponding to the preset identifier based on the operation parameter to obtain the data information.
Optionally, the method further comprises:
the identification module to be filled is used for identifying the text box to be filled in the carbon emission data table;
the formula identification module is used for acquiring a prestored calculation formula associated with the text box to be filled and a parameter text box participating in the calculation formula;
The data calculation module is used for extracting the data information in the parameter text box to the corresponding parameter position in the calculation formula, and calculating and outputting the data to be filled if all the parameter positions of the calculation formula extract the data information;
and the filling module is used for inputting the data to be filled into the text box to be filled.
Optionally, the method further comprises:
the unfilled identification module is used for identifying the data type of the parameter text box corresponding to the parameter position of the unfilled data information if the parameter position of the data information which is not extracted exists in the calculation formula;
the necessary filling item judging module is used for judging whether the data information in the parameter text box is necessary filling item or not based on the data type pre-associated with the parameter text box;
the supplementary reminding module is used for sending a data supplementary message to the user side if the supplementary reminding module is a necessary item;
and the calculation skip module is used for skipping the step of calculating the data to be filled if the data are not necessary to fill in, and sending the carbon emission data table to the data marking model.
Optionally, the method further comprises:
the target acquisition module is used for acquiring target data of accuracy to be checked in the carbon emission data table;
the verification module is used for sending the target data to a data verification model associated with the data type based on the data type preset by the target data and outputting a verification result;
The first verification judging module is used for generating an error report based on abnormal target data and sending the error report to the user side if the verification result is that the target data is abnormal;
and the second checking and judging module is used for sending the carbon emission data table to the data marking model if the checking results of the data checking model are all that the target data are normal.
Optionally, the verification module includes:
a position determination sub-module for determining a data type of the target data based on position information of the target data in the carbon emission data table;
the model identification sub-module is used for identifying a data verification model with a data type pre-associated and sending target data to the corresponding associated data verification model;
and the rule screening and identifying sub-module is used for screening corresponding verification rules based on the data type of the target data by the data verification model to verify and outputting a verification result.
Optionally, the rule screening is that the sub-module includes:
the character verification unit is used for screening preset keyword groups based on the data types when the character verification model receives character data, judging whether the character data contains necessary keywords in the keyword groups, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal;
The numerical value verification unit is used for screening a preset numerical value interval based on the data type when the numerical value verification model receives the numerical value data, judging whether the numerical value data are located in the numerical value interval, if yes, judging that the verification result is normal, and if not, judging that the verification result is abnormal.
For specific limitations on the data extraction system of a carbon emission data storage analysis platform, reference may be made to the above limitations on the data extraction method of a carbon emission data storage analysis platform, and the details are not repeated here. The above-mentioned various modules in the data extraction system of the carbon emission data storage analysis platform may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 8. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer equipment is used for storing a carbon emission data table, a data marking model and a data proofreading model. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program when executed by a processor implements a data extraction method for a carbon emission data storage analysis platform.
In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing a data extraction method of a carbon emission data storage analysis platform when executing the computer program;
in one embodiment, a computer readable storage medium having a computer program stored thereon, the computer program when executed by a processor implementing a data extraction method for a carbon emission data storage analysis platform is provided.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (10)
1. A data extraction method of a carbon emission data storage analysis platform is characterized by comprising the following steps of: the method comprises the following steps:
when a data import request sent by a user terminal representing a checking mechanism is received, acquiring a carbon emission data table of an enterprise to be imported to a platform terminal;
Transmitting the carbon emission data table to a data marking model, and identifying and extracting repeated data information in the carbon emission data table when the data marking model receives the carbon emission data table, and marking the repeated data;
replacing the marked repeated data with preset marks, wherein the preset marks replaced by different repeated data are inconsistent;
transmitting the carbon emission data table with the preset identification replaced and the repeated data to a platform terminal, and identifying the format of the carbon emission data table by the platform terminal to determine the mapping relation between the data information and the storage position of the platform terminal;
and when the mapping relation between the storage position of the platform terminal and the data information is determined, replacing the repeated data to a preset mark at the corresponding storage position.
2. The data extraction method of the carbon emission data storage analysis platform according to claim 1, wherein: the data marking model further comprises a preset operation rule, the carbon emission data table is sent to the data marking model, when the data marking model receives the carbon emission data table, repeated data information in the carbon emission data table is identified and extracted, and after the step of marking the repeated data, the following steps are further executed:
When the numerical value marking sub-model receives the carbon emission data table with the repeated data marking, identifying data information which does not belong to the repeated data in the carbon emission data table;
judging whether any marked repeated data can be obtained after the data information which does not belong to the repeated data is calculated through an operation rule;
if yes, replacing the data information with the preset identification of the repeated data obtained by calculation and the operation parameters of the operation rule;
when the mapping relation between the storage position of the platform terminal and the data information is determined, the platform terminal identifies the preset identifier with the operation parameter, and the repeated data corresponding to the preset identifier is calculated based on the operation parameter to obtain the data information.
3. The data extraction method of the carbon emission data storage analysis platform according to claim 1, wherein: the carbon emission data table is sent to a data marking model, when the data marking model receives the carbon emission data table, repeated data information in the carbon emission data table is identified and extracted, and before the step of marking the repeated data, the following steps are executed:
identifying a text box to be filled in a carbon emission data table;
acquiring a prestored calculation formula associated with the text box to be filled in and a parameter text box participating in the calculation formula;
Extracting data information in the parameter text box to corresponding parameter positions in a calculation formula, and calculating and outputting data to be filled if all the parameter positions of the calculation formula extract the data information;
and inputting the data to be filled into the text box to be filled.
4. A method for data extraction of a carbon emission data storage analysis platform according to claim 3, wherein: the step of extracting the data information in the parameter text box to the corresponding parameter position in the calculation formula, and if all the parameter positions of the calculation formula extract the data information, calculating and outputting the data to be filled in comprises the following steps:
if the parameter position of the data information which is not extracted exists in the calculation formula, the data type of the parameter text box corresponding to the parameter position of the unfilled data information is identified;
judging whether the data information in the parameter text box is a filling item or not based on the data type pre-associated with the parameter text box;
if the item is the necessary item, sending a data supplement message to the user side;
if the data is not necessary to be filled, the step of calculating the data to be filled is skipped, and the carbon emission data table is sent to the data marking model.
5. The data extraction method of the carbon emission data storage analysis platform according to claim 1, wherein: the carbon emission data table is sent to a data marking model, when the data marking model receives the carbon emission data table, repeated data information in the carbon emission data table is identified and extracted, and before the step of marking the repeated data, the following steps are executed:
Acquiring target data of accuracy to be checked in a carbon emission data table;
based on a data type preset by the target data, sending the target data to a data verification model associated with the data type, and outputting a verification result;
if the verification result is that the target data is abnormal, generating an error report based on the abnormal target data and sending the error report to the user side;
and if the verification results of the data verification model are all that the target data are normal, transmitting the carbon emission data table to the data marking model.
6. The data extraction method of the carbon emission data storage analysis platform according to claim 4, wherein: the data verification model comprises a text verification model and a numerical verification model, the target data comprises text data and numerical data, the target data is sent to the data verification model associated with the data type based on the preset data type of the target data, and the step of outputting a verification result comprises the steps of:
when the character verification model receives character data, screening preset keyword groups based on the data types, judging whether the character data contains necessary keywords in the keyword groups, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal;
When the numerical value verification model receives numerical value data, screening a preset numerical value interval based on the data type, judging whether the numerical value data are located in the numerical value interval, if so, judging that the verification result is normal, and if not, judging that the verification result is abnormal.
7. The data extraction system of the carbon emission data storage analysis platform is characterized in that: comprising the following steps:
the data table acquisition module is used for acquiring a carbon emission data table of an enterprise to be imported to the platform terminal when a data import request sent by a user terminal representing the checking mechanism is received;
the data marking module is used for sending the carbon emission data table to the data marking model, and when the data marking model receives the carbon emission data table, the data marking model identifies and extracts repeated data information in the carbon emission data table and marks the repeated data;
the replacing module is used for replacing the marked repeated data with preset marks, and the preset marks replaced by different repeated data are inconsistent;
the data transmission module is used for transmitting the carbon emission data table with the preset identification replaced and the repeated data to the platform terminal, and the platform terminal identifies the format of the carbon emission data table to determine the mapping relation between the data information and the storage position of the platform terminal;
And the platform replacement module is used for replacing the repeated data to a preset identifier at the corresponding storage position when the mapping relation between the storage position of the platform terminal and the data information is determined.
8. The data extraction system of a carbon emissions data storage analysis platform of claim 7, wherein the data tagging module comprises:
the identification sub-module is used for identifying the data information one by one according to the format of the carbon emission data table when the data marking model receives the carbon emission data table;
the association sub-module is used for associating the data information appearing for the first time with preset identifiers in the identification process, wherein the preset identifiers associated with all the data information appearing for the first time are different;
the association sub-module is used for associating the data information with a preset identifier associated with the last occurrence when the data information appears again;
and the marking sub-module is used for marking all the data information associated with the same preset identifier as a group of repeated data.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, carries out the steps of a data extraction method of a carbon emission data storage analysis platform according to any one of claims 1 to 6.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of a data extraction method of a carbon emission data storage analysis platform according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310389827.5A CN116401239A (en) | 2023-04-12 | 2023-04-12 | Data extraction method and system of carbon emission data storage analysis platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310389827.5A CN116401239A (en) | 2023-04-12 | 2023-04-12 | Data extraction method and system of carbon emission data storage analysis platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116401239A true CN116401239A (en) | 2023-07-07 |
Family
ID=87015659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310389827.5A Pending CN116401239A (en) | 2023-04-12 | 2023-04-12 | Data extraction method and system of carbon emission data storage analysis platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116401239A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160162507A1 (en) * | 2014-12-05 | 2016-06-09 | International Business Machines Corporation | Automated data duplicate identification |
| WO2019232832A1 (en) * | 2018-06-06 | 2019-12-12 | 平安科技（深圳）有限公司 | Data monitoring method and device, computer device, and storage medium |
| CN114579610A (en) * | 2022-03-02 | 2022-06-03 | 南方电网数字电网研究院有限公司 | Heterogeneous data processing method and device, electronic equipment and storage medium |
| CN114942923A (en) * | 2022-07-11 | 2022-08-26 | 深圳新闻网传媒股份有限公司 | Cloud platform-based unified management system for big data calculation and analysis |
| CN114997843A (en) * | 2022-08-03 | 2022-09-02 | 清华大学苏州汽车研究院（吴江） | Enterprise operation management information storage system |
-
2023
- 2023-04-12 CN CN202310389827.5A patent/CN116401239A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160162507A1 (en) * | 2014-12-05 | 2016-06-09 | International Business Machines Corporation | Automated data duplicate identification |
| WO2019232832A1 (en) * | 2018-06-06 | 2019-12-12 | 平安科技（深圳）有限公司 | Data monitoring method and device, computer device, and storage medium |
| CN114579610A (en) * | 2022-03-02 | 2022-06-03 | 南方电网数字电网研究院有限公司 | Heterogeneous data processing method and device, electronic equipment and storage medium |
| CN114942923A (en) * | 2022-07-11 | 2022-08-26 | 深圳新闻网传媒股份有限公司 | Cloud platform-based unified management system for big data calculation and analysis |
| CN114997843A (en) * | 2022-08-03 | 2022-09-02 | 清华大学苏州汽车研究院（吴江） | Enterprise operation management information storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110309125B (en) | Data verification method, electronic device and storage medium | |
| CN111027295B (en) | Data processing method, device, computer equipment and storage medium | |
| CN109254969A (en) | Tables of data processing method, device, equipment and storage medium | |
| CN112418812A (en) | Distributed full-link automatic intelligent clearance system, method and storage medium | |
| CN111191435A (en) | Method and device for generating report form by using dynamic template of customs report form | |
| CN107908843B (en) | Management system based on drawing of map label information | |
| CN110674145B (en) | Data consistency detection method, device, computer equipment and storage medium | |
| CN113760775A (en) | Automatic message testing method and device | |
| CN117391306A (en) | Homeland space planning result examination method, device, equipment and storage medium | |
| CN116401239A (en) | Data extraction method and system of carbon emission data storage analysis platform | |
| CN109947639B (en) | Automatic test method for ESB interface | |
| CN108337133B (en) | Multi-type intermodal message analysis tool | |
| CN115168217A (en) | Defect discovery method and device for source code file | |
| CN114387429A (en) | Vehicle property right processing method, device, equipment and medium based on RPA and AI | |
| CN114169331A (en) | Address resolution method, device, computer equipment and storage medium | |
| CN104182396A (en) | Terminal as well as device and method of optimizing description of format document content | |
| CN114116268A (en) | Method and device for checking Flink SQL statement, computer equipment and storage medium | |
| CN109766727B (en) | Intelligent block network construction method and system | |
| CN114254022A (en) | RPA and AI-based process task processing method, device, system and server | |
| CN116503887B (en) | Goods generation order processing method based on image recognition | |
| CN117312880B (en) | Method and device for processing basic information data of vehicle | |
| CN114707958B (en) | Cloud network-based pricing list auditing method and device | |
| CN114359944A (en) | Method for standardizing oil tank volume table, storage medium and computer equipment | |
| CN111614749B (en) | Data transmission method, data transmission device, computer equipment and storage medium | |
| CN117171816B (en) | Data verification method based on industrial identification and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |