CN114979120B - Data uploading method, device, equipment and storage medium - Google Patents
Data uploading method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114979120B CN114979120B CN202210575850.9A CN202210575850A CN114979120B CN 114979120 B CN114979120 B CN 114979120B CN 202210575850 A CN202210575850 A CN 202210575850A CN 114979120 B CN114979120 B CN 114979120B
- Authority
- CN
- China
- Prior art keywords
- data
- field
- template
- uploading
- length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013500 data storage Methods 0.000 claims abstract description 51
- 238000012545 processing Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 36
- 230000014509 gene expression Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000005192 partition Methods 0.000 claims description 8
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 9
- 238000000605 extraction Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 238000013075 data extraction Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
- G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a data processing technology, and discloses a data uploading method, which comprises the following steps: acquiring form data to be uploaded, extracting data fields of the form data, and identifying form types of the form data; matching the data field with a template field in a pre-constructed data uploading template, and loading data corresponding to the data field which is successfully matched into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file; according to the table category, inquiring a data storage table of the data uploading file from a preset database; and storing the data uploading file into the data storage table. The invention also provides a data uploading method, a device, equipment and a storage medium. The invention can improve the data uploading efficiency.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data uploading method, a data uploading device, an electronic device, and a computer readable storage medium.
Background
Management of data often takes a significant weight in the development of enterprises, where there are a variety of sources of data, which may be manually entered data, imported data from a local data system, or downloaded data in a network, and the like, and the data typically collected is Excel data. The data analysis can extract the effective information in the Excel data, and can store the effective information in the database in a classified manner through the data fields in the effective information and the types of the data fields.
At present, the Excel data is uploaded manually, the data obtained by analysis are added with data fields according to management requirements, the Excel data are uploaded to different databases in a classified mode, and the addition of the data fields can cause lower efficiency of data uploading because the uploading personnel cannot understand the field addition requirements of the data providing personnel, and repeated communication between the uploading personnel and the data providing personnel occurs.
Disclosure of Invention
The invention provides a data uploading method, a device, equipment and a storage medium, and mainly aims to solve the problem of low data uploading efficiency.
In order to achieve the above object, the present invention provides a data uploading method, including:
acquiring form data to be uploaded, extracting data fields of the form data, and identifying form types of the form data;
matching the data field with a template field in a pre-constructed data uploading template, and loading data corresponding to the data field which is successfully matched into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file;
according to the table category, inquiring a data storage table of the data uploading file from a preset database;
And storing the data uploading file into the data storage table.
Optionally, the matching the data field with the template field in the pre-constructed data uploading template includes:
respectively identifying field structures of the data field and the template field, wherein the field structures comprise field lengths and field objects;
according to the field structure in the field structure, performing field length and field object matching on the data field and the model field to obtain a matching result;
and when the matching degree in the matching result reaches a preset matching degree threshold, the data field is successfully matched with the template field in the pre-constructed data uploading template.
Optionally, the performing field length and field object matching on the data field and the model field according to a field structure in the field structure to obtain a matching result includes:
identifying the character lengths of the data field and the template field to obtain the length of the data field and the length of the template field;
comparing the field length of the data field with the field length of the template to obtain a length comparison result;
And performing field object matching on the data fields and the template fields with the same field length in the length matching result to obtain a matching result.
Optionally, the performing field object matching on the data field and the template field with the same field length in the length matching result to obtain a matching result includes:
converting the field objects in the data field and the template field with the same field length into a data field vector and a template field vector by adopting a word vector conversion tool;
and (3) completing calculation of the similarity of the field vectors by adopting a pre-constructed similarity formula, and obtaining the matching result after the calculation is completed.
Optionally, the loading the data corresponding to the data field successfully matched into a data filling frame corresponding to a template field of the data uploading template to obtain a data uploading file includes:
extracting data under a data field corresponding to a template field in a successfully matched data uploading template;
loading the data into a data filling frame corresponding to the template data;
and summarizing the data in the data filling frame under each template field to generate a data uploading file.
Optionally, before the matching the data field with the template field in the pre-constructed data uploading template, the method includes:
Configuring configuration data fields in the data uploading template by using SQL language;
constructing a data uploading rule in the data uploading template;
formulating a data format of uploading data in the data uploading template based on the regular expression;
and generating the data uploading template by using the configuration data field, the data uploading rule and the data format of the uploading data.
Optionally, storing the data upload file in the data storage table includes:
receiving the data uploading file by utilizing the data storage table;
and loading the data uploading file into the data storage table according to the association relation between the data fields in the data uploading file and the data fields in the data storage table.
In order to solve the above problems, the present invention further provides a data uploading device, which includes:
the data processing module is used for acquiring form data to be uploaded, extracting data fields of the form data and identifying form types of the form data;
the data uploading file generation module is used for matching the data field with a template field in a pre-constructed data uploading template, and loading data corresponding to the data field which is successfully matched into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file;
The data uploading module is used for inquiring a data storage table of the data uploading file from a preset database according to the table type; and storing the data uploading file into the data storage table.
In order to solve the above-mentioned problems, the present invention also provides an electronic apparatus including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data uploading method described above.
In order to solve the above-mentioned problems, the present invention also provides a computer-readable storage medium having stored therein at least one computer program that is executed by a processor in an electronic device to implement the above-mentioned data uploading method.
According to the embodiment of the invention, accurate extraction of data under the corresponding data field in the form data is completed according to the data field in the form data to be uploaded, so that the extraction accuracy of the form data can be improved. Therefore, the data uploading method, the device, the electronic equipment and the computer readable storage medium can solve the problem of low data uploading efficiency.
Drawings
Fig. 1 is a flow chart of a data uploading method according to an embodiment of the application;
FIG. 2 is a functional block diagram of a data uploading device according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device for implementing the data uploading method according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The embodiment of the application provides a data uploading method. The execution body of the data uploading method includes, but is not limited to, at least one of a server, a terminal and the like capable of being configured to execute the method provided by the embodiment of the application. In other words, the data uploading method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a flow chart of a data uploading method according to an embodiment of the invention is shown.
In this embodiment, the data uploading method includes:
s1, acquiring form data to be uploaded, identifying form types of the form data, and extracting data fields of the form data;
in the embodiment of the present invention, the form data may be data that can be queried and indexed and needs to be stored in a database, which is generated based on different business scenarios, for example, in a business operation scenario, the form data to be uploaded may be data of competitors' information, advantages and disadvantages of products of enterprises themselves, preferences of users, and the like, and in a commodity transaction scenario, the form data to be uploaded may be order data for completing a transaction.
In the embodiment of the invention, the form category may be a category label for classifying form data based on different scenes, for example, the form category may be an order information category, a product detail category, etc. in a commodity transaction scene.
In the embodiment of the present invention, the data field may be an inherent field corresponding to data in the table data, for example, in order data, the data field may be "order creation time", "order product", and "order number", etc.
In detail, the identifying the form category of the form data includes: extracting a subject field of the form data; inquiring the category of the subject field in a preset subject category library table to obtain the form category of the form data.
In the embodiment of the present invention, the preset topic category library table may be a topic category table of table data constructed by using topic fields of a large number of data tables.
Further, the data fields and the specific data information corresponding to the data fields generally exist in the table data, and the data fields of the table data can be extracted to finish accurate extraction of the data corresponding to the data fields, so that the accuracy of data extraction is improved.
S2, matching the data field with a template field in a pre-constructed data uploading template, and loading data corresponding to the data field which is successfully matched into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file.
In the embodiment of the invention, the pre-constructed data uploading template can be a template for extracting important data information in form data, such as order data important data information, an order data uploading template constructed by order time, order number and order product, and the important data information in order data can be accurately acquired by using the order data uploading template.
In detail, the embodiment of the invention matches the data field with the template field in the pre-constructed data uploading template, loads the data corresponding to the successfully matched data field into the data filling frame corresponding to the template field of the data uploading template to obtain the data uploading file, and can realize the standardization of the data uploading file and finish the data uploading.
As an embodiment of the present invention, the matching the data field with the template field in the pre-constructed data upload template includes:
respectively identifying field structures of the data field and the template field, wherein the field structures comprise field lengths and field objects;
according to the field structure in the field structure, performing field length and field object matching on the data field and the model field to obtain a matching result;
and when the matching degree in the matching result reaches a preset matching degree threshold, the data field is successfully matched with the template field in the pre-constructed data uploading template.
In the embodiment of the present invention, the field object refers to a specific meaning of each field representation in the data field and the template field, such as an object of "name", "score" or "math".
It should be understood that there may be a small deviation between performing field matching on two fields, but the meaning of the expressions is consistent, that is, in the embodiment of the present invention, the preset matching degree threshold may be set to 95%.
Further, the performing field length and field object matching on the data field and the model field according to the field structure in the field structure to obtain a matching result includes:
identifying the character lengths of the data field and the template field to obtain the length of the data field and the length of the template field;
comparing the field length of the data field with the field length of the template to obtain a length comparison result;
and performing field object matching on the data fields and the template fields with the same field length in the length matching result to obtain a matching result.
Further, the matching of the field objects is performed by the data fields with the same field length in the length matching result and the template fields, and a word vector conversion tool can be adopted to respectively convert the data fields with the same field length and the field objects in the template fields into a data field vector and a template field vector; and (3) completing calculation of the similarity of the field vectors by adopting a pre-constructed similarity formula, and obtaining the matching result after the calculation is completed.
In the embodiment of the invention, the pre-constructed Word vector conversion tool can be a Word-vector conversion tool based on a Word2vec model.
In the embodiment of the invention, the similarity between the data field vector and the template field vector can be calculated by adopting the following similarity formula:
wherein x is the data field vector, y is the template field vector, θ is the angle between the data field vector and the template field vector, sim (x, y) is the similarity between the data field vector and the template field vector.
Further, the embodiment of the invention extracts the data successfully matched with the data fields corresponding to the template fields in the data uploading template; loading the data into a data filling frame corresponding to the template data; and summarizing the data in the data filling frame under each template field to generate a data uploading file.
In another optional embodiment of the present invention, before the matching the data field with the template field in the pre-constructed data upload template, the method includes: configuring configuration data fields in the data uploading template by using SQL language; constructing a data uploading rule in the data uploading template; formulating a data format of uploading data in the data uploading template based on the regular expression; and generating the data uploading template by using the configuration data field, the data uploading rule and the data format of the uploading data.
In the embodiment of the invention, the SQL language is a structured language, is a special purpose programming language, and is a database query and programming language used for accessing data and querying, updating and managing a database system.
In the embodiment of the invention, the regular expression is generally used for retrieving and replacing texts conforming to a certain mode (rule), for example, the regular expression can be used for limiting the format of uploading data, and the uploading data cannot be empty.
S3, inquiring a data storage table of the data uploading file from a preset database according to the table type;
in detail, according to the table type, the embodiment of the invention queries the data storage table of the data uploading file from the preset database, so that the accurate uploading of the data uploading file can be realized.
As an embodiment of the present invention, the querying, according to the table category, the data storage table of the data upload file from a preset database includes: indexing a target storage partition matched with the table category in a preset database by utilizing the table category; and extracting the data storage table in the target storage partition.
In the embodiment of the invention, the data storage table may be a storage format file configured by receiving data uploading files of different types in a database, such as an order form data configuration order type storage format file, and an item detail form data configuration item detail request storage format file.
In the embodiment of the present invention, the preset database may be a mysql database, an oracle database, a hiveoracle database, etc
S4, storing the data uploading file into the data storage table.
In detail, the embodiment of the invention receives the data uploading file by using the data storage table; and loading the data uploading file into the data storage table according to the association relation between the data fields in the data uploading file and the data fields in the data storage table, and completing uploading of the table data.
It should be understood that the association relationship refers to a matching relationship between the data fields in the data upload file and the same data fields in the data storage table, for example, the data fields in the data upload file are the same as the data fields in the data storage table, and since the data in the data upload file is standardized data obtained by performing data processing on table data, the data corresponding to the data fields in the data upload file are randomly arranged and are inconsistent with the arrangement order of the data fields in the data storage table, so that after the data fields in the data upload file and the data fields in the data storage table are matched in the same field, the data upload file is stored in the data storage table.
According to the embodiment of the invention, accurate extraction of data under the corresponding data field in the form data is completed according to the data field in the form data to be uploaded, so that the extraction accuracy of the form data can be improved. Therefore, the data uploading method provided by the invention can solve the problem of low data uploading efficiency.
Fig. 2 is a functional block diagram of a data uploading device according to an embodiment of the present invention.
The data uploading device 100 of the present invention may be installed in an electronic apparatus. The data uploading device 100 may include a data processing module 101, a data uploading file generating module 102, and a data uploading module 103 according to the implemented functions. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
The data processing module 101 is configured to obtain form data to be uploaded, extract a data field of the form data, and identify a form category of the form data;
in the embodiment of the present invention, the form data may be data that can be queried and indexed and needs to be stored in a database, which is generated based on different business scenarios, for example, in a business operation scenario, the form data to be uploaded may be data of competitors' information, advantages and disadvantages of products of enterprises themselves, preferences of users, and the like, and in a commodity transaction scenario, the form data to be uploaded may be order data for completing a transaction.
In the embodiment of the invention, the form category may be a category label for classifying form data based on different scenes, for example, the form category may be an order information category, a product detail category, etc. in a commodity transaction scene.
In the embodiment of the present invention, the data field may be an inherent field corresponding to data in the table data, for example, in order data, the data field may be "order creation time", "order product", and "order number", etc.
In detail, the identifying the form category of the form data includes: extracting a subject field of the form data; inquiring the category of the subject field in a preset subject category library table to obtain the form category of the form data.
In the embodiment of the present invention, the preset topic category library table may be a topic category table of table data constructed by using topic fields of a large number of data tables.
Further, the data fields and the specific data information corresponding to the data fields generally exist in the table data, and the data fields of the table data can be extracted to finish accurate extraction of the data corresponding to the data fields, so that the accuracy of data extraction is improved.
The data uploading file generating module 102 is configured to match the data field with a template field in a pre-constructed data uploading template, and load data corresponding to the data field that is successfully matched into a data filling frame corresponding to the template field of the data uploading template, so as to obtain a data uploading file;
in the embodiment of the invention, the pre-constructed data uploading template can be a template for extracting important data information in form data, such as order data important data information, an order data uploading template constructed by order time, order number and order product, and the important data information in order data can be accurately acquired by using the order data uploading template.
In detail, the embodiment of the invention matches the data field with the template field in the pre-constructed data uploading template, loads the data corresponding to the successfully matched data field into the data filling frame corresponding to the template field of the data uploading template to obtain the data uploading file, and can realize the standardization of the data uploading file and finish the data uploading.
As an embodiment of the present invention, the matching the data field with the template field in the pre-constructed data upload template includes:
respectively identifying field structures of the data field and the template field, wherein the field structures comprise field lengths and field objects;
according to the field structure in the field structure, performing field length and field object matching on the data field and the model field to obtain a matching result;
and when the matching degree in the matching result reaches a preset matching degree threshold, the data field is successfully matched with the template field in the pre-constructed data uploading template.
In the embodiment of the present invention, the field object refers to a specific meaning of each field representation in the data field and the template field, such as an object of "name", "score" or "math".
It should be understood that there may be a small deviation between performing field matching on two fields, but the meaning of the expressions is consistent, that is, in the embodiment of the present invention, the preset matching degree threshold may be set to 95%.
Further, the performing field length and field object matching on the data field and the model field according to the field structure in the field structure to obtain a matching result includes:
identifying the character lengths of the data field and the template field to obtain the length of the data field and the length of the template field;
comparing the field length of the data field with the field length of the template to obtain a length comparison result;
and performing field object matching on the data fields and the template fields with the same field length in the length matching result to obtain a matching result.
Further, the matching of the field objects is performed by the data fields with the same field length in the length matching result and the template fields, and a word vector conversion tool can be adopted to respectively convert the data fields with the same field length and the field objects in the template fields into a data field vector and a template field vector; and (3) completing calculation of the similarity of the field vectors by adopting a pre-constructed similarity formula, and obtaining the matching result after the calculation is completed.
In the embodiment of the invention, the pre-constructed Word vector conversion tool can be a Word-vector conversion tool based on a Word2vec model.
In the embodiment of the invention, the similarity between the data field vector and the template field vector can be calculated by adopting the following similarity formula:
wherein x is the data field vector, y is the template field vector, θ is the angle between the data field vector and the template field vector, sim (x, y) is the similarity between the data field vector and the template field vector.
Further, the embodiment of the invention extracts the data successfully matched with the data fields corresponding to the template fields in the data uploading template; loading the data into a data filling frame corresponding to the template data; and summarizing the data in the data filling frame under each template field to generate a data uploading file.
In another optional embodiment of the present invention, before the matching the data field with the template field in the pre-constructed data upload template, the method includes: configuring configuration data fields in the data uploading template by using SQL language; constructing a data uploading rule in the data uploading template; formulating a data format of uploading data in the data uploading template based on the regular expression; and generating the data uploading template by using the configuration data field, the data uploading rule and the data format of the uploading data.
In the embodiment of the invention, the SQL language is a structured language, is a special purpose programming language, and is a database query and programming language used for accessing data and querying, updating and managing a database system.
In the embodiment of the invention, the regular expression is generally used for retrieving and replacing texts conforming to a certain mode (rule), for example, the regular expression can be used for limiting the format of uploading data, and the uploading data cannot be empty.
The data uploading module 103 is configured to query a data storage table of the data uploading file from a preset database according to the table type; and storing the data uploading file into the data storage table.
In detail, according to the table type, the embodiment of the invention queries the data storage table of the data uploading file from the preset database, so that the accurate uploading of the data uploading file can be realized.
As an embodiment of the present invention, the querying, according to the table category, the data storage table of the data upload file from a preset database includes: indexing a target storage partition matched with the table category in a preset database by utilizing the table category; and extracting the data storage table in the target storage partition.
In the embodiment of the invention, the data storage table may be a storage format file configured by receiving data uploading files of different types in a database, such as an order form data configuration order type storage format file, and an item detail form data configuration item detail request storage format file.
In the embodiment of the present invention, the preset database may be a mysql database, an oracle database, a hiveoracle database, etc
In detail, the embodiment of the invention receives the data uploading file by using the data storage table; and loading the data uploading file into the data storage table according to the association relation between the data fields in the data uploading file and the data fields in the data storage table, and completing uploading of the table data.
It should be understood that the association relationship refers to a matching relationship between the data fields in the data upload file and the same data fields in the data storage table, for example, the data fields in the data upload file are the same as the data fields in the data storage table, and since the data in the data upload file is standardized data obtained by performing data processing on table data, the data corresponding to the data fields in the data upload file are randomly arranged and are inconsistent with the arrangement order of the data fields in the data storage table, so that after the data fields in the data upload file and the data fields in the data storage table are matched in the same field, the data upload file is stored in the data storage table.
According to the embodiment of the invention, accurate extraction of data under the corresponding data field in the form data is completed according to the data field in the form data to be uploaded, so that the extraction accuracy of the form data can be improved. Therefore, the data uploading method provided by the invention can solve the problem of low data uploading efficiency.
Fig. 3 is a schematic structural diagram of an electronic device for implementing a data uploading method according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program, such as a data upload program, stored in the memory 11 and executable on the processor 10.
The processor 10 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the entire electronic device using various interfaces and lines, and executes various functions of the electronic device and processes data by running or executing programs or modules (e.g., executing a data upload program, etc.) stored in the memory 11, and calling data stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. The memory 11 may in other embodiments also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only for storing application software installed in an electronic device and various types of data, such as codes of a data uploading program, but also for temporarily storing data that has been output or is to be output.
The communication bus 12 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
The communication interface 13 is used for communication between the electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface.
Fig. 3 shows only an electronic device with components, it being understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or may be arranged in different components.
For example, although not shown, the electronic device may further include a power source (such as a battery) for supplying power to the respective components, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device may further include various sensors, bluetooth modules, wi-Fi modules, etc., which are not described herein.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The data upload program stored in the memory 11 of the electronic device 1 is a combination of instructions that, when executed in the processor 10, can implement:
acquiring form data to be uploaded, extracting data fields of the form data, and identifying form types of the form data;
matching the data field with a template field in a pre-constructed data uploading template, and loading data corresponding to the data field which is successfully matched into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file;
According to the table category, inquiring a data storage table of the data uploading file from a preset database;
and storing the data uploading file into the data storage table.
In particular, the specific implementation method of the above instructions by the processor 10 may refer to the description of the relevant steps in the corresponding embodiment of the drawings, which is not repeated herein.
Further, the modules/units integrated in the electronic device 1 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present invention also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement:
acquiring form data to be uploaded, extracting data fields of the form data, and identifying form types of the form data;
Matching the data field with a template field in a pre-constructed data uploading template, and loading data corresponding to the data field which is successfully matched into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file;
according to the table category, inquiring a data storage table of the data uploading file from a preset database;
and storing the data uploading file into the data storage table.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present application without departing from the spirit and scope of the technical solution of the present application.
Claims (7)
1. A method of uploading data, the method comprising:
Acquiring form data to be uploaded, extracting data fields of the form data, identifying form types of the form data, configuring configuration data fields in a data uploading template by using SQL language, constructing a data uploading rule in the data uploading template, formulating a data format of uploading data in the data uploading template based on a regular expression, and generating the data uploading template by using the configuration data fields, the data uploading rule and the data format of uploading data;
respectively identifying field structures of the data field and the template field, identifying character lengths of the data field and the template field to obtain a data field length and a template field length, comparing the data field length with the template field length to obtain a length comparison result, executing field object matching on the data field and the template field with the same field length in the length comparison result to obtain a matching result, and when the matching degree in the matching result reaches a preset matching degree threshold value, successfully matching the data field with the template field in a pre-constructed data uploading template, loading data corresponding to the successfully matched data field into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file;
According to the table category, indexing a target storage partition matched with the table category from a preset database, and extracting a data storage table in the target storage partition;
and storing the data uploading file into the data storage table.
2. The method for uploading data according to claim 1, wherein the performing field object matching on the data field and the template field with the same field length in the length comparison result to obtain a matching result includes:
converting the field objects in the data field and the template field with the same field length into a data field vector and a template field vector by adopting a word vector conversion tool;
and (3) completing calculation of the similarity of the field vectors by adopting a pre-constructed similarity formula, and obtaining the matching result after the calculation is completed.
3. The method for uploading data according to claim 1, wherein the loading the data corresponding to the data field successfully matched into the data filling frame corresponding to the template field of the data uploading template to obtain the data uploading file includes:
extracting data under a data field corresponding to a template field in a successfully matched data uploading template;
Loading the data into a data filling frame corresponding to the template data;
and summarizing the data in the data filling frame under each template field to generate a data uploading file.
4. The data uploading method according to claim 1, wherein storing the data uploading file in the data storage table comprises:
receiving the data uploading file by utilizing the data storage table;
and loading the data uploading file into the data storage table according to the association relation between the data fields in the data uploading file and the data fields in the data storage table.
5. A data uploading apparatus, the apparatus comprising:
the data processing module is used for acquiring form data to be uploaded, extracting data fields of the form data, identifying form types of the form data, constructing a data uploading rule in the data uploading template by utilizing configuration data fields in an SQL language configuration data uploading template, formulating a data format of uploading data in the data uploading template based on a regular expression, and generating the data uploading template by utilizing the configuration data fields, the data uploading rule and the data format of the uploading data;
The data uploading file generation module is used for respectively identifying field structures of the data field and the template field, identifying character lengths of the data field and the template field, obtaining the length of the data field and the length of the template field, comparing the length of the data field with the length of the template field to obtain a length comparison result, executing field object matching on the data field and the template field with the same field length in the length matching result to obtain a matching result, and when the matching degree in the matching result reaches a preset matching degree threshold value, successfully matching the data field with the template field in a preset data uploading template, loading data corresponding to the successfully matched data field into a data filling frame corresponding to the template field of the data uploading template to obtain a data uploading file;
the data uploading module is used for indexing a target storage partition matched with the form category from a preset database according to the form category and extracting a data storage table in the target storage partition; and storing the data uploading file into the data storage table.
6. An electronic device, the electronic device comprising:
At least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data uploading method of any of claims 1 to 4.
7. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the data uploading method according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210575850.9A CN114979120B (en) | 2022-05-24 | 2022-05-24 | Data uploading method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210575850.9A CN114979120B (en) | 2022-05-24 | 2022-05-24 | Data uploading method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114979120A CN114979120A (en) | 2022-08-30 |
| CN114979120B true CN114979120B (en) | 2023-10-13 |
Family
ID=82954864
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210575850.9A Active CN114979120B (en) | 2022-05-24 | 2022-05-24 | Data uploading method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114979120B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116319718B (en) * | 2023-03-10 | 2023-12-12 | 北京振中电子技术有限公司 | Cloud data storage processing method, system, equipment and medium |
| CN116861865B (en) * | 2023-06-26 | 2024-10-01 | 江苏常熟农村商业银行股份有限公司 | EXCEL data processing method, device, equipment and storage medium |
| CN116912005A (en) * | 2023-07-17 | 2023-10-20 | 广东企企通科技有限公司 | Financing information tracking management method, device, equipment and medium |
| CN116842223B (en) * | 2023-08-29 | 2023-11-10 | 天津鑫宝龙电梯集团有限公司 | Working condition data management method, device, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109815276A (en) * | 2018-12-14 | 2019-05-28 | 中国平安财产保险股份有限公司 | Method, apparatus, computer equipment and the storage medium of declaration form extraction data |
| CN110399420A (en) * | 2019-07-30 | 2019-11-01 | 广州吉信网络科技开发有限公司 | A kind of deriving method, electronic equipment and the medium of configurableization Excel format |
| CN110569341A (en) * | 2019-07-25 | 2019-12-13 | 深圳壹账通智能科技有限公司 | method and device for configuring chat robot, computer equipment and storage medium |
| CN112565393A (en) * | 2020-12-01 | 2021-03-26 | 平安科技（深圳）有限公司 | File uploading method, file downloading method, file uploading device, file downloading device, computer equipment and storage medium |
| CN112965982A (en) * | 2021-03-16 | 2021-06-15 | 中国平安财产保险股份有限公司 | Table processing method, device, equipment and storage medium |
| CN114510912A (en) * | 2022-04-20 | 2022-05-17 | 佳瑛科技有限公司 | Method, system, and medium for classifying electronic forms based on a distributed system |
-
2022
- 2022-05-24 CN CN202210575850.9A patent/CN114979120B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109815276A (en) * | 2018-12-14 | 2019-05-28 | 中国平安财产保险股份有限公司 | Method, apparatus, computer equipment and the storage medium of declaration form extraction data |
| CN110569341A (en) * | 2019-07-25 | 2019-12-13 | 深圳壹账通智能科技有限公司 | method and device for configuring chat robot, computer equipment and storage medium |
| WO2021012382A1 (en) * | 2019-07-25 | 2021-01-28 | 深圳壹账通智能科技有限公司 | Method and apparatus for configuring chat robot, computer device and storage medium |
| CN110399420A (en) * | 2019-07-30 | 2019-11-01 | 广州吉信网络科技开发有限公司 | A kind of deriving method, electronic equipment and the medium of configurableization Excel format |
| CN112565393A (en) * | 2020-12-01 | 2021-03-26 | 平安科技（深圳）有限公司 | File uploading method, file downloading method, file uploading device, file downloading device, computer equipment and storage medium |
| CN112965982A (en) * | 2021-03-16 | 2021-06-15 | 中国平安财产保险股份有限公司 | Table processing method, device, equipment and storage medium |
| CN114510912A (en) * | 2022-04-20 | 2022-05-17 | 佳瑛科技有限公司 | Method, system, and medium for classifying electronic forms based on a distributed system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114979120A (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114979120B (en) | Data uploading method, device, equipment and storage medium | |
| CN112052370A (en) | Message generation method and device, electronic equipment and computer readable storage medium | |
| CN112541338A (en) | Similar text matching method and device, electronic equipment and computer storage medium | |
| CN112528616B (en) | Service form generation method and device, electronic equipment and computer storage medium | |
| CN113868528A (en) | Information recommendation method and device, electronic equipment and readable storage medium | |
| CN114491047A (en) | Multi-label text classification method and device, electronic equipment and storage medium | |
| CN112231417A (en) | Data classification method and device, electronic equipment and storage medium | |
| CN114386509A (en) | Data fusion method and device, electronic equipment and storage medium | |
| CN114612194A (en) | Product recommendation method and device, electronic equipment and storage medium | |
| CN113868529A (en) | Knowledge recommendation method and device, electronic equipment and readable storage medium | |
| CN114637811A (en) | Data table entity relation graph generation method, device, equipment and storage medium | |
| CN114398346A (en) | Data migration method, device, equipment and storage medium | |
| CN111538768A (en) | Data query method and device based on N-element model, electronic equipment and medium | |
| CN115409041B (en) | Unstructured data extraction method, device, equipment and storage medium | |
| CN116741358A (en) | Inquiry registration recommendation method, inquiry registration recommendation device, inquiry registration recommendation equipment and storage medium | |
| CN116521867A (en) | Text clustering method and device, electronic equipment and storage medium | |
| CN111414452B (en) | Search word matching method and device, electronic equipment and readable storage medium | |
| CN114840388A (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN114547011A (en) | Data extraction method and device, electronic equipment and storage medium | |
| CN113743802A (en) | Work order intelligent matching method and device, electronic equipment and readable storage medium | |
| CN113704616A (en) | Information pushing method and device, electronic equipment and readable storage medium | |
| CN113707302A (en) | Service recommendation method, device, equipment and storage medium based on associated information | |
| CN113360505B (en) | Time sequence data-based data processing method and device, electronic equipment and readable storage medium | |
| CN114185617B (en) | Service call interface configuration method, device, equipment and storage medium | |
| CN116881284A (en) | Data retrieval method, device and equipment for structured query statement and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |