CN113934710B - Data acquisition method and device - Google Patents
Data acquisition method and device Download PDFInfo
- Publication number
- CN113934710B CN113934710B CN202111207894.8A CN202111207894A CN113934710B CN 113934710 B CN113934710 B CN 113934710B CN 202111207894 A CN202111207894 A CN 202111207894A CN 113934710 B CN113934710 B CN 113934710B
- Authority
- CN
- China
- Prior art keywords
- data
- template
- data template
- created
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
- G06K7/1417—2D bar codes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Toxicology (AREA)
- General Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a data acquisition method and a data acquisition device, and belongs to the technical field of data processing. According to the scheme, the data template is created by creating basic elements of the data template in advance, then selecting the elements of the data template through a small program or a platform and freely combining the basic elements of the data template, so that the data template can be created, the movability of creating the data template is realized, the data template can be rapidly created when the data template is changed, and the creating efficiency of the data template is improved. By creating the data template, the efficiency of importing data is improved. Data acquisition is carried out through the mobile terminal, and real-time data acquisition is achieved. By adding the scheme of modifying the data template, the migration of the historical data of the data template to the database of the new data template is realized. By adding a mechanism for verifying the integrity of data in the process of loading the original data into the database of the corresponding data template, the integrity of the data acquired by the data template is ensured.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data acquisition method and device.
Background
In a digital management system of a manufacturer, data information of each product line is often required to be collected in the process of producing products.
At present, in a digital management system of a manufacturer, if data information of each product line, such as a model of the product line, production value information of the product line, production time of the product line, and the like, needs to be collected in the system, the data information needs to be added in the system one by one, or an excel (spreadsheet) file is manufactured according to a template specified by the system, and then the excel file is imported into the system. And corresponding to different forms, such as a product model table and a product output value information table, because the fields of the forms are different, the excel table of a unified template cannot be used. Therefore, different templates must be configured according to different forms, and thus, the corresponding configuration needs to be made at the beginning of designing the digital management system.
However, the plans cannot keep pace with the changes, and designers are required to change the designs when the templates need to be changed in the use process of enterprises in the future, so that the data template creation efficiency is low. Meanwhile, after the template is changed, the previously collected historical data is reintroduced into a new template database, so that the operation is very complicated, and the situations of data omission and the like are easy to occur. Therefore, the data acquisition mode in the prior art cannot flexibly meet the requirements of enterprises, and the maintenance cost is high.
Disclosure of Invention
The invention provides a data acquisition method and a data acquisition device. The technical scheme of the invention is as follows:
in a first aspect, a data acquisition method is provided, which includes:
when the platform detects that the data acquisition service is started, judging whether a new data template needs to be created or not;
if the new data template is determined to be created, judging whether the data template is created through the platform or the mobile terminal;
if the data template is established through the platform, configuring preset data template basic elements to generate a target data template; if the data template is established through the mobile terminal, acquiring a target data template generated by configuring basic elements of a preset data template through the mobile terminal;
generating a designated spreadsheet for data batch acquisition of the platform and a designated two-dimensional code for real-time data acquisition of the mobile terminal according to the target data template;
acquiring collected data, and judging whether the collected data is imported data through the specified spreadsheet or data collected in real time through the specified two-dimensional code;
selecting a data template if it is determined to be data imported through the specified spreadsheet; if the data acquired in real time through the specified two-dimensional code is determined, identifying a data template;
analyzing the collected data and storing the collected data in an original database;
according to the selected data template or the identified data template, migrating the acquired data from the original database to a database corresponding to the target data template;
and verifying whether the data transferred to the database corresponding to the target data template is complete, and finishing data acquisition if the data transferred to the database corresponding to the target data template is complete.
Optionally, after the determining whether a new data template needs to be created, the method further includes:
if the new data template is determined to need to be created, judging whether the basic elements of the new data template need to be created or not;
and if the new data template basic elements need to be created, creating the new data template basic elements according to the data acquisition requirements.
Optionally, the mobile terminal scans the designated two-dimensional code through an applet thereon to perform real-time data acquisition.
Optionally, after verifying whether the data migrated to the database corresponding to the target data template is complete, the method further includes:
and if the data migrated to the database corresponding to the target data template is incomplete, returning to the original database again for data synchronization.
Optionally, the parsing the collected data and saving the collected data to the original database includes:
analyzing the collected data into various pieces of data, generating a data identifier of each piece of data, and storing each piece of data and the data identifier thereof in an original database.
Optionally, after the migrating the collected data from the original database to the database corresponding to the target data template, the method further includes:
filtering the acquired data according to the elements in the target data template, and filtering out data corresponding to the elements which do not exist in the target data template;
and filtering the data in the target data template according to the data identification, and filtering out repeated data.
Optionally, the verifying whether the data migrated to the database corresponding to the target data template is complete includes:
verifying whether the data migrated to the database corresponding to the target data template contains the data identifier of each piece of acquired data, and if so, determining that the migrated data is complete; otherwise, the missing data is inquired and copied from the original database again according to the data identification.
In a second aspect, a data acquisition device is provided, comprising:
the first judgment unit is configured to judge whether a new data template needs to be created or not after the platform detects that the data acquisition service is started;
a second judging unit configured to judge whether to create the data template through the platform or the mobile terminal if it is determined that a new data template needs to be created;
the template configuration unit is configured to configure preset data template basic elements to generate a target data template if the data template is determined to be created through the platform; if the data template is established through the mobile terminal, acquiring a target data template generated by configuring basic elements of a preset data template through the mobile terminal;
the generating unit is configured to generate a designated spreadsheet for batch data acquisition of the platform and a designated two-dimensional code for real-time data acquisition of the mobile terminal according to the target data template;
the data acquisition and judgment unit is configured to acquire the acquired data and judge whether the acquired data is imported data through the designated spreadsheet or data acquired in real time through the designated two-dimensional code;
a data template selecting unit configured to select a data template if it is determined to be data imported through the specified electronic form; if the data acquired in real time through the specified two-dimensional code is determined, identifying a data template;
the data analysis and storage unit is configured to analyze the acquired data and store the acquired data in an original database;
the data migration unit is configured to migrate the acquired data from the original database to a database corresponding to the target data template according to the selected data template or the identified data template;
and the data verification unit is used for verifying whether the data transferred to the database corresponding to the target data template is complete or not, and finishing data acquisition if the data transferred to the database corresponding to the target data template is complete.
In a third aspect, a data acquisition device is provided, which includes: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine-readable program to perform the method of the first aspect.
In a fourth aspect, a computer readable medium is provided, having stored thereon computer instructions, which, when executed by a processor, cause the processor to perform the method of the first aspect described above.
According to the method and the device provided by the embodiment of the specification, the data template can be created by creating the basic elements of the data template in advance and then checking the basic elements of the data template through a small program or a platform and freely combining the basic elements of the data template, so that the mobility of the created data template is realized, the data template can be quickly created when the data template is changed, and the creation efficiency of the data template is improved. In addition, the efficiency of importing data is improved by creating the data template. Data acquisition is carried out through the mobile terminal, and real-time data acquisition is achieved. By adding the scheme of modifying the data template, the migration of the historical data of the data template to the database of the new data template is realized. By adding a mechanism for verifying the integrity of data in the process of loading the original data into the database of the corresponding data template, the integrity of the data acquired by the data template is ensured. The invention has the advantages of flexible adaptation to the requirements of enterprises, low maintenance cost and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a data acquisition method according to an embodiment of the present invention.
Fig. 2 is a flowchart of a data collection method according to an embodiment of the present invention.
Fig. 3 is a block diagram of a data acquisition device according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a data acquisition device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a data acquisition method which can be executed by any device, equipment, platform or equipment cluster with calculation and processing capabilities. As shown in fig. 1, the method may include the steps of:
The production line often needs different data templates when producing different products, so that the method can be realized according to whether the type of the product to be produced by the production line is the same as the type of the product produced last time when judging whether to create a new data template. Specifically, if the type of product that is about to be produced by the production line is not the same as the type of product that was last produced, it is determined that a new data template needs to be created. If the type of product that is being produced by the production line is the same as the type of product that was last produced, it is determined that no new data template need be created.
For example, the last product produced in the production line is a single chip board, the recorded data includes the production place of the single chip, the product to be produced in the production line is a beer machine, the recorded data includes the size of a beer barrel, and since the data to be recorded has respective product characteristics, if the production line is changed from the production single chip board to the production beer machine, a new data template needs to be created. Of course, if the manufacturing line is to produce a product that is another model of the one-chip board, it may not be necessary to create a new data template.
And 103, if the new data template is determined to need to be created, judging whether the data template is created through the platform or the mobile terminal.
And 105, if the data template is determined to be created through the platform, configuring preset data template basic elements to generate a target data template.
Specifically, before data acquisition, the method provided by the embodiment of the present invention presets various basic elements of the data template according to data acquisition requirements, and on this basis, when a data template needs to be created, a user can generate the data template by checking the basic elements of the data template through a platform or a mobile terminal and combining the basic elements. Furthermore, when the user selects the basic elements of the data template, the selection can be carried out on the platform or the mobile terminal. The mobile terminal may be an application installed thereon, or may be an applet embedded in one of the applications.
When a user selects the basic elements of the data template on the platform to generate the target data template, the platform configures one or more basic elements of the data template selected by the user to generate the target data template. When the target data template is created, some basic information may be configured in advance, such as a product model, a product specification, a production address, and the like.
And 107, if the data template is determined to be created through the mobile terminal, acquiring a target data template generated by configuring basic elements of the preset data template by the mobile terminal.
When a user selects basic elements of the data template at the mobile terminal to generate a target data template, the mobile terminal configures one or more basic elements of the data template selected by the user to generate the target data template, and sends the generated target data template to the platform.
And step 109, generating a specified spreadsheet for the platform to perform batch data acquisition and a specified two-dimensional code for the mobile terminal to perform real-time data acquisition according to the target data template.
In the embodiment of the present invention, after the data template is generated, the platform may further generate an electronic form or a two-dimensional code of the data template. The electronic form is used for factory filling, and basic information of the electronic form comprises model, specification, time, production line and product basic information. Basic information or general information about a product in the electronic form is generated in advance, and only data needs to be filled and written. After the information of the electronic form is filled, the information in the electronic form can be collected to an original database of the platform through platform import. The two-dimensional code is used for acquiring real-time data of the mobile terminal, and comprises form information and basic information in the form, wherein the basic information comprises model, specification, time, production line and product basic information. The two-dimensional code already comprises basic information or general information of a product, after the code is scanned by the mobile terminal, only current real-time key data needs to be filled, and after filling is completed, a submission button is clicked, namely the current real-time data is acquired to an original database of the platform.
And step 111, acquiring the acquired data, and judging whether the acquired data is the data imported through the designated spreadsheet or the data acquired in real time through the designated two-dimensional code.
Optionally, when the platform acquires the acquired data: when data acquired by a plurality of electronic forms or a plurality of two-dimensional codes are acquired simultaneously, a distributed parallel data acquisition mode is adopted. The data are imported in multiple paths, so that the efficiency of uploading the data is improved, and the data in the data uploading stage are not lost.
When the platform collects data, statistical personnel download the specified electronic forms according to the requirements, fill in the necessary information of the specified electronic forms, and after filling in is completed, import the batch data through the function of importing the specified electronic forms by the platform, so that the batch data collection is realized. When the mobile terminal collects data, a statistic staff downloads and prints the designated two-dimensional code as required and pastes the designated two-dimensional code on a corresponding production line, when the production line statistic staff collects real-time data, the production line statistic staff scans the designated two-dimensional code, then fills in the required collection information on the loaded page, and after filling is finished, the collection information is uploaded by clicking, so that single real-time data collection is realized.
The embodiment of the invention not only realizes the batch data acquisition by importing the data in the electronic form through the platform, but also adds the function of real-time data acquisition through the mobile terminal. When the mobile terminal collects data, the data collection is preferably realized through an applet on the mobile terminal. The advantages of using the mobile terminal for data acquisition are as follows:
A. the data template is created through remote operation of the mobile terminal, so that the mobility and the real-time performance of creating the data template can be realized, and the data template creation can be performed at any time and anywhere without depending on the office environment. The conventional data template can only be created by a computer, and the mobility of creating the data template cannot be realized.
B. Data are collected through the mobile terminal, and real-time data collection can be achieved. The conventional data acquisition mode has two main types, the first type is to count through a spreadsheet and then upload the spreadsheet to a server for data acquisition for a period of time, and the data acquired by the conventional data acquisition mode has poor timeliness. The second category is that the acquisition of real-time data is realized by adding a camera and scanning hardware, and the method has the advantages of quick timeliness, but has obvious defects that firstly, the purchase cost of the hardware needs to be increased, secondly, the development workload needs to be increased, and the purchased hardware needs to be accessed to a platform.
In summary, the applet end is adopted to collect data, and the mobility and the real-time performance of the data template can be realized in the data template creating stage. In the data acquisition stage, the real-time data acquisition can be realized, and extra hardware purchasing cost and hardware access development cost are not required to be added.
Since the data uploaded to the platform at the same time is more than that uploaded to the platform, and may be from a plurality of different data templates, in the embodiment of the present invention, in order to determine from which data template the collected data is, when the data acquired by the platform is the data imported through the spreadsheet, the data template is selected so as to correctly store the collected data in the original database.
And for the data acquired in real time by the appointed two-dimensional code, the platform can identify the data template when the mobile terminal identifies the two-dimensional code.
When data is collected, in order to prevent data omission, the data collection process can be divided into two steps, the first step is to upload data to the platform, and the second step is to analyze the data, so that data loss or omission caused by overlarge calculation amount when the data is uploaded and analyzed is avoided.
And step 117, migrating the collected data from the original database to the database corresponding to the target data template according to the selected data template or the identified data template.
And step 119, verifying whether the data transferred to the database corresponding to the target data template is complete, and if the data transferred to the database corresponding to the target data template is complete, finishing data acquisition.
And 121, if the data migrated to the database corresponding to the target data template is incomplete, returning to the original database again for data synchronization, and returning to the step of migrating the acquired data from the original database to the database corresponding to the target data template.
To sum up, the method provided by the embodiment of the invention includes a mobile terminal for realizing remote configuration of a data template and real-time data acquisition, and a platform responsible for adding basic elements of the data template, configuring and updating the data template, or migrating and verifying the integrity of acquired data in the data acquisition process. The mobile terminal is used for realizing real-time data acquisition and remote configuration of the data template, realizing the mobility of the data template and the real-time data acquisition, realizing the creation of the data acquisition template at any place through the mobile terminal, and acquiring required data in real time through the mobile terminal. The data template can be quickly created by the user only by checking the basic elements of the created data template through the elements of the data template created by the platform, the difficulty of creating the data template is reduced, the scheme of updating the data template is realized, and after the data template is updated, the data collected by the old data template is transferred to the database corresponding to the new data template. In addition, the integrity of the data acquired by the data template is ensured through a mechanism for verifying the integrity of the data.
According to the method provided by the embodiment of the invention, the data template creation can be realized by freely combining the data template basic elements through creating the data template basic elements in advance and then checking the data template elements through a small program or a platform, so that the mobility of the created data template is realized, the data template can be quickly created when the data template is changed, and the creation efficiency of the data template is improved. In addition, the efficiency of importing data is improved by creating the data template. Data acquisition is carried out through the mobile terminal, and real-time data acquisition is achieved. By adding the scheme of modifying the data template, the migration of the historical data of the data template to the database of the new data template is realized. By adding a mechanism for verifying the integrity of data in the process of loading the original data into the database of the corresponding data template, the integrity of the data acquired by the data template is ensured. The invention has the advantages of flexible adaptation to the requirements of enterprises, lower maintenance cost and the like.
The embodiment of the invention provides a data acquisition method which can be executed by any device, equipment, platform or equipment cluster with calculation and processing capabilities. As shown in fig. 2, the method may include the steps of:
And step 203, if the new data template is determined to need to be created, judging whether the new data template basic element needs to be created.
And step 205, if a new data template basic element needs to be created, creating the new data template basic element according to the data acquisition requirement.
And step 207, judging whether the data template is created through the platform or the mobile terminal.
And 209, if the data template is determined to be created through the platform, configuring preset data template basic elements to generate a target data template.
And step 211, if the data template is determined to be created through the mobile terminal, acquiring a target data template generated by configuring a preset data template basic element by the mobile terminal.
And step 213, generating a designated spreadsheet for the platform to perform batch data acquisition and a designated two-dimensional code for the mobile terminal to perform real-time data acquisition according to the target data template.
And 223, verifying whether the data transferred to the database corresponding to the target data template is complete, and if the data transferred to the database corresponding to the target data template is complete, finishing data acquisition.
And 225, if the data migrated to the database corresponding to the target data template is incomplete, returning to the original database again for data synchronization, and returning to the step of migrating the acquired data from the original database to the database corresponding to the target data template.
According to the method provided by the embodiment of the invention, the data template can be created by creating the basic elements of the data template in advance, then selecting the basic elements of the data template through a small program or a platform and freely combining the basic elements of the data template, so that the mobility of the created data template is realized, the data template can be rapidly created when the data template is changed, and the creation efficiency of the data template is improved. In addition, the efficiency of importing data is improved by creating the data template. Data acquisition is carried out through the mobile terminal, and real-time data acquisition is achieved. By adding the scheme of modifying the data template, the migration of the historical data of the data template to the database of the new data template is realized. By adding a mechanism for verifying the integrity of data in the process of loading the original data into the database of the corresponding data template, the integrity of the data acquired by the data template is ensured. The invention has the advantages of flexible adaptation to the requirements of enterprises, low maintenance cost and the like.
Specifically, in step 115 or step 219 in the above embodiment, when analyzing the collected data and storing the collected data in the original database, the collected data may be analyzed into pieces of data, a data identifier of each piece of data is generated, and each piece of data and the data identifier thereof are stored in the original database. The data identifier is used for distinguishing each piece of data.
On this basis, when verifying whether the data migrated to the database corresponding to the target data template is complete, step 119 or step 223 may verify whether the data migrated to the database corresponding to the target data template includes the data identifier of each piece of acquired data, and if so, determine that the migrated data is complete; otherwise, the missing data is inquired and copied from the original database again according to the data identification.
Optionally, after migrating the acquired data from the original database to the database corresponding to the target data template, the method provided in the embodiment of the present invention may further include: filtering the acquired data according to elements in the target data template, and filtering data corresponding to elements which do not exist in the target data template; and filtering the data in the target data template according to the data identification, and filtering out repeated data. By the operation, the data in the database corresponding to the target data template can be ensured not to be leaked.
To illustrate the technical content of the present invention in detail, a series of scenarios for acquiring data are described below.
The production value information of a product A in the same day, specifically including the model, time, production place and SN information of the product, needs to be collected by a Mitsubishi factory singlechip board production line, a client collects data through a platform, and the data collection steps are as follows:
1.1, starting a data acquisition service by a platform;
1.2, judging that a new data template needs to be created by a platform;
1.3, creating model, time, production place and SN basic elements required by a data template on a platform;
1.4, selecting basic elements of the data template through a platform by a user, and generating a target data template;
1.5, generating a designated spreadsheet and a designated two-dimensional code according to the target data template;
1.6, clicking a specified electronic form generated by export by a user;
1.7, a user fills in the output value information of the product A on the current day, the model, the production place information and the production time of the product A are determined and are directly generated by the system, and the user only needs to fill in the SN information of the product;
1.8, importing the filled specified electronic form of the production value information into a platform;
1.9, judging that the acquired data is imported data through a specified electronic form by the platform, and then selecting a corresponding data template of the acquired data;
1.10, analyzing the acquired data, and storing the acquired data in an original database;
1.11, updating the acquired data from an original database to a database corresponding to a target data template according to the selected data template;
1.12, verifying whether the data updated to the database corresponding to the target data template is complete, and if the data is incomplete, synchronizing the data in the original database again;
and 1.13, if the data is complete, ending the data acquisition of the current day.
The production line of the beer machine in the Mitsubishi factory needs to collect the output value information of the beer machine on the day, specifically comprising the model, time, production place and SN information of the product, a client collects data in real time through a small program end, and the data collection steps are as follows:
2.1, starting a data acquisition service by the platform;
2.2, judging that a new data template needs to be created by the platform;
2.3, creating model, time, production place and SN basic elements required by the data template on the platform;
2.4, the user selects basic elements of the data template through the small program and generates a target data template;
2.5, the platform generates a designated spreadsheet and a designated two-dimensional code according to the target data template;
2.6, printing the specified two-dimensional code by a user and pasting the two-dimensional code on a production line;
2.7, scanning the appointed two-dimensional code by a user to acquire data, popping up an acquired information page after scanning the appointed two-dimensional code, determining the model, the production place information and the production time of the acquired information page, and directly generating the acquired information page by the system, wherein the user only needs to fill in SN information of a product;
2.9, the platform judges that the acquired data is acquired by the small program end in real time without selecting a data template for acquiring the data;
2.10, storing the acquired data into a database of original data;
2.11, updating the data from the original database to a database corresponding to the target data template;
2.12, verifying whether the data updated to the database corresponding to the target data template is complete, and if not, synchronizing the data in the original database again;
and 2.13, if the data is complete, ending the data acquisition of the current day.
An embodiment of the present invention provides a data acquisition apparatus, and fig. 3 shows a schematic block diagram of the data acquisition apparatus according to an embodiment. It is understood that the data collection device may be any device, apparatus, platform or cluster of devices having computing and processing capabilities. As shown in fig. 3, the data acquisition apparatus 300 includes:
the first judging unit 301 is configured to judge whether a new data template needs to be created after the platform detects that the data acquisition service is started;
a second judging unit 303 configured to judge whether to create the data template through the platform or the mobile terminal if it is determined that a new data template needs to be created;
a template configuration unit 305 configured to configure a preset data template basic element to generate a target data template if it is determined that the data template is created by the platform; if the data template is determined to be created through the mobile terminal, acquiring a target data template generated by configuring basic elements of a preset data template through the mobile terminal;
the generating unit 307 is configured to generate a designated spreadsheet for batch data acquisition of the platform and a designated two-dimensional code for real-time data acquisition of the mobile terminal according to the target data template;
a data acquisition and determination unit 309 configured to acquire the acquired data and determine whether the acquired data is data imported through a designated spreadsheet or data acquired in real time through a designated two-dimensional code;
a data template selecting unit 311 configured to select a data template if it is determined to be data imported by specifying the electronic form; if the data acquired in real time through the specified two-dimensional code is determined, identifying a data template;
a data parsing and storing unit 313 configured to parse the collected data and store the collected data to an original database;
a data migration unit 315 configured to migrate the acquired data from the original database to a database corresponding to the target data template according to the selected data template or the identified data template;
and the data verification unit 317 is configured to verify whether the data migrated to the database corresponding to the target data template is complete, and complete data acquisition if the data migrated to the database corresponding to the target data template is complete.
According to the device provided by the embodiment of the invention, the data template can be created by freely combining the basic elements of the data template through pre-creating the basic elements of the data template and then checking the basic elements of the data template through a small program or a platform, so that the mobility of the created data template is realized, the data template can be rapidly created when the data template is changed, and the creation efficiency of the data template is improved. In addition, the efficiency of importing data is improved by creating the data template. Data acquisition is carried out through the mobile terminal, and real-time data acquisition is achieved. By adding the scheme of modifying the data template, the migration of the historical data of the data template to the database of the new data template is realized. By adding a mechanism for verifying the integrity of data in the process of loading the original data into the database of the corresponding data template, the integrity of data acquired by the data template is ensured. The invention has the advantages of flexible adaptation to the requirements of enterprises, low maintenance cost and the like.
Optionally, the data acquisition device further comprises:
a third judging unit configured to judge whether a new data template basic element needs to be created if it is determined that a new data template needs to be created;
and the template basic element generating unit is configured to create a new data template basic element according to the data acquisition requirement if the new data template basic element needs to be created.
Optionally, the mobile terminal scans the designated two-dimensional code through an applet thereon to perform real-time data acquisition.
Optionally, the data acquisition apparatus further includes a data synchronization unit configured to, if the data migrated to the database corresponding to the target data template is incomplete, return to the original database again for data synchronization.
Optionally, the data parsing and storing unit 307 is configured to: analyzing the acquired data into various pieces of data, generating a data identifier of each piece of data, and storing each piece of data and the data identifier thereof in an original database.
Optionally, the data acquisition device further includes a filtering unit configured to filter the acquired data according to the elements in the target data template, and filter out data corresponding to elements that do not exist in the target data template; and filtering the data in the target data template according to the data identification, and filtering out repeated data.
Optionally, the data verification unit 309 is further configured to: verifying whether the data migrated to the database corresponding to the target data template contains the data identification of each piece of acquired data, and if so, determining that the migrated data is complete; otherwise, the missing data is inquired and copied from the original database again according to the data identification.
It should be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the data acquisition device. In other embodiments of the invention, the data acquisition device may include more or fewer components than illustrated, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Because the information interaction, execution process, and other contents between the units in the device are based on the same concept as the method embodiment of the present invention, specific contents may refer to the description in the method embodiment of the present invention, and are not described herein again.
An embodiment of the present invention further provides a data acquisition apparatus, as shown in fig. 4, including: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine readable program to perform the data collection method in any embodiment of the invention.
An embodiment of the present invention further provides a computer-readable medium, where computer instructions are stored on the computer-readable medium, and when executed by a processor, the computer instructions cause the processor to execute the data acquisition method in any embodiment of the present invention. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion unit is caused to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the embodiments described above.
It should be noted that not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
In the above embodiments, the hardware unit may be implemented mechanically or electrically. For example, a hardware element may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware elements may also comprise programmable logic or circuitry, such as a general purpose processor or other programmable processor, that may be temporarily configured by software to perform the corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.
While the invention has been shown and described in detail in the drawings and in the preferred embodiments, it is not intended to limit the invention to the embodiments disclosed, and it will be apparent to those skilled in the art that various combinations of the code auditing means in the various embodiments described above may be used to obtain further embodiments of the invention, which are also within the scope of the invention.
Claims (10)
1. A method of data acquisition, comprising:
when the platform detects that the data acquisition service is started, judging whether a new data template needs to be created or not;
if the new data template is determined to be created, judging whether the data template is created through the platform or the mobile terminal;
if the data template is established through the platform, configuring preset data template basic elements to generate a target data template; if the data template is determined to be created through the mobile terminal, acquiring a target data template generated by configuring basic elements of a preset data template through the mobile terminal;
generating a designated spreadsheet for data batch acquisition of the platform and a designated two-dimensional code for real-time data acquisition of the mobile terminal according to the target data template;
acquiring collected data, and judging whether the collected data is imported data through the specified spreadsheet or data collected in real time through the specified two-dimensional code;
selecting a data template if it is determined to be data imported through the specified spreadsheet; if the data acquired in real time through the specified two-dimensional code is determined, identifying a data template;
analyzing the collected data and storing the collected data in an original database;
according to the selected data template or the identified data template, migrating the acquired data from the original database to a database corresponding to the target data template;
and verifying whether the data transferred to the database corresponding to the target data template is complete, and finishing data acquisition if the data transferred to the database corresponding to the target data template is complete.
2. The method of claim 1, wherein after determining whether a new data template needs to be created, the method further comprises:
if the new data template is determined to need to be created, judging whether the basic elements of the new data template need to be created or not;
and if the new data template basic elements need to be created, creating the new data template basic elements according to the data acquisition requirements.
3. The method according to claim 1, wherein the mobile terminal scans the specified two-dimensional code through an applet thereon for real-time data acquisition.
4. The method according to claim 1, wherein after verifying whether the data migrated to the database corresponding to the target data template is complete, the method further comprises:
and if the data transferred to the database corresponding to the target data template is incomplete, returning to the original database again for data synchronization.
5. The method of claim 1, wherein parsing the collected data and saving the collected data to a raw database comprises:
analyzing the collected data into various pieces of data, generating a data identifier of each piece of data, and storing each piece of data and the data identifier thereof in an original database.
6. The method according to claim 5, wherein after the migrating the collected data from the original database to the database corresponding to the target data template, further comprising:
filtering the acquired data according to the elements in the target data template, and filtering out data corresponding to elements which do not exist in the target data template;
and filtering the data in the target data template according to the data identification, and filtering out repeated data.
7. The method of claim 5, wherein the verifying whether the data migrated to the database corresponding to the target data template is complete comprises:
verifying whether the data migrated to the database corresponding to the target data template contains the data identifier of each piece of acquired data, and if so, determining that the migrated data is complete; otherwise, the missing data is inquired and copied from the original database again according to the data identification.
8. A data acquisition device, comprising:
the first judgment unit is configured to judge whether a new data template needs to be created or not after the platform detects that the data acquisition service is started;
a second judging unit configured to judge whether to create the data template through the platform or the mobile terminal if it is determined that a new data template needs to be created;
the template configuration unit is configured to configure preset data template basic elements to generate a target data template if the data template is determined to be created through the platform; if the data template is established through the mobile terminal, acquiring a target data template generated by configuring basic elements of a preset data template through the mobile terminal;
the generating unit is configured to generate a specified spreadsheet for the platform to perform batch data acquisition and a specified two-dimensional code for the mobile terminal to perform real-time data acquisition according to the target data template;
the data acquisition and judgment unit is configured to acquire the acquired data and judge whether the acquired data is imported data through the specified spreadsheet or data acquired in real time through the specified two-dimensional code;
a data template selecting unit configured to select a data template if it is determined to be data imported through the specified electronic form; if the data acquired in real time through the specified two-dimensional code is determined, identifying a data template;
the data analysis and storage unit is configured to analyze the acquired data and store the acquired data in an original database;
the data migration unit is configured to migrate the acquired data from the original database to a database corresponding to the target data template according to the selected data template or the identified data template;
and the data verification unit is used for verifying whether the data transferred to the database corresponding to the target data template is complete or not, and finishing data acquisition if the data transferred to the database corresponding to the target data template is complete.
9. A data acquisition device, comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor configured to invoke the machine readable program to perform the method of any of claims 1 to 7.
10. A computer readable medium having stored thereon computer instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111207894.8A CN113934710B (en) | 2021-10-18 | 2021-10-18 | Data acquisition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111207894.8A CN113934710B (en) | 2021-10-18 | 2021-10-18 | Data acquisition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113934710A CN113934710A (en) | 2022-01-14 |
| CN113934710B true CN113934710B (en) | 2023-04-07 |
Family
ID=79280120
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111207894.8A Active CN113934710B (en) | 2021-10-18 | 2021-10-18 | Data acquisition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113934710B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115903681B (en) * | 2022-10-26 | 2023-10-24 | 江苏合信环境发展有限公司 | Sewage plant energy consumption data acquisition method and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106651407A (en) * | 2015-07-17 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Data acquisition method and apparatus |
| CN109829146A (en) * | 2019-01-16 | 2019-05-31 | 腾讯科技（深圳）有限公司 | A kind of information collection management method and device |
| CN110008420A (en) * | 2019-04-16 | 2019-07-12 | 秒针信息技术有限公司 | A kind of generation method and device of electronic spreadsheet |
| CN111914132A (en) * | 2020-08-20 | 2020-11-10 | 广东数智信息科技有限公司 | Method and system for real-time correlation of metrological verification calibration data information |
| CN111931708A (en) * | 2020-09-16 | 2020-11-13 | 北京维数统计事务所有限公司 | Form generation method and device |
| CN112668298A (en) * | 2021-01-15 | 2021-04-16 | 上海杉互健康科技有限公司 | Questionnaire recording method, system, equipment and storage medium based on mobile terminal |
| CN112767105A (en) * | 2021-01-14 | 2021-05-07 | 国网吉林省电力有限公司 | Power system industry and property informatization processing method based on two-dimensional code management platform |
-
2021
- 2021-10-18 CN CN202111207894.8A patent/CN113934710B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106651407A (en) * | 2015-07-17 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Data acquisition method and apparatus |
| CN112598434A (en) * | 2015-07-17 | 2021-04-02 | 阿里巴巴集团控股有限公司 | Data acquisition method and device and commodity release method |
| CN109829146A (en) * | 2019-01-16 | 2019-05-31 | 腾讯科技（深圳）有限公司 | A kind of information collection management method and device |
| CN110008420A (en) * | 2019-04-16 | 2019-07-12 | 秒针信息技术有限公司 | A kind of generation method and device of electronic spreadsheet |
| CN111914132A (en) * | 2020-08-20 | 2020-11-10 | 广东数智信息科技有限公司 | Method and system for real-time correlation of metrological verification calibration data information |
| CN111931708A (en) * | 2020-09-16 | 2020-11-13 | 北京维数统计事务所有限公司 | Form generation method and device |
| CN112767105A (en) * | 2021-01-14 | 2021-05-07 | 国网吉林省电力有限公司 | Power system industry and property informatization processing method based on two-dimensional code management platform |
| CN112668298A (en) * | 2021-01-15 | 2021-04-16 | 上海杉互健康科技有限公司 | Questionnaire recording method, system, equipment and storage medium based on mobile terminal |
Non-Patent Citations (4)
| Title |
|---|
| 张奇 ; .计量检测业务实验室综合管理平台的构建.计量技术.2019,(第12期),65-68. * |
| 杨红艳 ; .web表单在职业教育信息化教学中的应用.河南农业.2018,(第03期),33-34+36. * |
| 梁柯 ; 李秉毅 ; 陈柯 ; .二维码在通信标识资源管理系统中的应用.土木建筑与环境工程.2016,(第S1期),186-190. * |
| 赵磊 ; .基于手机二维码的低压供电业务管理系统应用.农村电工.2013,(第12期),34-35. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113934710A (en) | 2022-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108509392B (en) | Multi-mechanism account checking method, system, computer equipment and storage medium | |
| CN110275861B (en) | Data storage method and device, storage medium and electronic device | |
| CN103473672A (en) | System, method and platform for auditing metadata quality of enterprise-level data center | |
| CN104572122A (en) | Software application data generating device and method | |
| CN111400186A (en) | Performance test method and system | |
| CN112799782B (en) | Model generation system, method, electronic device and storage medium | |
| CN113360975A (en) | View construction method and device of building information model and storage medium | |
| CN113934710B (en) | Data acquisition method and device | |
| CN111193622B (en) | Asymmetric configuration management method, device and equipment and readable storage medium | |
| CN112559525B (en) | Data checking system, method, device and server | |
| CN112241625A (en) | Method and system for automatically generating contract document based on java language | |
| CN113656897A (en) | Intelligent design method, system and device for central air conditioner | |
| CN113868219A (en) | Method and device for migrating account set data, electronic equipment and computer storage medium | |
| CN113064627B (en) | Service access data processing method, platform, terminal, equipment and system | |
| CN113467782B (en) | Method, device and equipment for determining business process | |
| CN115048359A (en) | Universal data processing architecture design method for multi-type access data sources | |
| CN110058845A (en) | A kind of management method, equipment and the system of model component file | |
| CN113741970B (en) | Method and device for realizing separation of data warehouse production environment and development environment | |
| CN116560722B (en) | Operation and maintenance flow processing method and device, electronic equipment and storage medium | |
| CN111522779B (en) | Configuration processing method for multi-system file interaction | |
| CN111158749B (en) | Software version replacement method, device, equipment and readable storage medium | |
| CN116204509A (en) | Data access script generation method and device, electronic equipment and storage medium | |
| CN116841605A (en) | Method for acquiring server asset data and server asset data system | |
| CN117420991A (en) | Code generation method and device, storage medium and electronic device | |
| CN115907982A (en) | System for generating dynamic transaction data in electric power transaction market |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |