CN110515955B - Data storage and query method and system, electronic equipment and storage medium - Google Patents
Data storage and query method and system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN110515955B CN110515955B CN201910821324.4A CN201910821324A CN110515955B CN 110515955 B CN110515955 B CN 110515955B CN 201910821324 A CN201910821324 A CN 201910821324A CN 110515955 B CN110515955 B CN 110515955B
- Authority
- CN
- China
- Prior art keywords
- data
- production test
- production
- module
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data storage and query method, a system, electronic equipment and a storage medium, wherein the control method comprises the steps of obtaining production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced; acquiring characterization data for characterizing the production test data; and storing the characterization data into a local database in the production test station in sequence according to time sequence. The invention greatly improves the stability of data transmission, shortens the production test time of each module and improves the production test efficiency of the module; acquiring locally stored production test data according to the characterization data of the local SQLite database and uploading the locally stored production test data to the MongoDB in an original format, so that the consistency of data access is ensured; in addition, the MongoDB, the production database and the abstract log database are respectively inquired through the product serial numbers to obtain the required data, and the method has the advantages of high inquiry speed and high efficiency.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a method and a system for storing and querying data, an electronic device, and a storage medium.
Background
MES (manufacturing process execution system) factories generate a large number of data files in the process of producing products (such as wireless communication modules), such as 2 million Log files (journal files) of 300B-4KB, 100 million module parameter files of 60KB-70KB and the like, which are reserved for at least 5 years, and related industries such as vehicle-mounted and the like require at least 10 years.
At present, log files and module parameter files are firstly stored locally through production test tools of all production test stations, and then related data files are synchronously classified and uploaded to an FTP server through an FTP protocol (file transfer protocol), so long as the production test tools can continue to produce and test the next module after the uploading of the current module data is completed; when the searching is needed, the FTP server is required to be logged in, the directory layer by layer screening searching target files are saved according to a set tree shape, and then the target files are downloaded for viewing.
The above-mentioned manner of storing the production test data of the wireless communication module in the production process has the following problems:
1) When the related data is synchronously uploaded to the FTP, if the uploading failure is the test failure, the test is required to be retested, and the test efficiency is reduced or even the line is stopped due to factors such as unstable network or FTP server problem in the process, so that the productivity is seriously influenced; or the uploading is failed without retesting, and thus the data file is easy to lose;
2) With each file upload taking 100ms to calculate, 2 million files total 55H per day, thus accumulating losses of capacity;
3) The massive small files are stored in the FTP server, so that the inquiry and the retrieval are quite difficult, the catalogue is easy to fail, and the inquiry efficiency is low;
4) The massive small files are stored in the FTP server, so that backup and transfer are inconvenient;
5) When the service is in a peak, the requirement on the FTP server is higher, and at the moment, the blocking phenomenon is easy to occur, so that the uploading time is prolonged, and even the condition of batch uploading failure is caused.
Disclosure of Invention
The invention aims to solve the technical problems that in the prior art, a mode of storing production test data of a wireless communication module in a production process is easy to fail in transmission, the test efficiency is reduced, and the productivity is seriously influenced; and the defects of easy failure of query, low query efficiency and the like exist, and the method, the system, the electronic equipment and the storage medium for storing and querying the data are provided.
The invention solves the technical problems by the following technical scheme:
the invention provides a control method for data storage in a product production process, which comprises the following steps:
s1, acquiring production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced;
S2, obtaining characterization data used for characterizing the production test data;
wherein, the data volume corresponding to the characterization data is smaller than the data volume corresponding to the production test data;
s3, storing the characterization data into a local database in the production test station in sequence according to a time sequence.
Preferably, when the local data transfer module in the production test station is the local data transfer module, step S3 includes:
s31, storing the characterization data to the local data transfer module;
s32, acquiring a data storage request;
s33, storing the characterization data in the local data transfer module into the local database in the production test station according to the data storage request.
Preferably, after step S1 and before step S2, the method further comprises:
s11, storing the production test data into a local storage module in the production test station;
the step S3 further comprises the following steps:
s4, when the local data transfer module starts a data transmission thread, the characterization data in the local database are sequentially read according to a time sequence;
s5, acquiring corresponding production test data in the local storage module according to the characterization data;
S6, sequentially uploading the acquired production test data to a file database according to a time sequence.
Preferably, the file database comprises MongoDB (database based on distributed file storage);
when the production test data is binary format data, step S6 includes:
and sequentially uploading the production test data in the binary format to a MongoDB according to the time sequence.
Preferably, the control method further includes:
uploading production test tool data corresponding to the production test tools in the production test station to a production database;
uploading abstract log data generated by the production test station in the production process to an abstract log database;
wherein the production database and the summary log database are both relational databases; and/or the number of the groups of groups,
the local database comprises the SQLite database (a lightweight database).
Preferably, step S3 further comprises:
after the current product is produced, controlling a production test tool at the production test station to continuously produce the next product, and re-executing the step S1; and/or the number of the groups of groups,
when the product comprises a wireless communication module, the production test data comprise module test data, radio frequency parameters of the module, station position data, a product serial number, a data storage path and timestamp data;
The characterization data includes a product serial number, a data storage path, and time stamp data.
The invention also provides a data query method, which is realized by adopting the control method for data storage in the production process of the product, and comprises the following steps:
obtaining target characterization data corresponding to a target product;
and inquiring the target characterization data to obtain corresponding target production test data in the file database.
Preferably, after uploading the production test tool data corresponding to the production test tools in the production test station to a production database and uploading the summary log data generated in the production process of the production test station to the summary log database, the data query method further comprises:
and inquiring from the production database and the summary log database according to the target characterization data to obtain corresponding target production test tool data and target summary log data.
The invention also provides a control system for data storage in the production process of the product, which comprises a production test data acquisition module, a characterization data acquisition module and a first storage module;
The production test data acquisition module is used for acquiring production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced;
the characterization data acquisition module is used for acquiring characterization data used for characterizing the production test data;
wherein, the data volume corresponding to the characterization data is smaller than the data volume corresponding to the production test data;
the first storage module is used for sequentially storing the characterization data into a local database in the production test station according to the time sequence.
Preferably, when the local data transfer module in the production test station is a local data transfer module, the first storage module comprises a transfer storage unit, a request acquisition unit and a local storage unit;
the transfer storage unit is used for storing the characterization data to the local data transfer module;
the request acquisition unit is used for acquiring a data storage request;
the local storage unit is used for storing the characterization data in the local data transfer module into the local database in the production test station according to the data storage request.
Preferably, the control system further comprises a second storage module and a third storage module;
The second storage module is used for storing the production test data into a local storage module in the production test station;
the characterization data acquisition module is further used for sequentially reading the characterization data in the local database according to a time sequence when the local data transfer module starts a data transmission thread;
the production test data acquisition module is further used for acquiring the corresponding production test data in the local storage module according to the characterization data;
and the third storage module is used for sequentially uploading the acquired production test data to a file database according to the time sequence.
Preferably, the file database comprises a MongoDB;
and when the production test data are binary format data, the third storage module is used for sequentially uploading the binary format production test data into the MongoDB according to a time sequence.
Preferably, the control system further comprises a fourth storage module and a fifth storage module;
the fourth storage module is used for uploading production test tool data corresponding to the production test tools in the production test station to a production database;
The fifth storage module is used for uploading abstract log data generated by the production test station in the production process to an abstract log database;
wherein the production database and the summary log database are both relational databases; and/or the number of the groups of groups,
the local database comprises a SQLite database.
Preferably, the control system further comprises a control module;
the control module is used for controlling the production test tool of the production test station to continue producing the next product after the current product is produced, and recalling the production test data acquisition module; and/or the number of the groups of groups,
when the product comprises a wireless communication module, the production test data comprise module test data, radio frequency parameters of the module, station position data, a product serial number, a data storage path and timestamp data;
the characterization data includes a product serial number, a data storage path, and time stamp data.
The invention also provides a data query system which is realized by adopting the control system for data storage in the production process of the product, and comprises a target data acquisition module and a query module;
the target data acquisition module is used for acquiring target characterization data corresponding to a target product;
And the query module is used for querying the target characterization data to obtain corresponding target production test data in the file database.
Preferably, after uploading production test tool data corresponding to the production test tools in the production test station to a production database and uploading summary log data generated in the production process of the production test station to the summary log database, the query module is further configured to query the production database and the summary log database for corresponding target production test tool data and target summary log data according to the target characterization data.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the control method of data storage in the production process of the product and realizes the data query method when executing the computer program.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the control method for data storage in a product manufacturing process described above, and implements the steps of the data query method described above.
The invention has the positive progress effects that:
in the invention, the characterization data (product serial number, data storage path and the like) corresponding to the production test data generated by different time stamps in the process of producing the wireless communication module by the production test station are uniformly stored into the local SQLite database through the local transfer storage module, so that the production test station can continue to produce and test the next module after the current product is produced, the process belongs to local data interaction and does not involve a network, thereby greatly improving the stability of data transmission, shortening the production test time of each module and improving the production test efficiency of the module; acquiring locally stored production test data according to the characterization data of the local SQLite database and uploading the locally stored production test data to the MongoDB in an original format, so that the consistency of data access is ensured; in addition, the MongoDB, the production database and the summary log database can be respectively queried through the product serial numbers to obtain the required data, so that the query speed is high and the query efficiency is high.
Drawings
Fig. 1 is a flowchart of a control method of data storage in the production process of the product of embodiment 1 of the present invention.
Fig. 2 is a flowchart of a control method of data storage in the production process of the product of embodiment 2 of the present invention.
Fig. 3 is a flowchart of a data query method in embodiment 3 of the present invention.
Fig. 4 is a schematic block diagram of a control system for data storage in the production process of the product according to embodiment 4 of the present invention.
FIG. 5 is a schematic block diagram of a control system for data storage in the production process of the product according to embodiment 5 of the present invention.
Fig. 6 is a schematic block diagram of a data query system according to embodiment 6 of the present invention.
Fig. 7 is a flow chart of the structure of an electronic device for implementing the control method of data storage in the production process of a product according to embodiment 7 of the present invention.
Detailed Description
The invention is further illustrated by means of the following examples, which are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, the control method for data storage in the product production process of the present embodiment includes:
s101, acquiring production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced;
s102, obtaining characterization data for characterizing production test data;
the data quantity corresponding to the characterization data is smaller than the data quantity corresponding to the production test data;
specifically, when the product includes a wireless communication module, the production test data includes, but is not limited to, module test data, radio frequency parameters of the module, station data, product serial number, data storage path, and time stamp data;
The characterization data includes a product serial number, a data storage path, and time stamp data, i.e., the characterization data is index information corresponding to the production test data.
And S103, sequentially storing the characterization data into a local database in the production test station according to the time sequence.
The local database comprises an SQLite database.
In this embodiment: 1) The data storage process does not depend on a network, belongs to local data interaction, and synchronizes the data and independently produces test tools, so that the stability of data transmission is greatly improved, and the conditions of test failure or data loss and even line stoppage caused by unstable network or FTP server problem are effectively avoided; the time of the existing synchronous data is saved, and the productivity is effectively increased; 2) The data are not required to be sent in real time, the characterization data corresponding to the production test tools of each production test station are respectively transmitted in real time and uniformly stored in a local SQLite database, and the data storage mode can ensure the timely and rapid storage of the data of each module and can avoid the condition of data loss; 3) The production test process of the next module is not delayed, and the next module can be directly and continuously produced and tested after the current product is produced, so that the production test time of each module is shortened, and the production test efficiency of producing the wireless communication module is improved.
Example 2
As shown in fig. 2, the control method of data storage in the product production process of this embodiment is a further improvement of embodiment 1, specifically:
after step S101, before step S102, the method further comprises:
s1011, storing the production test data into a local storage module in a production test station;
when the local data transfer module in the test station is produced, step S3 includes:
s1031, storing the characterization data to a local data transfer module;
s1032, obtaining a data storage request;
s1033, storing the characterization data in the local data transfer module into a local database in the production test station according to the data storage request.
During the service peak period, the local data transfer module can effectively cache the characterization data and transmit the characterization data to the SQLite database at regular time.
When the uploading fails, the method can automatically retransmit until the transmission is successful, and inform relevant management staff to check the problem, thereby effectively avoiding the occurrence of the condition of affecting the normal production process.
Step S103 further includes:
after the current product is produced, the production test tool of the production test station is controlled to continue to produce the next product, and step S101 is re-executed.
Step S103 further includes:
s104, when the local data transfer module starts a data transmission thread, the characterization data in the local database are sequentially read according to the time sequence;
s105, acquiring corresponding production test data in the local storage module according to the characterization data;
s106, sequentially uploading the acquired production test data to a file database according to the time sequence.
The file database comprises MongoDB, small file data generated in the production test process of the wireless communication module are uniformly stored by the file database, the backup transfer efficiency can be greatly improved, and the safety is higher.
Specifically, for each production test station, a data storage request is sent to a local data transfer module at regular time by setting a production test tool in each production test station, and the local data transfer module sequentially sends the stored characterization data to an SQLite database for unified storage according to the data storage request.
When the local data transfer module starts a data transmission thread, the characterization data are sequentially read from the SQLite database according to the time sequence, corresponding production test data are sequentially obtained from the local storage module of the production test station according to the data storage path and the time stamp information corresponding to the characterization data, and the corresponding production test data are uploaded to the MongoDB.
When the production test data is binary format data, step S106 specifically includes:
and the production test data in the binary format is sequentially stored and uploaded to the MongoDB according to the time sequence, namely, the original format of the data is stored in the MongoDB, so that the consistency of the data is not changed in access operation, the lossless storage is realized, and the data storage requirements of various formats are met. For example, excel format files corresponding to module test data, compressed format files corresponding to radio frequency parameters of the module, TXT format text corresponding to a production tool, and the like. In addition, the different text types corresponding to various data are stored in a distinguishing mode.
In addition, the control method of the present embodiment further includes:
uploading production test tool data corresponding to the production test tools in the production test station to a production database;
uploading abstract log data generated in the production process of the production test station to an abstract log database;
the production database and the summary log database are both relational databases, such as SQL Server databases.
In the embodiment, the characterization data (the product serial number, the data storage path and the like) corresponding to the production test data generated by different time stamps in the process of producing the wireless communication module are uniformly stored into the local SQLite database through the local transfer storage module, so that the production test station can continue to produce and test the next module after the current product is produced, the process belongs to local data interaction and does not involve a network, the stability of data transmission is greatly improved, the production test duration of each module is shortened, and the production test efficiency of the module is improved; and acquiring the production test data stored locally according to the characterization data of the local SQLite database and uploading the production test data to the MongoDB in the original format of the production test data, so that the consistency of data access is ensured.
Example 3
The data query method of this embodiment is implemented by using the control method for data storage in the product production process of embodiment 2.
As shown in fig. 3, the data query method of the present embodiment includes:
s201, obtaining target characterization data corresponding to a target product;
s202, inquiring target characterization data to obtain corresponding target production test data in a file database.
The target characterization data comprise target product serial numbers corresponding to target products, and original file data in the file database can be directly read by adopting the target product serial numbers. A data storage path and time stamp data may also be included.
After uploading the production test tool data corresponding to the production test tools in the production test station to the production database and uploading the summary log data generated by the production test station in the production process to the summary log database, the data query method further comprises the following steps:
s203, respectively inquiring from the production database and the abstract log database according to the target characterization data to obtain corresponding target production test tool data and target abstract log data.
Specifically, the production line of the MES factory is provided with a plurality of production test stations, and each production test station is provided with a plurality of PCs (personal computers) for production test. Each PC machine is provided with a production test tool and a local data transfer module for data collection; the local data transfer module uploads module data files and Log files generated by a local production test tool to a file database in a server at regular time, meanwhile, the production test tool also stores some parameter data reflecting the production test tool in the production database according to production requirements, and uploads summary Log data generated in the production test process to the summary Log database, wherein the production database and the summary Log database are both relational databases, and the file database adopts a non-relational document database (MongoDB).
The production database is mainly used for flow control in the production process, the summary log database is mainly used for quick inquiry and simple analysis after production, the data file database is mainly used for investigation and data storage of detailed problems, and the three databases are combined to efficiently realize control, monitoring, analysis and decision of the whole production process.
In this embodiment, the MongoDB, the production database and the summary log database may be queried respectively by the product serial numbers to obtain the required data, and the data storage architecture may greatly improve the efficiency of data retrieval and analysis statistics, for example, query all test conditions of a certain module, all test conditions of a certain work order, test conditions of a certain station, etc., and has advantages of fast query speed and high efficiency.
Example 4
As shown in fig. 4, the control system for data storage in the production process of the product of the present embodiment includes a production test data acquisition module 1, a characterization data acquisition module 2, and a first storage module 3.
The production test data acquisition module 1 is used for acquiring production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced;
the characterization data acquisition module 2 is used for acquiring characterization data for characterizing production test data;
The data quantity corresponding to the characterization data is smaller than the data quantity corresponding to the production test data;
specifically, when the product includes a wireless communication module, the production test data includes, but is not limited to, module test data, radio frequency parameters of the module, station data, product serial number, data storage path, and time stamp data;
the characterization data includes a product serial number, a data storage path, and time stamp data, i.e., the characterization data is index information corresponding to the production test data.
The first storage module 3 is used for sequentially storing the characterization data into a local database in the production test station according to the time sequence.
The local database comprises an SQLite database.
In this embodiment: 1) The data storage process does not depend on a network, belongs to local data interaction, and synchronizes the data and independently produces test tools, so that the stability of data transmission is greatly improved, and the conditions of test failure or data loss and even line stoppage caused by unstable network or FTP server problem are effectively avoided; the time of the existing synchronous data is saved, and the productivity is effectively increased; 2) The data are not required to be sent in real time, the characterization data corresponding to the production test tools of each production test station are respectively transmitted in real time and uniformly stored in a local SQLite database, and the data storage mode can ensure the timely and rapid storage of the data of each module and can avoid the condition of data loss; 3) The production test process of the next module is not delayed, and the next module can be directly and continuously produced and tested after the current product is produced, so that the production test time of each module is shortened, and the production test efficiency of producing the wireless communication module is improved.
Example 5
As shown in fig. 5, the control system for data storage in the product production process of this embodiment is a further improvement of embodiment 4, specifically:
when producing the local data transfer module in the test station, the first storage module 3 comprises a transfer storage unit 4, a request acquisition unit 5 and a local storage unit 6.
The relay storage unit 4 is used for storing the characterization data to the local data relay module.
The request acquisition unit 5 is used for acquiring a data storage request;
the local storage unit 6 is used for storing the characterization data in the local data transfer module into a local database in the production test station according to the data storage request.
During the service peak period, the local data transfer module can effectively cache the characterization data and transmit the characterization data to the SQLite database at regular time.
When the uploading fails, the method can automatically retransmit until the transmission is successful, and inform relevant management staff to check the problem, thereby effectively avoiding the occurrence of the condition of affecting the normal production process.
The control system further comprises a control module 7, wherein the control module 7 is used for controlling the production test tool of the production test station to continue producing the next product after the current product is produced, and recalling the production test data acquisition module 1.
The control system further comprises a second memory module 8 and a third memory module 9.
The second storage module 8 is used for storing the production test data into a local storage module in the production test station;
the characterization data acquisition module 2 is further configured to sequentially read the characterization data in the local database according to the time sequence when the local data transfer module starts a data transmission thread;
the production test data acquisition module 1 is further configured to acquire corresponding production test data in the local storage module according to the characterization data.
The third storage module 9 is configured to sequentially upload the acquired production test data to a file database according to a time sequence.
The file database comprises MongoDB, small file data generated in the production test process of the wireless communication module are uniformly stored by the file database, the backup transfer efficiency can be greatly improved, and the safety is higher.
Specifically, for each production test station, a data storage request is sent to a local data transfer module at regular time by setting a production test tool in each production test station, and the local data transfer module sequentially sends the stored characterization data to an SQLite database for unified storage according to the data storage request.
When the local data transfer module starts a data transmission thread, the characterization data are sequentially read from the SQLite database according to the time sequence, corresponding production test data are sequentially obtained from the local storage module of the production test station according to the data storage path and the time stamp information corresponding to the characterization data, and the corresponding production test data are uploaded to the MongoDB.
When the production test data are binary format data, the third storage module is used for sequentially uploading the binary format production test data to the MongoDB according to time sequence, namely, the original format of the data is stored in the MongoDB, so that the consistency of the data is not changed in access operation, lossless storage is realized, and the data storage requirements of various formats are met. For example, excel format files corresponding to module test data, compressed format files corresponding to radio frequency parameters of the module, TXT format text corresponding to a production tool, and the like. In addition, the different text types corresponding to various data are stored in a distinguishing mode.
In addition, the control system further comprises a fourth memory module 10 and a fifth memory module 11;
the fourth storage module 10 is configured to upload production test tool data corresponding to a production test tool in the production test station to the production database;
The fifth storage module 11 is used for uploading abstract log data generated by the production test station in the production process to an abstract log database;
the production database and the summary log database are both relational databases, such as SQL Server databases.
In the embodiment, the characterization data corresponding to the production test data generated by different time stamps in the process of producing the wireless communication module by the production test station are uniformly stored into the local SQLite database through the local transfer storage module, so that the production test station can continue to produce and test the next module after the current product is produced, the process belongs to local data interaction and does not involve a network, the stability of data transmission is greatly improved, the production test duration of each module is shortened, and the production test efficiency of the module is improved; and acquiring the production test data stored locally according to the characterization data of the local SQLite database and uploading the production test data to the MongoDB in the original format of the production test data, so that the consistency of data access is ensured.
Example 6
As shown in fig. 6, the data query system of this embodiment is implemented by using the control system for data storage in the product manufacturing process of embodiment 5, and the data query system includes a target data acquisition module 12 and a query module 13.
The target data acquisition module 12 is used for acquiring target characterization data corresponding to a target product;
the query module 13 is configured to query the target characterization data to obtain corresponding target production test data in the file database.
The target characterization data comprise target product serial numbers corresponding to target products, and original file data in the file database can be directly read by adopting the target product serial numbers. A data storage path and time stamp data may also be included.
After uploading the production test tool data corresponding to the production test tools in the production test station to the production database and uploading the summary log data generated by the production test station in the production process to the summary log database, the query module 13 is further configured to query from the production database and the summary log database according to the target characterization data to obtain corresponding target production test tool data and target summary log data.
Specifically, a plurality of production test stations are arranged on a production line of the MES factory, and each production test station is provided with a plurality of PCs for production test. Each PC machine is provided with a production test tool and a local data transfer module for data collection; the local data transfer module uploads module data files and Log files generated by a local production test tool to a file database in a server at regular time, meanwhile, the production test tool also stores some parameter data reflecting the production test tool in the production database according to production requirements, and uploads summary Log data generated in the production test process to the summary Log database, wherein the production database and the summary Log database are both relational databases, and the file database adopts a non-relational document database (MongoDB).
The production database is mainly used for flow control in the production process, the summary log database is mainly used for quick inquiry and simple analysis after production, the data file database is mainly used for investigation and data storage of detailed problems, and the three databases are combined to efficiently realize control, monitoring, analysis and decision of the whole production process.
In this embodiment, the MongoDB, the production database and the summary log database may be queried respectively by the product serial numbers to obtain the required data, and the data storage architecture may greatly improve the efficiency of data retrieval and analysis statistics, for example, query all test conditions of a certain module, all test conditions of a certain work order, test conditions of a certain station, etc., and has advantages of fast query speed and high efficiency.
Example 7
Fig. 7 is a schematic structural diagram of an electronic device according to embodiment 7 of the present invention. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the control method of data storage in the production process of the article in any one of embodiments 1 or 2 when executing the program. The electronic device 30 shown in fig. 7 is only an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 7, the electronic device 30 may be embodied in the form of a general purpose computing device, which may be a server device, for example. Components of electronic device 30 may include, but are not limited to: the at least one processor 31, the at least one memory 32, a bus 33 connecting the different system components, including the memory 32 and the processor 31.
The bus 33 includes a data bus, an address bus, and a control bus.
Memory 32 may include volatile memory such as Random Access Memory (RAM) 321 and/or cache memory 322, and may further include Read Only Memory (ROM) 323.
Memory 32 may also include a program/utility 325 having a set (at least one) of program modules 324, such program modules 324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The processor 31 executes various functional applications and data processing, such as a control method of data storage in the production process of the product in any of embodiments 1 or 2 of the present invention, by running a computer program stored in the memory 32.
The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). Such communication may be through an input/output (I/O) interface 35. Also, model-generating device 30 may also communicate with one or more networks, such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet, via network adapter 36. As shown in fig. 7, network adapter 36 communicates with the other modules of model-generating device 30 via bus 33. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in connection with the model-generating device 30, including, but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, data backup storage systems, and the like.
It should be noted that although several units/modules or sub-units/modules of an electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more units/modules described above may be embodied in one unit/module in accordance with embodiments of the present invention. Conversely, the features and functions of one unit/module described above may be further divided into ones that are embodied by a plurality of units/modules.
Example 8
An embodiment 8 of the present invention provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the data query method in embodiment 3 when executing the program, and the specific structure of the electronic device refers to the electronic device in embodiment 7, and the working principle of the electronic device is basically identical to that of the electronic device in embodiment 7, and is not described herein again.
Example 9
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps in the control method of data storage in the production process of a product in any one of embodiments 1 or 2.
More specifically, among others, readable storage media may be employed including, but not limited to: portable disk, hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In a possible implementation manner, the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps of a control method implementing the data storage in the production process of the product in any of embodiments 1 or 2, when said program product is run on the terminal device.
Wherein the program code for carrying out the invention may be written in any combination of one or more programming languages, which program code may execute entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on the remote device or entirely on the remote device.
Example 10
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data query method in embodiment 3.
More specifically, among others, readable storage media may be employed including, but not limited to: portable disk, hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In a possible embodiment, the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps of implementing the data query method in embodiment 3, when said program product is run on the terminal device.
Wherein the program code for carrying out the invention may be written in any combination of one or more programming languages, which program code may execute entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on the remote device or entirely on the remote device.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.
Claims (14)
1. A method for controlling data storage in a product manufacturing process, the method comprising:
s1, acquiring production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced;
s2, obtaining characterization data used for characterizing the production test data;
wherein, the data volume corresponding to the characterization data is smaller than the data volume corresponding to the production test data;
S3, sequentially storing the characterization data into a local database in the production test station according to a time sequence;
when the local data transfer module in the production test station is the local data transfer module, step S3 includes:
s31, storing the characterization data to the local data transfer module;
s32, acquiring a data storage request;
s33, storing the characterization data in the local data transfer module into the local database in the production test station according to the data storage request;
after step S1 and before step S2, the method further comprises:
s11, storing the production test data into a local storage module in the production test station;
the step S3 further comprises the following steps:
s4, when the local data transfer module starts a data transmission thread, the characterization data in the local database are sequentially read according to a time sequence;
s5, acquiring corresponding production test data in the local storage module according to the characterization data;
s6, sequentially uploading the acquired production test data to a file database according to a time sequence.
2. The method for controlling data storage in a production process of a product according to claim 1, wherein the file database includes mongo db;
When the production test data is binary format data, step S6 includes:
and sequentially uploading the production test data in the binary format to a MongoDB according to the time sequence.
3. The method for controlling data storage in a product manufacturing process of claim 1, further comprising:
uploading production test tool data corresponding to the production test tools in the production test station to a production database;
uploading abstract log data generated by the production test station in the production process to an abstract log database;
wherein the production database and the summary log database are both relational databases; and/or the number of the groups of groups,
the local database comprises a SQLite database.
4. The method for controlling data storage in a product manufacturing process according to claim 1, further comprising, after step S3:
after the current product is produced, controlling a production test tool at the production test station to continuously produce the next product, and re-executing the step S1; and/or the number of the groups of groups,
when the product comprises a wireless communication module, the production test data comprise module test data, radio frequency parameters of the module, station position data, a product serial number, a data storage path and timestamp data;
The characterization data includes a product serial number, a data storage path, and time stamp data.
5. A data query method, wherein the data query method is implemented by using the control method for data storage in the product production process according to claim 1, and the data query method comprises:
obtaining target characterization data corresponding to a target product;
and inquiring the target characterization data to obtain corresponding target production test data in the file database.
6. The data query method of claim 5, wherein after uploading production test tool data corresponding to a production test tool in the production test site to a production database and uploading summary log data generated by the production test site during production to a summary log database, the data query method further comprises:
and inquiring from the production database and the summary log database according to the target characterization data to obtain corresponding target production test tool data and target summary log data.
7. The control system for data storage in the production process of the product is characterized by comprising a production test data acquisition module, a characterization data acquisition module and a first storage module;
The production test data acquisition module is used for acquiring production test data corresponding to different time stamps of a production test tool of a production test station when a current product is produced;
the characterization data acquisition module is used for acquiring characterization data used for characterizing the production test data;
wherein, the data volume corresponding to the characterization data is smaller than the data volume corresponding to the production test data;
the first storage module is used for sequentially storing the characterization data into a local database in the production test station according to a time sequence;
when the local data transfer module in the production test station is used, the first storage module comprises a transfer storage unit, a request acquisition unit and a local storage unit;
the transfer storage unit is used for storing the characterization data to the local data transfer module;
the request acquisition unit is used for acquiring a data storage request;
the local storage unit is used for storing the characterization data in the local data transfer module into the local database in the production test station according to the data storage request;
the control system also comprises a second storage module and a third storage module;
The second storage module is used for storing the production test data into a local storage module in the production test station;
the characterization data acquisition module is further used for sequentially reading the characterization data in the local database according to a time sequence when the local data transfer module starts a data transmission thread;
the production test data acquisition module is further used for acquiring the corresponding production test data in the local storage module according to the characterization data;
and the third storage module is used for sequentially uploading the acquired production test data to a file database according to the time sequence.
8. The control system for data storage in a product manufacturing process of claim 7, wherein said file database comprises mongo db;
and when the production test data are binary format data, the third storage module is used for sequentially uploading the binary format production test data into the MongoDB according to a time sequence.
9. The control system for data storage in a product manufacturing process of claim 7, wherein the control system further comprises a fourth memory module and a fifth memory module;
The fourth storage module is used for uploading production test tool data corresponding to the production test tools in the production test station to a production database;
the fifth storage module is used for uploading abstract log data generated by the production test station in the production process to an abstract log database;
wherein the production database and the summary log database are both relational databases; and/or the number of the groups of groups,
the local database comprises a SQLite database.
10. The control system for data storage in a product manufacturing process of claim 7, wherein the control system further comprises a control module;
the control module is used for controlling the production test tool of the production test station to continue producing the next product after the current product is produced, and recalling the production test data acquisition module; and/or the number of the groups of groups,
when the product comprises a wireless communication module, the production test data comprise module test data, radio frequency parameters of the module, station position data, a product serial number, a data storage path and timestamp data;
the characterization data includes a product serial number, a data storage path, and time stamp data.
11. A data query system, characterized in that the data query system is implemented by the control system for data storage in the production process of the product according to claim 8, and the data query system comprises a target data acquisition module and a query module;
the target data acquisition module is used for acquiring target characterization data corresponding to a target product;
and the query module is used for querying the target characterization data to obtain corresponding target production test data in the file database.
12. The data query system of claim 11, wherein the query module is further configured to query the production database and the summary log database for corresponding target production test tool data and target summary log data, respectively, based on the target characterization data after uploading production test tool data corresponding to production test tools in the production test site to the production database and uploading summary log data generated by the production test site during production to the summary log database.
13. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements a method for controlling data storage during production of a product according to any one of claims 1-4 or a method for querying data according to claim 5 or 6 when executing the computer program.
14. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of a method for controlling data storage in a production process of a product according to any one of claims 1-4, or the steps of a data query method according to claim 5 or 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910821324.4A CN110515955B (en) | 2019-09-02 | 2019-09-02 | Data storage and query method and system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910821324.4A CN110515955B (en) | 2019-09-02 | 2019-09-02 | Data storage and query method and system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110515955A CN110515955A (en) | 2019-11-29 |
| CN110515955B true CN110515955B (en) | 2023-09-15 |
Family
ID=68630207
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910821324.4A Active CN110515955B (en) | 2019-09-02 | 2019-09-02 | Data storage and query method and system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110515955B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473260A (en) * | 2013-06-25 | 2013-12-25 | 北京控制工程研究所 | Concurrency OLAP (On-Line Analytical Processing)-oriented test data hierarchy cluster query processing system and method |
| CN105550333A (en) * | 2015-12-22 | 2016-05-04 | 歌尔声学股份有限公司 | MongoDB-based test data storage query method and system |
| CN109117375A (en) * | 2018-08-30 | 2019-01-01 | 上海携程金融信息服务有限公司 | Database interface test method, system, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200535604A (en) * | 2004-04-16 | 2005-11-01 | Hon Hai Prec Ind Co Ltd | Test data collection and management system and method |
-
2019
- 2019-09-02 CN CN201910821324.4A patent/CN110515955B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473260A (en) * | 2013-06-25 | 2013-12-25 | 北京控制工程研究所 | Concurrency OLAP (On-Line Analytical Processing)-oriented test data hierarchy cluster query processing system and method |
| CN105550333A (en) * | 2015-12-22 | 2016-05-04 | 歌尔声学股份有限公司 | MongoDB-based test data storage query method and system |
| CN109117375A (en) * | 2018-08-30 | 2019-01-01 | 上海携程金融信息服务有限公司 | Database interface test method, system, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于MongoDB的Web信息采集系统应用研究;孙美卫;《湖南邮电职业技术学院学报》;20170615(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110515955A (en) | 2019-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9645736B2 (en) | Processing time series data from multiple sensors | |
| WO2021091489A1 (en) | Method and apparatus for storing time series data, and server and storage medium thereof | |
| Lai et al. | Towards a framework for large-scale multimedia data storage and processing on Hadoop platform | |
| CN102411533A (en) | Log-management optimizing method for clustered storage system | |
| CN104599032A (en) | Distributed memory power grid construction method and system for resource management | |
| CN109299121B (en) | Efficient historical alarm query method for SCADA system | |
| CN102750350A (en) | Monitoring system and method | |
| CN109669975B (en) | Industrial big data processing system and method | |
| CN110569142A (en) | ORACLE data increment synchronization system and method | |
| CN105162414A (en) | Photovoltaic power station power prediction system based on time sequence database platform | |
| CN103117878A (en) | Design method of Nagios-based distribution monitoring system | |
| CN112035563A (en) | Real-time database system based on shared storage | |
| CN103440302B (en) | The method and system of Real Data Exchangs | |
| CN115048452A (en) | Big data management system based on block chain | |
| CN112131214A (en) | Method, system, equipment and storage medium for data writing and data query | |
| CN104601374A (en) | Network fault processing method and device for numerical control machine tool | |
| CN110515955B (en) | Data storage and query method and system, electronic equipment and storage medium | |
| CN111104441A (en) | Data acquisition method and system | |
| CN112417050A (en) | Data synchronization method and device, system, storage medium and electronic device | |
| CN112685486B (en) | Data management method and device for database cluster, electronic equipment and storage medium | |
| CN115391286A (en) | Link tracking data management method, device, equipment and storage medium | |
| CN115080666A (en) | Data synchronization method, system, electronic device and storage medium | |
| CN109344192B (en) | Optimized CIMISS database system and adaptation method thereof | |
| US9250839B1 (en) | Printing system for data handling having a primary server for storing active and passive data and a second server for storing normalized and analytical data | |
| CN113886472A (en) | Data access system, access method, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |