CN111737327A - Automobile production action data acquisition method, system, device and storage medium - Google Patents
Automobile production action data acquisition method, system, device and storage medium Download PDFInfo
- Publication number
- CN111737327A CN111737327A CN202010371060.XA CN202010371060A CN111737327A CN 111737327 A CN111737327 A CN 111737327A CN 202010371060 A CN202010371060 A CN 202010371060A CN 111737327 A CN111737327 A CN 111737327A
- Authority
- CN
- China
- Prior art keywords
- data
- action
- action data
- value
- redis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009471 action Effects 0.000 title claims abstract description 223
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000003860 storage Methods 0.000 title claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 42
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 230000008569 process Effects 0.000 claims description 10
- 238000004140 cleaning Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000013480 data collection Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 abstract description 7
- 238000009825 accumulation Methods 0.000 abstract description 3
- 230000008901 benefit Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005111 flow chemistry technique Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 125000002015 acyclic group Chemical group 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- General Factory Administration (AREA)
Abstract
The invention discloses a method, a system, a device and a storage medium for collecting automobile production action data, wherein the method comprises the following steps: collecting action data of automobile production equipment; uploading the action data to a Kafka message queue; consuming the action data of the Kafka message queue through a Flink cluster; and processing the action data in a distributed mode, wherein the action data is streaming data. According to the invention, by utilizing the architectural advantage of distributed streaming processing real-time data of the Flink platform assembly, the traditional single-thread Cycle computing algorithm is correspondingly modified and then applied to the computing algorithm for processing cycles in a distributed mode provided by the Flink, and by adopting the application mode of the new architecture, the performance of real-time data monitoring and processing is improved, the current data accumulation and performance bottleneck are solved, and the real-time monitoring of the action data of automobile production is realized. The invention is widely applied to the technical field of automobile production.
Description
Technical Field
The invention relates to the technical field of automobile production, in particular to an automobile production action data acquisition method, system, device and storage medium.
Background
Data storage and data analysis mining in the automobile production process are in a starting stage, most of the current data storage is single-machine storage, so that the current data analysis and mining application is less, and the performance of data analysis mining has a great influence on the production efficiency, so that the production cost is high in the automobile production process. On the other hand, in the daily automobile production process, the real-time monitoring delay in operation and maintenance is long, because most of the data are firstly stored in a time sequence database or a mysql database after being collected from production and then are uniformly calculated and output, and thus, the performance of calculating massive real-time data by a single machine is low.
In the production process of automobiles, the time for finishing one working procedure is called a beat, which is also called a Cycle. Each Cycle may consist of one action group or multiple action groups, and each action group consists of many actions. The action data uploaded by the collectors are unordered within a specific time, and after the system is simultaneously accessed to the data uploaded by the collectors, the data are processed by the single server, the average delay reaches 3-5 seconds, and the standard of real-time monitoring in the production process cannot be met. In the process of monitoring the current automobile production, the current real-time production condition and the production efficiency of each station under each production line can be reflected through the real-time monitoring of each Cycle. When the Cycle is calculated at present, the performance of single-thread processing of the Cycle data is very slow, the data of the Cycle are very seriously accumulated in the cache queue, and when the data are particularly serious, the delay of accumulating the data in the cache queue reaches 2 hours, so that the monitoring of the action data of automobile production is seriously influenced.
Disclosure of Invention
In view of the above, the present invention provides a method, a system, a device and a storage medium for collecting motion data of automobile production, so as to monitor the motion data of automobile production in real time.
The first technical scheme adopted by the invention is as follows:
a method for collecting automobile production action data comprises the following steps:
collecting action data of automobile production equipment;
uploading the action data to a Kafka message queue;
consuming the action data of the Kafka message queue through a Flink cluster;
and processing the action data in a distributed mode, wherein the action data is streaming data.
Further, the action data comprises action marks, action duration, generation time, a production model value, a station ID value, an action group ID value and a line body ID value.
Further, the processing the motion data includes:
when the action mark of the action data is a process mark, writing the action data into Redis;
when the action mark of the action data is a start mark and the action data exists in Redis, outputting the action data with all station ID values in the Redis being the same as the work ID values of the action data, and deleting the action data with all station ID values in the Redis being the same as the work ID values of the action data;
and when the action mark of the action data is a start mark and the action data does not exist in the Redis, writing the action data into the Redis.
Further, the uploading the action data to a Kafka message queue includes:
and uploading the action data to a Kafka message queue in sequence according to the generation time of the action data.
Further, the method also comprises the following steps:
and performing data cleaning on the action data.
Further, outputting the action data with the same station ID value as the work ID value of the action data in the Redis includes:
acquiring action data with the same station ID value as the action data;
classifying the action data with the same station ID value and the same action data into different action groups according to the ID value of the action group;
sorting the action data in the action group according to the generation time, and calculating the starting time and the ending time of the action group by combining the action duration of the action data;
and outputting the starting time and the ending time of the action group, the produced vehicle type value, the station ID value, the action group ID value and the line body ID value.
Further, the deleting of the action data in which all workstation ID values in the Redis are the same as the work ID values of the action data includes:
and deleting the action duration, the generation time and the produced vehicle type value of the action data with the same station ID value and the action data in the Redis.
The second technical scheme adopted by the invention is as follows:
an automotive production action data acquisition system comprising:
the acquisition module is used for acquiring the action data of the automobile production equipment;
the transmission module is used for uploading the action data to a Kafka message queue;
the consumption module is used for consuming the action data of the Kafka message queue through a Flink cluster;
a monitoring module for processing the action data in a distributed manner;
the motion data is streaming data.
The third technical scheme adopted by the invention is as follows:
an automotive production motion data acquisition device comprising:
the PLC collectors are used for collecting action data of the automobile production equipment;
the data server is used for uploading the action data to a Kafka message queue, consuming the action data of the Kafka message queue through a Flink cluster and processing the action data in a distributed mode;
the motion data is streaming data.
The fourth technical scheme adopted by the invention is as follows:
a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the method for collecting automotive production behavior data.
Compared with the prior art, the invention utilizes the architectural advantage of the Flink platform assembly for processing the real-time streaming data in a distributed manner, applies the traditional single-thread Cycle computing algorithm to the computing algorithm for processing the cycles in a distributed manner provided by the Flink after correspondingly modifying the traditional single-thread Cycle computing algorithm, improves the performance of monitoring and processing the real-time data, solves the current data accumulation and performance bottleneck and realizes the real-time monitoring of the action data of the automobile production by the application manner of the new architecture.
Drawings
FIG. 1 is a first flowchart of a method for collecting data of automobile production operations according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a method for collecting data of vehicle production operations according to an embodiment of the present invention;
FIG. 3 is a block diagram of an embodiment of a data collection system for automobile production operations;
fig. 4 is a structural diagram of an automobile production operation data acquisition device according to an embodiment of the invention.
Detailed Description
The conception, the specific structure and the technical effects of the present invention will be clearly and completely described in conjunction with the embodiments and the accompanying drawings to fully understand the objects, the schemes and the effects of the present invention.
The embodiment of the invention provides a method for acquiring automobile production action data, which comprises the following steps of with reference to FIG. 1:
s1, collecting the action data of the automobile production equipment;
s2, uploading the action data to a Kafka message queue;
s3, consuming the action data of the Kafka message queue through a Flink cluster;
s4, processing the action data in a distributed mode;
the motion data is streaming data.
Specifically, the automobile production equipment is production equipment for performing operation on an automobile production line, and each automobile production equipment can act according to set parameters.
Kafka is a high throughput distributed publish-subscribe messaging system with the following characteristics: persistence of messages is provided by a disk data structure that maintains stable performance for long periods of time even for TB message storage; high throughput: even the very common hardware Kafka can support millions of messages per second; supporting partitioning of messages by Kafka server and consumer clusters; and Hadoop parallel data loading is supported. The Kafka can realize the implementation transmission of the action data.
Flink is a stream processing framework, and the Flink program is mapped to stream data streams after execution, each of which starts with one or more sources and ends with one or more receivers. The Flink can perform any number of transformations on the flow, the flows can be arranged into directed acyclic data flow diagrams, application program branching and data flow combination are allowed, distributed flow processing on action data can be achieved by using a Flink framework, and data accumulation and performance bottleneck caused by original single-thread processing are avoided.
The action data in the embodiment of the invention is stream data, and the action data is processed in a distributed mode, namely the action data is processed in a distributed streaming mode. The distributed stream processing refers to dynamic processing of stream data based on a distributed technology, and has good flexibility, real-time performance and openness. Distributed streaming processing is typically deployed in large-scale clusters, and the stream data processing process is typically abstracted into a directed acyclic graph. The scheduling algorithm is responsible for the logical distribution of the components in the directed acyclic graph to the available servers in the cluster. The stream processing system is used as a platform for processing the operation by the stream and is responsible for managing and distributing all cluster resources. For a flow processing job submitted by a user, the flow processing system needs to consider the data volume processed by the job and the load of different nodes in a cluster and reasonably distribute the data volume to the different job nodes of the cluster. Thus how to maximize the utilization of the cluster, i.e., maximize the number of stream processing jobs that the cluster can accommodate. The good task allocation strategy can accelerate the data processing rate, improve the overall throughput of the system, balance the load in the cluster, or reduce the resource occupancy rate in the cluster.
According to the embodiment of the invention, the acquired action data of the automobile equipment is subjected to distributed processing by matching the high throughput of the Kafka message queue with the Flink cluster, the Flink cluster distributes streaming consumption Kafka queue data, the performance of real-time processing of mass data of the whole platform product is ensured, and even if the geometric data is increased, the platform achieves stable expansibility by hot deploying of a data node machine, namely the performance and stability of the algorithm are ensured.
Further as an optional implementation manner, the action data includes an action flag, an action duration, a generation time, a production model value, a station ID value, an action group ID value, and a line body ID value.
Specifically, the action mark is used for judging and determining the end of each process action and the start of the next process; the action mark is divided into a start mark and a process mark, and the corresponding parameters are 1 and 0 respectively; judging the start and the end of a Cycle by combining the action mark with data in the Redis cache; for example, 10000001001 is the uploaded parameter flag value generated in the production process under a workstation, then there are 2 cycles under the workstation, i.e. 10000001 and 1001.
The action duration is used for recording the time consumed by the automobile production equipment for executing a single action; the generation time is used for recording the generation time of the action data of the automobile production equipment; the produced vehicle model value is used for recording the vehicle model of the vehicle currently produced by the vehicle production equipment; the station ID value is used for positioning the specific position of the production equipment; an action group ID value is used to classify the action data; and the line body ID value is used for positioning the production line position of the production equipment.
As a further optional implementation, the processing the action data, with reference to fig. 2, includes:
when the action mark of the action data is a process mark, writing the action data into Redis;
when the action mark of the action data is a start mark and the action data exists in Redis, outputting the action data with all station ID values in the Redis being the same as the work ID values of the action data, and deleting the action data with all station ID values in the Redis being the same as the work ID values of the action data;
and when the action mark of the action data is a start mark and the action data does not exist in the Redis, writing the action data into the Redis.
Specifically, since data of a plurality of automobile production facilities needs to be processed, a plurality of pieces of motion data need to be classified by Cycle at the time of processing. In the processing of the operation data, first, the operation flag of the operation data is determined. When the action mark is a process mark, the collected action data is an intermediate process in the whole Cycle, and the action data is normally cached in Redis; when the action mark is a start mark, the collected action data is indicated to be at the beginning or the end of a Cycle, the action data is required to be compared with all the action data in Redis, when data of a station where the action data is located exist in the Redis, the collected action data is indicated to be at the end of the Cycle, the Cycle data is output, and the corresponding Cycle data is deleted; when data of the station where the action data are located does not exist in Redis, the collected action data are indicated to be at the beginning of a Cycle, and the action data are normally cached in Redis. By the judgment method shown in fig. 2, ordered Cycle output can be realized for unordered automobile data.
Redis is a Key-Value storage system. Similar to Memcached, it supports relatively more Value types for storage, including strings, linked lists, collections, ordered collections, and hash types. These data types all support push/pop, add/remove, and intersect union and difference, and richer operations, and these operations are all atomic. On this basis, Redis supports various different ways of ordering. Like Memcached, data is cached in memory to ensure efficiency. The difference is that Redis periodically writes updated data to disk or writes modify operations to an additional recording file, and realizes master-slave synchronization based on the updated data or the modify operations.
In this embodiment, the action data includes a Key Value and a Value when being recorded after being input to Redis, where the Key is a character string superposition of a line body ID Value, a station ID, and an action group ID, and the Value includes an action duration, a generation time, and a vehicle type Value of production.
Further as an optional implementation manner, the uploading the action data to a Kafka message queue includes:
and uploading the action data to a Kafka message queue in sequence according to the generation time of the action data.
Specifically, the data are sequentially transmitted according to the generation time of the data, so that the data can be prevented from being disordered in sequence, and the data of different cycles are prevented from being aliased. Since each Cycle is executed strictly in steps, the generation time corresponds to the time stamp in one Cycle, by means of which the different cycles can be separated.
Further as an optional implementation, the method further comprises the following steps:
and performing data cleaning on the action data.
Specifically, data cleansing refers to the last procedure to find and correct recognizable errors in data files, including checking data consistency, processing invalid and missing values, and the like. In the embodiment, the data with errors is identified through data cleaning and deleted or corrected, and meanwhile, the data cleaning code is nested in the data processing code. The data cleaning and the data processing are realized on the same data server node, namely the data cleaning is realized in a Flink streaming processing framework. The design solves the bottleneck of data cleaning and reduces the delay of network transmission, because the cleaning and the data processing are processed on the same data server according to the platform and code design principle.
As a further optional implementation manner, outputting the action data in which all workstation ID values in Redis the same as the work ID value of the action data includes:
acquiring action data with the same station ID value as the action data;
classifying the action data with the same station ID value and the same action data into different action groups according to the ID value of the action group;
sorting the action data in the action group according to the generation time, and calculating the starting time and the ending time of the action group by combining the action duration of the action data;
and outputting the starting time and the ending time of the action group, the produced vehicle type value, the station ID value, the action group ID value and the line body ID value.
Specifically, after the action data of one Cycle is cached, the action data in the Cycle needs to be output, and the output result can be used for generating a report or storing in a database for use. When outputting the data of one Cycle, all the data cached in Redis of the workstation where the Cycle is located need to be output, that is, the data is output according to the workstation ID value. In this embodiment, the motion data is classified according to the motion group ID of the motion data, the motion data of one motion group is classified into one type, and then the start time and the end time of the output motion group, the model value of production, the station ID value, the motion group ID value, and the line body ID value are calculated according to the motion data of each motion group.
In some embodiments, more specific start time and end time of each action, a production vehicle type value, a station ID value, an action group ID value and a line body ID value can be selected and output, but doing so results in a larger output data volume and lower efficiency, and output of the action group data by calculation is more in line with production needs, so that not only can a fault point be located, but also the action data processing efficiency can be improved.
Further as an optional implementation manner, the deleting action data in which all workstation ID values in Redis the same as the work ID value of the action data includes:
and deleting the action duration, the generation time and the produced vehicle type value of the action data with the same station ID value and the action data in the Redis.
Specifically, when the outputted motion data is deleted, only Value values are deleted, that is, only changed data are deleted, because the station ID Value, the motion group ID Value, and the line body ID Value are fixed data in the production process of the automobile production equipment, and they can be retained during deletion to reduce the data modification amount during the next entry, but correspondingly, in the subsequent data entry, corresponding Key values, that is, the station ID Value, the motion group ID Value, and the line body ID Value, need to be queried in the Redis, and data modification is performed in the Value corresponding to the Key Value. However, when the retrieval efficiency is higher than the modification efficiency, the effect of improving the overall efficiency can be achieved by deleting the action data in part of the Redis.
The embodiment of the invention also provides an automobile production action data acquisition system, which refers to fig. 3 and comprises the following components:
the acquisition module 301 is used for acquiring action data of the automobile production equipment;
a transmission module 302, configured to upload the action data to a Kafka message queue;
a consuming module 303, configured to consume, by a Flink cluster, the action data of the Kafka message queue;
a monitoring module 304 for processing the action data in a distributed manner;
the motion data is streaming data.
Specifically, the contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
The layers, modules, units, platforms, and/or the like included in the system may be implemented or embodied by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
The system may be implemented in any type of computing platform operatively connected to a suitable connection, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. The data processing flows correspondingly executed by the layers, modules, units and/or platforms included in the inventive system may be implemented in machine readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and/or write storage medium, a RAM, a ROM, etc., such that it may be read by a programmable computer, and when the storage medium or device is read by the computer, may be used to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.
The embodiment of the present invention further provides an automobile production action data acquisition apparatus, referring to fig. 4, including:
the PLC collectors 401 are used for collecting action data of the automobile production equipment;
the data server 402 is used for uploading the action data to a Kafka message queue, consuming the action data of the Kafka message queue through a Flink cluster, and processing the action data in a distributed mode;
the motion data is streaming data.
Specifically, the contents in the above method embodiments are all applicable to the present apparatus embodiment, the functions specifically implemented by the present apparatus embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present apparatus embodiment are also the same as those achieved by the above method embodiments.
The embodiment of the invention also provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the method for acquiring the automobile production action data is realized.
In particular, the storage medium stores processor-executable instructions, which when executed by the processor, are configured to perform the steps of the method for processing mutual information according to any one of the above-mentioned method embodiments. For the storage medium, it may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. It can be seen that the contents in the foregoing method embodiments are all applicable to this storage medium embodiment, the functions specifically implemented by this storage medium embodiment are the same as those in the foregoing method embodiments, and the advantageous effects achieved by this storage medium embodiment are also the same as those achieved by the foregoing method embodiments.
The above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment, and any modifications, equivalent substitutions, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention as long as the technical effects of the present invention are achieved by the same means. The invention is capable of other modifications and variations in its technical solution and/or its implementation, within the scope of protection of the invention.
Claims (10)
1. A method for collecting automobile production action data is characterized by comprising the following steps:
collecting action data of automobile production equipment;
uploading the action data to a Kafka message queue;
consuming the action data of the Kafka message queue through a Flink cluster;
and processing the action data in a distributed mode, wherein the action data is streaming data.
2. The method of claim 1, wherein the action data includes an action flag, an action duration, a generation time, a production model value, a station ID value, an action group ID value, and a line body ID value.
3. The automotive production motion data collection method of claim 2, wherein the processing the motion data comprises:
when the action mark of the action data is a process mark, writing the action data into Redis;
when the action mark of the action data is a start mark and the action data exists in Redis, outputting the action data with all station ID values in the Redis being the same as the work ID values of the action data, and deleting the action data with all station ID values in the Redis being the same as the work ID values of the action data;
and when the action mark of the action data is a start mark and the action data does not exist in the Redis, writing the action data into the Redis.
4. The automobile production motion data collection method according to claim 1, wherein the uploading the motion data to a Kafka message queue comprises:
and uploading the action data to a Kafka message queue in sequence according to the generation time of the action data.
5. The automobile production motion data collection method according to claim 1, further comprising the steps of:
and performing data cleaning on the action data.
6. The method for collecting automotive production motion data according to claim 3, wherein outputting motion data in which all workstation ID values in Redis are the same as the work ID values of the motion data comprises:
acquiring action data with the same station ID value as the action data;
classifying the action data with the same station ID value and the same action data into different action groups according to the ID value of the action group;
sorting the action data in the action group according to the generation time, and calculating the starting time and the ending time of the action group by combining the action duration of the action data;
and outputting the starting time and the ending time of the action group, the produced vehicle type value, the station ID value, the action group ID value and the line body ID value.
7. The method for collecting automotive production motion data according to claim 3, wherein the deleting motion data in which all workstation ID values in Redis are the same as the work ID values of the motion data comprises:
and deleting the action duration, the generation time and the produced vehicle type value of the action data with the same station ID value and the action data in the Redis.
8. An automotive production action data acquisition system, comprising:
the acquisition module is used for acquiring the action data of the automobile production equipment;
the transmission module is used for uploading the action data to a Kafka message queue;
the consumption module is used for consuming the action data of the Kafka message queue through a Flink cluster;
a monitoring module for processing the action data in a distributed manner;
the motion data is streaming data.
9. The utility model provides an automobile production action data acquisition device which characterized in that includes:
the PLC collectors are used for collecting action data of the automobile production equipment;
the data server is used for uploading the action data to a Kafka message queue, consuming the action data of the Kafka message queue through a Flink cluster and processing the action data in a distributed mode;
the motion data is streaming data.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for collecting automotive production behavior data according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010371060.XA CN111737327A (en) | 2020-05-06 | 2020-05-06 | Automobile production action data acquisition method, system, device and storage medium |
| PCT/CN2020/140079 WO2021223451A1 (en) | 2020-05-06 | 2020-12-28 | Method, system and device for acquiring action data of automobile production and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010371060.XA CN111737327A (en) | 2020-05-06 | 2020-05-06 | Automobile production action data acquisition method, system, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111737327A true CN111737327A (en) | 2020-10-02 |
Family
ID=72647003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010371060.XA Pending CN111737327A (en) | 2020-05-06 | 2020-05-06 | Automobile production action data acquisition method, system, device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111737327A (en) |
| WO (1) | WO2021223451A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112417015A (en) * | 2020-11-18 | 2021-02-26 | 青岛海尔科技有限公司 | Data distribution method and device, storage medium and electronic device |
| CN113110310A (en) * | 2021-03-04 | 2021-07-13 | 广州市友乃德金属制品有限公司 | Port AGV production process |
| WO2021223451A1 (en) * | 2020-05-06 | 2021-11-11 | 广州明珞装备股份有限公司 | Method, system and device for acquiring action data of automobile production and storage medium |
| CN114546031A (en) * | 2021-12-31 | 2022-05-27 | 广州明珞装备股份有限公司 | Process beat calculation method, system, equipment and storage medium |
| WO2023125082A1 (en) * | 2021-12-31 | 2023-07-06 | 广州明珞装备股份有限公司 | Process action determination method, system, device, and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114546274B (en) * | 2022-02-22 | 2023-10-27 | 云智慧（北京）科技有限公司 | Big data processing dimension table calculation system and method based on cache |
| CN115460062A (en) * | 2022-08-04 | 2022-12-09 | 内蒙古蒙商消费金融股份有限公司 | Data monitoring method and device and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107045316A (en) * | 2016-10-25 | 2017-08-15 | 湘潭智联技术转移促进有限责任公司 | A kind of precision machined PLC integrated control systems |
| CN109829765A (en) * | 2019-03-05 | 2019-05-31 | 北京博明信德科技有限公司 | Method, system and device based on Flink and Kafka real time monitoring sales data |
| CN110334070A (en) * | 2019-05-21 | 2019-10-15 | 中国人民财产保险股份有限公司 | Data processing method, system, equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10275278B2 (en) * | 2016-09-14 | 2019-04-30 | Salesforce.Com, Inc. | Stream processing task deployment using precompiled libraries |
| CN109710731A (en) * | 2018-11-19 | 2019-05-03 | 北京计算机技术及应用研究所 | A kind of multidirectional processing system of data flow based on Flink |
| CN110798525A (en) * | 2019-11-01 | 2020-02-14 | 哈工大机器人（合肥）国际创新研究院 | Industrial robot multisource data cloud storage system |
| CN111737327A (en) * | 2020-05-06 | 2020-10-02 | 广州明珞汽车装备有限公司 | Automobile production action data acquisition method, system, device and storage medium |
-
2020
- 2020-05-06 CN CN202010371060.XA patent/CN111737327A/en active Pending
- 2020-12-28 WO PCT/CN2020/140079 patent/WO2021223451A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107045316A (en) * | 2016-10-25 | 2017-08-15 | 湘潭智联技术转移促进有限责任公司 | A kind of precision machined PLC integrated control systems |
| CN109829765A (en) * | 2019-03-05 | 2019-05-31 | 北京博明信德科技有限公司 | Method, system and device based on Flink and Kafka real time monitoring sales data |
| CN110334070A (en) * | 2019-05-21 | 2019-10-15 | 中国人民财产保险股份有限公司 | Data processing method, system, equipment and storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021223451A1 (en) * | 2020-05-06 | 2021-11-11 | 广州明珞装备股份有限公司 | Method, system and device for acquiring action data of automobile production and storage medium |
| CN112417015A (en) * | 2020-11-18 | 2021-02-26 | 青岛海尔科技有限公司 | Data distribution method and device, storage medium and electronic device |
| CN113110310A (en) * | 2021-03-04 | 2021-07-13 | 广州市友乃德金属制品有限公司 | Port AGV production process |
| CN114546031A (en) * | 2021-12-31 | 2022-05-27 | 广州明珞装备股份有限公司 | Process beat calculation method, system, equipment and storage medium |
| WO2023125082A1 (en) * | 2021-12-31 | 2023-07-06 | 广州明珞装备股份有限公司 | Process action determination method, system, device, and storage medium |
| CN114546031B (en) * | 2021-12-31 | 2023-10-13 | 广州明珞装备股份有限公司 | Process beat calculation method, system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021223451A1 (en) | 2021-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111737327A (en) | Automobile production action data acquisition method, system, device and storage medium | |
| CN104915407B (en) | A kind of resource regulating method based under Hadoop multi-job environment | |
| CN107018042B (en) | Tracking method and tracking system for online service system | |
| US10467569B2 (en) | Apparatus and method for scheduling distributed workflow tasks | |
| US9619430B2 (en) | Active non-volatile memory post-processing | |
| US9286123B2 (en) | Apparatus and method for managing stream processing tasks | |
| CN111324445A (en) | Task scheduling simulation system | |
| CN103516807A (en) | Cloud computing platform server load balancing system and method | |
| CN102722432B (en) | Follow the trail of the method and apparatus of internal storage access | |
| CN104077224B (en) | Software function analyzes System and method for | |
| KR20170134723A (en) | Event processing system paging | |
| CN104639347A (en) | Multi-cluster monitoring method and device, and system | |
| CN103941662A (en) | Task scheduling system and method based on cloud computing | |
| CN107870763A (en) | For creating the method and its device of the real-time sorting system of mass data | |
| CN103761262A (en) | Repetition log control method based on syslogd | |
| CN108829505A (en) | A kind of distributed scheduling system and method | |
| CN109783441A (en) | Mass data inquiry method based on Bloom Filter | |
| CN105049524B (en) | A method of the large-scale dataset based on HDFS loads | |
| US10558391B2 (en) | Data processing system and data processing method | |
| Zhang et al. | A spark scheduling strategy for heterogeneous cluster | |
| Yang et al. | Redis automatic performance tuning based on eBPF | |
| CN102867023A (en) | Method for storing and reading grid data and device | |
| CN111580950A (en) | Self-adaptive feedback resource scheduling method for improving cloud reliability | |
| CN107609129B (en) | Log real-time processing system | |
| CN115794783A (en) | Data deduplication method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201002 |
|
| RJ01 | Rejection of invention patent application after publication |