CN112507003A - Internet of vehicles data analysis platform based on big data architecture - Google Patents
Internet of vehicles data analysis platform based on big data architecture Download PDFInfo
- Publication number
- CN112507003A CN112507003A CN202110146529.4A CN202110146529A CN112507003A CN 112507003 A CN112507003 A CN 112507003A CN 202110146529 A CN202110146529 A CN 202110146529A CN 112507003 A CN112507003 A CN 112507003A
- Authority
- CN
- China
- Prior art keywords
- data
- data analysis
- task
- analysis task
- analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24561—Intermediate data storage techniques for performance improvement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a vehicle networking data analysis platform based on a big data architecture, which comprises a data receiving module, a data analysis task management module and a data pushing module; the data receiving module receives the collected data reported by the vehicle-mounted terminal, judges and processes the collected data according to certain data preprocessing logic, and writes the data meeting certain conditions into the message middleware Kafka; the data analysis task management module submits the data analysis tasks to a Spark distributed computing engine, the data analysis tasks are uniformly monitored and managed in the execution process, and monitoring indexes and final analysis results of the data analysis tasks during the operation period are written into a database Clickhouse; the data pushing module receives a request for subscribing a certain data analysis task output result from a user based on a Websocket protocol; the problems that a traditional data analysis platform cannot be in butt joint with an online big data scene environment, the model training analysis process is not standard and the like are solved.
Description
Technical Field
The invention relates to a vehicle networking data analysis platform based on a big data architecture, and belongs to the technical field of big data vehicle networking data analysis.
Background
Along with the continuous falling and perfection of industrial policies and the technical maturity and popularization of vehicle-mounted intelligent terminals, more and more enterprises complete the receiving and storing of vehicle data; however, in the face of mass data reported by terminals, most of the current applications are processing current real-time data or performing simple statistical analysis on historical data, but in contrast to this, the cost of platform construction and maintenance and storage is high; the method is characterized in that 100 thousands of vehicles are measured in a certain vehicle enterprise, information is reported every 10s, each piece of information is calculated by 1KB (after compression), the storage requirement of one year is about 3PB, and the storage cost is nearly millions of yuan.
In order to better explore the value of received data and solve the problems that a traditional data analysis platform cannot be in butt joint with an online environment, a model training analysis process is not standard, the single-machine processing performance is low, model monitoring cannot be visualized and the like, a new vehicle networking data analysis platform based on a big data architecture is urgently needed.
Disclosure of Invention
The invention provides a vehicle networking data analysis platform based on a big data architecture, and aims to solve the problems that a traditional data analysis platform cannot be in butt joint with an online environment, a model training analysis process is not standard, and the single-machine processing performance is low.
The technical solution of the invention is as follows: a big data architecture-based Internet of vehicles data analysis platform structurally comprises a data receiving module, a data analysis task management module and a data pushing module; the data receiving module is used for receiving the collected data reported by the vehicle-mounted terminal, judging and processing the received collected data according to certain data preprocessing logic, and writing the data meeting certain conditions into the message middleware Kafka; the data analysis task management module is used for submitting the data analysis tasks to the Spark distributed computing engine, uniformly monitoring and managing the data analysis tasks in the execution process, and writing the monitoring indexes and the final analysis results of the data analysis tasks in the running period into a database Clickhouse; the data pushing module receives a request of a user for subscribing an output result of a certain data analysis task based on a Websocket protocol, accepts or rejects the subscription request according to the affiliated relationship between the user and the data analysis task, and pushes monitoring indexes and analysis results in the execution process of the corresponding data analysis task to the user after accepting the subscription request of the user for subscribing the output result of the certain data analysis task.
Further, the received collected data is judged and processed according to a certain data preprocessing logic, and the data meeting a certain condition is written into the message middleware Kafka, specifically: and carrying out null value inspection, missing value filling and abnormal value detection on the acquired data, and packaging the data meeting the conditions in a segmented manner according to the driving cycle and writing the data into the message middleware Kafka.
Further, the vehicle networking data analysis platform based on the big data architecture structurally further comprises a data visualization module, wherein the data visualization module is used for constructing the monitoring indexes and the analysis results into a visualization chart and organizing the visualization chart into a data panel for displaying according to actual requirements; preferably, the data visualization module displays each item of data in the database Clickhouse in real time based on the Redash; further preferably, the data visualization module constructs data in the database Clickhouse into a visualization chart based on the Redash and organizes the visualization chart into a data panel for displaying according to actual requirements.
Further, the data analysis task management module can also customize a visualization page for monitoring indexes and displaying analysis results for each data analysis task based on the data visualization module.
Further, the data analysis task management module divides the data analysis tasks into 3 types of tasks; the 3 types of tasks comprise an offline training task, an online analysis task and an offline analysis task; preferably, the data analysis task management module divides the data analysis tasks into an offline training task, an online analysis task and an offline analysis task according to different scheduling requirements; the off-line training task is a first type task, the on-line analysis task is a second type task, and the off-line analysis task is a third type task.
Further, the offline training task, the online analysis task and the offline analysis task are submitted to the Spark distributed computing engine by the data analysis task management module.
Furthermore, the off-line training task has the characteristics of single execution and manual submission, the data source of the off-line training task is a file on the distributed file system HDFS, the off-line training task is usually used in the model training stage, and the trained model file is output after the off-line training task is finished.
Further, the execution of the offline training task comprises the following steps:
the method comprises the steps of firstly, uploading a task code jar package for off-line training;
secondly, correlating a target vehicle number set to be analyzed;
thirdly, selecting a data set range to be analyzed, namely a date directory on the HDFS;
fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with a corresponding data analysis task;
fifthly, before starting the data analysis task, adjusting relevant parameters of the data analysis task according to actual requirements;
sixthly, assembling a data analysis task, designating a data source as a file in a specified date directory on the distributed file system HDFS, and submitting the data analysis task to a Spark distributed computing engine;
and seventhly, writing the monitoring index into a database Clickhouse in the data analysis task execution process, writing an analysis result into the database Clickhouse after the data analysis task is executed, and outputting the trained model file to a specified directory on the distributed file system HDFS.
Furthermore, the online analysis task performs persistence analysis on unbounded data streams, and is usually used in a data online analysis stage, a data source of the online analysis task is a message stream in a message middleware Kafka, and the online analysis task calls a model file output by an offline training task continuously according to data reported by a terminal in real time, and gives an analysis result persistently.
Further, the online analysis task corresponds to the offline training task one to one, and the execution of the online analysis task comprises the following steps:
the method comprises the steps of firstly, uploading a task code jar package for online analysis;
secondly, correlating a target vehicle number set to be analyzed;
thirdly, correlating the model files output by the corresponding off-line training tasks;
fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with a corresponding data analysis task;
fifthly, before starting the data analysis task, adjusting relevant parameters of the data analysis task according to actual requirements;
sixthly, assembling a data analysis task, designating a data source as a message middleware Kafka, and submitting the data analysis task to a Spark distributed computing engine;
and seventhly, writing the monitoring index into a database Clickhouse in the data analysis task execution process, and writing an analysis result into the database Clickhouse after the data analysis task is finished.
Furthermore, the offline analysis task has the characteristic of periodic automatic execution, is suitable for the periodic data analysis task, the data source of the offline analysis task is a file on a distributed file system (HDFS), and the function of timing analysis of data in a certain period is completed according to a set scheduling strategy.
Further, the offline analysis task is used for periodically executing a single data analysis task, and the execution of the offline analysis task includes the following steps:
the method comprises the steps of firstly, uploading a task code jar package for off-line analysis;
secondly, correlating a target vehicle number set to be analyzed;
thirdly, correlating the model files output by the corresponding off-line training tasks;
fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with a corresponding data analysis task;
fifthly, configuring a periodic scheduling strategy to be executed periodically according to a day or week or month mode;
sixthly, assembling a data analysis task, designating a data source as a file in a specified date directory on the distributed file system HDFS, and submitting the task to a Spark distributed computing engine;
seventhly, writing the monitoring index into a database Clickhouse in the data analysis task execution process, and writing an analysis result into the database Clickhouse after the data analysis task is finished;
the model file output by associating the corresponding offline training task in the third step can be omitted; and when model files output by the corresponding offline training tasks are not needed to be associated in the third step, after the offline analysis task executes the second step of associating the target vehicle number set to be analyzed, directly executing the fourth step of customizing the visual instrument panel in the data visualization module, and associating the visual instrument panel with the corresponding data analysis task.
A method for analyzing data of internet of vehicles by using a data analysis platform of internet of vehicles based on a big data architecture integrates a message middleware Kafka and a distributed file system HDFS based on a Spark distributed computing engine, abstracts and normalizes the whole process of data analysis, thereby realizing model training, online analysis and offline timing analysis aiming at large data volume; in the whole process of executing the analysis task, various monitoring indexes and analysis results are output to a database Clickhouse, and the monitoring indexes and the analysis results are converted into various charts according to actual requirements and flexibly organized into a data panel for displaying; for each data analysis task running on the internet of vehicles data analysis platform based on the big data architecture, a user can subscribe the running monitoring index and analysis result data of a certain data analysis task by adopting a WebSocket protocol.
The invention has the beneficial effects that:
1) aiming at the characteristics of data analysis and application of the Internet of vehicles, data analysis tasks are abstracted into three types, namely an offline training task, an online analysis task and an offline analysis task, and the whole data analysis process is standardized: an off-line training task: reading historical data from a distributed file system (HDFS) in a single execution mode to perform model training, and outputting a model file; and (3) on-line analysis task: reading unbounded data flow in the message middleware Kafka in a continuous operation mode, and providing online data analysis service for equipment; an offline analysis task: according to a user-defined scheduling strategy, periodically reading historical data on a distributed file system (HDFS), and performing timing analysis service on equipment;
2) further writing the monitoring index and the analysis result into a database Clickhouse in the execution process of each analysis task, and providing a user-defined instrument panel for the data visualization module based on the Redash to monitor the operation index and the analysis result of the data visualization module;
3) the data pushing module identifies and judges a subscription request of a user, and pushes an analysis result and a monitoring index to the user in real time, so that high-efficiency, standard and visual data analysis for the Internet of vehicles data is completed;
4) the invention solves the problems that the traditional data analysis platform can not be in butt joint with an online big data environment, the model training and analyzing process is not standard, the single machine processing performance is low, the model monitoring can not be visualized, the operation index is difficult to monitor during the model operation period and the like.
Drawings
FIG. 1 is a schematic architecture diagram of a data analysis platform of the Internet of vehicles.
FIG. 2 is a flowchart of an offline training task execution of the data analysis platform of the Internet of vehicles.
FIG. 3 is a flow chart of the online analysis task execution of the data analysis platform of the Internet of vehicles.
FIG. 4 is a flowchart of an offline analysis task execution of the data analysis platform of the Internet of vehicles.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
Example 1
Fig. 1 is a schematic diagram of a platform architecture of a data analysis platform of a vehicle networking system based on a big data architecture, the data analysis platform of the vehicle networking system comprises a data receiving module, a data analysis task management module, and a data pushing module;
the data receiving module is used for receiving real-time data reported by the terminal, carrying out preprocessing operations such as null value check, missing value filling, abnormal value detection and the like on the data, and carrying out segmented packaging and writing on the data meeting the conditions into a message middleware Kafka and a distributed file system HDFS according to a driving cycle;
the data analysis task management module is used for assembling data analysis tasks according to different task types, submitting the data analysis tasks to a Spark distributed computation engine for computation according to the difference of scheduling parameters, monitoring various indexes executed by the data analysis tasks in real time in the data analysis task execution process, wherein the indexes comprise running states, processing rates, data volumes, average absolute errors, verification set mean square deviations and the like, and writing the indexes and final analysis results into a database Clickhouse; the data visualization module reads data from a database Clickhouse and visually displays the data according to the user-defined visualization module;
the data pushing module pushes monitoring index data and analysis data stored in a database Clickhouse to a subscription user in real time based on a Websocket protocol, each analysis task can automatically generate an accessToken corresponding to each analysis task when being created on the data analysis task management module, the user needs to attach the accessToken to a request information header before requesting to subscribe running data of a certain task based on the Websocket protocol, and the data pushing module determines whether to accept the subscription request of the user by verifying the legality of the accessToken.
Example 2
Fig. 2 to 4 are task execution flowcharts of a vehicle networking data analysis platform based on a big data architecture according to the present invention, where data analysis tasks are classified into 3 types: the off-line training task, the on-line analysis task and the off-line analysis task have certain difference in corresponding execution flow according to different task types.
An off-line training task executing step: the method comprises the steps of firstly, uploading a task code jar packet of off-line training; secondly, associating a target vehicle number set to be analyzed; thirdly, selecting a data set range to be analyzed, namely a date catalogue on the HDFS; fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with the analysis task; fifthly, before starting the task, adjusting and analyzing relevant parameters of the task according to actual requirements; sixthly, assembling an analysis task, designating a data source as a file under a specified date directory on the HDFS, and submitting the task to a Spark distributed computing engine; and seventhly, writing the monitoring indexes into a database Clickhouse in the process of executing the analysis task, writing an analysis result into the database Clickhouse after the task is executed, pushing the analysis result and the monitoring indexes to a subscriber through a pushing module, and analyzing and outputting the trained model to a specified directory on the HDFS.
The on-line analysis task, which usually corresponds to the off-line training task one by one, executes the steps of: the method comprises the steps of firstly, uploading a task code jar package for online analysis; secondly, associating a target vehicle number set to be analyzed; thirdly, correlating the model files output by the off-line training tasks corresponding to the model files; fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with the analysis task; fifthly, before starting the task, adjusting and analyzing relevant parameters of the task according to actual requirements; sixthly, assembling an analysis task, designating a data source as a message middleware Kafka, and submitting the task to a Spark distributed computing engine; and seventhly, writing the monitoring indexes into a database Clickhouse in the execution process of the analysis task, writing an analysis result into the database Clickhouse after the execution of the task is finished, and pushing the analysis result and the monitoring indexes to a subscriber through a pushing module.
The off-line analysis task is used for periodically executing a single analysis task and comprises the following execution steps: the method comprises the steps of firstly, uploading a task code jar packet analyzed offline; secondly, associating a target vehicle number set to be analyzed; the third step is optionally associated with the model file output by the off-line training task corresponding to the third step; fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with the analysis task; step five, configuring a periodic scheduling strategy to be executed periodically according to a day or week or month mode; sixthly, assembling an analysis task, designating a data source as a file under a specified date directory on the HDFS, and submitting the task to a Spark distributed computing engine; and seventhly, writing the monitoring indexes into a database Clickhouse in the execution process of the analysis task, writing an analysis result into the database Clickhouse after the execution of the task is finished, and pushing the analysis result and the monitoring indexes to a subscriber through a pushing module.
Claims (10)
1. A vehicle networking data analysis platform based on big data architecture is characterized by comprising a data receiving module, a data analysis task management module and a data pushing module; the data receiving module is used for receiving the collected data reported by the vehicle-mounted terminal, judging and processing the received collected data according to certain data preprocessing logic, and writing the data meeting certain conditions into the message middleware Kafka; the data analysis task management module is used for submitting the data analysis tasks to the Spark distributed computing engine, uniformly monitoring and managing the data analysis tasks in the execution process, and writing the monitoring indexes and the final analysis results of the data analysis tasks in the running period into a database Clickhouse; the data pushing module receives a request of a user for subscribing an output result of a certain data analysis task based on a Websocket protocol, accepts or rejects the subscription request according to the affiliated relationship between the user and the data analysis task, and pushes monitoring indexes and analysis results in the execution process of the corresponding data analysis task to the user after accepting the subscription request of the user for subscribing the output result of the certain data analysis task.
2. The vehicle networking data analysis platform based on big data architecture as claimed in claim 1, wherein the received collected data is judged and processed according to a certain data preprocessing logic, and the data meeting a certain condition is written into a message middleware Kafka, specifically: and carrying out null value inspection, missing value filling and abnormal value detection on the acquired data, and packaging the data meeting the conditions in a segmented manner according to the driving cycle and writing the data into the message middleware Kafka.
3. The Internet of vehicles data analysis platform based on big data architecture as claimed in claim 1, further comprising a data visualization module, wherein the data visualization module displays each item of data in the database Clickhouse in real time based on Redash; the data analysis task management module can customize a visualization page for monitoring indexes and displaying analysis results for each data analysis task based on the data visualization module.
4. The Internet of vehicles data analysis platform based on big data architecture as claimed in any of claims 1-3, wherein the data analysis task management module divides data analysis tasks into 3 types of tasks; the 3 types of tasks comprise an offline training task, an online analysis task and an offline analysis task.
5. The Internet of vehicles data analysis platform based on big data architecture as claimed in claim 4, wherein the off-line training task, the on-line analysis task and the off-line analysis task are submitted to the Spark distributed computing engine by the data analysis task management module.
6. The Internet of vehicles data analysis platform based on big data architecture as claimed in claim 4, wherein the data source of the off-line training task is a file on a distributed file system (HDFS), and the trained model file is output after the off-line training task is finished.
7. The big data architecture-based vehicle networking data analysis platform according to claim 4, wherein the offline training task is performed by the steps of:
the method comprises the steps of firstly, uploading a task code jar package for off-line training;
secondly, correlating a target vehicle number set to be analyzed;
thirdly, selecting a data set range to be analyzed, namely a date directory on the HDFS;
fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with a corresponding data analysis task;
fifthly, before starting the data analysis task, adjusting relevant parameters of the data analysis task according to actual requirements;
sixthly, assembling a data analysis task, designating a data source as a file in a specified date directory on the distributed file system HDFS, and submitting the data analysis task to a Spark distributed computing engine;
and seventhly, writing the monitoring index into a database Clickhouse in the data analysis task execution process, writing an analysis result into the database Clickhouse after the data analysis task is executed, and outputting the trained model file to a specified directory on the distributed file system HDFS.
8. The Internet of vehicles data analysis platform based on big data architecture as claimed in claim 4, wherein the online analysis task is in one-to-one correspondence with the offline training task, and the execution of the online analysis task comprises the following steps:
the method comprises the steps of firstly, uploading a task code jar package for online analysis;
secondly, correlating a target vehicle number set to be analyzed;
thirdly, correlating the model files output by the corresponding off-line training tasks;
fourthly, customizing a visual instrument panel in a data visualization module in a self-defined mode, and associating the visual instrument panel with a corresponding data analysis task;
fifthly, before starting the data analysis task, adjusting relevant parameters of the data analysis task according to actual requirements;
sixthly, assembling a data analysis task, designating a data source as a message middleware Kafka, and submitting the data analysis task to a Spark distributed computing engine;
and seventhly, writing the monitoring index into a database Clickhouse in the data analysis task execution process, and writing an analysis result into the database Clickhouse after the data analysis task is finished.
9. The big data architecture-based vehicle networking data analysis platform according to claim 4, wherein the offline analysis task is performed by the steps of:
the method comprises the steps of firstly, uploading a task code jar package for off-line analysis;
secondly, correlating a target vehicle number set to be analyzed;
step three, customizing a visual instrument panel in a data visualization module, and associating the visual instrument panel with a corresponding data analysis task;
fourthly, configuring a periodic scheduling strategy to be executed periodically according to a day or week or month mode;
fifthly, assembling a data analysis task, designating a data source as a file in a specified date directory on the distributed file system HDFS, and submitting the task to a Spark distributed computing engine;
and sixthly, writing the monitoring index into a database Clickhouse in the data analysis task execution process, and writing an analysis result into the database Clickhouse after the data analysis task is executed.
10. The big data architecture-based vehicle networking data analysis platform according to claim 9, wherein the second step is replaced by: and associating a target vehicle number set to be analyzed and associating a model file output by the corresponding off-line training task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110146529.4A CN112507003A (en) | 2021-02-03 | 2021-02-03 | Internet of vehicles data analysis platform based on big data architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110146529.4A CN112507003A (en) | 2021-02-03 | 2021-02-03 | Internet of vehicles data analysis platform based on big data architecture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112507003A true CN112507003A (en) | 2021-03-16 |
Family
ID=74952654
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110146529.4A Withdrawn CN112507003A (en) | 2021-02-03 | 2021-02-03 | Internet of vehicles data analysis platform based on big data architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507003A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113159731A (en) * | 2021-05-12 | 2021-07-23 | 河南雪城软件有限公司 | Intelligent analysis system and method for automatic monitoring data of pollution source |
| CN113285957A (en) * | 2021-06-15 | 2021-08-20 | 广州数智网络科技有限公司 | Gambling website detection method based on clickhouse |
| CN113282608A (en) * | 2021-06-10 | 2021-08-20 | 湖南力唯中天科技发展有限公司 | Intelligent traffic data analysis and storage method based on column database |
| CN113642300A (en) * | 2021-07-30 | 2021-11-12 | 南京星云数字技术有限公司 | Report generation method and device, electronic equipment and computer readable medium |
| CN113867844A (en) * | 2021-10-09 | 2021-12-31 | 中邮科通信技术股份有限公司 | Offline data storage and calculation method based on time slice intelligent inspection control |
| CN115221134A (en) * | 2022-07-18 | 2022-10-21 | 陕西天行健车联网信息技术有限公司 | Distributed real-time compression method and device for Internet of vehicles data and storage medium |
| CN116626505A (en) * | 2023-07-21 | 2023-08-22 | 江苏海平面数据科技有限公司 | Battery pack consistency anomaly detection method based on Internet of vehicles big data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107832913A (en) * | 2017-10-11 | 2018-03-23 | 微梦创科网络科技（中国）有限公司 | The Forecasting Methodology and system to monitoring data trend based on deep learning |
| CN107846468A (en) * | 2017-11-16 | 2018-03-27 | 北京卫星信息工程研究所 | Car networking application system and its control method based on cloud computing technology |
| CN108595605A (en) * | 2018-04-20 | 2018-09-28 | 上海蓥石汽车技术有限公司 | A kind of construction method of car networking platform database |
| CN111209261A (en) * | 2020-01-02 | 2020-05-29 | 邑客得（上海）信息技术有限公司 | User travel track extraction method and system based on signaling big data |
| CN111966885A (en) * | 2019-05-20 | 2020-11-20 | 腾讯科技（深圳）有限公司 | User portrait construction method and device |
-
2021
- 2021-02-03 CN CN202110146529.4A patent/CN112507003A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107832913A (en) * | 2017-10-11 | 2018-03-23 | 微梦创科网络科技（中国）有限公司 | The Forecasting Methodology and system to monitoring data trend based on deep learning |
| CN107846468A (en) * | 2017-11-16 | 2018-03-27 | 北京卫星信息工程研究所 | Car networking application system and its control method based on cloud computing technology |
| CN108595605A (en) * | 2018-04-20 | 2018-09-28 | 上海蓥石汽车技术有限公司 | A kind of construction method of car networking platform database |
| CN111966885A (en) * | 2019-05-20 | 2020-11-20 | 腾讯科技（深圳）有限公司 | User portrait construction method and device |
| CN111209261A (en) * | 2020-01-02 | 2020-05-29 | 邑客得（上海）信息技术有限公司 | User travel track extraction method and system based on signaling big data |
Non-Patent Citations (1)
| Title |
|---|
| 徐战辉: "CDN海量日志实时分析问题研究与平台开发", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113159731A (en) * | 2021-05-12 | 2021-07-23 | 河南雪城软件有限公司 | Intelligent analysis system and method for automatic monitoring data of pollution source |
| CN113282608A (en) * | 2021-06-10 | 2021-08-20 | 湖南力唯中天科技发展有限公司 | Intelligent traffic data analysis and storage method based on column database |
| CN113285957A (en) * | 2021-06-15 | 2021-08-20 | 广州数智网络科技有限公司 | Gambling website detection method based on clickhouse |
| CN113642300A (en) * | 2021-07-30 | 2021-11-12 | 南京星云数字技术有限公司 | Report generation method and device, electronic equipment and computer readable medium |
| CN113867844A (en) * | 2021-10-09 | 2021-12-31 | 中邮科通信技术股份有限公司 | Offline data storage and calculation method based on time slice intelligent inspection control |
| CN115221134A (en) * | 2022-07-18 | 2022-10-21 | 陕西天行健车联网信息技术有限公司 | Distributed real-time compression method and device for Internet of vehicles data and storage medium |
| CN116626505A (en) * | 2023-07-21 | 2023-08-22 | 江苏海平面数据科技有限公司 | Battery pack consistency anomaly detection method based on Internet of vehicles big data |
| CN116626505B (en) * | 2023-07-21 | 2023-10-13 | 江苏海平面数据科技有限公司 | Battery pack consistency anomaly detection method based on Internet of vehicles big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112507003A (en) | Internet of vehicles data analysis platform based on big data architecture | |
| CN113361663B (en) | Artificial intelligence-based power grid event diagnosis autonomous learning method and system | |
| CN110321273A (en) | A kind of business statistical method and device | |
| CN113935497A (en) | Intelligent operation and maintenance fault processing method, device and equipment and storage medium thereof | |
| CN110765189A (en) | Exception management method and system for Internet products | |
| CN109491343A (en) | The long-range Internet of Things monitoring system of industrial manufacturing process | |
| CN114048055A (en) | Time series data abnormal root cause analysis method and system | |
| CN110502538A (en) | Label of drawing a portrait generates method, system, equipment and the storage medium of logical mappings | |
| CN111245671A (en) | Automatic integrated test system for ground test of satellite laser communication terminal | |
| CN116295587A (en) | Sensor simulation semi-automatic test system and method | |
| CN115660382A (en) | Vehicle section debugging management system based on Internet of things | |
| CN115759977A (en) | Link automation operation and maintenance system based on data center station | |
| CN112965793B (en) | Identification analysis data-oriented data warehouse task scheduling method and system | |
| CN114358910A (en) | Abnormal financial data processing method, device, equipment and storage medium | |
| CN112699022A (en) | Real-time efficient automatic contract testing method and system | |
| CN112433909A (en) | Processing method of real-time monitoring data based on kafka | |
| CN116415662B (en) | Factory expert system based on knowledge discovery | |
| CN115826503B (en) | Machine tool remote diagnosis method based on industrial internet big data | |
| CN117833296B (en) | Energy storage device performance optimization system and method based on electric power spot transaction data | |
| CN113570333B (en) | Process design method suitable for integration | |
| CN117520172A (en) | Application program testing method, system, device, equipment and storage medium | |
| CN116049167A (en) | Big data-based equipment management platform and processing method thereof | |
| CN114546764A (en) | Method and system for remotely controlling T-BOX system log | |
| CN116775981A (en) | System recommendation method, device, computer equipment and storage medium | |
| CN117172347A (en) | Carbon emission prediction method based on energy big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210316 |
|
| WW01 | Invention patent application withdrawn after publication |