CN116614794A - Mass data access method, electronic equipment and vehicle - Google Patents
Mass data access method, electronic equipment and vehicle Download PDFInfo
- Publication number
- CN116614794A CN116614794A CN202310545444.2A CN202310545444A CN116614794A CN 116614794 A CN116614794 A CN 116614794A CN 202310545444 A CN202310545444 A CN 202310545444A CN 116614794 A CN116614794 A CN 116614794A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- map data
- edge node
- node
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 94
- 230000008569 process Effects 0.000 claims abstract description 29
- 238000004891 communication Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 description 15
- 230000005540 biological transmission Effects 0.000 description 14
- 238000013523 data management Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000013480 data collection Methods 0.000 description 7
- 238000007726 management method Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
- H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0925—Management thereof using policies
- H04W28/0942—Management thereof using policies based on measured or predicted load of entities- or links
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a mass data access method, electronic equipment and a vehicle, and belongs to the technical field of data access, wherein the method comprises the following steps: receiving crowd-sourced map data acquired by a sensor of the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle; determining a target edge node in a plurality of edge nodes, and establishing communication connection with the target edge node; the edge nodes are nodes which are within a preset distance range from the vehicle; the target edge node uploads the crowd sourced map data. According to the method provided by the invention, the deployment and management of the edge nodes are carried out, the data receiving pressure of the cloud is shared, the centralized mass data access is optimized to a plurality of edge node accesses, the rapid receiving and storage management of the mass data is realized, and the real-time performance and the reliability of the server are improved.
Description
Technical Field
The invention belongs to the technical field of data access, and particularly relates to a mass data access method, electronic equipment and a vehicle.
Background
The crowdsourcing map is used for acquiring high-precision map data in a crowdsourcing mode, and the high-precision map is manufactured based on the vehicle-end map data acquired by crowdsourcing. Compared with the traditional data acquisition mode of the professional surveying and mapping vehicle, the crowdsourcing map has the advantages of low cost, quick updating and the like, meanwhile, the defect of low precision exists, and in order to make up the defect of low precision, the crowdsourcing map improves the precision at the cloud end through the technical means by acquiring mass data.
However, the crowdsourcing map data transmission terminal is a vehicle, is often in a dynamic state, has rapid state change and inconsistent surrounding network conditions, further increases the difficulty of data transmission, and is a problem to be solved in order to ensure that massive crowdsourcing map data is successfully uploaded to the cloud.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention provide a mass data access method, an electronic device, and a vehicle, so as to overcome or at least partially solve the foregoing problems.
The embodiment of the invention discloses a mass data access method, which is applied to a vehicle, and comprises the following steps:
receiving crowd-sourced map data acquired by a sensor of the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle;
Determining a target edge node in a plurality of edge nodes, and establishing communication connection with the target edge node; the edge nodes are nodes which are within a preset distance range from the vehicle;
uploading the crowd sourced map data to the target edge node.
Optionally, the determining the target edge node among the plurality of edge nodes includes:
determining the network delay sizes corresponding to a plurality of edge nodes respectively;
and determining the edge node corresponding to the minimum network delay as the target edge node based on the network delay.
Optionally, the determining the network delay sizes corresponding to the edge nodes respectively includes:
acquiring a node list, wherein the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes;
the node list is issued to the vehicle by the cloud, or is a node list acquired by the vehicle at a historical moment, or is the node list acquired by the vehicle through an Internet domain name system.
Optionally, before the uploading the crowd-sourced map data to the target edge node, the method further comprises:
after the crowd-sourced map data uploaded last time is obtained, the running track of the vehicle is obtained;
Determining the data amount of the crowdsourcing map data which is accumulated again after the crowdsourcing map data is uploaded last time based on the driving track;
and uploading the crowdsourcing map data to the target edge node when the data volume is greater than or equal to a preset data volume.
The second aspect of the embodiment of the invention discloses a mass data access method, which is applied to an edge node and comprises the following steps:
receiving crowd-sourced map data uploaded by a vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle;
caching the crowdsourcing map data in a message queue of the edge node;
the crowd-sourced map data in the message queue is used for being acquired by a cloud.
Optionally, the receiving crowd-sourced map data uploaded by the vehicle includes:
creating at least one receiving thread through a load balancer according to the bandwidth utilization rate of the edge node;
and receiving the crowdsourcing map data uploaded by the vehicle through at least one receiving thread.
In a third aspect of the embodiment of the present invention, a method for accessing mass data is disclosed, which is applied to a cloud, where the cloud is communicatively connected to a plurality of edge nodes, and the method includes:
Periodically acquiring cached crowdsourcing map data from a message queue of each edge node;
the crowdsourcing map data is sent to the edge node by a vehicle and is cached in the message queue by the edge node, and at least comprises collected route identifiers and semantic information corresponding to the route identifiers in the running process of the vehicle.
Optionally, the method further comprises:
detecting the network delay of each edge node in real time, and writing the network delay into a node list; the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes;
and when a node list acquisition request sent by the vehicle is received, sending the node list to the vehicle.
In a fourth aspect of the embodiment of the present invention, a mass data access device is disclosed, applied to a vehicle, the device comprising:
the first receiving module is used for receiving crowdsourcing map data acquired by a sensor of the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle;
The establishing module is used for determining a target edge node in a plurality of edge nodes and establishing communication connection with the target edge node; the edge nodes are nodes which are within a preset distance range from the vehicle;
and the uploading module is used for uploading the crowdsourcing map data to the target edge node.
Optionally, the first receiving module includes:
a determining module, configured to determine the network delay sizes corresponding to the edge nodes respectively;
and determining the edge node corresponding to the minimum network delay as the target edge node based on the network delay.
Optionally, the determining module includes:
the acquisition module is used for acquiring a node list, wherein the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes;
the node list is issued to the vehicle by the cloud, or is a node list acquired by the vehicle at a historical moment, or is the node list acquired by the vehicle through an Internet domain name system.
Optionally, before the uploading the crowdsourcing map data to the target edge node, the apparatus further comprises:
the judging module is used for acquiring the driving track of the vehicle after the crowdsourcing map data is uploaded last time;
Determining the data amount of the crowdsourcing map data which is accumulated again after the crowdsourcing map data is uploaded last time based on the driving track;
and uploading the crowdsourcing map data to the target edge node when the data volume is greater than or equal to a preset data volume.
In a fifth aspect of the embodiment of the present invention, a mass data access device is disclosed, applied to an edge node, the device includes:
the second receiving module is used for receiving crowdsourcing map data uploaded by the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle;
the caching module is used for caching the crowdsourcing map data in a message queue of the edge node;
the crowd-sourced map data in the message queue is used for being acquired by a cloud.
Optionally, the second receiving module includes:
the creating module is used for creating at least one receiving thread through a load balancer according to the bandwidth utilization rate of the edge node;
and receiving the crowdsourcing map data uploaded by the vehicle through at least one receiving thread.
In a sixth aspect of the embodiment of the present invention, a mass data access device is disclosed, and is applied to a cloud, where the cloud is communicatively connected to a plurality of edge nodes, and the device includes:
The period acquisition module is used for periodically acquiring cached crowdsourcing map data from the message queue of each edge node;
the crowdsourcing map data is sent to the edge node by a vehicle and is cached in the message queue by the edge node, and at least comprises collected route identifiers and semantic information corresponding to the route identifiers in the running process of the vehicle.
Optionally, the apparatus further comprises:
the sending module is used for detecting the network delay of each edge node in real time and writing the network delay into a node list; the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes;
and when a node list acquisition request sent by the vehicle is received, sending the node list to the vehicle.
In a seventh aspect of the embodiment of the present invention, an electronic device is provided, where the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for accessing mass data according to the first aspect of the embodiment of the present invention, the method for accessing mass data according to the second aspect of the embodiment of the present invention, or the method for accessing mass data according to the third aspect of the embodiment of the present invention when executing the computer program.
An eighth aspect of the embodiment of the present application provides a vehicle, including a mass data access module, where the mass data access module is configured to implement a mass data access method according to the first aspect of the embodiment of the present application, or a mass data access method according to the second aspect of the embodiment of the present application, or a mass data access method according to the third aspect of the embodiment of the present application.
The mass data access method provided by the application comprises the steps of firstly receiving crowdsourcing map data acquired by a sensor of a vehicle, wherein the crowdsourcing map data at least comprises a acquired route identifier and semantic information corresponding to the route identifier in the running process of the vehicle, then determining a target edge node in a plurality of edge nodes, and establishing communication connection with the target edge node; the edge nodes are nodes which are within a preset distance range from the vehicle; and then uploading the crowdsourcing map data to the target edge node by the vehicle, and pulling the data to the target edge node by the cloud.
Mainly comprises the following advantages:
firstly, a target edge node is selected and used as an access node of crowdsourcing map data, so that stability and instantaneity of uploading the crowdsourcing map data by a vehicle can be guaranteed, the crowdsourcing data acquired by the vehicle is transmitted to the target edge node, after the target edge node receives the data, the data is transmitted to a data receiving service through load balancing, the data receiving service writes the data into a message queue, throughput of the message queue is fully utilized, gu Pingfeng is realized on massive data, and the data transmission process is more stable.
In addition, through the deployment and management of the edge nodes, the problem of network change transmission of the vehicle in a motion state can be effectively solved, the transmission quality is guaranteed, meanwhile, the mass data receiving pressure of the cloud is shared, the centralized mass data access is optimized to be accessed by a plurality of edge access nodes, the rapid receiving and storage management of mass map data is realized, and the real-time performance and the reliability of a server side are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of steps of a mass data access method applied to a vehicle according to an embodiment of the present application;
fig. 2 is a flowchart of steps of a method for accessing mass data applied to an edge node according to an embodiment of the present application;
FIG. 3 is a flowchart of steps of a method for accessing mass data according to an embodiment of the present application;
Fig. 4 is a diagram of a mass data access system according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a mass data access device applied to a vehicle according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a mass data access device applied to an edge node according to an embodiment of the present application;
fig. 7 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings in the embodiments of the present application. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.
In the related art, for a mass data processing system of a crowdsourcing map, access data is transmitted to a message queue, but with rapid increase of data volume, load of a server is further improved. Meanwhile, the fact that the data transmission terminal of the crowdsourcing map is a vehicle is considered, the data transmission terminal is always in a dynamic state, the state change is fast, the situation of the surrounding network is inconsistent, and the difficulty of data transmission is further increased.
The accessed mass data is rapidly processed and stored in real time, but the bottleneck problem still exists at the entrance, if the data volume is too large, the congestion occurs at the load balancing position of the entrance, and meanwhile, the quality of the communication network of the vehicle terminal is possibly unstable under the condition of rapid change and uncertainty, so that the data access failure is caused. The invention discloses a mass data access method to solve the problems.
Example 1
Referring to fig. 1, fig. 1 is a flowchart of steps of a method for accessing mass data applied to a vehicle, where the steps of the method include:
step S101: receiving crowd-sourced map data acquired by a sensor of the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle.
In this embodiment, the vehicle is a vehicle with environment awareness, and in a driving process of the vehicle, a vehicle end of the vehicle receives crowd-sourced map data collected by a sensor of the vehicle, specifically, collects road information while driving, and in this embodiment, the crowd-sourced map data at least includes a collected route identifier and semantic information corresponding to the route identifier in a driving process of the vehicle, where the route identifier includes: the semantic information corresponding to the route identification refers to the direction and the running indication of the route and the characteristics and the attributes of the route, such as the road grade, the number of lanes, the speed limit information, the road type and the like, and the semantic information can help a driver to understand the current position and the running direction and the characteristics and the rules of the road, so that the vehicle can run more safely and efficiently.
On-board sensors that collect information are performed to collect, for example: a GPS (Global Positioning System ) receiver for acquiring position information of the vehicle; the inertial measurement unit is used for acquiring acceleration and angular velocity information of the vehicle; radar and lidar for acquiring obstacle information around the vehicle; the camera and the vision sensor are used for acquiring images and video information around the vehicle; the temperature, humidity and air pressure sensors are used for acquiring weather information around the vehicle; a sound sensor for acquiring sound information around the vehicle, and the like.
Step S102: determining a target edge node in a plurality of edge nodes, and establishing communication connection with the target edge node; the edge node is a node which is within a preset distance range from the vehicle.
In this embodiment, in the process of driving the vehicle, there are a plurality of edge nodes that can be communicatively connected with the vehicle around the current position of the vehicle, but for stability and instantaneity of uploading crowd-sourced map data by the vehicle, a target edge node that is most suitable for establishing communication connection with the vehicle is selected from the plurality of edge nodes, and after the target edge node is determined, communication connection is established with the target edge node, so as to prepare for uploading the crowd-sourced map data.
In addition, the edge node refers to a node within a preset distance range from the vehicle, and can be understood as a service computing resource close to the current vehicle running environment, which can be a public cloud computing resource, a self-built machine room node, a service computing resource of other systems such as V2X (Vehicle to Everything ), 5G MEC (Mobile Edge Computing, moving to the edge) and the like.
Step S103: uploading the crowd sourced map data to the target edge node.
In this embodiment, after determining that the target edge node is completed, the vehicle may acquire the IP address of the target edge node using the internet domain name system, and upload the crowd-sourced map data to the target edge node by sending the crowd-sourced map data to the IP address of the target edge node.
In one embodiment, the determining the target edge node among the plurality of edge nodes includes: determining the network delay sizes corresponding to a plurality of edge nodes respectively; and determining the edge node corresponding to the minimum network delay as the target edge node based on the network delay.
In this embodiment, a target edge node is determined among a plurality of edge nodes around a current vehicle, first, the network delay corresponding to each of the plurality of edge nodes needs to be determined, where the network delay refers to the time required for sending crowd-sourced map data from a sending end to a receiving end, that is, the time required for uploading the crowd-sourced map to the edge nodes by the vehicle, and in order to effectively reduce the delay of data transmission and improve the efficiency of data transmission, the target edge node is determined according to the magnitude of the network delay. The closest edge node refers to the node with the lowest network delay through the test, and is usually the node with the closest physical distance.
In one embodiment, the determining the network delay sizes corresponding to the plurality of edge nodes respectively includes: acquiring a node list, wherein the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes; the node list is issued to the vehicle by the cloud, or is a node list acquired by the vehicle at a historical moment, or is the node list acquired by the vehicle through an Internet domain name system.
In this embodiment, the network delay sizes of the plurality of edge nodes around the vehicle are determined by reading information in a node list, where the node list generally includes information such as node IDs, node positions, node states, network delay sizes, available bandwidths, etc., and the network delay sizes corresponding to the plurality of edge nodes around the vehicle can be known through the node list, and in this embodiment, three modes of obtaining the node list are described below:
firstly, the vehicle sends a request to the cloud to acquire a node list of edge nodes which can be in communication connection around the vehicle, and the cloud can detect the network delay state of each edge node in real time and update the network delay state, so that the node list can be acquired by sending the request to the cloud.
Second, the vehicle selects according to the node list acquired at the current position at the historical moment, and since the edge nodes are fixed and the vehicle is not fixed, if a plurality of edge nodes existing on the node list acquired at the same position are fixed, the network delay of acquiring each node list can be performed according to the node list acquired at the current position at the historical moment.
Third, the vehicle may map the hostname of the edge node to a corresponding IP address according to the internet domain name system, thereby obtaining a node list of the edge node.
In one embodiment, before the uploading of the crowd sourced map data to the target edge node, the method further comprises: after the crowd-sourced map data uploaded last time is obtained, the running track of the vehicle is obtained; determining the data amount of the crowdsourcing map data which is accumulated again after the crowdsourcing map data is uploaded last time based on the driving track; and uploading the crowdsourcing map data to the target edge node when the data volume is greater than or equal to a preset data volume.
In this embodiment, before the crowd-sourced map data is uploaded, it needs to be determined whether the currently collected crowd-sourced map data meets a preset data amount, and the collected crowd-sourced map data is uploaded under the condition that the preset data amount is met, firstly, the time after the last time of uploading the crowd-sourced map data is determined as a starting time point, the driving track of the vehicle is collected, the data amount of the crowd-sourced map data collected in the driving track is accumulated again, the crowd-sourced map data refers to geographic position information of a road on which the vehicle is driving, state information of traffic lights, environmental information around the vehicle, road speed limit information and the like, the data amount refers to that the information needs to be converted into binary data streams when the data is transmitted, and is packaged according to a certain protocol to form a data packet, and when the data amount is greater than or equal to the preset data amount, the crowd-sourced map data is uploaded to a target edge node.
Example two
Referring to fig. 2, fig. 2 is a flowchart of steps of a method for accessing mass data applied to an edge node according to an embodiment of the present invention, where the steps of the method include:
step S201: receiving crowd-sourced map data uploaded by a vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle.
In this embodiment, an edge node receives crowd-sourced map data uploaded by a vehicle, where the crowd-sourced map data at least includes a collected route identifier and semantic information corresponding to the route identifier in a running process of the vehicle, and the route identifier includes: the semantic information corresponding to the route identification refers to the direction and the running indication of the route and the characteristics and the attributes of the route, such as the road grade, the number of lanes, the speed limit information, the road type and the like, and the semantic information can help a driver to understand the current position and the running direction and the characteristics and the rules of the road, so that the vehicle can run more safely and efficiently.
Step S202: caching the crowdsourcing map data in a message queue of the edge node; the crowd-sourced map data in the message queue is used for being acquired by a cloud.
In this embodiment, the edge node caches the crowd-sourced map data in a message queue in the edge node, where the crowd-sourced map data cached in the message queue is for being acquired by the cloud.
In one embodiment, the receiving crowd sourced map data uploaded by a vehicle includes:
creating at least one receiving thread through a load balancer according to the bandwidth utilization rate of the edge node; and receiving the crowdsourcing map data uploaded by the vehicle through at least one receiving thread.
In this embodiment, before the edge node receives the crowd-sourced data uploaded by the vehicle, a data receiving service, a message queue and a load balancer are disposed in the edge node, after the vehicle establishes a communication connection with the edge node, the edge node may create at least one receiving thread according to its current bandwidth usage rate through the load balancer to receive the crowd-sourced map data uploaded by the vehicle, the number of the receiving threads is determined according to the current bandwidth usage rate of the edge node, generally, when the bandwidth usage rate of one edge node is fixed, if the bandwidth usage rate is already half of the bandwidth usage rate, only the remaining part of the bandwidth is used to receive the crowd-sourced map data uploaded by the vehicle, after the load balancer receives the resource load condition of the message queue in the edge node, an optimal load balancing is selected to create a plurality of running instances, that is, the receiving threads may also dynamically adjust the number of instances of the data receiving service according to the size of the crowd-sourced map data, so as to implement smooth expansion, to ensure the stability of the data access to store the received crowd-sourced data in the message queue, and implement a load balancing algorithm (e.g., a hash algorithm, a load balancing method, a virtual polling method, and a virtual polling method, can be implemented by using a plurality of load balancers, and the like. How many threads are specifically created can be specifically allocated according to actual situations, and the application is not limited.
Example III
The mass data access method provided by the embodiment of the invention is applied to a cloud, wherein the cloud is in communication connection with a plurality of edge nodes, and the method comprises the following steps: periodically acquiring cached crowdsourcing map data from a message queue of each edge node; the crowdsourcing map data is sent to the edge node by a vehicle and is cached in the message queue by the edge node, and at least comprises collected route identifiers and semantic information corresponding to the route identifiers in the running process of the vehicle.
In this embodiment, the cloud end includes a data collection service, where the purpose of the data collection service is to periodically pull cached crowd-sourced map data from a message queue of an edge node, where the crowd-sourced map data is sent from a vehicle to the edge node, and the edge node caches data in the message queue, where the crowd-sourced map data at least includes route identifiers collected by sensors on the vehicle and semantic information corresponding to the route identifiers during a driving process of the vehicle.
In addition, the cloud end also comprises a data management service, wherein the data management service is responsible for processing and version management of the crowd-sourced map data, extracting route information and marking version information. The data collection service and the data management service in this embodiment can dynamically expand the capacity according to the data size, so as to cope with the increase of the data size.
After the data management service processes the crowd-sourced map data, version information is marked, and the crowd-sourced map data of the last version is updated, so that when a vehicle wants to acquire the crowd-sourced map from the cloud, the cloud can send the latest version of the crowd-sourced map to the vehicle or can send information about whether to update the version of the crowd-sourced map to a user.
In one embodiment, the method further comprises: detecting the network delay of each edge node in real time, and writing the network delay into a node list; the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes; and when a node list acquisition request sent by the vehicle is received, sending the node list to the vehicle.
In this embodiment, the cloud end detects the network delay sizes of a plurality of edge nodes which are in communication connection with the cloud end in real time, writes the network delay sizes into a node list, and when receiving a node obtaining request sent by a vehicle, the cloud end sends the vehicle a node list of edge nodes which can be in communication connection with the vehicle around the vehicle. The node list comprises network delay sizes corresponding to a plurality of edge nodes.
According to the method provided by the invention, firstly, the target edge node is selected and used as the access node of the crowdsourcing map data, so that the stability and instantaneity of the crowdsourcing map data uploaded by the vehicle can be ensured, the crowdsourcing data acquired by the vehicle is transmitted to the target edge node, the target edge node transmits the data to the data receiving service through load balancing after receiving the data, the data receiving service writes the data into the message queue, the throughput of the message queue is fully utilized, and Gu Pingfeng is realized on the massive data, so that the data transmission process is more stable.
In addition, through the deployment and management of the edge nodes, the problem of network change transmission of the vehicle in a motion state can be effectively solved, the transmission quality is guaranteed, meanwhile, the mass data receiving pressure of the cloud is shared, the centralized mass data access is optimized to be accessed by a plurality of edge access nodes, the rapid receiving and storage management of mass map data is realized, and the real-time performance and the reliability of a server side are improved.
For example, the first, second and third embodiments of the present invention are described below with reference to fig. 3 and 4, and are applied to a vehicle, an edge node and a cloud as a completed updated crowd-sourced map.
Referring to fig. 3, fig. 3 is a flowchart of steps of a method for accessing mass data according to an embodiment of the present invention, which mainly includes the following steps:
step S301: first, the vehicle selects a target edge node and uploads crowd-sourced map data. The vehicle end determines that the crowdsourcing map data needs to be uploaded, firstly acquires a group of access nodes, and selects a node closest to the access nodes from the access nodes for transmission.
Step S302: the target edge node receives the crowd-sourced map data and receives the data receiving service of the back end through the load balancer.
Step S303: the data receiving service processes the data and writes it into the message queue. And the receiving service creation thread receives the crowdsourcing map data. The embodiment of the invention selects the kafka message queue, and writes the data into the kafka message queue after the receiving is completed. The kafka message queue is a distributed stream processing platform, is a high throughput, low latency publish-subscribe message system, and the other kafka message queues are referred to herein and will not be repeated.
Step S304: the cloud end regularly pulls crowd-sourced map data to the message queue. The cloud data collection service pulls crowd-sourced map data from the kafka message queues, maintains information of a set of access nodes, and pulls data to the kafka message queues in each edge node at regular time.
Step S305: and updating the crowdsourcing map by the cloud end, and marking version information. The data management service in the cloud processes the pulled crowdsourcing map data, extracts line information, updates the crowdsourcing map, and marks version information.
Step S306: and storing the updated crowdsourcing map. And writing the crowdsourcing map data, the corresponding route information and version information into an object storage. The object storage is a key-value-based storage system, supports reading and writing of mass files, and provides low-delay and high-reliability storage service. The selected object storage system may be a support s3 protocol interface.
Referring to fig. 4, fig. 4 is a system diagram of mass data access according to an embodiment of the present invention, where the system diagram includes an access node 401, a cloud 402, and a data storage 403 for explanation.
The access node 401, i.e. the edge node, is structured by a load balancer, a data receiving service, a kafka message queue. In the load balancer, an optimal load balancing algorithm can be selected according to the resource load condition of the data receiving service. The data receiving service creates a plurality of running instances, and the running instances are processed by the load balancing selection instance. According to the size of the received data volume, the number of instances of the data receiving service can be dynamically adjusted, the increasing data volume is dealt with, smooth expansion is achieved, and the stability of data access is ensured.
Cloud 402 includes data collection services and data management services. The data collection service is responsible for maintaining the state and information of the access nodes and for data pulling from the kafka message queues in the access nodes. The data management service is responsible for processing and version management of the crowd-sourced map data, extracting route information and marking version information. Both the data collection service and the data management service can dynamically expand the capacity according to the data size so as to cope with the increase of the data size.
The data storage 403 adopts an object storage system, the object storage system can provide storage service of mass files, the number of the object storage system can reach billions, the storage capacity can support PB-level storage, and the object storage system can also provide hundred millisecond-level low-delay access. The object storage system provides multiple data backup modes such as double-copy, triple-copy, erasure codes and the like, and extremely high data reliability is provided. The object storage system supports dynamic expansion to cope with an increase in data size.
Example IV
Referring to fig. 5, fig. 5 is a schematic structural diagram of a mass data access device applied to a vehicle according to an embodiment of the present invention, where the device includes: a first receiving module 501, a setting up module 502 and an uploading module 503.
A first receiving module 501, configured to receive crowd-sourced map data acquired by a sensor of the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle.
An establishing module 502, configured to determine a target edge node from a plurality of edge nodes, and establish a communication connection with the target edge node; the edge node is a node which is within a preset distance range from the vehicle.
An uploading module 503, configured to upload the crowd-sourced map data to the target edge node.
In this embodiment, the first receiving module 501 includes: a determining module, configured to determine the network delay sizes corresponding to the edge nodes respectively; and determining the edge node corresponding to the minimum network delay as the target edge node based on the network delay.
In this embodiment, the determining module includes: the acquisition module is used for acquiring a node list, wherein the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes; the node list is issued to the vehicle by the cloud, or is a node list acquired by the vehicle at a historical moment, or is the node list acquired by the vehicle through an Internet domain name system.
In this embodiment, before the uploading the crowd-sourced map data to the target edge node, the apparatus further includes: the judging module is used for acquiring the driving track of the vehicle after the crowdsourcing map data is uploaded last time; determining the data amount of the crowdsourcing map data which is accumulated again after the crowdsourcing map data is uploaded last time based on the driving track; and uploading the crowdsourcing map data to the target edge node when the data volume is greater than or equal to a preset data volume.
Example five
Referring to fig. 6, a schematic structural diagram of a mass data access device applied to an edge node according to an embodiment of the present invention includes: a second receiving module 601 and a buffering module 602.
A second receiving module 601, configured to receive crowd-sourced map data uploaded by a vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle.
A caching module 602, configured to cache the crowd-sourced map data in a message queue of the edge node; the crowd-sourced map data in the message queue is used for being acquired by a cloud.
In this embodiment, the second receiving module 601 includes: the creating module is used for creating at least one receiving thread through a load balancer according to the bandwidth utilization rate of the edge node; and receiving the crowdsourcing map data uploaded by the vehicle through at least one receiving thread.
Example six
The mass data access device provided by the embodiment of the invention is applied to a cloud, wherein the cloud is in communication connection with a plurality of edge nodes, and the device comprises: and a period acquisition module.
The period acquisition module is used for periodically acquiring cached crowdsourcing map data from the message queue of each edge node; the crowdsourcing map data is sent to the edge node by a vehicle and is cached in the message queue by the edge node, and at least comprises collected route identifiers and semantic information corresponding to the route identifiers in the running process of the vehicle.
In this embodiment, the apparatus further includes:
the sending module is used for detecting the network delay of each edge node in real time and writing the network delay into a node list; the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes; and when a node list acquisition request sent by the vehicle is received, sending the node list to the vehicle.
Example seven
The embodiment of the invention provides electronic equipment, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the mass data access method is realized when the processor executes the computer program.
In this embodiment, referring to fig. 7, fig. 7 is a schematic diagram of an electronic device according to an embodiment of the present invention; as shown in fig. 7, the electronic device 100 includes: the memory 110 and the processor 120 are connected through bus communication, and the memory 110 stores a computer program in the memory 110, and the computer program can run on the processor 120, so as to implement the mass data access method according to the embodiment of the invention.
Example eight
The embodiment of the invention provides a vehicle, which comprises a mass data access module, wherein the mass data access module is used for executing and realizing the mass data access method according to the embodiment of the invention.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The above describes in detail a mass data access method, an electronic device and a vehicle provided by the present invention, and specific examples are applied to describe the principle and implementation of the present invention, where the description of the above examples is only used to help understand the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (10)
1. A mass data access method, characterized by being applied to a vehicle, the method comprising:
receiving crowd-sourced map data acquired by a sensor of the vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle;
determining a target edge node in a plurality of edge nodes, and establishing communication connection with the target edge node; the edge nodes are nodes which are within a preset distance range from the vehicle;
uploading the crowd sourced map data to the target edge node.
2. The method of claim 1, wherein said determining a target edge node among a plurality of edge nodes comprises:
determining the network delay sizes corresponding to a plurality of edge nodes respectively;
and determining the edge node corresponding to the minimum network delay as the target edge node based on the network delay.
3. The method of claim 2, wherein determining the network delay sizes respectively corresponding to the plurality of edge nodes comprises:
acquiring a node list, wherein the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes;
the node list is issued to the vehicle by the cloud, or is a node list acquired by the vehicle at a historical moment, or is the node list acquired by the vehicle through an Internet domain name system.
4. The method of claim 1, wherein prior to the uploading the crowd-sourced map data to the target edge node, the method further comprises:
after the crowd-sourced map data uploaded last time is obtained, the running track of the vehicle is obtained;
determining the data amount of the crowdsourcing map data which is accumulated again after the crowdsourcing map data is uploaded last time based on the driving track;
And uploading the crowdsourcing map data to the target edge node when the data volume is greater than or equal to a preset data volume.
5. A method for accessing mass data, applied to an edge node, the method comprising:
receiving crowd-sourced map data uploaded by a vehicle; the crowdsourcing map data at least comprises collected route identifications and semantic information corresponding to the route identifications in the running process of the vehicle;
caching the crowdsourcing map data in a message queue of the edge node; the crowd-sourced map data in the message queue is used for being acquired by a cloud.
6. The method of claim 5, wherein receiving crowd sourced map data uploaded by a vehicle comprises:
creating at least one receiving thread through a load balancer according to the bandwidth utilization rate of the edge node;
and receiving the crowdsourcing map data uploaded by the vehicle through at least one receiving thread.
7. The mass data access method is characterized by being applied to a cloud end, wherein the cloud end is in communication connection with a plurality of edge nodes, and the method comprises the following steps:
periodically acquiring cached crowdsourcing map data from a message queue of each edge node;
The crowdsourcing map data is sent to the edge node by a vehicle and is cached in the message queue by the edge node, and at least comprises collected route identifiers and semantic information corresponding to the route identifiers in the running process of the vehicle.
8. The method of claim 7, wherein the method further comprises:
detecting the network delay of each edge node in real time, and writing the network delay into a node list; the node list comprises a plurality of network delay sizes respectively corresponding to the edge nodes;
and when a node list acquisition request sent by the vehicle is received, sending the node list to the vehicle.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the mass data access method of any one of claims 1 to 4, or the mass data access method of any one of claims 5 to 6, or the mass data access method of any one of claims 7 to 8 when the computer program is executed.
10. A vehicle comprising a mass data access module, characterized in that the mass data access module is configured to perform a mass data access method according to any one of claims 1 to 4, or a mass data access method according to any one of claims 5 to 6, or a mass data access method according to any one of claims 7 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310545444.2A CN116614794A (en) | 2023-05-15 | 2023-05-15 | Mass data access method, electronic equipment and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310545444.2A CN116614794A (en) | 2023-05-15 | 2023-05-15 | Mass data access method, electronic equipment and vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116614794A true CN116614794A (en) | 2023-08-18 |
Family
ID=87675880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310545444.2A Pending CN116614794A (en) | 2023-05-15 | 2023-05-15 | Mass data access method, electronic equipment and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116614794A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117729206A (en) * | 2024-02-07 | 2024-03-19 | 湖南三湘银行股份有限公司 | Multimedia access method based on Internet of things |
-
2023
- 2023-05-15 CN CN202310545444.2A patent/CN116614794A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117729206A (en) * | 2024-02-07 | 2024-03-19 | 湖南三湘银行股份有限公司 | Multimedia access method based on Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9231998B2 (en) | Vehicle-specific computation management system for cloud computing | |
| EP3594811B1 (en) | Managing computational tasks in vehicle context | |
| WO2020147487A1 (en) | Method, device and system for map data updating and storage medium | |
| JP7059362B2 (en) | Map data construction method, vehicle terminal, and server | |
| US8897806B2 (en) | Dynamic data publication and dissemination | |
| JP7062892B2 (en) | Dynamic map updater, dynamic map update method, dynamic map update program | |
| JP6922937B2 (en) | Edge data transmission for connected vehicles | |
| EP3809647B1 (en) | Network management device, method and program | |
| CN111221679B (en) | Vehicle-to-vehicle distributed data storage system lost data recovery | |
| CN116614794A (en) | Mass data access method, electronic equipment and vehicle | |
| US10681639B2 (en) | Systems and methods for receiving sensor data from a mobile device | |
| US20230115290A1 (en) | In-vehicle/out-vehicle cooperation device and method | |
| US11689622B2 (en) | Efficient real time vehicular traffic reporting and sharing | |
| WO2019182656A1 (en) | Connecting and managing vehicles using a publish-subscribe system | |
| CN115174490B (en) | Data transmission method and network application terminal | |
| JP2021167986A (en) | Sensor sharing system, sensor sharing device, sensor sharing method, and computer program | |
| CN111291775B (en) | Vehicle positioning method, device and system | |
| WO2018175421A1 (en) | Collaborative geo-positioning of electronic devices | |
| CN108770014B (en) | Calculation evaluation method, system and device of network server and readable storage medium | |
| JP2023029600A (en) | wireless communication system | |
| Wang et al. | Minerva: Information-centric programming for social sensing | |
| CN114501374A (en) | Dynamic service deployment method, system, device and storage medium for Internet of vehicles | |
| JP7000884B2 (en) | Data acquisition system and server | |
| Eryilmaz et al. | An architecture for dynamic context recognition in an autonomous driving testing environment | |
| CN117539916B (en) | Data display method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |