WO2021057324A1 - 数据处理方法、装置、芯片系统及介质 - Google Patents
数据处理方法、装置、芯片系统及介质 Download PDFInfo
- Publication number
- WO2021057324A1 WO2021057324A1 PCT/CN2020/109187 CN2020109187W WO2021057324A1 WO 2021057324 A1 WO2021057324 A1 WO 2021057324A1 CN 2020109187 W CN2020109187 W CN 2020109187W WO 2021057324 A1 WO2021057324 A1 WO 2021057324A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- target
- attribute description
- sensor
- feedback
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 148
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 105
- 238000000034 method Methods 0.000 claims abstract description 77
- 238000005259 measurement Methods 0.000 claims abstract description 17
- 230000003993 interaction Effects 0.000 claims description 23
- 230000002452 interceptive effect Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 230000005540 biological transmission Effects 0.000 abstract description 11
- 230000002457 bidirectional effect Effects 0.000 abstract description 2
- 230000004927 fusion Effects 0.000 description 26
- 230000008569 process Effects 0.000 description 19
- 238000004891 communication Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 11
- 238000001914 filtration Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 8
- 230000001133 acceleration Effects 0.000 description 6
- 238000012937 correction Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 238000009434 installation Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/35—Data fusion
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/45—External transmission of data to or from the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
Definitions
- This application relates to the field of automatic driving technology, and specifically relates to a data processing method.
- this application also relates to a data processing device, a chip system, and a computer-readable storage medium.
- FIG. 1 is a schematic diagram of a typical data processing architecture of an autonomous vehicle.
- the processing unit of the autonomous vehicle is responsible for data processing, and it can include three modules: a data fusion module, a planning decision module, and a behavior control module.
- a sensor such as a camera, lidar, sonar, etc.
- captures an electrical signal in a traffic scene it digitizes it and converts it into digital signal data, that is, raw data.
- the sensor uses the recognition algorithm to process the original data to obtain abstract data.
- the abstract data generally includes the traffic target identified from the traffic scene, as well as the attribute description data of the traffic target, such as the color and size of the vehicle, and the indication content of the traffic sign.
- Each sensor sends its own abstract data to the data fusion module in the processing unit.
- the data fusion module fuses these abstract data, and integrates the abstract data of multiple sensors to re-identify the traffic target and the attribute description data of the traffic target to obtain the fused data.
- the attribute description data of the traffic target contained in the fusion data may be the same or different from the attribute description data of the traffic target identified by each sensor.
- the fusion data can be used to construct a world model (also known as an environmental model) in a computer, so as to simulate and restore the situation in the real world.
- the planning and decision-making module plans and decides the driving route of the vehicle according to the world model.
- the behavior control module instructs the actuators (such as accelerator, brake, steering wheel, car window, car lights, etc.) to perform operations based on the decision, so as to realize the control of the vehicle's trajectory.
- a sensor uses a recognition algorithm to process the original data acquired in a certain traffic scene, and after obtaining the abstract data, the sensor has no way of knowing whether the abstract data is consistent with the real situation. If the abstract data is inconsistent with the real situation, then every time the sensor obtains the same original data, it will process the same abstract data that is inconsistent with the real situation. This makes it difficult to improve the recognition accuracy of the sensor.
- the present application provides a data processing method.
- the sensor using this method can perform two-way data transmission with the processing unit, and the recognition algorithm is optimized according to the feedback data fed back by the processing unit, thereby improving the recognition accuracy of the sensor.
- the present application provides a data processing method, the method includes: obtaining first abstract data from first original data through a first recognition algorithm, the first abstract data including attribute description data of a first target; Receiving first feedback data, the first feedback data including the attribute description data of the second target; optimizing the first recognition algorithm according to the first feedback data; wherein, the first raw data is the measurement data of the scene, The first target and the second target are targets in the scene.
- the first sensor not only sends the information recognized from the traffic scene, that is, the first abstract data, to the outside, but can also receive the first feedback data from the outside, realizing two-way transmission.
- the first sensor can then optimize the first recognition algorithm based on the attribute description data of the second target in the same traffic scene, so as to improve the accuracy of obtaining the first abstract data from the first original data by the first recognition algorithm, that is, improve The recognition accuracy of the first sensor.
- the second target includes at least one first specific target, and the first target has the same target as the first specific target.
- the first specific target is a target that is also recognized by the first sensor.
- the first sensor uses the attribute description data of the first specific target determined from other ways to optimize the first recognition algorithm, thereby improving the performance of the first sensor. Recognition accuracy rate.
- the second target includes at least one second specific target, and the first target does not exist in the same manner as the second specific target.
- the second specific target is a target not recognized by the first sensor.
- the first sensor uses the attribute description data of the second specific target determined from other ways to optimize the first recognition algorithm, thereby improving the performance of the first sensor. Recognition accuracy rate.
- the first feedback data includes location information and attribute description data of the second specific target;
- the step of optimizing the first recognition algorithm with the first feedback data includes: optimizing the first recognition algorithm according to the location information and attribute description data of the at least one second specific target. For the second specific target, since the first sensor does not recognize the target, the location information is used to associate the second specific target with the feature points of the corresponding location in the first raw data. In this way, the first sensor can use the attribute description data of the second specific target to optimize the first recognition algorithm, thereby improving the recognition accuracy of the first sensor.
- the attribute description data of the second target is determined based on the first abstract data, or based on data from the cloud Data determined by the interaction information of the sensor.
- the attribute description data of the second target is determined based on the first abstract data or the interaction information from the cloud sensor, which is closer to the actual situation of the target in the scene. Feeding such data back to the first sensor helps the first sensor optimize the first recognition algorithm, thereby improving the recognition accuracy.
- the first original data, the first abstract data, and the first feedback data include a time stamp; wherein, The timestamp is used to indicate the time information of the first original data obtained from the scene; the step of optimizing the first recognition algorithm according to the first feedback data includes: corresponding to the timestamp The first original data, the first abstract data, and the first feedback data are optimized for the first recognition algorithm.
- the first sensor can find the first original data, the first abstract data and the first feedback data corresponding to the timestamp, avoiding the influence caused by the confusion of the first original information, the first abstract data and the first feedback data at different times The recognition accuracy of the first sensor.
- the step of optimizing the first recognition algorithm according to the first feedback data includes: according to the second The attribute description data of the target and the confidence level corresponding to the attribute description data of the second target are optimized to optimize the first recognition algorithm; wherein the confidence level is used to characterize the credibility of the attribute description data of the second target degree.
- the first sensor can adjust the correction range of the parameters of the first recognition algorithm according to the difference in the confidence of the second attribute description data, thereby further improving the recognition accuracy of the optimized first sensor.
- the attribute description data of the second target includes a source tag
- the source tag is used to identify the second target's The source of the attribute description data, and there is a correspondence between the source of the attribute description data of the second target and the confidence of the attribute description data of the second target.
- the first sensor can determine the corresponding confidence level according to the source tag of the attribute description data of the second target, thereby adjusting the correction range of the parameters of the first recognition algorithm, and further improving the recognition accuracy of the optimized first sensor rate.
- the present application provides a data processing method, including: receiving first abstract data from a first sensor, the first abstract data is derived from first raw data, and the first abstract data includes the data of the first target Attribute description data; determine the first feedback data, the first feedback data includes the attribute description data of the second target; send the first feedback data to the first sensor; wherein, the first raw data is the scene Measurement data, the first target and the second target are targets in the scene.
- the processing unit can not only receive the information recognized by the first sensor from the traffic scene, that is, the first abstract data, but also send the determined first feedback data to the first sensor, thereby realizing two-way transmission.
- the recognition algorithm used by the first sensor can be optimized according to the first feedback data, and the recognition accuracy of the first sensor can be improved.
- the method further includes: determining at least one first specific target based on the first abstract data or based on interaction information from cloud sensors;
- the first target has the same target as the first specific target; wherein the first feedback data includes attribute description data of the at least one first specific target.
- the processing unit can determine the first specific target that can also be recognized by the first recognition sensor based on the first abstract data, or based on the interaction information from the cloud sensor, so that the first specific target determined by other means can be used.
- the attribute description data of the target is fed back to the first sensor. In this way, the first sensor can use the attribute description data of the first specific target to optimize the first recognition algorithm, thereby improving the recognition accuracy of the first sensor.
- the method further includes: determining at least one first based on the second abstract data, or based on interactive information from cloud sensors Two specific targets; wherein the second abstract data is derived from second original data, the second original data is measurement data of the scene, and the second abstract data includes the attribute description data of the third target, the The third target is the target in the scene; the first target does not have the same target as the second specific target; wherein, the first feedback data includes the attribute description of the at least one second specific target data.
- the processing unit can determine the second specific target that is not recognized by the first recognition sensor based on the second abstract data, or based on the interactive information from the cloud sensor, so that the second specific target determined by other means can be used.
- the attribute description data of the target is fed back to the first sensor.
- the first sensor can use the attribute description data of the second specific target to optimize the first recognition algorithm, thereby improving the recognition accuracy of the first sensor.
- the first feedback data further includes location information of the at least one second specific target.
- the processing unit may also feed back the position information of the second specific target to the first sensor.
- the first sensor can associate the second specific target with the feature points of the corresponding location in the first raw data according to the location information, and use the attribute description data of the second specific target to optimize the first recognition algorithm , Improve the recognition accuracy of the first sensor.
- the step of determining the first feedback data includes: based on the first abstract data, or based on a cloud sensor To determine the attribute description data of the second target.
- the processing unit can determine the attribute description data of the second target based on the first abstract data, or based on the interaction information from the cloud sensor, so as to feed back the attribute description data of the second target determined by these methods to The first sensor.
- the first sensor can use the attribute description data of the second target fed back by these processing units to optimize the first recognition algorithm, thereby improving the recognition accuracy of the first sensor.
- the first original data, the first abstract data, and the first feedback data include a time stamp; wherein, The time stamp is used to indicate the time information of the first raw data measured from the scene; the step of sending the first feedback data to the first sensor includes: communicating with the first sensor The original data and the first feedback data corresponding to the timestamp of the first abstract data are sent to the first sensor.
- the processing unit sends the first feedback data corresponding to the timestamp to the first sensor, so that the first sensor can find the first original data, the first abstract data, and the first feedback data corresponding to the timestamp, avoiding different The confusion of the first original information, the first abstract data, and the first feedback data of time results in a situation that affects the recognition accuracy of the first sensor.
- the first feedback data includes a confidence level corresponding to each piece of attribute description data of the second target, where: The confidence is used to characterize the credibility of the attribute description data of the second target.
- the first feedback data contains the confidence level corresponding to the attribute description data of each second target, so that the first sensor can adjust the first recognition algorithm according to the difference in the confidence level of the second attribute description data.
- the parameter correction range further improves the recognition accuracy of the optimized first sensor.
- each piece of the attribute description data of the second target includes a corresponding source tag, and the source tag is used to identify the The attribute description data of the second target is the source of the data, and there is a correspondence between the source of the attribute description data of the second target and the confidence of the attribute description data of the second target.
- each piece of the attribute description data of the second target contains a corresponding source tag, so that the first sensor can determine the corresponding confidence level according to the source tag of the attribute description data of the second target, and then adjust the first The correction range of the parameters of the recognition algorithm further improves the recognition accuracy of the optimized first sensor.
- the present application provides a data processing device, including: a first transceiver module, configured to receive first feedback data, where the first feedback data includes attribute description data of a second target; at least one first processing module, For obtaining first abstract data from the first original data through a first recognition algorithm; and optimizing the first recognition algorithm according to the first feedback data; wherein the first abstract data includes the attributes of the first target Descriptive data, the first raw data is measurement data of a scene, and the first target and the second target are targets in the scene.
- the present application provides a data processing device, including: a second transceiver module for receiving first abstract data from a first sensor; and sending first feedback data to the first sensor; wherein The first abstract data is derived from first original data, the first abstract data includes attribute description data of a first target, the first feedback data includes attribute description data of a second target, and the first original data is a scene
- the first target and the second target are targets identified from the scene; at least one second processing module is used to determine the first feedback data.
- the present application provides a chip system, including at least one processor and an interface, where the interface is used to receive code instructions and transmit the at least one processor; the at least one processor runs the code instructions, To achieve any method of the first aspect.
- the present application provides a chip system, including at least one processor and an interface, the interface is used to receive code instructions and transmit the at least one processor; the at least one processor runs the code instructions, To achieve any method of the second aspect.
- the present application provides a computer-readable storage medium for storing computer programs or instructions.
- the computer programs or instructions When the computer programs or instructions are run on an electronic device, the electronic device realizes any of the aspects of the first aspect. a way.
- the present application provides a computer-readable storage medium for storing a computer program or instruction.
- the computer program or the instruction runs on an electronic device, the electronic device realizes any of the second aspect. a way.
- Figure 1 is a schematic diagram of a typical data processing architecture of an autonomous vehicle
- FIG. 2 is a schematic diagram of the architecture of a data processing system of a vehicle in an embodiment of the application
- FIG. 3 is a flowchart of one of the implementation modes of the data processing method of this application.
- FIG. 4 is a flowchart of the second implementation manner of the data processing method of this application.
- FIG. 5 is a flowchart of the third implementation manner of the data processing method of this application.
- FIG. 6 is a schematic structural diagram of a data processing device of this application.
- FIG. 7 is a schematic structural diagram of another data processing device of this application.
- FIG. 8 is a schematic structural diagram of one implementation of the chip system of this application.
- sensors used in areas such as autonomous driving and smart driving all send data unidirectionally to the processing unit, as shown in Figure 1. This causes the sensor to be unable to optimize its own recognition algorithm, which makes it difficult to improve the recognition accuracy of the sensor.
- sensors are installed on a vehicle, including internal sensors and external sensors.
- internal sensors refer to sensors used to measure some status data of the vehicle itself, such as gyroscopes, accelerometers, steering angle sensors, wiper activity sensors, steering indicators, etc. These internal sensors can be installed inside or outside the vehicle.
- External sensors refer to sensors used to measure environmental data around the vehicle, such as radar, lidar, ultrasonic radar, camera, global positioning system, sonar, etc. These external sensors can be installed outside or inside the vehicle.
- sensors such as microwave radars, millimeter-wave radars, cameras, etc. may also be installed on the roadside and other locations, which are mainly used to measure the condition of targets on the road.
- cloud sensors can also be used in areas such as autonomous driving and smart driving. Cloud sensors can also be called metadata sensors.
- Vehicles can establish communication connections and interact with cloud sensors, so as to obtain information about traffic targets in traffic scenes from cloud sensors.
- the cloud sensor here may be a cloud server, other terminals or base stations connected to the vehicle through vehicle-to-everything (V2X) technology.
- V2X vehicle-to-everything
- the vehicle can establish a communication connection with the cloud server, and obtain information on the transportation infrastructure near the current location of the vehicle from the high-precision map of the cloud server.
- a vehicle can establish a communication connection with another vehicle in a traffic scene, and then obtain information such as the size, color, and speed of the vehicle.
- the sensor that executes the data processing method of the embodiment of the present application may be the aforementioned sensor installed in a vehicle or a roadside position.
- any sensor that executes the data processing method of the present application is referred to as a first sensor; sensors other than the first sensor are referred to as second sensors.
- FIG. 2 is a schematic diagram of the architecture of a data processing system of a vehicle in an embodiment of the application.
- the system includes a first sensor, one or more second sensors, and one or more cloud sensors. These sensors are all connected to the processing unit, and the processing unit is also connected to multiple actuators.
- the first embodiment of the present application provides a data processing method, which is executed by a first sensor.
- the method may be executed by an electronic control unit (ECU) in the first sensor.
- ECU electronice control unit
- the first sensor can perform two-way transmission with the processing unit, receive feedback data returned, and use the feedback data to optimize the recognition algorithm used by the sensor, so as to avoid the same recognition error from recurring , Improve the recognition accuracy of the sensor.
- FIG. 3 is a flowchart of one implementation of the data processing method of this application.
- the steps performed by the first sensor include the following S101 to S103.
- the first sensor obtains first abstract data from the first raw data by using the first recognition algorithm.
- the scene mainly refers to the traffic scene, which can include traffic participants or traffic environment in the traffic scene, such as other vehicles on the road, traffic signs, etc., and can also include something related to automatic driving and smart driving. Information that is not directly related, such as the green belts on both sides of the road.
- the original data is the measurement data of the scene, which is used to digitally describe the traffic scene. Sensors can capture electrical signals from traffic scenes, and then digitize the electrical signals to obtain raw data. The specific method of capturing and processing the original data can be accomplished by the existing method, which will not be repeated here. This application does not limit the method of obtaining the original data.
- the data measured by the first sensor from the traffic scene is referred to as the first raw data.
- the specific data form of the first raw data may be different.
- the camera converts light signals captured in a traffic scene into electrical signals. Then, these electrical signals are digitally processed and converted into data that can describe the picture of the traffic scene, that is, the first raw data.
- the lidar when the first sensor is a lidar, the lidar emits laser light to scan the traffic scene. People, objects, etc. in the traffic scene reflect the laser light back. Since the intensity and time of the light reflected by different people, objects, etc., are different, the lidar can obtain information in the traffic scene accordingly. That is, the lidar receives the reflected light and converts the optical signal into an electrical signal. Then, these electrical signals are digitally processed to obtain data that can describe people and objects in the traffic scene, that is, the point cloud information of the lidar, which is the first raw data in the embodiment of the present application.
- the target in this application mainly refers to the traffic target, which can be a traffic participant or a traffic environment.
- traffic participants include dynamic people or objects, such as vehicles, pedestrians, and animals on the road, and the traffic environment includes static objects, such as traffic signs (including lane lines, traffic lights, traffic signs, etc.), guardrails, etc.
- the recognition algorithm in this application can use existing recognition algorithms, such as feature recognition algorithms, target recognition algorithms, etc., more specifically, the recognition algorithm can use Kalman filter tracking algorithms, etc.
- the sensor uses the recognition algorithm to process the original data. It can identify the traffic target from the traffic scene and the attribute description data of the traffic target.
- the attribute description data is correspondingly different.
- the traffic target is a vehicle driving on a road
- its attribute description data may be the size of the vehicle, the distance between the vehicle, surface material, direction angle, speed, acceleration, etc.
- its attribute description data may be the content and location of the traffic sign.
- the identification algorithms used to process the raw data may be different or the same, which is not limited in this application.
- the recognition algorithm used by the first sensor is called the first recognition algorithm
- the traffic target recognized by the first sensor from the traffic scene through the first recognition algorithm is called the first target.
- the abstract data obtained by the first sensor from the first raw data through the first recognition algorithm is referred to as the first abstract data.
- the first abstract data includes attribute description data of the first target, such as the category, size, distance between the first target and the vehicle, surface material, direction angle, speed, acceleration, content, or location.
- the first sensor may identify one or more first targets, and each first target may correspond to one or more attribute description data.
- the first abstract data may include attribute description data corresponding to all the first targets. It is also possible that the first sensor did not recognize any traffic targets. At this time, the attribute description data of the first target in the first abstract data is empty.
- the first sensor is a camera, and the camera recognizes multiple first targets, including traffic targets with target IDs T001 and T002.
- the attributes corresponding to T001 include type, size, color, etc.
- the attributes corresponding to T002 include type, content, and so on.
- the first abstract data includes attribute description data of multiple first targets, and part of the attribute description data of two first targets is shown in Table 1.
- the first sensor receives first feedback data.
- the first sensor sends the first abstract data to the processing unit, and then receives the first feedback data sent by the processing unit, where the first feedback data includes the attribute description data of the second target.
- the second target here refers to the traffic target identified from the same traffic scene mentioned above, which may be the same or different from the first target. Similar to the attribute description data of the first target, the attribute description data of the second target can be the category, size, distance from the first target, surface material, direction angle, speed, acceleration, content of the second target Or location, etc.
- the processing unit may determine the attribute description data of the second target included in the first feedback data in various ways.
- the attribute description data of the second target is data determined based on the first abstract data, or data determined based on interactive information from cloud sensors.
- the processing unit may fuse abstract data from multiple sensors to obtain the fused data, and then determine the attribute description data of the second target based on the fused data.
- the processing unit may interact with a cloud sensor, receive interaction information from the cloud sensor, and then determine the attribute description data of the second target based on the interaction information.
- the attribute description data of the second target determined by the fused data or interactive information is closer to the actual situation of the traffic target in the traffic scene. Feeding such data back to the first sensor helps the first sensor optimize the first recognition algorithm and improve the recognition accuracy.
- the processing unit may first determine the second target, which may specifically be determined in a variety of ways. For example, in an implementation manner, the processing unit may fuse abstract data from multiple sensors to obtain the fused data, and then determine the second target based on the fused data. For another example, in another implementation manner, the processing unit may determine the second target based on interactive information from cloud sensors.
- multiple sensors are generally installed at locations such as vehicles or roadsides, including a first sensor and at least one second sensor.
- the first sensor and the second sensor each obtain the original data from the traffic scene, and use their corresponding recognition algorithms to process the original data to obtain the corresponding abstract data.
- the data measured by the second sensor from the traffic scene is referred to as the second raw data.
- Each second sensor uses the corresponding second recognition algorithm to process the second original data measured respectively to obtain the corresponding second abstract data. That is, for any second sensor, the steps it executes may include the step of S301.
- the second sensor obtains second abstract data from the second original data by using the second recognition algorithm.
- the process of obtaining the second abstract data is also the process of each second sensor identifying the traffic target from the traffic scene.
- the traffic target recognized by the second sensor in the traffic scene is referred to as the third target in the embodiment of the present application.
- the second abstract data includes the attribute description data of the third object.
- the second original data is similar to the first original data
- the second recognition algorithm is similar to the first recognition algorithm
- the second abstract data is similar to the first abstract data. You can refer to the previous related descriptions, which will not be repeated here.
- Each second sensor sends its respective second abstract data to the processing unit.
- one of the second sensors 1 is a lidar, and the lidar has identified three third targets with target IDs of T001, T002, and T003.
- the attributes corresponding to T001 include type, size, color, etc.
- the attributes corresponding to T002 include type, content, etc.
- the attributes corresponding to T003 include type, size, and so on.
- the other second sensor 2 is a millimeter wave radar, which also recognizes the same three third targets T001, T002, and T003 as the first sensor 1.
- the second abstract data 1 corresponding to the second sensor 1 includes the attribute description data of three third targets
- the second abstract data 2 corresponding to the second sensor 2 also includes the attribute description data of three third targets, as shown in Table 2. Shown.
- Different sensors have different characteristics. Some are good at recognizing short-distance traffic targets, some are good at recognizing long-distance traffic targets, and some are good at recognizing traffic targets in inclement weather.
- the traffic target identified by a single sensor may not be accurate and reliable, and there may be omissions.
- the processing unit may fuse abstract data received from multiple sensors to obtain the fused data. After the fusion of the data, the recognition situation of multiple sensors is considered comprehensively, and the traffic target is re-determined in order to reduce the probability of missing traffic targets.
- the re-determined traffic target may be used as the second target in the embodiment of the present application.
- the data fusion in this application can use the existing data fusion method, which will not be repeated here, and the method of data fusion is not limited in this application.
- the processing unit may obtain the attribute description data of the second target from the fused data, or may communicate with the cloud sensor to obtain the attribute description data of the second target. Then, the processing unit sends the first feedback data to the first sensor, and the first feedback data includes the aforementioned attribute description data of the second target.
- the processing unit receives the abstract data sent by multiple sensors such as cameras, lidar, millimeter wave radar, etc. shown in Table 1 and Table 2, data fusion is performed on these data to obtain the fused data.
- the fusion data includes the attribute description data corresponding to the three second targets, and the attribute description data of each second target is shown in Table 3.
- the processing unit sends the first feedback data to the first sensor, and the first feedback data includes part or all of the attribute description data of the three second targets, T001, T002, and T003.
- the processing unit may also establish a communication connection with the cloud sensor, and receive interactive information from the cloud sensor.
- the interaction information may include the attribute description data of the traffic target in the traffic scene stored on the cloud sensor, for example, the attribute description data of the vehicles and traffic signs in the traffic scene.
- the processing unit may determine the second target based on the interaction information.
- the cloud sensor may be a cloud server, and a high-precision map is stored on the cloud server.
- the high-precision map in the embodiments of the present application is generally an electronic map with higher accuracy and more data dimensions than a general navigation map. Higher accuracy is reflected in the accuracy that can reach the centimeter level, and the data dimension is more reflected in the fact that it includes surrounding static information related to traffic in addition to road information.
- High-precision maps store a large amount of driving assistance information as structured data, which mainly includes two types: the first type is road data, such as lane information such as the location, type, width, slope, and curvature of the lane line.
- the second category is information about static traffic targets around the lane, such as one or more of traffic signs, traffic lights, sewer crossings, obstacles, elevated objects, guardrails, and so on. Assuming that the processing unit obtains the attribute description data of the obstacle near its current location from the cloud server, as shown in Table 4, the obstacle can be determined as the second target, and the target ID is T004.
- the cloud sensor may be vehicle B in a traffic scene.
- Vehicle A where the processing unit is located can establish a communication connection with vehicle B, and receive information sent by vehicle B.
- Vehicle B can pre-set some of its own attribute description data that is allowed to be sent to other terminals, such as the size and color of vehicle B or the direction angle, speed, acceleration, etc. of the current time point.
- vehicle B can send this information to vehicle A.
- the processing unit can receive the attribute description data of vehicle B, as shown in the attribute description data corresponding to T001 in Table 4. Accordingly, the processing unit can determine the vehicle B as a second target, and the target ID is T001.
- the processing unit may determine the attribute description data of the second target based on the interaction information from the cloud sensor.
- the processing unit may also obtain the attribute description data of the second target from the fusion data. Then, the processing unit sends the first feedback data to the first sensor, and the first feedback data includes the attribute description data of the second target. For example, in the foregoing example, the processing unit determines the first feedback data based on the interaction information shown in Table 4, which may include part or all of the attribute description data of the two second targets, T001 and T004.
- the location of the traffic target in the traffic scene can be used to determine whether the traffic targets determined by the two methods are the same. If it is the same traffic target, the same target ID can be used to identify, for example, the traffic target with ID T001 in Table 3 and Table 4. If it is not the same traffic target, use different target IDs to distinguish, such as T002 and T003 in Table 3 and T004 in Table 4.
- the processing unit can select the attribute description data of the second target determined by one of the methods according to different application scenarios or preset rules, or combine the attribute description data of the second target determined by the two methods, and then include The first feedback data of the attribute description data of these second targets is sent to the first sensor.
- FIG. 5 is a flowchart of the third implementation manner of the data processing method of this application.
- the processing unit not only receives the second abstract data from the second sensor, but also receives interactive information from the cloud sensor. After data fusion is performed on the first abstract data and the second abstract data, and the fused data is obtained, the processing unit performs the processing of the attribute description data of the second target in the fused data and the attribute description data of the second target in the interactive information. Combine, determine the first feedback data that needs to be sent to the first sensor.
- the first feedback data may include the attribute description data of the three second targets in Table 3, so as to be subsequently used to optimize the first recognition algorithm and improve the recognition accuracy of the first sensor.
- the first feedback data may include the attribute description data of the two second targets in Table 4, so as to be subsequently used to optimize the first recognition algorithm and improve the recognition accuracy of the first sensor.
- the processing unit may also determine the first feedback data according to a preset rule, so that the first feedback data includes the attribute description data of T002 and T003 in Table 3, and the attribute description data in Table 4 The attribute description data of T001 and T004.
- the attribute description data of the second target determined in the above two methods may also be different.
- the processing unit determines the first feedback data, it can select the attribute description data of the second target determined by one of the methods according to the different application scenarios or preset rules, or combine the two methods The determined attribute description data of the second target.
- the foregoing Table 3 and Table 4 both include the attribute description data of T001.
- the processing unit may preferentially use the attribute description data sent by T001.
- the processing unit may combine the attribute description data determined in two ways, using the type and color data of T001 in Table 4, and the size data of T001 in Table 3.
- the second target may include the target recognized by the first sensor, or it may include the target that the first sensor should recognize but did not recognize.
- the traffic targets in the second target that are the same as the first target, that is, the traffic targets recognized by the first sensor are called the first specific target;
- the first feedback data may include attribute description data of at least one first specific target.
- the first specific target may be a traffic target determined by the processing unit based on fused data obtained by data fusion of the first abstract data and the second abstract data.
- the first specific target may also be a traffic target determined by the processing unit based on the interaction information obtained from the cloud sensor.
- the first feedback data may include attribute description data of at least one second specific target.
- the second specific target may be a traffic target determined by the processing unit based on fused data obtained by data fusion of the second abstract data and the first abstract data. That is, the second specific target may be a target that the first sensor fails to recognize, but the second sensor recognizes.
- the second specific target may also be a traffic target determined by the processing unit based on the interaction information obtained from the cloud sensor.
- the ID number and the like can be used to associate the two.
- the second specific target since the first sensor does not recognize the target, it is necessary to use location information to associate the second specific target with the feature points of the corresponding location in the first original data for subsequent optimization. That is, when the first feedback data includes attribute description data of one or more second specific targets, the first feedback data also includes location information of the one or more second specific targets.
- the position information here can adopt pixel position, coordinate position, or relative position with the vehicle, etc. In this way, the first sensor can associate the second specific target with the feature points of the corresponding position in the first raw data, so that the first sensor can use the attribute description data of the second specific target to optimize the first recognition. algorithm.
- S103 The first sensor optimizes the first recognition algorithm according to the first feedback data.
- the attribute description data of the traffic target in the first abstract data can be compared, Whether it is the same as the attribute description data of the traffic target in the first feedback data. If they are the same, it means that the attribute description data of the traffic target recognized by the first sensor are all accurate, and no optimization is needed. If they are different, it can be considered that the attribute description data of the second target is different from the attribute description data of the first target.
- the second target is a second specific target, that is, the first sensor fails to recognize the same traffic target, then its attribute description data must be different from the attribute description data of all the first targets in the first abstract data, that is, the second The attribute description data of the target is different from the attribute description data of the first target.
- the processing unit can send the first feedback data containing the attribute description data of the second target to the first sensor.
- the first sensor determines whether the two are the same.
- the processing unit can receive the attribute description data of the first target recognized by the first sensor, the processing unit can determine whether the attribute description data of the second target and the attribute description data of the first target are the same. If they are different, the first feedback data including the attribute description data of the second target is sent to the first sensor.
- the attribute description data of the second target is different from the attribute description data of the first target
- data that can reflect the actual situation that is, the attribute description data of the second target
- the optimization in the embodiment of the present application may be to optimize some of the parameters that can be modified in the first recognition algorithm, or other optimization methods, which is not limited in the present application.
- Optimizing the parameters in the first recognition algorithm can also be understood as correcting the parameters of the first recognition algorithm.
- its function itself is unchanged, and its parameters can be adjusted.
- Training is to adjust some parameters in the recognition algorithm, so that the recognition algorithm can obtain a better prediction effect in a specific application scenario.
- the aforementioned first feedback data is also used to re-correct these parameters in the recognition algorithm, so as to further improve the prediction effect.
- the specific method for correcting the parameters can be an existing method.
- the parameter selection in the first recognition algorithm can be updated according to the difference between the attribute description data of the first specific target and the attribute description data of the same traffic target in the first abstract data. value.
- the value of the parameter in the first recognition algorithm can be updated according to the attribute description data of the second specific target and the first original data.
- the corrected parameters also have corresponding differences.
- the following will briefly describe the Kalman filtering tracking algorithm as an example.
- Kalman filter tracking algorithm is an optimal linear state estimation method, that is, using the linear system state equation, through the system input and output observation data, the system state is optimally estimated. Since the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process.
- Kalman filter tracking algorithm is often applied in the fields of communication, radar, navigation and so on. Five formulas are needed to realize the Kalman filter tracking algorithm. Among them, the prediction equation includes the following two formulas (1) and (2), and the update equation includes the following three formulas (3), (4) and (5), which are as follows:
- A represents the state transition matrix, which is an n ⁇ n order matrix.
- A is actually a conjecture model for the transition of the target state.
- u k-1 represents the control gain at k-1, which can be set to 0 in general application scenarios.
- B represents the gain of the optional control input u, which is an n ⁇ l order matrix. In most practical situations, the gain is not controlled, and B is 0 at this time.
- P k-1 represents the posterior estimated covariance at time k-1
- P k represents the posterior estimated covariance at time k, both of which are the results of filtering.
- Q represents the process excitation noise covariance. This parameter is used to represent the error between the state transition matrix and the actual process.
- K k represents the Kalman gain, which is the intermediate calculation result of filtering.
- H represents the measurement matrix, which is an m ⁇ n order matrix, which is responsible for converting the measured value of m dimension to n dimension to make it conform to the mathematical form of the state variable, which is one of the prerequisites for filtering.
- R represents the measurement noise covariance
- z k represents the measured value, which is an m-order vector and is the input of the filter.
- the value of I can be 1.
- training samples need to be used for training in the training stage, so that the parameters such as Q, R, A, B, and H in the algorithm are adjusted to improve the prediction effect.
- the parameters such as Q, R, A, B, and H in the algorithm are adjusted to improve the prediction effect.
- historical data of the target and real data of the target are usually used as training samples, which are substituted into the above equations in order to calculate parameter values suitable for the actual scene.
- these parameters are usually unchanged.
- the aforementioned first feedback data can also be used as the target's real data, and the first raw data can be used as the target's historical data to correct these parameters, thereby further improving the prediction effect.
- the first sensor first obtains the first abstract data including the attribute description data of the first target from the first original data by the first recognition algorithm, and then receives the first abstract data including the second target from the processing unit.
- the first recognition algorithm is optimized according to the first feedback data.
- the first sensor not only sends the information recognized from the traffic scene, that is, the first abstract data, to the processing unit, but can also receive the first feedback data from the processing unit, realizing bidirectional transmission.
- the first sensor can then optimize the first recognition algorithm based on the attribute description data of the second target in the same traffic scene, so as to improve the accuracy of obtaining the first abstract data from the first original data by the first recognition algorithm, that is, improve The recognition accuracy of the first sensor.
- the above-mentioned second sensor may be the same as a general sensor, only transmitting data in one direction with the processing unit, or it may execute the data processing method of this application to transmit data in both directions with the processing unit, such as the second sensor in FIG. 2 As shown in 1, this application does not limit this.
- the first raw data in the above data processing method refers to the data measured from the traffic scene at a certain point in time.
- Both the first abstract data and the first feedback data are data corresponding to the first raw data, that is, the data corresponding to the first raw data.
- the first sensor when the first sensor is working, it usually continuously measures data in different traffic scenes at a certain frequency. For example, when the first sensor is a camera, it can collect pictures of a traffic scene according to different frame rates (for example, 30 frames per second or 60 frames per second, etc.). For another example, when the first sensor is a lidar, it can scan the traffic scene in a certain period (for example, a period of tens of milliseconds).
- both the first original data and the corresponding first abstract data may include corresponding time stamps.
- the second abstract data sent may also include a corresponding time stamp.
- the timestamp in the embodiment of the present application is used to indicate the time information of the original data measured by the sensor from the scene.
- the time stamp contained in the first raw data and the first abstract data is the time information of the first raw data measured by the first sensor from the scene.
- the time stamp contained in the second original data and the second abstract data is used to indicate that the second sensor obtains the time information of the second original data measured from the scene.
- the processing unit can fuse abstract data with the same timestamp, without erroneously fusing abstract data with different timestamps.
- the time stamp included in the first feedback data is the same as the time stamp of the first abstract data and the first original data corresponding to the first feedback data. That is, when sending the first feedback data, the processing unit sends the first feedback data corresponding to the timestamps of the first original data and the first abstract data to the first sensor. Based on this, the first sensor can find the corresponding first abstract data, first raw data, and first feedback data through the time stamp to optimize the first recognition algorithm.
- the first sensor carries at least one identification tag, and the identification tag is used to identify the attribute type of the target that can be recognized by the first sensor.
- the first sensor is a lidar
- it can recognize the size of the vehicle in the traffic scene, the distance to the vehicle, etc., but it cannot recognize the color of the vehicle.
- the lidar can be configured with identification tags such as "size” and "distance from the vehicle”.
- the attribute description data of the second target is obtained based on the fusion data or interactive information. These attribute description data may include "size", etc., or may include "color”.
- the lidar cannot use it to optimize the first recognition algorithm. For this reason, when the processing unit sends the first feedback data, the first feedback data only contains the attribute description data of the second target that matches the at least one identification tag of the first sensor.
- the first feedback data can include the second target's "size”, “distance from the vehicle” and other attribute description data, instead of including "color” and other attributes Describe the data.
- the first feedback data includes a confidence level corresponding to each piece of the attribute description data of the second target, and the confidence level is used to characterize the credibility of the attribute description data of the second target.
- the corresponding confidence level can be included in the second target’s attribute description data, and a certain piece of the second target’s attribute description data is used. It can be represented by one field, or it can be included in the first feedback data, and represented by an independent character string, which is not limited in this application.
- the confidence is represented by an independent character string, a corresponding relationship is established between the attribute description data of the second target and the corresponding confidence.
- the confidence level of the attribute description data of a certain second target can be determined according to its source. For example, if the attribute description data of the second target comes from the fusion data, its confidence level can be set to 80%; if the attribute description data of the second target comes from the cloud server, its confidence level can be set to 95%; if The attribute description data of the second target comes from the second target, and its confidence level can be set to 99%. That is, the attribute description data of the second target includes a corresponding source tag, and the source label is used to identify the source of the attribute description data of the second target.
- the source tag may be represented by a certain field in the attribute description data of the second target. Since there is a correspondence between the source of the attribute description data of the second target and the confidence level of the attribute description data of the second target, the first sensor can determine the corresponding confidence level according to the source of the attribute description data of the second target .
- the aforementioned step of S103 may include: optimizing the first recognition algorithm according to the attribute description data of the second target and the confidence level corresponding to the attribute description data of the second target.
- the first sensor can adjust the correction range of the parameters of the first recognition algorithm, thereby further improving the recognition accuracy of the optimized first recognition algorithm.
- a communication interface is provided in the embodiment of the present application.
- the communication interface can be used between the first sensor and the processing unit, and between the second sensor and the processing unit.
- the communication interface adopts the form of "data packet header + data body".
- the data packet header includes the protocol version number, the aforementioned time stamp and other information.
- the data packet header may also include a period counter, the installation position of the sensor, the number of data bodies, etc., as shown in Table 5.
- the period counter is used to indicate the period of data transmission
- the installation position of the sensor is used to indicate the installation position of the sensor in a vehicle or roadside
- the number of data bodies is used to indicate the number of data bodies contained in the currently sent data .
- the data body includes the attribute description data of the traffic target, such as the type, size, position, direction angle, speed, acceleration, etc. of the aforementioned traffic target, as shown in Table 5. It should be noted that, according to different application scenarios, the specific content in the data packet header and data body can be adjusted.
- the processing unit can be connected to a variety of different sensors, and can receive data sent by the sensor or send data to the sensor.
- the communication interface can also be used for data transmission between the processing unit and the cloud sensor. In this way, a unified interface can be adopted between each sensor and the processing unit, thereby simplifying the software design of the processing unit.
- a data processing method is provided.
- the method can be executed by the processing unit.
- the processing unit may be a processing unit with a data fusion function.
- the processing unit can perform two-way transmission with the sensor, receive abstract data sent by the sensor, and send feedback data to the sensor, so that the sensor can use the feedback data to optimize the recognition algorithm used by the sensor.
- the data processing method executed by the processing unit may include the following steps S201 to S203.
- the processing unit receives first abstract data from the first sensor.
- the first abstract data comes from the first original data.
- the first abstract data is obtained by the first sensor using the first recognition algorithm to process the first raw data.
- the first abstract data includes attribute description data of the first target.
- the first target is a traffic target recognized by the first sensor from the traffic scene according to the first raw data.
- the first raw data is the data measured by the first sensor from the traffic scene.
- S202 The processing unit determines the first feedback data.
- the first feedback data includes attribute description data of a second target
- the second target is a traffic target identified from the foregoing traffic scene.
- the attribute description data of the second target, etc. reference may also be made to the aforementioned related description, which will not be repeated here.
- the processing unit can determine the attribute description data of the second target in a variety of ways.
- it may further include: a step of determining the second target by the processing unit.
- the second target may include the first specific target and/or the second specific target.
- the processing unit may determine the first specific target based on the first abstract data, or based on interactive information from cloud sensors.
- the processing unit may determine the second specific target based on the second abstract data, or based on the interaction information from the cloud sensor.
- aforementioned related description please refer to the aforementioned related description, which will not be repeated here.
- S203 The processing unit sends the first feedback data to the first sensor.
- the processing unit sends the first feedback data to the first sensor, so that the first sensor uses the first feedback data to optimize the first recognition algorithm.
- the data processing method may further include:
- S204 The processing unit receives second abstract data from the second sensor.
- the processing unit After the processing unit receives the abstract data from multiple sensors, it performs data fusion to obtain the fused data.
- One or more second targets can be determined from the fused data, which may include the traffic targets recognized by the first sensor, that is, the first specific target, or may include traffic targets that are not recognized by the first sensor, that is, The second specific goal. If the second specific target is included, the second specific target is a traffic target recognized by the second sensor, that is, the second specific target is determined by the processing unit based on the second abstract data.
- the data processing method may further include:
- S205 The processing unit receives interactive information from the cloud sensor.
- the interaction information obtained by the processing unit from the cloud sensor contains the attribute description data of the traffic target in the traffic scene stored on the cloud sensor, such as the attribute description data of the vehicle and traffic signs in the traffic scene. Accordingly, the processing unit may determine one or more second targets, which may include the traffic targets recognized by the first sensor, that is, the aforementioned first specific target, or may include traffic targets not recognized by the first sensor, That is the aforementioned second specific goal.
- the processing unit may determine the attribute description data of the second target from the fused data or the interaction information.
- the first feedback data is sent to the first sensor, and the first feedback data includes the attribute description data of these second targets.
- the processing unit and the cloud sensor may interact one or more times, which is not limited in this application. It should also be noted that the processing unit and the cloud sensor can interact indirectly with the transceiver module in the T-BOX (Telematics BOX), which is not limited in this application.
- the above data processing method may include the steps of S204 and S205 at the same time, as shown in FIG. 5.
- the first feedback data further includes location information of the second specific target.
- location information of the second specific target reference may be made to the related description in the first embodiment, which will not be repeated here.
- the first feedback data includes a confidence level corresponding to the attribute description data of each of the second targets.
- the confidence level is used to characterize the credibility of the attribute description data of the second target.
- the first abstract data, the first raw data, and the first feedback data further include a time stamp, so that the first sensor uses the time stamp to determine the first abstract data and the first raw data corresponding to the first feedback data.
- a time stamp so that the first sensor uses the time stamp to determine the first abstract data and the first raw data corresponding to the first feedback data.
- the third embodiment of the present application provides a data processing device.
- the device can be the sensor itself, the ECU in the sensor, or the chip in the sensor. Please refer to FIG. 6.
- FIG. 6 is a schematic structural diagram of one implementation of a data processing apparatus in an embodiment of the application.
- the data processing device 400 includes:
- the first transceiver module 401 is configured to receive first feedback data, where the first feedback data includes attribute description data of the second target;
- At least one first processing module 402 configured to obtain first abstract data from first original data through a first recognition algorithm; and, optimize the first recognition algorithm according to the first feedback data;
- the first abstract data includes attribute description data of a first target
- the first original data is measurement data of a scene
- the first target and the second target are targets in the scene.
- the second target includes at least one first specific target, and the first target has the same target as the first specific target.
- the second target includes at least one second specific target, and there is no target that is the same as the second specific target in the first target.
- the first feedback data includes the location information and attribute description data of the second specific target; the at least one first processing module 402 is further configured to perform according to the location information and attribute description data of the at least one second specific target.
- the attribute description data optimizes the first recognition algorithm.
- the attribute description data of the second target is data determined based on the first abstract data, or data determined based on interaction information from cloud sensors.
- the first original data, the first abstract data, and the first feedback data include a timestamp; wherein, the timestamp is used to indicate that the first original data is obtained from the scene Time information; the at least one first processing module 402 is also configured to optimize the first recognition algorithm according to the first original data, the first abstract data, and the first feedback data corresponding to the timestamp .
- the at least one first processing module 402 is further configured to optimize the first recognition algorithm according to the attribute description data of the second target and the confidence corresponding to the attribute description data of the second target; Wherein, the confidence level is used to characterize the credibility of the attribute description data of the second target.
- the attribute description data of the second target includes a source tag
- the source tag is used to identify the source of the attribute description data of the second target
- the source of the attribute description data of the second target is related to the There is a correspondence between the attributes of the second target describing the confidence of the data.
- the third embodiment of the present application provides another data processing device.
- the device is a processing device with data fusion function.
- the processing device may be the on-board central processing unit itself, or a chip or component in the on-board central processing unit.
- the processing device may also be a fusion unit, or a chip or component in the fusion unit.
- the processing device can also be in other product forms.
- the data processing device will be explained in terms of logic function with reference to FIG. 7. Please refer to FIG. 7.
- FIG. 7 is a schematic structural diagram of one of the implementation manners of another data processing apparatus in an embodiment of the application.
- the data processing device 500 includes:
- the second transceiver module 501 is configured to receive the first abstract data from the first sensor; and send the first feedback data to the first sensor; wherein, the first abstract data is derived from the first raw data, and the The first abstract data includes the attribute description data of the first target, the first feedback data includes the attribute description data of the second target, the first raw data is the measurement data of the scene, the first target and the second target The target is the target identified from the scene;
- At least one second processing module 502 is used to determine the first feedback data.
- the at least one second processing module 502 is further configured to determine at least one first specific target based on the first abstract data, or based on interactive information from cloud sensors;
- the first specific target is the same target.
- the first feedback data includes attribute description data of the at least one first specific target.
- At least one second specific target is determined based on second abstract data, or based on interaction information from cloud sensors; wherein, the second abstract data is derived from second original data, and the second original data is The measurement data of the scene, the second abstract data includes the attribute description data of a third target, the third target is a target in the scene; the first target does not exist in the second specific target The same goal.
- the first feedback data includes attribute description data of the at least one second specific target.
- the first feedback data further includes location information of the at least one second specific target.
- the at least one second processing module 502 is further configured to determine the attribute description data of the second target based on the first abstract data, or based on interaction information from cloud sensors.
- the first raw data, the first abstract data, and the first feedback data include a time stamp; wherein the time stamp is used to indicate that the first raw data is measured from the scene Time information;
- the at least one second processing module 502 is further configured to send the first feedback data corresponding to the timestamps of the first raw data and the first abstract data to the first sensor.
- the first feedback data includes a confidence level corresponding to each piece of the attribute description data of the second target, wherein the confidence level is used to characterize the credibility of the attribute description data of the second target .
- each piece of the attribute description data of the second target includes a corresponding source tag
- the source tag is used to identify the source of the attribute description data of the second target, and the attribute description data of the second target There is a correspondence between the source and the confidence of the attribute description data of the second target.
- the division of the above modules is only a logical function division.
- the functions of the first transceiver module or the second transceiver module can be implemented by the transceiver, the first processing module or the second processing module
- the function can be realized by the processor.
- the transceiver may include a receiving antenna and a transmitting antenna to perform receiving and transmitting functions respectively.
- first transceiver module 401 and second transceiver module 501 may be implemented through a data interface or other possible forms, which is not limited in this application.
- the above-mentioned first processing module 402 and second processing module 502 may be general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof .
- the aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a general array logic (generic array logic, GAL), or any combination thereof.
- the general-purpose processor may be a microprocessor or any conventional processor or the like.
- FIG. 8 is a schematic structural diagram of an implementation manner of a chip system provided by an embodiment of the present application.
- the chip system 600 includes: at least one processor 601 and an interface 602.
- the interface 602 is used to receive code instructions and transmit them to at least one processor 601.
- the at least one processor 601 runs the code instructions to implement what is executed by the aforementioned sensor or processing unit. Either way.
- the aforementioned processor 601 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof.
- the aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a general array logic (generic array logic, GAL), or any combination thereof.
- the general-purpose processor may be a microprocessor or any conventional processor or the like.
- chip system 600 may include one chip or a chip module composed of multiple chips, which is not limited in this application.
- This embodiment also provides a vehicle, which may include any of the aforementioned data processing devices 400.
- the vehicle may also include any of the aforementioned data processing devices 500. This application does not limit the specific implementation form of the vehicle.
- This embodiment also provides a terminal, which can be installed in a traffic scene, for example, installed on a roadside.
- the terminal may include any of the foregoing data processing apparatus 400, and the terminal may also include any of the foregoing data processing apparatus 500.
- This application does not limit the specific implementation form of the terminal.
- This embodiment also provides a system, which may include any of the aforementioned data processing apparatus 400.
- the system may also include any of the aforementioned data processing devices 500.
- This embodiment also provides a computer-readable storage medium.
- the computer-readable storage medium is used to store a computer program or instruction.
- the computer program or instruction runs on an electronic device, the electronic device realizes part or all of any method executed by the aforementioned sensor or processing unit step.
- the readable storage medium here may be a magnetic disk, an optical disc, a DVD, a USB, a read-only memory (ROM) or a random access memory (RAM), etc.
- the application does not limit the specific storage medium form.
- the computer may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
- software it can be implemented in the form of a computer program product in whole or in part.
- the computer program product includes one or more computer instructions.
- the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
- the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
- the computer instructions may be transmitted from a website, computer, server, or data center.
- the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
- the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
- the execution order of each step should be determined by its function and internal logic, and the size of the sequence number of each step does not mean the order of the execution order, and does not limit the implementation process of the embodiment.
- the “plurality” in this specification refers to two or more.
- words such as “first” and “second” are used to distinguish the same items or similar items that have substantially the same function and effect.
- the words “first”, “second” and the like do not limit the quantity and order of execution, and the words “first” and “second” do not limit the difference.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Aviation & Aerospace Engineering (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Traffic Control Systems (AREA)
Abstract
一种数据处理方法、装置、芯片系统及介质。该方法可以应用于自动驾驶或者智能驾驶领域。该方法包括:通过第一识别算法从第一原始数据中获取第一抽象数据,所述第一抽象数据包括第一目标的属性描述数据;接收第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;根据所述第一反馈数据优化所述第一识别算法;其中,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。采用上述技术方案中的方法,传感器可以与处理单元之间进行双向数据传输,并根据接收到的反馈数据来优化识别算法,提高传感器的识别准确率。
Description
本申请涉及自动驾驶技术领域,具体涉及一种数据处理方法。此外,本申请还涉及一种数据处理装置、一种芯片系统,以及一种计算机可读存储介质。
随着汽车自动驾驶程度的提高,汽车自身所产生的数据将越来越庞大。据英特尔公司估算,一辆自动驾驶汽车如果配置了全球定位系统(Global Positioning System,GPS)、声纳、摄像头、毫米波雷达(millimeter-wave radar)和激光雷达(Light Detection and Ranging,LiDAR)等传感器,每天可以产生超过4000GB的数据。这就需要一个数据处理的计算架构来处理如此海量的数据。
请参见图1,图1为一个典型的自动驾驶车辆的数据处理架构示意图。自动驾驶车辆的处理单元负责数据处理,其可以包括三个模块:数据融合模块、规划决策模块和行为控制模块。传感器(例如摄像头、激光雷达、声呐等)在一个交通场景中捕获到电信号之后,会将其进行数字化处理,转换为数字信号数据,即原始数据(raw data)。然后,传感器利用识别算法处理原始数据,得到抽象数据(abstract data)。抽象数据中一般包括了从交通场景中识别出来的交通目标,以及交通目标的属性描述数据,例如车辆的颜色、尺寸、交通标识的指示内容等。每个传感器分别将各自的抽象数据发送给处理单元中的数据融合模块。数据融合模块将这些抽象数据进行融合,综合多个传感器的抽象数据来重新识别交通目标,以及交通目标的属性描述数据,得到融合数据。融合数据中包含的交通目标的属性描述数据,与各个传感器自己识别出的交通目标的属性描述数据可能相同,也可能不同。通过数据融合,可以在一定程度上提高交通目标和交通目标的属性的识别准确率。融合数据可以用于在计算机中构建出世界模型(又名环境模型),从而模拟还原出真实世界中的情况。规划决策模块根据世界模型对车辆的行驶路线进行规划和决策。最后,行为控制模块根据决策来指示执行器(例如油门、刹车、方向盘、车窗、车灯等)执行操作,从而实现对车辆行驶轨迹的控制。
从上述数据处理的过程可见,传感器所提供的抽象数据是否准确,会直接影响到后续行驶路线的规划决策的准确性。一般地,传感器在利用识别算法处理在某一种交通场景中获取到的原始数据,得到抽象数据之后,无论抽象数据与真实情况是否一致,传感器都无从知晓。如果抽象数据与真实情况不一致,那么每一次当传感器获取到同样的原始数据之后,其都会处理得到相同的、与真实情况不一致的抽象数据。这就导致传感器的识别准确率难以提高。
发明内容
本申请提供一种数据处理方法,采用该方法的传感器可以与处理单元之间进行双向数据传输,根据处理单元反馈回来的反馈数据来优化识别算法,从而提高传感器的 识别准确率。
第一方面,本申请提供一种数据处理方法,所述方法包括:通过第一识别算法从第一原始数据中获取第一抽象数据,所述第一抽象数据包括第一目标的属性描述数据;接收第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;根据所述第一反馈数据优化所述第一识别算法;其中,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
采用本实现方式,第一传感器不但向外部发送从交通场景中识别出来的信息,即第一抽象数据,也能够从外部接收第一反馈数据,实现了双向传输。第一传感器进而可以根据同一个交通场景中的第二目标的属性描述数据来优化第一识别算法,以提高通过第一识别算法从第一原始数据中获取第一抽象数据的准确率,即提高第一传感器的识别准确率。
结合第一方面,在第一方面第一种可能的实现方式中,所述第二目标包括至少一个第一特定目标,所述第一目标中存在与所述第一特定目标相同的目标。第一特定目标是第一传感器也识别出来的目标,采用本实现方式,第一传感器利用从其他途径确定的第一特定目标的属性描述数据,来优化第一识别算法,从而提高第一传感器的识别准确率。
结合第一方面及上述可能的实现方式,在第一方面第二种可能的实现方式中,所述第二目标包括至少一个第二特定目标,所述第一目标中不存在与所述第二特定目标相同的目标。第二特定目标是第一传感器未识别出来的目标,采用本实现方式,第一传感器利用从其他途径确定的第二特定目标的属性描述数据,来优化第一识别算法,从而提高第一传感器的识别准确率。
结合第一方面及上述可能的实现方式,在第一方面第三种可能的实现方式中,所述第一反馈数据包括所述第二特定目标的位置信息和属性描述数据;所述根据所述第一反馈数据优化所述第一识别算法的步骤,包括:根据所述至少一个第二特定目标的位置信息和属性描述数据,优化所述第一识别算法。对于第二特定目标而言,由于第一传感器没有识别出该目标,故而采用位置信息将第二特定目标和第一原始数据中对应位置的特征点关联起来。通过这样的方式,第一传感器就可以利用该第二特定目标的属性描述数据,来优化第一识别算法,从而提高第一传感器的识别准确率。
结合第一方面及上述可能的实现方式,在第一方面第四种可能的实现方式中,所述第二目标的属性描述数据是基于所述第一抽象数据确定的数据,或者,基于来自云端传感器的交互信息确定的数据。基于第一抽象数据,或者基于自云端传感器的交互信息确定第二目标的属性描述数据,更加接近场景中的目标的实际情况。将这样的数据反馈给第一传感器,有利于第一传感器优化第一识别算法,从而提高识别准确率。
结合第一方面及上述可能的实现方式,在第一方面第五种可能的实现方式中,所述第一原始数据、所述第一抽象数据和所述第一反馈数据包含时间戳;其中,所述时间戳用于指示从所述场景中得到所述第一原始数据的时间信息;所述根据所述第一反馈数据优化所述第一识别算法的步骤,包括:根据所述时间戳对应的所述第一原始数据、所述第一抽象数据和所述第一反馈数据,优化所述第一识别算法。通过时间戳,第一传感器可以找到时间戳对应的第一原始数据、第一抽象数据和第一反馈数据,避 免将不同时间的第一原始信息、第一抽象数据和第一反馈数据混淆导致影响第一传感器的识别准确率的情况。
结合第一方面及上述可能的实现方式,在第一方面第六种可能的实现方式中,所述根据所述第一反馈数据优化所述第一识别算法的步骤,包括:根据所述第二目标的属性描述数据,以及所述第二目标的属性描述数据对应的置信度,优化所述第一识别算法;其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。采用本实现方式,第一传感器可以根据第二属性描述数据的置信度的不同,来调节第一识别算法的参数的校正幅度,从而进一步提高优化后的第一传感器的识别准确率。
结合第一方面及上述可能的实现方式,在第一方面第七种可能的实现方式中,所述第二目标的属性描述数据包含来源标签,所述来源标签用于标识所述第二目标的属性描述数据的来源,所述第二目标的属性描述数据的来源与所述第二目标的属性描述数据的置信度之间存在对应关系。采用本实现方式,第一传感器可以根据第二目标的属性描述数据的来源标签来确定对应的置信度,从而调节第一识别算法的参数的校正幅度,进一步提高优化后的第一传感器的识别准确率。
第二方面,本申请提供一种数据处理方法,包括:接收来自第一传感器的第一抽象数据,所述第一抽象数据来源于第一原始数据,所述第一抽象数据包括第一目标的属性描述数据;确定第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;发送所述第一反馈数据给所述第一传感器;其中,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
采用本实现方式,处理单元可以不但可以接收第一传感器从交通场景中识别出来的信息,即第一抽象数据,而且可以将其确定的第一反馈数据发送给第一传感器,从而实现双向传输。这样,第一传感器就可以根据第一反馈数据来对其使用的识别算法进行优化,提高第一传感器的识别准确率。
结合第二方面,在第二方面第一种可能的实现方式中,所述方法还包括:基于所述第一抽象数据,或者,基于来自云端传感器的交互信息,确定至少一个第一特定目标;所述第一目标中存在与所述第一特定目标相同的目标;其中,所述第一反馈数据包括所述至少一个第一特定目标的属性描述数据。采用本实现方式,处理单元可以基于第一抽象数据,或者,基于来自云端传感器的交互信息,来确定第一识别传感器也能够识别出来的第一特定目标,从而将利用其他途径确定的第一特定目标的属性描述数据反馈给第一传感器。通过这样的方式,第一传感器就可以利用第一特定目标的属性描述数据,来优化第一识别算法,从而提高第一传感器的识别准确率。
结合第二方面及上述可能的实现方式,在第二方面第二种可能的实现方式中,所述方法还包括:基于第二抽象数据,或者,基于来自云端传感器的交互信息,确定至少一个第二特定目标;其中,所述第二抽象数据来源于第二原始数据,所述第二原始数据为所述场景的测量数据,所述第二抽象数据包括第三目标的属性描述数据,所述第三目标为所述场景中的目标;所述第一目标中不存在与所述第二特定目标相同的目标;其中,所述第一反馈数据包括所述至少一个第二特定目标的属性描述数据。采用本实现方式,处理单元可以基于第二抽象数据,或者,基于来自云端传感器的交互信息,来确定第一识别传感器未能识别出来的第二特定目标,从而将利用其他途径确定 的第二特定目标的属性描述数据反馈给第一传感器。通过这样的方式,第一传感器就可以利用第二特定目标的属性描述数据,来优化第一识别算法,从而提高第一传感器的识别准确率。
结合第二方面及上述可能的实现方式,在第二方面第三种可能的实现方式中,所述第一反馈数据还包括所述至少一个第二特定目标的位置信息。对于第二特定目标而言,由于第一传感器没有识别出该目标,故而处理单元可以将第二特定目标的位置信息也反馈给第一传感器。通过这样的方式,第一传感器就能够根据位置信息将第二特定目标和第一原始数据中对应位置的特征点关联起来,并利用该第二特定目标的属性描述数据,来优化第一识别算法,提高第一传感器的识别准确率。
结合第二方面及上述可能的实现方式,在第二方面第四种可能的实现方式中,所述确定第一反馈数据的步骤,包括:基于所述第一抽象数据,或者,基于来自云端传感器的交互信息,确定所述第二目标的属性描述数据。采用本实现方式,处理单元可以基于第一抽象数据,或者,基于来自云端传感器的交互信息,来确定第二目标的属性描述数据,从而将利用这些途径确定的第二目标的属性描述数据反馈给第一传感器。通过这样的方式,第一传感器就可以利用这些处理单元反馈回来的第二目标的属性描述数据,来优化第一识别算法,从而提高第一传感器的识别准确率。
结合第二方面及上述可能的实现方式,在第二方面第五种可能的实现方式中,所述第一原始数据、所述第一抽象数据和所述第一反馈数据包含时间戳;其中,所述时间戳用于指示从所述场景中测量得到所述第一原始数据的时间信息;所述发送所述第一反馈数据给所述第一传感器的步骤,包括:将与所述第一原始数据和所述第一抽象数据的时间戳对应的所述第一反馈数据,发送给所述第一传感器。采用本实现方式,处理单元将时间戳对应的第一反馈数据发送给第一传感器,以便第一传感器可以找到时间戳对应的第一原始数据、第一抽象数据和第一反馈数据,避免将不同时间的第一原始信息、第一抽象数据和第一反馈数据混淆导致影响第一传感器的识别准确率的情况。
结合第二方面及上述可能的实现方式,在第二方面第六种可能的实现方式中,所述第一反馈数据包括与每一条所述第二目标的属性描述数据对应的置信度,其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。采用本实现方式,第一反馈数据中包含每一条第二目标的属性描述数据对应的置信度,从而使第一传感器可以根据第二属性描述数据的置信度的不同,来调节第一识别算法的参数的校正幅度,进一步提高优化后的第一传感器的识别准确率。
结合第二方面及上述可能的实现方式,在第二方面第七种可能的实现方式中,每一条所述第二目标的属性描述数据包含对应的来源标签,所述来源标签用于标识所述第二目标的属性描述数据的来源,所述第二目标的属性描述数据的来源与所述第二目标的属性描述数据的置信度之间存在对应关系。采用本实现方式,每一条所述第二目标的属性描述数据包含对应的来源标签,从而使第一传感器可以根据第二目标的属性描述数据的来源标签来确定对应的置信度,进而调节第一识别算法的参数的校正幅度,进一步提高优化后的第一传感器的识别准确率。
第三方面,本申请提供一种数据处理装置,包括:第一收发模块,用于接收第一 反馈数据,所述第一反馈数据包括第二目标的属性描述数据;至少一个第一处理模块,用于通过第一识别算法从第一原始数据中获取第一抽象数据;以及,根据所述第一反馈数据优化所述第一识别算法;其中,所述第一抽象数据包括第一目标的属性描述数据,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
第四方面,本申请提供一种数据处理装置,包括:第二收发模块,用于接收来自第一传感器的第一抽象数据;以及,发送第一反馈数据给所述第一传感器;其中,所述第一抽象数据来源于第一原始数据,所述第一抽象数据包括第一目标的属性描述数据,所述第一反馈数据包括第二目标的属性描述数据,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为从所述场景中识别出的目标;至少一个第二处理模块,用于确定所述第一反馈数据。
第五方面,本申请提供一种芯片系统,包括至少一个处理器以及接口,所述接口用于接收代码指令,并传输所述至少一个处理器;所述至少一个处理器运行所述代码指令,以实现第一方面的任一种方法。
第六方面,本申请提供一种芯片系统,包括至少一个处理器以及接口,所述接口用于接收代码指令,并传输所述至少一个处理器;所述至少一个处理器运行所述代码指令,以实现第二方面的任一种方法。
第七方面,本申请提供一种计算机可读存储介质,用于存储计算机程序或指令,当所述计算机程序或所述指令在电子设备上运行时,使所述电子设备实现第一方面的任一种方法。
第八方面,本申请提供一种计算机可读存储介质,用于存储计算机程序或指令,当所述计算机程序或所述指令在电子设备上运行时,使所述电子设备实现第二方面的任一种方法。
为了更清楚地说明本申请的技术方案,下面将对实施例中的附图作简单地介绍。
图1为一个典型的自动驾驶车辆的数据处理架构示意图;
图2为本申请的一个实施例中车辆的数据处理系统架构示意图;
图3为本申请的数据处理方法的实现方式之一的流程图;
图4为本申请的数据处理方法的实现方式之二的流程图;
图5为本申请的数据处理方法的实现方式之三的流程图;
图6为本申请的一种数据处理装置的结构示意图;
图7为本申请的另一种数据处理装置的结构示意图;
图8为本申请的芯片系统的实现方式之一的结构示意图。
一般地,在自动驾驶、智慧驾驶等领域应用的传感器都是单向向处理单元发送数据,如图1所示。这导致传感器无法对自身的识别算法进行优化,进而导致传感器的 识别准确率难以提高。
车辆上一般会安装多个传感器,包括内部传感器和外部传感器。其中,内部传感器是指用于测量车辆本身的一些状态数据的传感器,例如陀螺仪、加速度计、转向角传感器、雨刮器活动传感器、转向指示器等。这些内部传感器可以安装在车辆的内部或者外部。外部传感器是指用于测量车辆周围的环境数据的传感器,例如雷达、激光雷达、超声波雷达、摄像头、全球定位系统、声呐等。这些外部传感器可以安装在车辆的外部或者内部。此外,在路侧等其他位置也可能会安装传感器,例如微波雷达、毫米波雷达、摄像头等,主要用于测量道路上的目标的状况。
除了安装在车辆或者路侧的传感器之外,在自动驾驶、智慧驾驶等领域还可以应用云端传感器。云端传感器,也可以称为元数据传感器,车辆可以与云端传感器建立通信连接,进行交互,从而从云端传感器上获取交通场景中的交通目标的信息。这里的云端传感器可以是云端服务器、与车辆通过车用无线通信(vehicle to everything,V2X)技术连接的其他终端或者基站等。例如,车辆可以与云端服务器建立通信连接,从云端服务器的高精度地图中获取车辆当前所处的位置附近的交通基础设施的信息等。又例如,车辆可以与交通场景中的另一个车辆建立通信连接,然后获取该车辆的尺寸、颜色、速度等信息。
执行本申请实施例的数据处理方法的传感器,可以是前述的安装在车辆或者路侧等位置的传感器。为便于区分,本申请实施例中将任意一个执行本申请的数据处理方法的传感器,称为第一传感器;将除第一传感器以外的其他传感器称为第二传感器。请参见图2,图2为本申请的一个实施例中车辆的数据处理系统架构示意图。其中,该系统包括一个第一传感器、一个或多个第二传感器,以及一个或多个云端传感器。这些传感器均与处理单元连接,处理单元还与多个执行器连接。
本申请的第一个实施例提供一种数据处理方法,该方法由第一传感器来执行。可选地,该方法可以由第一传感器中的电子控制单元(Electronic Control Unit,ECU)来执行。第一传感器通过执行本实施例中的数据处理方法,可以与处理单元进行双向传输,接收反馈回来的反馈数据,并利用反馈数据来优化传感器所使用的识别算法,从而避免相同的识别错误重复发生,提高传感器的识别准确率。
请参见图3,图3为本申请的数据处理方法的实现方式之一的流程图。其中由第一传感器执行的步骤包括以下S101至S103。
S101:第一传感器通过第一识别算法从第一原始数据中获取第一抽象数据。
在本申请实施例中,场景主要指的是交通场景,其可以包含交通场景中的交通参与者或者交通环境,例如马路上的其他车辆、交通标识等,也可以包含一些与自动驾驶、智慧驾驶等没有直接关联的信息,例如马路两侧的绿化带等。
原始数据为场景的测量数据,用于数字化描述交通场景。传感器可以从交通场景中捕获电信号,然后将电信号进行数字化处理,得到原始数据。捕获和处理得到原始数据的具体方法可以通过现有的方法来完成,此处不再赘述。本申请不对获取原始数据的方式进行限定。
为便于区分不同传感器中的原始数据,在本申请实施例中,将第一传感器从交通场景中测量到的数据,称为第一原始数据。当第一传感器为不同的具体传感器时,第 一原始数据的具体数据形式可能存在差别。
例如,当第一传感器为摄像头时,摄像头将从交通场景中捕获的光信号转换为电信号。然后,再将这些电信号进行数字化处理,就转换成了能够描述交通场景的画面的数据,即第一原始数据。
又例如,当第一传感器为激光雷达时,激光雷达发射激光,对交通场景进行扫描。交通场景中的人、物等将激光反射回来。由于不同的人、物等反射回来的光线的强度和时间都不同,故而激光雷达可以据此来获取交通场景中的信息。即,激光雷达接收这些反射回来的光线,将光信号转换为电信号。然后,再将这些电信号进行数字化处理,得到能够描述交通场景中的人、物的数据,即激光雷达的点云信息,也就是本申请实施例中的第一原始数据。
本申请中的目标主要指的是交通目标,其可以是交通参与者,也可以是交通环境。其中,交通参与者包括动态的人或物,例如马路上的车辆、行人、动物等,交通环境包括静态的物,例如交通标识(包括车道线、交通灯、交通标识牌等)、护栏等。
本申请中的识别算法,可以采用现有的识别算法,例如特征识别算法、目标识别算法等,更具体地,识别算法可以采用卡尔曼滤波跟踪算法等。
传感器利用识别算法处理原始数据的过程,可以从交通场景中识别出交通目标,以及交通目标的属性描述数据。当交通目标为不同类别的交通参与者或者交通环境时,其属性描述数据也相应不同。例如,当交通目标是马路上行驶的车辆时,其属性描述数据可以是车辆的尺寸、和车辆之间相隔的距离、表面材料、方向角、速度、加速度等。又例如,当交通目标是交通标识时,其属性描述数据可以是交通标识的内容、位置等。
对于不同的传感器而言,其处理原始数据所采用的识别算法可能不同,也可能相同,本申请对此不作限定。为便于区分,在本申请中将第一传感器所采用的识别算法,称为第一识别算法,将第一传感器通过第一识别算法从交通场景中识别出来的交通目标,称为第一目标,将第一传感器通过第一识别算法从第一原始数据中获取的抽象数据,称为第一抽象数据。第一抽象数据包括第一目标的属性描述数据,例如第一目标的类别、尺寸、和车辆之间相隔的距离、表面材料、方向角、速度、加速度、内容或者位置等。
需要说明的是,第一传感器可能识别出一个或多个第一目标,每个第一目标可能对应一个或者多个属性描述数据。此时,第一抽象数据中可以包括全部的第一目标各自对应的属性描述数据。第一传感器还可能未识别出任何交通目标。此时,第一抽象数据中的第一目标的属性描述数据为空。
例如,第一传感器为摄像头,摄像头识别出了多个第一目标,包括目标ID为T001、T002的交通目标。T001对应的属性包括类型、尺寸、颜色等,T002对应的属性包括类型、内容等。第一抽象数据中包括多个第一目标的属性描述数据,其中2个第一目标的部分属性描述数据如表1所示。
表1 第一目标的属性描述数据局部示例
S102:所述第一传感器接收第一反馈数据。
第一传感器会将第一抽象数据发送给处理单元,然后接收处理单元发送来的第一反馈数据,其中,第一反馈数据包括第二目标的属性描述数据。这里的第二目标,指的是从前述的同一个交通场景中识别出的交通目标,其与第一目标可能相同,也可能不同。与第一目标的属性描述数据类似地,第二目标的属性描述数据,可以是第二目标的类别、尺寸、和第一目标之间相隔的距离、表面材料、方向角、速度、加速度、内容或者位置等。
处理单元可以通过多种方式来确定第一反馈数据中包括的第二目标的属性描述数据。第二目标的属性描述数据是基于第一抽象数据确定的数据,或者,基于来自云端传感器的交互信息确定的数据。例如,在一种实现方式中,处理单元可以融合来自多个传感器的抽象数据,得到融合后数据,然后基于融合后数据确定第二目标的属性描述数据。又例如,在另一种实现方式中,处理单元可以与云端传感器进行交互,从云端传感器接收交互信息,然后基于交互信息确定第二目标的属性描述数据。通过融合后数据或者交互信息所确定的第二目标的属性描述数据,更加接近交通场景中的交通目标的实际情况。将这样的数据反馈给第一传感器,有利于第一传感器优化第一识别算法,提高识别准确率。
可选地,在确定第一反馈数据之前,处理单元可以先确定第二目标,具体可以采用多种方式来确定。例如,在一种实现方式中,处理单元可以融合来自多个传感器的抽象数据,得到融合后数据,然后基于融合后数据来确定第二目标。又例如,在另一种实现方式中,处理单元可以基于来自云端传感器的交互信息来确定第二目标。
需要说明的是,前述的多种不同的确定第二目标的方式,与多种不同的确定第二目标的属性描述信息的方式可以相互组合。以下将结合例子来对确定第二目标,以及确定第二目标的属性描述数据的其中几种实现方式作进一步说明。
如前所述,车辆或者路侧等位置一般会安装有多个传感器,包括第一传感器和至少一个第二传感器。第一传感器和第二传感器各自从交通场景中测量得到原始数据,并利用各自对应的识别算法处理原始数据,得到对应的抽象数据。
在本申请实施例中,将第二传感器从交通场景中测量到的数据,称为第二原始数据。每个第二传感器利用各自对应的第二识别算法处理各自测量到的第二原始数据,得到对应的第二抽象数据。即,对任一个第二传感器而言,其执行的步骤可以包括S301的步骤。
S301:第二传感器通过第二识别算法从第二原始数据中获取第二抽象数据。
获取第二抽象数据的过程也是每个第二传感器各自从交通场景中识别交通目标的过程。第二传感器在交通场景中识别出的交通目标,在本申请实施例中称为第三目标。第二抽象数据包括了第三目标的属性描述数据。第二原始数据与第一原始数据类似,第二识别算法与第一识别算法类似,第二抽象数据与第一抽象数据类似,可以参考前 述相关的描述,此处不再赘述。每个第二传感器都将各自的第二抽象数据发送给处理单元。
例如,其中一个第二传感器1为激光雷达,激光雷达识别出了目标ID为T001、T002、T003共3个第三目标。T001对应的属性包括类型、尺寸、颜色等,T002对应的属性包括类型、内容等,T003对应的属性包括类型、尺寸等。另一个第二传感器2为毫米波雷达,其也识别出了与第一传感器1相同的T001、T002、T003这3个第三目标。第二传感器1对应的第二抽象数据1中包括3个第三目标的属性描述数据,第二传感器2对应的第二抽象数据2中也包括3个第三目标的属性描述数据,如表2所示。
表2 第三目标的属性描述数据局部示例
不同的传感器的特点不同,有的善于识别近距离的交通目标,有的善于识别远距离的交通目标,有的善于在恶劣天气中识别交通目标等。单个传感器所识别出的交通目标不一定准确可靠,可能存在遗漏的情况。
为此,在一种实现方式中,处理单元可以融合从多个传感器接收的抽象数据,得到融合后数据。融合后数据综合考虑了多个传感器各自的识别情况,重新确定出交通目标,以便降低遗漏交通目标的概率。重新确定的交通目标,可以作为本申请实施例中的第二目标。本申请中数据融合可以采用现有的数据融合方法,此处不再赘述,本申请对于数据融合的方式不作限定。
在确定第二目标之后,处理单元可以从融合后数据中获得第二目标的属性描述数据,也可以与云端传感器进行通信,来获取第二目标的属性描述数据。然后,处理单元将第一反馈数据发送给第一传感器,第一反馈数据中包括了前述的第二目标的属性描述数据。
例如,处理单元在接收到表1和表2所示的摄像头、激光雷达、毫米波雷达等多个传感器分别发送过来的抽象数据之后,对这些数据进行数据融合,得到融合后数据。融合后数据包括了3个第二目标对应的属性描述数据,每一个第二目标的属性描述数据如表3所示。处理单元将第一反馈数据发送给第一传感器,第一反馈数据中包括了T001、T002和T003这三个第二目标的部分或者全部属性描述数据。
表3 融合后数据中包含的第二目标的属性描述数据示例
请参见图4,图4为本申请的数据处理方法的实现方式之二的流程图。在另一种实现方式中,处理单元还可以与云端传感器建立通信连接,从云端传感器中接收交互信息。交互信息中可以包含存储在云端传感器上的交通场景中的交通目标的属性描述数据,例如交通场景中的车辆、交通标识等的属性描述数据。处理单元可以基于交互信息确定第二目标。
例如,云端传感器可以为云端服务器,云端服务器上存储有高精度地图。本申请实施例中的高精度地图,通俗来讲就是比一般的导航地图精度更高、数据维度更多的电子地图。精度更高体现在其精度可以到达厘米级别,数据维度更多体现在其包括了除道路信息之外的与交通相关的周围静态信息。高精度地图将大量的行车辅助信息存储为结构化数据,主要包括两类:第一类是道路数据,比如车道线的位置、类型、宽度、坡度和曲率等车道信息。第二类是车道周边的静态的交通目标的信息,比如交通标志、交通信号灯、下水道口、障碍物、高架物体、防护栏等的信息中的一个或多个。假设处理单元从云端服务器上获取其当前所处的位置附近的障碍物的属性描述数据,如表4所示,则可以将该障碍物确定为第二目标,目标ID为T004。
又例如,云端传感器可以为交通场景中的车辆B。处理单元所在的车辆A可以与车辆B建立通信连接,接收车辆B发送的信息。车辆B可以预先设定好自身的一些允许发送给其他终端的属性描述数据,例如车辆B的尺寸、颜色或者当前时间点的方向角、速度、加速度等。当车辆A与车辆B建立通信连接之后,车辆B就可以将这些信息发送给车辆A。由此,处理单元可以接收到车辆B的属性描述数据,如表4中T001对应的属性描述数据所示。据此,处理单元可以将该车辆B确定为一个第二目标,目标ID为T001。
表4 从云端传感器获取的交互信息示例
在确定第二目标之后,处理单元可以基于来自云端传感器的交互信息确定第二目标的属性描述数据。此外,在融合后数据中包括第二目标的属性描述数据的情况下,处理单元也可以从融合后数据中获得第二目标的属性描述数据。然后,处理单元将第一反馈数据发送给第一传感器,第一反馈数据中包括第二目标的属性描述数据。例如,在前述的例子中,处理单元基于表4所示的交互信息,确定第一反馈数据,其中可以 包含T001和T004这两个第二目标的部分或者全部的属性描述数据。
需要说明的是,上述两种方式所确定的第二目标,可能存在部分或全部目标相同的情况。在一种实现方式中,可以通过交通目标在交通场景中的位置,来判断两种方法所确定的交通目标是否为同一个。如果为同一个交通目标,可以采用相同的目标ID来标识,例如表3和表4中的ID为T001的交通目标。如果不是同一个交通目标,则采用不同的目标ID来区分,例如表3中的T002、T003和表4中的T004。
处理单元可以根据应用场景或者预设规则的不同,来选择其中一种方式所确定的第二目标的属性描述数据,或者,组合两种方式所确定的第二目标的属性描述数据,然后将包含这些第二目标的属性描述数据的第一反馈数据发送给第一传感器。
请参见图5,图5为本申请的数据处理方法的实现方式之三的流程图。其中,处理单元既从第二传感器接收第二抽象数据,也从云端传感器接收交互信息。在对第一抽象数据和第二抽象数据进行数据融合,得到融合后数据之后,处理单元将融合后数据中的第二目标的属性描述数据,以及交互信息中的第二目标的属性描述数据进行组合,确定需要发送给第一传感器的第一反馈数据。
例如,在前述表3的例子中,处理单元进行数据融合后确定的T001和T002与第一传感器识别出的第一目标相同,T003则是第一传感器未识别出的交通目标。在一种实现方式中,第一反馈数据可以包括表3中的这三个第二目标的属性描述数据,以便后续用于优化第一识别算法,提高第一传感器的识别准确率。
又例如,在前述表4的例子中,处理单元基于交互信息确定的T001与第一传感器识别出的第一目标相同,T004则是第一传感器未识别出的交通目标。在一种实现方式中,第一反馈数据可以包括表4中的这两个第二目标的属性描述数据,以便后续用于优化第一识别算法,提高第一传感器的识别准确率。
还例如,在另一种实现方式中,处理单元还可以根据预设的规则来确定第一反馈数据,使第一反馈数据包括表3中的T002和T003的属性描述数据,以及表4中的T001和T004的属性描述数据。
还需要说明的是,针对同一个第二目标,上述两种方式所确定的第二目标的属性描述数据也可能存在区别。在这种情况下,当处理单元确定第一反馈数据时,可以根据应用场景或者预设规则的不同,来选择其中一种方式所确定的第二目标的属性描述数据,或者,组合两种方式所确定的第二目标的属性描述数据。
例如,前述表3和表4都包括了T001的属性描述数据。在一种实现方式中,在确定第一反馈数据时,处理单元可以优先采用T001发送过来的属性描述数据。在另一种实现方式中,处理单元可以将两种方式确定的属性描述数据进行组合,采用表4中的T001的类型和颜色的数据,以及表3中的T001的尺寸数据。
第一反馈数据中所涉及的第二目标可能有一个或者多个。从上述两种确定第二目标的方式可以发现,第二目标中可能包含了第一传感器识别出的目标,也有可能包含了第一传感器本应当识别出,而未识别出的目标。为了便于说明,在本申请的实施例中,将第二目标中的与第一目标相同的这些交通目标,也就是第一传感器识别出的这些交通目标,称为第一特定目标;将第二目标中的与第一目标不相同的这部分交通目标,也就是第一传感器未识别出的这些交通目标,称为第二特定目标。
第一反馈数据中可以包括至少一个第一特定目标的属性描述数据。其中,第一特定目标可能是处理单元基于对第一抽象数据和第二抽象数据进行数据融合得到的融合后数据,所确定的交通目标。第一特定目标也可能是处理单元基于从云端传感器获取的交互信息而确定的交通目标。
第一反馈数据中可以包括至少一个第二特定目标的属性描述数据。其中,第二特定目标可能是处理单元基于对第二抽象数据和第一抽象数据进行数据融合得到的融合后数据,所确定的交通目标。即第二特定目标可能是第一传感器未能识别出,而第二传感器识别出的目标。第二特定目标也可能是处理单元基于从云端传感器获取的交互信息而确定的交通目标。
这里的第二抽象数据、数据融合、交互信息,以及与第二抽象数据相关的第二原始数据、第三目标、第三目标的属性描述数据等,可以参考前述的相关描述,此处不再赘述。
对于一个第一特定目标而言,由于第一传感器也识别出了该目标(即第一目标中的某一个目标),故而可以采用ID号等将二者关联起来。对于第二特定目标而言,由于第一传感器没有识别出该目标,故而需要采用位置信息将第二特定目标和第一原始数据中对应位置的特征点关联起来,以便后续进行优化。即当第一反馈数据包括一个或多个第二特定目标的属性描述数据时,所述第一反馈数据还包括这一个或多个第二特定目标的位置信息。这里的位置信息可以采用像素位置、坐标位置,或者与车辆之间的相对位置等。通过这样的方式,第一传感器就可以将第二特定目标与第一原始数据中对应位置的特征点关联起来,从而使第一传感器利用该第二特定目标的属性描述数据,来优化第一识别算法。
S103:第一传感器根据第一反馈数据优化第一识别算法。
当第二目标的属性描述数据与第一目标的属性描述数据不相同时,说明第一传感器利用第一识别算法来处理第一原始数据所得到的结果与实际情况不一致。
具体来说,对于某一个第二目标来说,如果其为第一特定目标,即第一传感器也识别出了相同的交通目标,那么可以对比第一抽象数据中该交通目标的属性描述数据,与第一反馈数据中该交通目标的属性描述数据是否相同。如果相同,说明第一传感器所识别的该交通目标的属性描述数据都是准确的,则无需优化。如果不同,则可以认为第二目标的属性描述数据与第一目标的属性描述数据不相同。如果该第二目标为第二特定目标,即第一传感器未能识别出相同的交通目标,那么其属性描述数据必然与第一抽象数据中所有第一目标的属性描述数据都不同,即第二目标的属性描述数据与第一目标的属性描述数据不相同。
在一种实现方式中,无论第二目标的属性描述数据与第一目标的属性描述数据是否相同,处理单元都可以将包含第二目标的属性描述数据的第一反馈数据,发送给第一传感器,由第一传感器来判断二者是否相同。
在另一种实现方式中,由于处理单元能够接收到第一传感器识别出的第一目标的属性描述数据,故而,处理单元可以判断第二目标的属性描述数据与第一目标的属性描述数据是否相同。如果不同,再将包含该第二目标的属性描述数据的第一反馈数据, 发送给第一传感器。
当第二目标的属性描述数据与第一目标的属性描述数据不相同时,可以利用能够体现实际情况的数据,也就是第二目标的属性描述数据,来优化第一识别算法。本申请实施例中优化,可以是优化第一识别算法中的一些可被修改的参数,也可以是其他优化方式,本申请对此不作限定。
优化第一识别算法中的参数,也可以理解为校正第一识别算法的参数。一般地,对于一个特定的识别算法而言,其函数本身是不变的,而其中的参数是可以调整的。当将该识别算法应用到不同的应用场景中时,就需要先利用特定的训练样本,来对该识别算法进行训练。训练,也即对该识别算法中的一些参数进行调整,以便使该识别算法在特定的应用场景中获得较好的预测效果。在训练完成之后就可以进入使用阶段。在使用阶段,这些参数通常是不变的。而本申请实施例的技术方案,是在使用阶段,也利用前述的第一反馈数据来再校正识别算法中的这些参数,从而进一步提升预测效果。
校正参数的具体方法可以采用现有的方法。例如,在一种实现方式中,可以根据第一特定目标的属性描述数据和第一抽象数据中同一个交通目标的属性描述数据之间的差值,来更新第一识别算法中的参数的取值。又例如,在另一种实现方式中,可以根据第二特定目标的属性描述数据和第一原始数据,来更新第一识别算法中的参数的取值。
需要说明的是,当第一识别算法为不同的算法时,被校正的参数也相应地存在差异。为便于进一步理解参数校正的过程,以下将以卡尔曼滤波(Kalman filtering)跟踪算法为例来对作简要说明。
卡尔曼滤波跟踪算法是一种最优线性状态估计方法,即利用线性系统状态方程,通过系统输入输出观测数据,对系统状态进行最优估计的算法。由于观测数据中包括系统中的噪声和干扰的影响,所以最优估计也可看作是滤波过程。卡尔曼滤波跟踪算法常被应用在通信、雷达、导航等领域。实现卡尔曼滤波跟踪算法需要五个公式,其中,预测方程包括以下(1)和(2)两个公式,更新方程包括以下(3)、(4)和(5)三个公式,具体如下:
A表示状态转移矩阵,是一个n×n阶的矩阵。A实际上是对目标状态转换的一种猜想模型。
u
k-1表示k-1时刻的控制增益,在一般的应用场景中可以设置为0。
B表示可选的控制输入u的增益,是一个n×l阶的矩阵。在大多数实际情况下并没有控制增益,此时B为0。
P
k-1表示k-1时刻的后验估计协方差,P
k表示k时刻的后验估计协方差,二者都是滤波的结果之一。
Q表示过程激励噪声协方差,该参数用来表示状态转移矩阵与实际过程之间的误差。
K
k表示卡尔曼增益,是滤波的中间计算结果。
H表示测量矩阵,是一个m×n阶矩阵,负责将m维的测量值转换到n维,使之符合状态变量的数学形式,是滤波的前提条件之一。
R表示测量噪声协方差。
z
k表示测量值,是一个m阶向量,是滤波的输入。
对于单模型单测量,I取值可以为1。
通过更新方程(3)、(4)、(5)计算出
和P
k,然后将其代入到预测方程(1)和(2)中,从而得到新的预测状态值(即k+1时刻的先验状态估计值),以及新的预测预计协方差值(即k+1时刻的先验估计协方差)。
当将上述卡尔曼滤波跟踪算法应用到实际场景中时,在训练阶段需要采用训练样本来进行训练,从而对算法中的Q、R、A、B、H等参数进行调整,以便提升预测效果。一般来说,在训练阶段校正这些参数时,通常采用目标的历史数据和目标的真实数据作为训练样本,将其代入上述方程中,以便计算得到适合实际场景的参数值。而在使用阶段,这些参数通常是不变的。结合本申请的方案,在使用阶段,也可以将前述的第一反馈数据作为目标的真实数据,将第一原始数据作为目标的历史数据,来校正这些参数,从而进一步提升预测效果。
在上述技术方案中,第一传感器首先通过第一识别算法从所述第一原始数据中获取包含了第一目标的属性描述数据的第一抽象数据,然后从处理单元接收包含了第二目标的属性描述数据的第一反馈数据,再根据第一反馈数据来优化第一识别算法。通过这样的方式,第一传感器不但向处理单元发送其从交通场景中识别出来的信息,即 第一抽象数据,也能够从处理单元接收第一反馈数据,实现了双向传输。第一传感器进而可以根据同一个交通场景中的第二目标的属性描述数据来优化第一识别算法,以提高通过第一识别算法从第一原始数据中获取第一抽象数据的准确率,即提高第一传感器的识别准确率。
需要说明的是,上述第二传感器可以与一般的传感器一样,仅与处理单元单向传输数据,也可以执行本申请的数据处理方法,与处理单元双向传输数据,例如图2中的第二传感器1所示,本申请对此不作限定。
上述数据处理方法中的第一原始数据指的是某一个时间点从交通场景中测量得到的数据,第一抽象数据和第一反馈数据均是与该第一原始数据对应的数据,即与该时间点对应的数据。而实际上第一传感器在工作时,其通常按照一定的频率在不断地测量不同的交通场景中的数据。例如,当第一传感器为摄像头时,其可以根据不同的帧率(比如30帧每秒或者60帧每秒等),来采集交通场景的图片。又例如,当第一传感器为激光雷达时,其可以按照一定的周期(比如以几十毫秒为一个周期),对交通场景进行扫描。因此,在不同的时间点,第一传感器在不断测量得到第一原始数据。在不同的第一原始数据所对应的交通场景中,交通参与者、交通环境可能发生变化。为了便于区分不同时间点获取到的第一原始数据,可选地,第一原始数据、对应的第一抽象数据都可以包含相应的时间戳。其他传感器在向处理单元传输数据时,发送的第二抽象数据也可以包含相应的时间戳。
本申请实施例中的时间戳,用于指示传感器从场景中测量得到原始数据的时间信息。第一原始数据、第一抽象数据所包含的时间戳,是第一传感器从场景中测量得到第一原始数据的时间信息。相应地,第二原始数据、第二抽象数据所包含的时间戳,用于指示第二传感器从场景中测量得到第二原始数据的时间信息。
这样,处理单元就可以把时间戳相同的抽象数据进行融合,而不会错误地把时间戳不同的抽象数据融合起来。第一反馈数据包含的时间戳,与该第一反馈数据对应的第一抽象数据、第一原始数据的时间戳相同。即在发送第一反馈数据时,处理单元将与所述第一原始数据和所述第一抽象数据的时间戳对应的所述第一反馈数据,发送给第一传感器。基于此,第一传感器就可以通过时间戳,找到对应的第一抽象数据、第一原始数据和第一反馈数据,来优化第一识别算法。
可选地,在前述S103的步骤中,第一传感器携带至少一个识别标签,所述识别标签用于标识所述第一传感器能够识别的目标的属性种类。例如,如果第一传感器为激光雷达,其能够识别出交通场景中的车辆的尺寸、与车辆之间的距离等,但是无法识别出车辆的颜色。这样,就可以为激光雷达配置“尺寸”、“与车辆之间的距离”等识别标签。第二目标的属性描述数据是基于融合后数据或者交互信息得到的,这些属性描述数据中可能包含了“尺寸”等,也可能包含了“颜色”。由于不同传感器的实现原理不同,即便将“颜色”的数值发送给激光雷达,激光雷达也无法用它来优化第一识别算法。为此,处理单元在发送第一反馈数据时,第一反馈数据中仅包含与第一传感器的至少一个识别标签相匹配的第二目标的属性描述数据。沿用前述的例子,当第一传感器为激光雷达时,第一反馈数据中可以包括第二目标的“尺寸”、“与车辆之间的距离”等属性描述数据,而无需包括“颜色”等属性描述数据。
可选地,第一反馈数据包括与其中的每一条第二目标的属性描述数据对应的置信度,所述置信度用于表征第二目标的属性描述数据的可信程度。
需要说明的是,对于某一条第二目标的属性描述数据而言,其对应的置信度可以包含在该条第二目标的属性描述数据中,采用该条第二目标的属性描述数据中的某一个字段来表示,也可以包含在第一反馈数据中,采用一个独立的字符串来表示,本申请对此不作限定。当置信度采用独立的字符串来表示时,第二目标的属性描述数据与相应的置信度之间建立有对应关系。
在一种实现方式中,某一条第二目标的属性描述数据的置信度,可以根据其来源来确定。例如,如果第二目标的属性描述数据来自融合后数据,则可以将其置信度设置为80%;如果第二目标的属性描述数据来自云端服务器,则可以将其置信度设置为95%;如果第二目标的属性描述数据来自第二目标,则可以将其置信度设置为99%。也就是说,第二目标的属性描述数据包含对应的来源标签,该来源标签用于标识第二目标的属性描述数据的来源。可选地,上述来源标签可以采用第二目标的属性描述数据中的某一个字段来表示。由于第二目标的属性描述数据的来源与第二目标的属性描述数据的置信度之间存在对应关系,故而第一传感器可以根据第二目标的属性描述数据的来源,来确定其相应的置信度。
在这种情况下,前述S103的步骤可以包括:根据所述第二目标的属性描述数据,以及所述第二目标的属性描述数据对应的置信度,优化所述第一识别算法。通过这样的方式,第一传感器可以调节第一识别算法的参数的校正幅度,从而进一步提高优化后的第一识别算法的识别准确率。
可选地,为了使第一传感器与处理单元之间实现双向传输,本申请实施例中提供一种通信接口。第一传感器与处理单元之间、第二传感器与处理单元之间均可以采用该通信接口。
该通信接口采用“数据包头+数据体”的形式。其中,数据包头包括了协议版本号、前述的时间戳等信息。此外,数据包头还可以包括周期计数器、传感器的安装位置、数据体个数等,如表5所示。其中,周期计数器用于指示数据传输的周期,传感器的安装位置用于指示传感器在车辆或者路侧等地方的安装位置,数据体个数用于指示当前发送的数据中所包含的数据体的数量。数据体包括了交通目标的属性描述数据,例如前述的交通目标的类别、尺寸、位置、方向角、速度、加速度等,如表5所示。需要说明的是,根据不同的应用场景,可以调整数据包头和数据体中的具体内容。
表5 通信接口的实现形式举例
内容 | |
数据包头 | 协议版本号、时间戳、周期计数器、传感器的安装位置、数据体个数…… |
数据体 | 交通目标的位置、方向角、速度、加速度、交通目标的类别、灯光信息…… |
通过采用该通信接口,处理单元可以对接多种不同的传感器,既可以接收传感器发送的数据,也可以向传感器发送数据。此外,处理单元与云端传感器之间,也可以采用该通信接口来进行数据传输。这样,各个传感器与处理单元之间就可以采用统一的接口,从而简化处理单元的软件设计。
在本申请的第二个实施例中提供一种数据处理方法。该方法可以由处理单元来执行。该处理单元可以是具有数据融合功能的处理单元。通过执行该数据处理方法,处理单元可以与传感器进行双向传输,接收传感器发送来的抽象数据,并向传感器发送反馈数据,使传感器可以利用反馈数据来优化传感器所使用的识别算法。
请参见图3,由处理单元所执行的数据处理方法可以包括以下S201至S203的步骤。
S201:处理单元接收来自第一传感器的第一抽象数据。
其中,第一抽象数据来源于第一原始数据。在一种实现方式中,第一抽象数据由第一传感器利用第一识别算法处理第一原始数据得到。第一抽象数据包括第一目标的属性描述数据。第一目标为第一传感器根据第一原始数据从交通场景中识别出的交通目标。第一原始数据为第一传感器从交通场景中测量到的数据。这里的第一原始数据、第一目标、交通目标、第一目标的属性描述数据、第一抽象数据,以及第一识别算法等,均可以参考前述的相关描述,此处不再赘述。
S202:处理单元确定第一反馈数据。
其中,第一反馈数据包括第二目标的属性描述数据,所述第二目标为从前述交通场景中识别出的交通目标。这里的第二目标、第二目标的属性描述数据等,也可以参考前述的相关描述,此处不再赘述。
如前所述,处理单元可以通过多种方式来确定第二目标的属性描述数据。可选地,在S202的步骤之前,还可以包括:处理单元确定第二目标的步骤。第二目标中可以包括第一特定目标和/或第二特定目标。处理单元可以基于第一抽象数据,或者,基于来自云端传感器的交互信息,确定第一特定目标。处理单元可以基于第二抽象数据,或者,基于来自云端传感器的交互信息,确定第二特定目标。具体可以参考前述的相关描述,此处不再赘述。
S203:处理单元发送第一反馈数据给第一传感器。
处理单元将第一反馈数据发送给第一传感器,以使第一传感器利用第一反馈数据优化第一识别算法,具体可以参考前述相关描述,此处不再赘述。
可选地,该数据处理方法还可以包括:
S204:处理单元接收来自第二传感器的第二抽象数据。
第二抽象数据、其包含的第三目标的属性描述数据、第二原始数据等,可以参考第一个实施例中的相关描述,此处不再赘述。
处理单元接收了多个传感器的抽象数据之后,进行数据融合,得到融合后数据。从融合后数据中可以确定出一个或者多个第二目标,其中可能包括了第一传感器识别出的交通目标,即第一特定目标,也可能包括了第一传感器未识别出的交通目标,即第二特定目标。如果其中包括了第二特定目标,那么该第二特定目标为第二传感器识别出的交通目标,即该第二特定目标由处理单元基于第二抽象数据而确定的目标。
请参见图4,图4为本申请的数据处理方法的实现方式之二的流程图。可选地,该数据处理方法还可以包括:
S205:处理单元接收来自云端传感器的交互信息。
处理单元从云端传感器获取的交互信息,其中包含了存储在云端传感器上的交通 场景中的交通目标的属性描述数据,例如交通场景中的车辆、交通标识等的属性描述数据。据此,处理单元可以确定一个或者多个第二目标,其中可能包括了第一传感器识别出的交通目标,即前述的第一特定目标,也可能包括了第一传感器未识别出的交通目标,即前述的第二特定目标。
在确定了第二目标之后,处理单元可以从融合后数据,或者交互信息中确定第二目标的属性描述数据。将第一反馈数据发送给第一传感器,第一反馈数据中包括了这些第二目标的属性描述数据。
需要说明的是,在确定第二目标及其属性描述数据时,处理单元与云端传感器之间可以交互一次或者多次,本申请对此不作限定。还需要说明的是,处理单元与云端传感器之间可以T-BOX(Telematics BOX)中的收发模块等间接地进行交互,本申请对此不作限定。上述数据处理方法可以同时包括S204和S205的步骤,如图5所示。
可选地,对于第二目标中的第二特定目标,第一反馈数据中还包括该第二特定目标的位置信息。第二特定目标的位置信息可以参考第一个实施例中的相关描述,此处不再赘述。
可选地,第一反馈数据包括与其中的每一条第二目标的属性描述数据对应的置信度。其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。关于置信度可以参考前述第一个实施例中的相关描述,此处不再赘述。
可选地,第一抽象数据、第一原始数据和第一反馈数据还包含时间戳,以便第一传感器利用时间戳来确定与该第一反馈数据对应的第一抽象数据和第一原始数据。具体可以参考第一个实施例中的相关描述,此处不再赘述。
本申请的第三个实施例提供一种数据处理装置。该装置可以为传感器本身,也可以为传感器中的ECU,还可以传感器中的芯片。请参见图6,图6为本申请实施例中一种数据处理装置的实现方式之一的结构示意图。该数据处理装置400包括:
第一收发模块401,用于接收第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;
至少一个第一处理模块402,用于通过第一识别算法从第一原始数据中获取第一抽象数据;以及,根据所述第一反馈数据优化所述第一识别算法;
其中,所述第一抽象数据包括第一目标的属性描述数据,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
可选地,所述第二目标包括至少一个第一特定目标,所述第一目标中存在与所述第一特定目标相同的目标。
可选地,所述第二目标包括至少一个第二特定目标,所述第一目标中不存在与所述第二特定目标相同的目标。
可选地,所述第一反馈数据包括所述第二特定目标的位置信息和属性描述数据;所述至少一个第一处理模块402还用于根据所述至少一个第二特定目标的位置信息和属性描述数据,优化所述第一识别算法。
可选地,所述第二目标的属性描述数据是基于所述第一抽象数据确定的数据,或者,基于来自云端传感器的交互信息确定的数据。
可选地,所述第一原始数据、所述第一抽象数据和所述第一反馈数据包含时间戳;其中,所述时间戳用于指示从所述场景中得到所述第一原始数据的时间信息;所述至少一个第一处理模块402还用于根据所述时间戳对应的所述第一原始数据、所述第一抽象数据和所述第一反馈数据,优化所述第一识别算法。
可选地,所述至少一个第一处理模块402还用于根据所述第二目标的属性描述数据,以及所述第二目标的属性描述数据对应的置信度,优化所述第一识别算法;其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。
可选地,所述第二目标的属性描述数据包含来源标签,所述来源标签用于标识所述第二目标的属性描述数据的来源,所述第二目标的属性描述数据的来源与所述第二目标的属性描述数据的置信度之间存在对应关系。
本申请的第三个实施例提供另一种数据处理装置。该装置为具有数据融合功能的处理装置。该处理装置可以为车载的中央处理单元本身,也可以是车载的中央处理单元中的芯片或者元件。该处理装置还可以为融合单元,或者融合单元中的芯片或者元件。该处理装置还可以是其它的产品形态。以下通过图7从逻辑功能上对所述数据处理装置进行阐述。请参见图7,图7为本申请实施例中另一种数据处理装置的实现方式之一的结构示意图。该数据处理装置500包括:
第二收发模块501,用于接收来自第一传感器的第一抽象数据;以及,发送第一反馈数据给所述第一传感器;其中,所述第一抽象数据来源于第一原始数据,所述第一抽象数据包括第一目标的属性描述数据,所述第一反馈数据包括第二目标的属性描述数据,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为从所述场景中识别出的目标;
至少一个第二处理模块502,用于确定所述第一反馈数据。
可选地,所述至少一个第二处理模块502还用于基于所述第一抽象数据,或者,基于来自云端传感器的交互信息,确定至少一个第一特定目标;所述第一目标中存在与所述第一特定目标相同的目标。在这种情况下,所述第一反馈数据包括所述至少一个第一特定目标的属性描述数据。
可选地,基于第二抽象数据,或者,基于来自云端传感器的交互信息,确定至少一个第二特定目标;其中,所述第二抽象数据来源于第二原始数据,所述第二原始数据为所述场景的测量数据,所述第二抽象数据包括第三目标的属性描述数据,所述第三目标为所述场景中的目标;所述第一目标中不存在与所述第二特定目标相同的目标。在这种情况下,所述第一反馈数据包括所述至少一个第二特定目标的属性描述数据。
可选地,所述第一反馈数据还包括所述至少一个第二特定目标的位置信息。
可选地,所述至少一个第二处理模块502还用于基于所述第一抽象数据,或者,基于来自云端传感器的交互信息,确定所述第二目标的属性描述数据。
可选地,所述第一原始数据、所述第一抽象数据和所述第一反馈数据包含时间戳;其中,所述时间戳用于指示从所述场景中测量得到所述第一原始数据的时间信息;
所述至少一个第二处理模块502还用于将与所述第一原始数据和所述第一抽象数据的时间戳对应的所述第一反馈数据,发送给所述第一传感器。
可选地,所述第一反馈数据包括与每一条所述第二目标的属性描述数据对应的置 信度,其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。
可选地,每一条所述第二目标的属性描述数据包含对应的来源标签,所述来源标签用于标识所述第二目标的属性描述数据的来源,所述第二目标的属性描述数据的来源与所述第二目标的属性描述数据的置信度之间存在对应关系。
可以理解的是,以上各个模块的划分仅仅是一种逻辑功能的划分,在实际实现时,第一收发模块或第二收发模块的功能可以由收发器实现,第一处理模块或第二处理模块的功能可以由处理器实现。当装置为传感器本身时,收发器可以包括接收天线和发射天线,分别执行接收和发射功能。
在实际应用中,上述第一收发模块401、第二收发模块501均可以通过数据接口或者其他可能的形式来实现,本申请对此不作限定。
上述第一处理模块402、第二处理模块502可以是通用处理器、数字信号处理器、专用集成电路(application-specific integrated circuit,ASIC),可编程逻辑器件(programmable logic device,PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logic device,CPLD),现场可编程逻辑门阵列(field-programmable gate array,FPGA),通用阵列逻辑(generic array logic,简称GAL)或其任意组合。通用处理器可以是微处理器或者任何常规的处理器等。
本申请的第三个实施例还提供一种芯片系统。请参见图8,图8是本申请实施例提供的芯片系统的一种实现方式的结构示意图。该芯片系统600包括:至少一个处理器601和接口602,接口602用于接收代码指令,并传输至至少一个处理器601,至少一个处理器601运行代码指令以实现前述传感器或者处理单元所执行的任一种方法。
上述处理器601可以是通用处理器、数字信号处理器、专用集成电路(application-specific integrated circuit,ASIC),可编程逻辑器件(programmable logic device,PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logic device,CPLD),现场可编程逻辑门阵列(field-programmable gate array,FPGA),通用阵列逻辑(generic array logic,简称GAL)或其任意组合。通用处理器可以是微处理器或者任何常规的处理器等。
应理解,上述芯片系统600可以包括一个芯片,也可以包括多个芯片组成的芯片模组,本申请对此不作限定。
本实施例还提供一种车辆,该车辆可以包括前述的任一种数据处理装置400。此外,该车辆还可以包括前述的任一种数据处理装置500。本申请对于车辆的具体实现形式不作限定。
本实施例还提供一种终端,该终端可以设置在交通场景中,例如安装在路侧等位置。该终端可以包括前述任一种数据处理装置400,该终端还可以包括前述的任一种数据处理装置500。本申请对于终端的具体实现形式不作限定。
本实施例还提供一种系统,该系统可以包括前述的任一种数据处理装置400。此外,该系统还可以包括前述的任一种数据处理装置500。
本实施例还提供一种计算机可读存储介质。该计算机可读存储介质用于存储计算机程序或指令,当所述计算机程序或指令在电子设备上运行时,使得所述电子设备实现前述传感器或者处理单元所执行的任一种方法的部分或全部步骤。
这里的可读存储介质可为磁碟、光盘、DVD、USB、只读存储记忆体(ROM)或随机存储记忆体(RAM)等,本申请对具体的存储介质形式不作限定。
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘(solid state disk,SSD))等。
应理解,在本申请的各种实施例中,各步骤的执行顺序应以其功能和内在逻辑确定,各步骤序号的大小并不意味着执行顺序的先后,不对实施例的实施过程构成限定。
除非另外说明,本说明书中的“多个”,指的是两个或者两个以上。在本申请的实施例中,采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解,“第一”、“第二”等字样并不对数量和执行次序构成限定,并且“第一”、“第二”等字样也并不限定一定不同。
应理解,本申请实施例的架构示意图中,各个功能模块之间的连接仅是示意,不代表实际应用中的物理走线和网络连接方式。
应理解,本说明书中各个实施例之间相同相似的部分互相参见即可。尤其,对于数据处理装置、芯片系统、车辆、终端、计算机可读存储介质的实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,相关之处参见方法实施例中的说明即可。以上所述的本发明实施方式并不构成对本发明保护范围的限定。
Claims (22)
- 一种数据处理方法,其特征在于,所述方法包括:通过第一识别算法从第一原始数据中获取第一抽象数据,所述第一抽象数据包括第一目标的属性描述数据;接收第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;根据所述第一反馈数据优化所述第一识别算法;其中,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
- 根据权利要求1所述的方法,其特征在于,所述第二目标包括至少一个第一特定目标,所述第一目标中存在与所述第一特定目标相同的目标。
- 根据权利要求1或2所述的方法,其特征在于,所述第二目标包括至少一个第二特定目标,所述第一目标中不存在与所述第二特定目标相同的目标。
- 根据权利要求3所述的方法,其特征在于,所述第一反馈数据包括所述第二特定目标的位置信息和属性描述数据;所述根据所述第一反馈数据优化所述第一识别算法的步骤,包括:根据所述至少一个第二特定目标的位置信息和属性描述数据,优化所述第一识别算法。
- 根据权利要求1-4任一项所述的方法,其特征在于,所述第二目标的属性描述数据是基于所述第一抽象数据确定的数据,或者,基于来自云端传感器的交互信息确定的数据。
- 根据权利要求1-5任一项所述的方法,其特征在于,所述第一原始数据、所述第一抽象数据和所述第一反馈数据包含时间戳;其中,所述时间戳用于指示从所述场景中得到所述第一原始数据的时间信息;所述根据所述第一反馈数据优化所述第一识别算法的步骤,包括:根据所述时间戳对应的所述第一原始数据、所述第一抽象数据和所述第一反馈数据,优化所述第一识别算法。
- 根据权利要求1-6任一项所述的方法,其特征在于,所述根据所述第一反馈数据优化所述第一识别算法的步骤,包括:根据所述第二目标的属性描述数据,以及所述第二目标的属性描述数据对应的置信度,优化所述第一识别算法;其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。
- 根据权利要求7所述的方法,其特征在于,所述第二目标的属性描述数据包含来源标签,所述来源标签用于标识所述第二目标的属性描述数据的来源,所述第二目标的属性描述数据的来源与所述第二目标的属性描述数据的置信度之间存在对应关系。
- 一种数据处理方法,其特征在于,包括:接收来自第一传感器的第一抽象数据,所述第一抽象数据来源于第一原始数据, 所述第一抽象数据包括第一目标的属性描述数据;确定第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;发送所述第一反馈数据给所述第一传感器;其中,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
- 根据权利要求9所述的方法,其特征在于,所述方法还包括:基于所述第一抽象数据,或者,基于来自云端传感器的交互信息,确定至少一个第一特定目标;所述第一目标中存在与所述第一特定目标相同的目标;其中,所述第一反馈数据包括所述至少一个第一特定目标的属性描述数据。
- 根据权利要求9或10所述的方法,其特征在于,所述方法还包括:基于第二抽象数据,或者,基于来自云端传感器的交互信息,确定至少一个第二特定目标;其中,所述第二抽象数据来源于第二原始数据,所述第二原始数据为所述场景的测量数据,所述第二抽象数据包括第三目标的属性描述数据,所述第三目标为所述场景中的目标;所述第一目标中不存在与所述第二特定目标相同的目标;其中,所述第一反馈数据包括所述至少一个第二特定目标的属性描述数据。
- 根据权利要求11所述的方法,其特征在于,所述第一反馈数据还包括所述至少一个第二特定目标的位置信息。
- 根据权利要求9-12任一项所述的方法,其特征在于,所述确定第一反馈数据的步骤,包括:基于所述第一抽象数据,或者,基于来自云端传感器的交互信息,确定所述第二目标的属性描述数据。
- 根据权利要求9-13任一项所述的方法,其特征在于,所述第一原始数据、所述第一抽象数据和所述第一反馈数据包含时间戳;其中,所述时间戳用于指示从所述场景中测量得到所述第一原始数据的时间信息;所述发送所述第一反馈数据给所述第一传感器的步骤,包括:将与所述第一原始数据和所述第一抽象数据的时间戳对应的所述第一反馈数据,发送给所述第一传感器。
- 根据权利要求9-14任一项所述的方法,其特征在于,所述第一反馈数据包括与每一条所述第二目标的属性描述数据对应的置信度,其中,所述置信度用于表征所述第二目标的属性描述数据的可信程度。
- 根据权利要求15所述的方法,其特征在于,每一条所述第二目标的属性描述数据包含对应的来源标签,所述来源标签用于标识所述第二目标的属性描述数据的来源,所述第二目标的属性描述数据的来源与所述第二目标的属性描述数据的置信度之间存在对应关系。
- 一种数据处理装置,其特征在于,包括:第一收发模块,用于接收第一反馈数据,所述第一反馈数据包括第二目标的属性描述数据;至少一个第一处理模块,用于通过第一识别算法从第一原始数据中获取第一抽象数据;以及,根据所述第一反馈数据优化所述第一识别算法;其中,所述第一抽象数 据包括第一目标的属性描述数据,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为所述场景中的目标。
- 一种数据处理装置,其特征在于,包括:第二收发模块,用于接收来自第一传感器的第一抽象数据;以及,发送第一反馈数据给所述第一传感器;其中,所述第一抽象数据来源于第一原始数据,所述第一抽象数据包括第一目标的属性描述数据,所述第一反馈数据包括第二目标的属性描述数据,所述第一原始数据为场景的测量数据,所述第一目标和所述第二目标为从所述场景中识别出的目标;至少一个第二处理模块,用于确定所述第一反馈数据。
- 一种芯片系统,其特征在于,包括至少一个处理器以及接口,所述接口用于接收代码指令,并传输所述至少一个处理器;所述至少一个处理器运行所述代码指令,以实现权利要求1-8任一项所述的方法。
- 一种芯片系统,其特征在于,包括至少一个处理器以及接口,所述接口用于接收代码指令,并传输所述至少一个处理器;所述至少一个处理器运行所述代码指令,以实现权利要求9-16任一项所述的方法。
- 一种计算机可读存储介质,其特征在于,用于存储计算机程序或指令,当所述计算机程序或所述指令在电子设备上运行时,使所述电子设备实现权利要求1至8任一项所述的方法。
- 一种计算机可读存储介质,其特征在于,用于存储计算机程序或指令,当所述计算机程序或所述指令在电子设备上运行时,使所述电子设备实现权利要求9至16任一项所述的方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP20869237.6A EP4024974A4 (en) | 2019-09-29 | 2020-08-14 | DATA PROCESSING METHOD AND DEVICE, CHIP SYSTEM AND MEDIUM |
US17/706,351 US20220215197A1 (en) | 2019-09-29 | 2022-03-28 | Data processing method and apparatus, chip system, and medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910934078.3A CN112578781B (zh) | 2019-09-29 | 2019-09-29 | 数据处理方法、装置、芯片系统及介质 |
CN201910934078.3 | 2019-09-29 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/706,351 Continuation US20220215197A1 (en) | 2019-09-29 | 2022-03-28 | Data processing method and apparatus, chip system, and medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021057324A1 true WO2021057324A1 (zh) | 2021-04-01 |
Family
ID=75111059
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/109187 WO2021057324A1 (zh) | 2019-09-29 | 2020-08-14 | 数据处理方法、装置、芯片系统及介质 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20220215197A1 (zh) |
EP (1) | EP4024974A4 (zh) |
CN (1) | CN112578781B (zh) |
WO (1) | WO2021057324A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113252881A (zh) * | 2021-05-17 | 2021-08-13 | 陕西中易泰能源科技有限公司 | 一种油品含水分析系统和信息融合分析方法 |
CN114217316A (zh) * | 2021-12-13 | 2022-03-22 | 深圳市道通智能航空技术股份有限公司 | 一种数据处理方法、设备、系统及介质 |
CN117376391B (zh) * | 2023-09-26 | 2024-04-02 | 广东省水文局江门水文分局 | 应用于水质监测仪的传感器自动识别方法及装置 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060204107A1 (en) * | 2005-03-04 | 2006-09-14 | Lockheed Martin Corporation | Object recognition system using dynamic length genetic training |
CN102303605A (zh) * | 2011-06-30 | 2012-01-04 | 中国汽车技术研究中心 | 基于多传感器信息融合的碰撞及偏离预警装置及预警方法 |
CN105682222A (zh) * | 2016-03-01 | 2016-06-15 | 西安电子科技大学 | 一种基于车辆自组织网络的车辆位置定位信息融合方法 |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6915239B2 (en) * | 2001-01-19 | 2005-07-05 | International Business Machines Corporation | Method and apparatus for opportunistic decision support from intermittent interconnected sensors and data archives |
US7065465B2 (en) * | 2002-03-26 | 2006-06-20 | Lockheed Martin Corporation | Method and system for multi-sensor data fusion |
DE102004038494A1 (de) * | 2004-08-07 | 2006-03-16 | Robert Bosch Gmbh | Verfahren und Vorrichtung zum Betreiben eines Sensorsystems |
KR101860966B1 (ko) * | 2010-06-23 | 2018-07-05 | 콘티넨탈 테베스 아게 운트 코. 오하게 | 가속 물체 인식 및/또는 가속 물체 속성 인식을 위한 방법 및 시스템 그리고 그 방법의 이용 |
US9563199B1 (en) * | 2013-11-27 | 2017-02-07 | Google Inc. | Assisted perception for autonomous vehicles |
US9916703B2 (en) * | 2015-11-04 | 2018-03-13 | Zoox, Inc. | Calibration for autonomous vehicle operation |
CN115686005A (zh) * | 2016-06-28 | 2023-02-03 | 柯尼亚塔有限公司 | 训练自动驾驶系统的虚拟模型的系统及计算机执行方法 |
US10262234B2 (en) * | 2017-04-24 | 2019-04-16 | Baidu Usa Llc | Automatically collecting training data for object recognition with 3D lidar and localization |
US11042155B2 (en) * | 2017-06-06 | 2021-06-22 | Plusai Limited | Method and system for closed loop perception in autonomous driving vehicles |
CN107908655B (zh) * | 2017-10-12 | 2019-03-19 | 北京三快在线科技有限公司 | 一种定位的方法和装置 |
US10591914B2 (en) * | 2017-11-08 | 2020-03-17 | GM Global Technology Operations LLC | Systems and methods for autonomous vehicle behavior control |
US10535138B2 (en) * | 2017-11-21 | 2020-01-14 | Zoox, Inc. | Sensor data segmentation |
CN108196535B (zh) * | 2017-12-12 | 2021-09-07 | 清华大学苏州汽车研究院(吴江) | 基于增强学习和多传感器融合的自动驾驶系统 |
CN108229366B (zh) * | 2017-12-28 | 2021-12-14 | 北京航空航天大学 | 基于雷达和图像数据融合的深度学习车载障碍物检测方法 |
CN109901574B (zh) * | 2019-01-28 | 2021-08-13 | 华为技术有限公司 | 自动驾驶方法及装置 |
CN109816033A (zh) * | 2019-01-31 | 2019-05-28 | 清华四川能源互联网研究院 | 一种基于优化的监督学习进行台区用户识别的方法 |
DE102019102769A1 (de) * | 2019-02-05 | 2020-08-06 | Bayerische Motoren Werke Aktiengesellschaft | Verfahren und eine Vorrichtung zur Sensordatenfusion für ein Fahrzeug |
CN110263324B (zh) * | 2019-05-16 | 2021-02-12 | 华为技术有限公司 | 文本处理方法、模型训练方法和装置 |
CN110160804B (zh) * | 2019-05-31 | 2020-07-31 | 中国科学院深圳先进技术研究院 | 一种自动驾驶车辆的测试方法、装置及系统 |
EP3805998A1 (de) * | 2019-10-11 | 2021-04-14 | Elektrobit Automotive GmbH | Verarbeitung von sensordaten in einem kraftfahrzeug |
-
2019
- 2019-09-29 CN CN201910934078.3A patent/CN112578781B/zh active Active
-
2020
- 2020-08-14 WO PCT/CN2020/109187 patent/WO2021057324A1/zh unknown
- 2020-08-14 EP EP20869237.6A patent/EP4024974A4/en active Pending
-
2022
- 2022-03-28 US US17/706,351 patent/US20220215197A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060204107A1 (en) * | 2005-03-04 | 2006-09-14 | Lockheed Martin Corporation | Object recognition system using dynamic length genetic training |
CN102303605A (zh) * | 2011-06-30 | 2012-01-04 | 中国汽车技术研究中心 | 基于多传感器信息融合的碰撞及偏离预警装置及预警方法 |
CN105682222A (zh) * | 2016-03-01 | 2016-06-15 | 西安电子科技大学 | 一种基于车辆自组织网络的车辆位置定位信息融合方法 |
Non-Patent Citations (2)
Title |
---|
DAN HU, CHANGFU ZONG, HSIAOHSIANG NA: "Research on information fusion algorithm for vehicle speed information and road adhesion property estimation", MECHATRONICS AND AUTOMATION, 2009. ICMA 2009. INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 1 August 2009 (2009-08-01), Piscataway, NJ, USA, pages 3229 - 3234, XP055795981, ISBN: 978-1-4244-2692-8, DOI: 10.1109/ICMA.2009.5246209 * |
See also references of EP4024974A4 * |
Also Published As
Publication number | Publication date |
---|---|
EP4024974A4 (en) | 2022-11-02 |
CN112578781B (zh) | 2022-12-30 |
CN112578781A (zh) | 2021-03-30 |
US20220215197A1 (en) | 2022-07-07 |
EP4024974A1 (en) | 2022-07-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021057324A1 (zh) | 数据处理方法、装置、芯片系统及介质 | |
WO2021057612A1 (zh) | 传感器的标定方法和装置 | |
CN109086788B (zh) | 用于从异构装置感测的多种不同格式的数据的多模式融合处理的设备、方法和系统 | |
CN109212530B (zh) | 用于确定障碍物速度的方法和装置 | |
US11940804B2 (en) | Automated object annotation using fused camera/LiDAR data points | |
CN105793669B (zh) | 车辆位置推定系统、装置、方法以及照相机装置 | |
CN114051628B (zh) | 一种确定目标对象点云集的方法及装置 | |
US20210325901A1 (en) | Methods and systems for automated driving system monitoring and management | |
US20210354708A1 (en) | Online perception performance evaluation for autonomous and semi-autonomous vehicles | |
US20200341150A1 (en) | Systems and methods for constructing a high-definition map based on landmarks | |
US11410429B2 (en) | Image collection system, image collection method, image collection device, recording medium, and vehicle communication device | |
CN113885062A (zh) | 基于v2x的数据采集融合设备、方法和系统 | |
CN112883058A (zh) | 用于车辆定位的标定方法、装置、设备、车辆和介质 | |
CN113129382A (zh) | 一种确定坐标转换参数的方法及装置 | |
WO2024078265A1 (zh) | 多图层高精地图生成方法和装置 | |
CN107545760B (zh) | 提供在定位地点处定位车辆的定位信息的方法和通过另一车辆提供定位车辆的信息的方法 | |
US11312393B2 (en) | Artificially falsifying sensor data to initiate a safety action for an autonomous vehicle | |
CN117804470A (zh) | 基于车联万物辅助的车辆定位方法、装置、设备及介质 | |
JP2023539643A (ja) | 車両の確認および検証のためのクリティカルシナリオの識別 | |
CN116935640A (zh) | 一种基于多传感器的路侧感知方法、装置、设备及介质 | |
CN114724116B (zh) | 车辆通行信息生成方法、装置、设备和计算机可读介质 | |
CN111480165A (zh) | 在考虑对象的特征结构的情况下创建用于车辆的基于特征的定位地图的方法 | |
CN112733778B (zh) | 一种车辆前导车确定方法、装置及计算机设备 | |
CN112698315B (zh) | 移动设备定位系统、方法及设备 | |
US11433920B2 (en) | Map-based prediction and mitigation of performance limitations for autonomous vehicles |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20869237 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2020869237 Country of ref document: EP Effective date: 20220331 |