CN109547693A - Multi-sensor data synchronization system and image data synchronous method - Google Patents
Multi-sensor data synchronization system and image data synchronous method Download PDFInfo
- Publication number
- CN109547693A CN109547693A CN201811437890.7A CN201811437890A CN109547693A CN 109547693 A CN109547693 A CN 109547693A CN 201811437890 A CN201811437890 A CN 201811437890A CN 109547693 A CN109547693 A CN 109547693A
- Authority
- CN
- China
- Prior art keywords
- data
- frame
- sensor
- bus
- host
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 23
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 238000004321 preservation Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 4
- 230000001419 dependent effect Effects 0.000 claims description 2
- VIKNJXKGJWUCNN-XGXHKTLJSA-N norethisterone Chemical compound O=C1CC[C@@H]2[C@H]3CC[C@](C)([C@](CC4)(O)C#C)[C@@H]4[C@@H]3CCC2=C1 VIKNJXKGJWUCNN-XGXHKTLJSA-N 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 19
- 238000012546 transfer Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000013507 mapping Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Small-Scale Networks (AREA)
- Traffic Control Systems (AREA)
Abstract
The application provides a kind of multi-sensor data synchronization system and image data synchronous method, the multiple sensors connecting the system comprises host, with host；Host, the data inputted for acquiring multiple sensors and image acquiring device；Multiple sensors, including first kind sensor, the second class sensor and third class sensor；First kind sensor is connect with vehicle body CAN bus, and the second class sensor is connect with privately owned CAN bus, and third class sensor includes signal end and data terminal, signal end is connect with the vehicle body CAN bus, for exporting frame signal, data terminal is connect by data line with host, to host output data；Vehicle body CAN bus, privately owned CAN bus pass through same data converter respectively and connect with host, so that the data for being input to host are synchronous.The embodiment of the present application can be greatly reduced data in transmission process generated delay, improve the instantaneity of data, and data collected can be made to realize that time label is unified.
Description
Technical field
This application involves data collecting field, in particular to a kind of multi-sensor data synchronization system and image data are same
One step process.
Background technique
In automatic Pilot field, vehicle can all install multiple sensors to perceive and detect complicated road environment, such as
Camera, millimetre-wave radar, ultrasonic probe etc..
And automatic Pilot algorithm model is studied using the method for deep learning, need the true road conditions data of magnanimity to instruct
Practice algorithm model, and either algorithm research and development, algorithm improvement need a large amount of data, test, analog simulation in product
The data for also needing magnanimity are supported.
Under existing vehicle condition, data or unification is transferred to by CAN bus agreement that each sensor generates
ECU(Electronic Control Unit) it is handled, or handled equipped with respective ECU.But each sensing
The data transfer mode and interface of device are possibly different from, and external host and numerous sensors can not be directly connected to and communicate.
Simultaneously as each sensor not necessarily has clock system, it is difficult to accomplish to not simultaneous interpretation particular sensor has clock system
The clock system of sensor carries out relatively accurate clock synchronization, thus all data of host acquisition or without time label or
Because of data delay is larger, is recorded time and physical presence error.Multiple sensing datas can not be real on the time marks
Now unify, increases the difficulty of researcher's subsequent execution data analysis.
Summary of the invention
The application provides a kind of multi-sensor data synchronization system and image data synchronous method, can make sensor institute
The data of acquisition realize time synchronization.
The application provides a kind of multi-sensor data synchronization system, comprising:
Host, the image acquiring device being connect respectively with the host and multiple sensors；
The host, the data inputted for acquiring multiple sensors and described image acquisition device；
The multiple sensor, including first kind sensor, the second class sensor and third class sensor；
The first kind sensor is connect with vehicle body CAN bus, and the second class sensor is connect with privately owned CAN bus, described
Third class sensor includes signal end and data terminal, and the signal end is connect with the vehicle body CAN bus, is believed for output frame
Number, the data terminal is connect by data line with the host, Xiang Suoshu host output data；It is the vehicle body CAN bus, privately owned
CAN bus passes through same data converter respectively and connect with the host, so that the data for being input to the host are synchronous；
Wherein, the first kind sensor, the second class sensor and third class sensor are generated flat in section at work
Equal data volume is respectively the first data volume, the second data volume and third data volume, first data volume, the second data volume with
And third data volume is incremented by successively.
Optionally, the data converter, the data for sending the vehicle body CAN bus and privately owned CAN bus
Converged, and by the data after converging from the data conversion of CAN protocol is met at the data for meeting Ethernet protocol；By host
Data from the data conversion of Ethernet protocol is met at the data for meeting CAN protocol, and to corresponding vehicle body CAN bus or
Privately owned CAN bus is sent.
Optionally, the third class sensor is image acquiring device, and described image acquisition device includes that can be connected with each other
The camera module and signal processing mould group of communication；
The camera module is connect with the data terminal, and the signal processing mould group is total by signal end and the vehicle body CAN
Line connection, for generating and exporting the frame signal to the vehicle body CAN bus by the signal end.
Optionally, the data line is equipped with adapter, and the adapter is used in the interface of the data terminal and described
It transfers between the interface of host.
Optionally, the adapter is for the switching between HDMI interface and USB interface；
The adapter HDMI interface is connect with the output end of the camera module, and the USB interface is connect with the host.
Optionally, stating first kind sensor includes vehicle speed sensor, steering wheel angle sensor, turn signal sensor, oil
The combination of the one or more of them of door jaw opening sensor and brake sensor.
Optionally, the second class sensor includes millimetre-wave radar.
Disclosed herein as well is a kind of image data synchronous method, applied to more described in claim 1-7 any one
Sensing data synchronization system, which comprises
Frame image is obtained, frame identification corresponding with the frame image is obtained according to the frame image；
Frame signal is generated according to the frame identification, the frame signal carries the frame identification；
The frame signal is exported to vehicle body CAN bus, to send the frame signal to master by the vehicle body CAN bus
Machine, and other data in the frame signal and the vehicle body CAN bus is made to realize time synchronization；
The frame image is sent to host by data line.
It is optionally, described that frame identification corresponding with the frame image is obtained according to the frame image, comprising:
Frame number corresponding with the frame image, the acquisition time elder generation of the frame number and frame image are obtained according to the frame image
It is order dependent afterwards；
Corresponding frame identification is obtained based on the frame number.
It is optionally, described that the frame image is sent to host by data line, comprising:
Frame image and the frame identification are associated, the frame image after being associated with；
Frame image after the association is sent to host by data line.
Optionally, described to be associated frame image and the frame identification, the frame image after being associated with, comprising:
It will be in the predeterminable area of the information preservation of the frame identification to the frame image.
Optionally, the information preservation by the frame identification is into the predeterminable area of the frame image, comprising:
Obtain target color values corresponding with the information of the frame identification；
Obtain the object pixel in the predeterminable area of the frame image；
The color value of the object pixel is switched to the target color values.
Optionally, the method also includes:
The frame signal is obtained in the timing node of the data flow of the vehicle body CAN bus；
Obtain target frame image associated by frame identification entrained by the frame signal；
Target frame image timing node corresponding with the frame signal is associated.
From the foregoing, it will be observed that multi-sensor data synchronization system, image data synchronous method in the embodiment of the present application, more biographies
Connection type between host used by sensor data synchronous system, multiple sensors is passed by a plurality of data transfer channel
Data generated instantaneity for postponing, improving data in transmission process can be greatly reduced in transmission of data, by multiple sensors
Signal be respectively connected in a plurality of CAN bus, and after the data of a plurality of CAN bus are summarized using data converter with
Host carries out data transmission, and data collected can be made to realize that time label is unified.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of multi-sensor data synchronization system provided by the embodiments of the present application.
Fig. 2 is another structural schematic diagram of multi-sensor data synchronization system provided by the embodiments of the present application.
Fig. 3 is the implementation flow chart of image data synchronous method provided by the embodiments of the present application.
Fig. 4 is the implementation flow chart provided by the embodiments of the present application for obtaining frame identification.
Fig. 5 is the implementation flow chart provided by the embodiments of the present application for saving frame identification information.
Fig. 6 is the implementation flow chart of disassociation frame image and timing node provided by the embodiments of the present application.
Specific embodiment
The preferred embodiment of the application is described in detail with reference to the accompanying drawing so that the advantages of the application and feature more
It is easily readily appreciated by one skilled in the art, to make apparent define to the protection scope of the application.
Referring to Fig. 1, showing the structure of multi-sensor data synchronization system provided by the embodiments of the present application in figure.
The multi-sensor data synchronization system can be used for vehicle electronic device, which can pass through multiple biographies
Sensor 20 realizes the acquisition to automotive environment parameter.
The multi-sensor data synchronization system includes host 10 and multiple sensors 20.
Specifically, the host 10, the data inputted for acquiring multiple sensors 20.The sensor 20 may include pressure
Force snesor, acceleration transducer, ultrasonic probe, imaging sensor or millimetre-wave radar etc., can also be for obtaining
The other sensors of environmental parameter, do not make exhaustion herein.The host 10 can be PC(Personal Computer), it can be with
It is other data acquisition devices, e.g. server, vehicle driving computer etc..
Multiple sensor 20, including first kind sensor 21, the second class sensor 22 and third class sensor 23, should
First kind sensor 21, the second class sensor 22 and third class sensor 23 generated average amount in section at work
Respectively the first data volume, the second data volume and third data volume, first data volume, the second data volume and third data
It measures incremented by successively.Wherein, the signal end of the first kind sensor 21 and the third class sensor 23 and vehicle body CAN bus 31 connect
It connects；The second class sensor 22 is connect with privately owned CAN bus 32；The vehicle body CAN bus 31, privately owned CAN bus 32 pass through respectively
Same data converter 40 is connect with the host 10, so that the data for being input to the host are synchronous.
In some embodiments, the first kind sensor 21 can be the number generated after the completion of each data acquisition movement
It may include vehicle speed sensor 20, steering wheel angle sensor 20, turn signal sensor 20, oil according to lesser sensor 20 is measured
The combination of the one or more of them of door jaw opening sensor 20 and brake sensor 20.When multiple first kind sensors 21 are linked into
After vehicle body CAN bus 31, the data of first kind sensor 21 can pass through vehicle body CAN bus because of the lesser reason of data volume
31 are transmitted directly to the case where host 10 is without will cause data congestion.
In some embodiments, the second class sensor 22 can be the number generated after the completion of each data acquisition movement
According to the biggish sensor 20 of amount.In this embodiment, which may include millimetre-wave radar, the millimeter wave thunder
Data volume caused by dyne is larger, and the data congestion of vehicle body CAN bus 31 can be avoided by privately owned CAN bus 32.
Certainly, traditional 30 maximum transmission rate of CAN bus can also be other biographies in addition to millimetre-wave radar for 1Mbit/s
Defeated rate meets the sensor 20 of 30 transmission conditions of CAN bus.The privately owned CAN bus 32 can have one or more, specifically
Quantity and the 20 type the application of sensor connect are not limited thereto.
The third class sensor 23, including signal end and data terminal, the data terminal are connect by data line with host 10, to
10 output data of host, the signal end is for exporting frame signal.Wherein, which can obtain dress for image
Set, the image acquiring device include camera and with the camera connection communication, in obtained data signal at
Manage mould group.The camera can be CMOS camera either CCD camera, and specific camera type, model can bases
Practical application is adjusted.
It should be understood that the third class sensor can also be the biggish biography of other data volumes in addition to image acquiring device
Sensor, the sensing such as the sensor, such as array radar of especially generated data traffic beyond the limitation of common CAN bus bandwidth
Equipment.
Wherein, which is the signal that single is sent, e.g. the signal comprising certain frame message or certain pulse letter
Number etc., if third class sensor be image acquiring device, the frame signal be when image acquiring device gets a frame image,
The signal corresponding with the frame image generated simultaneously.Namely the corresponding frame signal of frame image, the frame number and frame of the image are believed
Number quantity it is suitable.
In some embodiments, which can be dedicated serial/parallel mouth data line, such as HDMI data line, USB number
According to line, DVI data line or AV data line etc., concrete type the application is not construed as limiting.The image acquiring device passes through data line
It is connect with host 10, so that the biggish image of image acquiring device data volume obtained or image, may not need occupancy has
30 resource of CAN bus of limit is directly transmitted between host 10, and efficiency of transmission is improved.
Specifically, sending phase by signal end after the frame image transmitting acquired in the image acquiring device enters host 10
The frame signal answered can use frame signal in the time that CAN bus 30 is transmitted and mark the sending instant to determine frame signal.So
Afterwards, the sending instant of frame signal and the frame image are associated, allow host 10 by time of frame signal label come
The specific sending time of frame image is known, so that when the frame image and other data sent through CAN bus 30 are realized
Between the unification that marks.
In addition, host 10 marks in order to obtain each data specific time, it can be by vehicle body CAN bus 31 and other
Privately owned CAN bus 32 is attached by data converter 40, so that vehicle body CAN bus 31 and other privately owned CAN bus 32 are logical
The data converter 40 is crossed by data summarization into same route, and realizes that inputted data all have phase using CAN protocol
The time label answered, so that the data in multiple and different CAN bus 30 realize the unification of time label.
In some embodiments, which may include multiple CAN ports, and different CAN bus 30 passes through
The CAN port of the CAN port of itself and the data converter 40 is attached.The data converter 40 can pass through internal number
According to conversion function, the data mode that the data conversion for meeting CAN protocol is read at host 10 is transferred on host 10,
To facilitate host 10 to be received, saved to above-mentioned data.
It is host used by the system, multiple from the foregoing, it will be observed that the multi-sensor data synchronization system in the embodiment of the present application
Connection type between sensor, data can be greatly reduced in transmission process by transmitting data by a plurality of data transfer channel
Generated delay, the instantaneity for improving data, the signal of multiple sensors are respectively connected in a plurality of CAN bus, and benefit
Carry out data transmission after being summarized the data of a plurality of CAN bus with data converter with host, number collected can be made
It is marked between factually current unified.
Referring to Fig. 2, showing another structure of multi-sensor data synchronization system provided by the embodiments of the present application in figure.
As shown in Fig. 2, in one embodiment, which includes host 10, image acquisition dress
Set 23 and electronic control unit (ECU, Electronic Control Unit).
Wherein, the host 10 be PC machine 10, the PC machine 10 may include processor, memory and with the processor, deposit
The relevant interface of reservoir connection, such as USB interface, the network card interface that is connect with Ethernet (Ethernet).The PC machine 10 is used for
Obtain the sensing data and image data of multi-sensor data synchronization system.
The image acquiring device 23 includes camera module 231 and signal processing mould group 232, the image acquiring device 23
It further include the SOC for data acquisition, transmission.The camera module 231 includes camera and relevant peripheral circuit, is used for
Realization is photosensitive to external ambient light progress, to obtain frame image.The SOC passes through data for handling the frame image
Line is transferred in PC machine 10.
Herein, because may be because that interface problem cannot be directly connected between PC machine 10 and image acquiring device 23, at this time
On the data line can be equipped with adapter, the adapter be used for image acquiring device 23 data terminal interface and PC machine
It transfers between 10 interface.Specifically, adapter is that HDMI turns USB device 50 in this embodiment, it to be used for HDMI interface
Switching between USB interface, the adapter HDMI interface are connect with the output end of camera module 231, namely defeated with SOC
Outlet connection, the USB interface are connect with PC.Certainly, the interface class which can need to transfer determines according to actual conditions
Switching between type, such as HDMI interface, USB interface, DVI interface or AV interface.
The signal processing module of the image acquiring device 23 is attached with vehicle body CAN bus 31, to the vehicle body CAN bus
31 send corresponding frame signal, can use frame signal in the time that CAN bus 30 is transmitted and mark the transmission to determine frame signal
Moment.Then, the sending instant of frame signal and the frame image are associated, allow time of the host 10 by frame signal
Label is to know the specific sending time of frame image, so that the frame image and other data sent through CAN bus 30
Realize the unification of time label.
The ECU may include vehicle speed sensor 20, steering wheel angle sensor 20, turn signal sensor 20, accelerator open degree
The one or more of them of sensor 20 and brake sensor 20 combines, and the sensor 20 in the ECU can be linked into vehicle body
In CAN bus 31, and sensing data is transmitted using vehicle body CAN bus 31.
It should be understood that in addition to the sensor 20 in ECU, other sensors 20 such as acceleration sensing of setting in systems
The lesser sensor 20 of data volume caused by device 20, pressure sensor 20 etc. is accessible in the vehicle body CAN bus 31,
This is not enumerated.
The sensor 20 can also include millimetre-wave radar 22, the millimetre-wave radar 22 because generate data volume it is larger,
It is then linked into privately owned CAN bus 32, to carry out independent biography by data of the privately owned CAN bus 32 to millimetre-wave radar 22
It is defeated, the data of millimetre-wave radar 22 and conflicting for other sensors data are avoided, data delay can be effectively reduced.In addition to milli
Metre wave radar 22, other can use privately owned CAN bus 32 there may be the biggish sensor 20 of data volume and carry out data biography
It is defeated.
The data of the vehicle body CAN bus 31 and privately owned CAN bus 32 are required to be attached with PC machine 10, in order to provide
Preferably compatibility, and guarantee the transmission speed of data, the vehicle body CAN bus 31 and privately owned CAN bus 32 can pass through number
Turn Ethernet equipment 40 according to converter 40 namely CAN to be attached with PC machine 10, the data of each CAN bus are utilized into number
It is unified after summarizing according to converter 40 that PC machine 10 is transferred to by Ethernet.Specifically, the data converter 40, is used for vehicle body
The data that CAN bus 31 and privately owned CAN bus 32 are sent are converged, and by the data after converging from meeting CAN protocol
Data conversion is at the data for meeting Ethernet protocol；Or by the data of host 10 from meet the data conversion of Ethernet protocol at
Meet the data of CAN protocol, and is sent to corresponding vehicle body CAN bus 31 or privately owned CAN bus 32.
The data of each CAN bus can realize data summarization by data converter 40, it is unified using CAN protocol to its into
Row time label is solved the problems, such as because using time label caused by a plurality of CAN bus transmission data skimble-scamble, and the number
Networking can be carried out by Ethernet according to converter 40, without having to worry about the problem of CAN port deficiency, expansibility is strong.
The Ethernet can have 100Mbit/s or more than message transmission rate, it is ensured that the number of a plurality of CAN bus
Reduced levels are maintained according to the delay of transmission process, and Ethernet is communicated by agreements such as TCP, UDP with PC machine 10, it can be with
It is connect by Network Cable Connector with PC machine 10, so that 10 need not have special interface of PC machine can carry out data communication with CAN bus,
Compatibility is higher.
From the foregoing, it will be observed that the implementation of the multi-sensor data synchronization system makes data transmission more reliable, delay is low,
Compatible different host, and a variety of data are carried out to summarize unification using data adapter unit, it can solve in the prior art
Time caused by data source disunity accessed by host marks skimble-scamble problem.
Referring to Fig. 3, showing the implementation process of image data synchronous method provided by the embodiments of the present application in figure.
Wherein, the image data synchronous method can be applied to the more of any one embodiment offer as described in Fig. 1-2
Third class sensor in sensing data synchronization system, the third class sensor are image acquiring device.In order to avoid repeating,
The specific descriptions of the multi-sensor data synchronization system may refer to above-described embodiment.
The image data synchronous method, comprising:
101, frame image is obtained, frame identification corresponding with frame image is obtained according to frame image.
Wherein, which can use the frame image that camera module obtains single frames, which is pre-
If image acquired in the moment, multiple continuous times, frame image obtained can form image.The frame identification and frame image it
Between uniquely correspond to.
In some embodiments, which can be the time letter at current time corresponding when obtaining the frame image
Breath, or frame number is formed by based on the temporal information, which indicates the frame image in the shadow currently recorded
Frame sequential as in.
In further embodiments, which can also be the sequence of the acquisition based on image.For example, when starting camera
Afterwards, the frame identification of first frame image obtained is 1, and the frame identification of the second frame image obtained is 2, and so on.
It should be understood that the frame identification specifically set up mode can according to the actual situation depending on, the application is not construed as limiting.
102, frame signal is generated according to frame identification, the frame signal carries frame identification.
Wherein, frame signal is the signal for meeting CAN transport protocol, and converting frame identification to can be in vehicle body after frame signal
It is transmitted in CAN bus.
The information of frame identification is carried in the signal, so that host can know frame identification therein according to the frame signal
Information.
103, frame signal is exported to vehicle body CAN bus, to send frame signal to host by vehicle body CAN bus,
And other data in frame signal and vehicle body CAN bus is made to realize time synchronization.
Wherein, which can be set CAN interface, and be connected using the CAN interface and vehicle body CAN bus
It connects.
In some embodiments, after the image acquiring device gets frame image, image acquiring device can simultaneously by
Frame signal corresponding with the frame image is sent in vehicle body CAN bus, and utilizes vehicle body CAN bus and turn of data converter
It changes and is sent on host.
104, frame image is sent to host by data line.
Wherein, frame image is sent to host using data line, may not need the resource for occupying vehicle body CAN bus, improved
Transmission rate reduces the data transfer delay of vehicle body CAN bus.
If frame image is sent by CAN bus, it can cause and produced between other data because the data volume of image is excessive
Raw data congestion；And if only frame image is transmitted using data line, will appear frame image and other data generation times
Mark skimble-scamble problem.
Therefore, it when which is transferred to host using data line for the frame image, while being sent to CAN bus
Frame signal, because in CAN bus further including the data of other sensors, which can be obtained using CAN protocol and it
The opposite time label of his data, so that acquisition system is completed to mark the time between the frame image and other sensors data
Unification.
From the foregoing, it will be observed that frame signal is sent when passing through image acquiring device getting frame image, while to vehicle body CAN bus,
Data generated instantaneity for postponing, improving data in transmission process can not only be greatly reduced, frame image can also be made
Time label and the time labels of other data of vehicle body CAN bus realize unified, researcher's subsequent execution is reduced with this
The difficulty of data analysis.
Referring to Fig. 4, showing the implementation flow chart provided by the embodiments of the present application for obtaining frame identification in figure.
In some embodiments, this obtains frame identification corresponding with frame image according to frame image, may include:
201, frame number corresponding with frame image is obtained according to frame image, the acquisition time of the frame number and frame image is successively suitable
Sequence is related.
In some embodiments, frame number can be number, letter either other parameters comprising ordinal relation.
Specifically, the frame number can be related to the chronological order of frame image is obtained.
202, corresponding frame identification is obtained based on frame number.
It in some embodiments, can directly be using frame number as frame identification, it can also be by mapping pass accordingly
Be feature based on the frame number to obtain frame identification, concrete implementation mode can according to the actual situation depending on.
Using the implementation of above-mentioned acquisition frame identification, the acquisition time sequencing based on frame image can be passed through
Frame number is ranked up frame image, it is possible to reduce researcher reduces and analyzes to the workload of frame image analysis process
Wrong chance.
Referring to Fig. 5, showing the implementation flow chart provided by the embodiments of the present application for saving frame identification information in figure.
In some embodiments, frame image and the frame identification are associated by this, the frame image after being associated with, can be with
Include:
It will be in the predeterminable area of the information preservation of frame identification to frame image.
Wherein, which can be certain pixels in the predeterminated position of frame image.The information of the frame identification can be with
It is the sequence information of frame number, or the mark information for distinguishing different frame.
For example, can show a serial number corresponding with the frame image in the lower right corner of frame image, or frame identification
Information preservation into certain pixels of the frame image.
In some embodiments, which may include: into the predeterminable area of frame image
301, target color values corresponding with the information of frame identification are obtained.
Wherein, each color value is determining parameter, which can be read by host system.
After getting the information of frame identification, it can be passed through according to the serial number either label in relation to sequence of frame identification
Preset mapping table chooses from preservable color value to corresponding target color values.
For example, can then obtain the face of nth frame by mapping table when the serial number of frame identification shows that the frame image is nth frame
Color value M is as target color values.
302, the object pixel in the predeterminable area of frame image is obtained.
The object pixel can be some or certain several pixels of predeterminated position in frame image, which can pass through
Default means are set in advance.
303, the color value of object pixel is switched to target color values.
In frame image, each pixel has corresponding color value to be corresponding to it, by the way that the color value to be changed to and frame mark
After the corresponding target color values of the information of knowledge, host can be determined and the frame figure by reading the color value on the object pixel
As corresponding frame identification.
After getting the frame identification, researcher can be extracted received by host, therewith based on the frame identification
Corresponding frame signal to its time label is associated, analyzes.
Obviously, by being saved in mesh by the corresponding target color values of the information acquisition of frame identification, and by the target color values
It marks in pixel, under the premise of the image information that can be saved in not influencing frame image as far as possible, to the frame mark of the frame image
Knowledge is saved, is associated with, so that researcher when analyzing the frame image, can quickly determine the frame figure
Generated time label, improves the analysis efficiency of data when as obtaining.
Referring to Fig. 6, showing the implementation process of disassociation frame image and timing node provided by the embodiments of the present application in figure.
In this embodiment, the image data synchronous method, further includes:
401, frame signal is obtained in the timing node of the data flow of vehicle body CAN bus.
Wherein, the equipment for receiving the data flow transmitted in the CAN bus can carry out the timing node of wherein data flow
Label, to form the time label of data.
In some embodiments, after obtaining frame signal, can by the frame signal vehicle body CAN bus data flow when
Intermediate node extracts, with by the time of the timing node mark determine the frame signal relative to other data it is opposite when
Between, which can show the precedence relationship between the frame signal and other data in transmission process.
402, target frame image associated by frame identification entrained by frame signal is obtained.
In some embodiments, after obtaining frame signal, entrained frame can be extracted from the parameter of the frame signal
Mark.Because frame signal and associated frame image are almost to send simultaneously to host, the frame identification of the frame signal can with connect
The a certain frame image of the multiple frame images received uniquely corresponds to, and the target frame figure being associated can be obtained based on the frame identification
Picture.
403, target frame image timing node corresponding with frame signal is associated.
It should be understood that target frame image timing node corresponding with frame signal is associated, can to study people
Member quickly determines that the frame image relative to the precedence relationship between host other data obtained, greatly facilitates researcher
Data are researched and analysed.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of above-described embodiment is can
It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storage
Medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random
Access Memory), disk or CD etc..
In the embodiment of the present application, one of the lane state alert device and foregoing embodiments lane shape
State based reminding method belongs to same design, and the lane state can be run in the lane state alert device and is mentioned
Either offer method step in embodiment of the method for waking up, specific implementation process are detailed in the implementation of lane state based reminding method
Example, and the alternative embodiment of the application can be formed using any combination, details are not described herein again.
Presently filed embodiment is explained in detail above in conjunction with attached drawing, but the application is not limited to above-mentioned implementation
Mode within the knowledge of a person skilled in the art can also be under the premise of not departing from the application objective
Various changes can be made.
Claims (13)
1. a kind of multi-sensor data synchronization system characterized by comprising
Host, the image acquiring device being connect respectively with the host and multiple sensors；
The host, the data inputted for acquiring multiple sensors and described image acquisition device；
The multiple sensor, including first kind sensor, the second class sensor and third class sensor；
The first kind sensor is connect with vehicle body CAN bus, and the second class sensor is connect with privately owned CAN bus, described
Third class sensor includes signal end and data terminal, and the signal end is connect with the vehicle body CAN bus, is believed for output frame
Number, the data terminal is connect by data line with the host, Xiang Suoshu host output data；It is the vehicle body CAN bus, privately owned
CAN bus passes through same data converter respectively and connect with the host, so that the data for being input to the host are synchronous；
Wherein, the first kind sensor, the second class sensor and third class sensor are generated flat in section at work
Equal data volume is respectively the first data volume, the second data volume and third data volume, first data volume, the second data volume with
And third data volume is incremented by successively.
2. multi-sensor data synchronization system as described in claim 1, which is characterized in that the data converter, being used for will
The data that the vehicle body CAN bus and privately owned CAN bus are sent are converged, and the data after converging are assisted from CAN is met
The data conversion of view is at the data for meeting Ethernet protocol；By the data of host from the data conversion of Ethernet protocol is met at full
The data of sufficient CAN protocol, and sent to corresponding vehicle body CAN bus or privately owned CAN bus.
3. multi-sensor data synchronization system as described in claim 1, which is characterized in that the third class sensor is image
Acquisition device, described image acquisition device include the camera module and signal processing mould group that can be connected with each other communication；
The camera module is connect with the data terminal, and the signal processing mould group is total by signal end and the vehicle body CAN
Line connection, for generating and exporting the frame signal to the vehicle body CAN bus by the signal end.
4. multi-sensor data synchronization system as claimed in claim 3, which is characterized in that the data line is equipped with adapter,
The adapter between the interface of the data terminal and the interface of the host for transferring.
5. multi-sensor data synchronization system as claimed in claim 4, which is characterized in that the adapter is used for HDMI interface
Switching between USB interface；
The adapter HDMI interface is connect with the output end of the camera module, and the USB interface is connect with the host.
6. multi-sensor data synchronization system as described in claim 1, which is characterized in that the first kind sensor includes vehicle
The one of which of fast sensor, steering wheel angle sensor, turn signal sensor, accelerator open degree sensor and brake sensor
Or multiple combinations.
7. multi-sensor data synchronization system as described in claim 1, which is characterized in that the second class sensor includes milli
Metre wave radar.
8. a kind of image data synchronous method, synchronous applied to multi-sensor data described in claim 1-7 any one to be
System, which comprises
Frame image is obtained, frame identification corresponding with the frame image is obtained according to the frame image；
Frame signal is generated according to the frame identification, the frame signal carries the frame identification；
The frame signal is exported to vehicle body CAN bus, to send the frame signal to master by the vehicle body CAN bus
Machine, and other data in the frame signal and the vehicle body CAN bus is made to realize time synchronization；
The frame image is sent to host by data line.
9. image data synchronous method as claimed in claim 8, which is characterized in that described according to frame image acquisition and institute
State the corresponding frame identification of frame image, comprising:
Frame number corresponding with the frame image, the acquisition time elder generation of the frame number and frame image are obtained according to the frame image
It is order dependent afterwards；
Corresponding frame identification is obtained based on the frame number.
10. image data synchronous method as claimed in claim 8, which is characterized in that described that the frame image is passed through data
Line is sent to host, comprising:
Frame image and the frame identification are associated, the frame image after being associated with；
Frame image after the association is sent to host by data line.
11. image data synchronous method as claimed in claim 10, which is characterized in that described by frame image and the frame identification
It is associated, the frame image after being associated with, comprising:
It will be in the predeterminable area of the information preservation of the frame identification to the frame image.
12. image data synchronous method as claimed in claim 11, which is characterized in that the information by the frame identification is protected
It is stored in the predeterminable area of the frame image, comprising:
Obtain target color values corresponding with the information of the frame identification；
Obtain the object pixel in the predeterminable area of the frame image；
The color value of the object pixel is switched to the target color values.
13. image data synchronous method as claimed in claim 10, which is characterized in that the method also includes:
The frame signal is obtained in the timing node of the data flow of the vehicle body CAN bus；
Obtain target frame image associated by frame identification entrained by the frame signal；
Target frame image timing node corresponding with the frame signal is associated.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811437890.7A CN109547693A (en) | 2018-11-29 | 2018-11-29 | Multi-sensor data synchronization system and image data synchronous method |
| PCT/CN2018/120210 WO2020107527A1 (en) | 2018-11-29 | 2018-12-11 | Multi-sensor data synchronization system and image data synchronization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811437890.7A CN109547693A (en) | 2018-11-29 | 2018-11-29 | Multi-sensor data synchronization system and image data synchronous method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109547693A true CN109547693A (en) | 2019-03-29 |
Family
ID=65850944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811437890.7A Pending CN109547693A (en) | 2018-11-29 | 2018-11-29 | Multi-sensor data synchronization system and image data synchronous method |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109547693A (en) |
| WO (1) | WO2020107527A1 (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111193568A (en) * | 2019-12-19 | 2020-05-22 | 北汽福田汽车股份有限公司 | Time synchronization method, device, system, storage medium and vehicle |
| CN111554004A (en) * | 2020-04-26 | 2020-08-18 | 东风汽车集团有限公司 | Vehicle-mounted data acquisition equipment supporting bus data and video synchronization |
| CN111897330A (en) * | 2020-07-28 | 2020-11-06 | 惠州市德赛西威智能交通技术研究院有限公司 | Multi-sensor information processing method in different time domains |
| CN111985772A (en) * | 2020-07-15 | 2020-11-24 | 惠州市德赛西威智能交通技术研究院有限公司 | Method for realizing robustness and integrity of evaluation standard protocol |
| CN112203030A (en) * | 2019-07-08 | 2021-01-08 | 联詠科技股份有限公司 | Receiving device, image recording system and method for reducing image delay of image recording system |
| CN113132042A (en) * | 2020-01-15 | 2021-07-16 | 宏碁股份有限公司 | Time synchronization method and time synchronization system |
| CN114095110A (en) * | 2022-01-24 | 2022-02-25 | 天津安力信通讯科技有限公司 | Method and system for improving synchronism of spectrum data transmission |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003264567A (en) * | 2002-03-11 | 2003-09-19 | Toyota Motor Corp | Can communication method and system |
| CN101841446A (en) * | 2010-05-25 | 2010-09-22 | 许继集团有限公司 | Pilot channel transmission device, optical fiber differential protective device and method |
| CN102652288A (en) * | 2009-12-08 | 2012-08-29 | (株)阿尔迪泰 | System and method for synchronizing video and sensor signals |
| CN103684944A (en) * | 2012-09-10 | 2014-03-26 | 西门子信号有限公司 | Embedded gateway, railway monitoring system adopting gateway, and railway monitoring method |
| CN106292626A (en) * | 2016-08-04 | 2017-01-04 | 中国汽车工程研究院股份有限公司 | Data acquisition unit and method for new-energy automobile test evaluation |
| CN106443607A (en) * | 2016-10-25 | 2017-02-22 | 华域汽车系统股份有限公司 | Vehicular radar data processing algorithm verification system |
| CN206114004U (en) * | 2016-10-10 | 2017-04-19 | 交通运输部公路科学研究所 | Synchronous sampling and device of handling vehicle operation information and operating information |
| CN107743054A (en) * | 2017-08-25 | 2018-02-27 | 杭州德泽机器人科技有限公司 | System during a kind of synchronous pair of multisensor |
| CN108241590A (en) * | 2016-12-27 | 2018-07-03 | 中国船舶重工集团公司七五○试验场 | A kind of serial data bus applied to submarine navigation device |
| CN211656247U (en) * | 2018-11-29 | 2020-10-09 | 惠州市德赛西威智能交通技术研究院有限公司 | Multi-sensor data synchronization system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101067768B1 (en) * | 2009-12-04 | 2011-09-28 | (주) 알디텍 | Method and system for synchronizing video signal and sensor signal |
| CN103780697B (en) * | 2014-01-23 | 2017-07-21 | 广州睿嵌电子技术有限公司 | Vehicle electronics processing unit common platform system and its data communication method |
| CN103995266A (en) * | 2014-06-05 | 2014-08-20 | 南通鸿鹄信息技术有限公司 | Data collecting system based on CAN buses |
| CN107659367B (en) * | 2017-08-30 | 2018-12-21 | 安徽江淮汽车集团股份有限公司 | More sensing unit method for synchronizing time and system |
-
2018
- 2018-11-29 CN CN201811437890.7A patent/CN109547693A/en active Pending
- 2018-12-11 WO PCT/CN2018/120210 patent/WO2020107527A1/en active Application Filing
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003264567A (en) * | 2002-03-11 | 2003-09-19 | Toyota Motor Corp | Can communication method and system |
| CN102652288A (en) * | 2009-12-08 | 2012-08-29 | (株)阿尔迪泰 | System and method for synchronizing video and sensor signals |
| CN101841446A (en) * | 2010-05-25 | 2010-09-22 | 许继集团有限公司 | Pilot channel transmission device, optical fiber differential protective device and method |
| CN103684944A (en) * | 2012-09-10 | 2014-03-26 | 西门子信号有限公司 | Embedded gateway, railway monitoring system adopting gateway, and railway monitoring method |
| CN106292626A (en) * | 2016-08-04 | 2017-01-04 | 中国汽车工程研究院股份有限公司 | Data acquisition unit and method for new-energy automobile test evaluation |
| CN206114004U (en) * | 2016-10-10 | 2017-04-19 | 交通运输部公路科学研究所 | Synchronous sampling and device of handling vehicle operation information and operating information |
| CN106443607A (en) * | 2016-10-25 | 2017-02-22 | 华域汽车系统股份有限公司 | Vehicular radar data processing algorithm verification system |
| CN108241590A (en) * | 2016-12-27 | 2018-07-03 | 中国船舶重工集团公司七五○试验场 | A kind of serial data bus applied to submarine navigation device |
| CN107743054A (en) * | 2017-08-25 | 2018-02-27 | 杭州德泽机器人科技有限公司 | System during a kind of synchronous pair of multisensor |
| CN211656247U (en) * | 2018-11-29 | 2020-10-09 | 惠州市德赛西威智能交通技术研究院有限公司 | Multi-sensor data synchronization system |
Non-Patent Citations (2)
| Title |
|---|
| 张昊;崔永俊;沈三民;叶勇;: "基于CAN总线的传感器网络设计", 计算机测量与控制, no. 11, 25 November 2013 (2013-11-25) * |
| 王俊华;何乐;赵银;夏群生;: "基于CAN总线的车载测量系统设计与实现", 汽车技术, no. 12, 24 December 2009 (2009-12-24) * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112203030A (en) * | 2019-07-08 | 2021-01-08 | 联詠科技股份有限公司 | Receiving device, image recording system and method for reducing image delay of image recording system |
| CN112203030B (en) * | 2019-07-08 | 2022-05-03 | 联詠科技股份有限公司 | Receiving device, image recording system and method for reducing image delay of image recording system |
| CN111193568A (en) * | 2019-12-19 | 2020-05-22 | 北汽福田汽车股份有限公司 | Time synchronization method, device, system, storage medium and vehicle |
| CN113132042A (en) * | 2020-01-15 | 2021-07-16 | 宏碁股份有限公司 | Time synchronization method and time synchronization system |
| CN111554004A (en) * | 2020-04-26 | 2020-08-18 | 东风汽车集团有限公司 | Vehicle-mounted data acquisition equipment supporting bus data and video synchronization |
| CN111554004B (en) * | 2020-04-26 | 2021-11-12 | 东风汽车集团有限公司 | Vehicle-mounted data acquisition equipment supporting bus data and video synchronization |
| CN111985772A (en) * | 2020-07-15 | 2020-11-24 | 惠州市德赛西威智能交通技术研究院有限公司 | Method for realizing robustness and integrity of evaluation standard protocol |
| CN111985772B (en) * | 2020-07-15 | 2024-02-20 | 惠州市德赛西威智能交通技术研究院有限公司 | Method for realizing robustness and integrity of evaluation standard protocol |
| CN111897330A (en) * | 2020-07-28 | 2020-11-06 | 惠州市德赛西威智能交通技术研究院有限公司 | Multi-sensor information processing method in different time domains |
| CN111897330B (en) * | 2020-07-28 | 2023-05-26 | 惠州市德赛西威智能交通技术研究院有限公司 | Different time domain multi-sensor information processing method |
| CN114095110A (en) * | 2022-01-24 | 2022-02-25 | 天津安力信通讯科技有限公司 | Method and system for improving synchronism of spectrum data transmission |
| CN114095110B (en) * | 2022-01-24 | 2022-04-08 | 天津安力信通讯科技有限公司 | Method and system for improving synchronism of spectrum data transmission |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020107527A1 (en) | 2020-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109547693A (en) | Multi-sensor data synchronization system and image data synchronous method | |
| US10565458B2 (en) | Simulation system, simulation program and simulation method | |
| CN105912980B (en) | Unmanned plane and UAV system | |
| WO2018196655A1 (en) | Public network communication method for on-board electronic processing unit system | |
| CN113479218B (en) | Roadbed automatic driving auxiliary detection system and control method thereof | |
| Basil et al. | IoT based traffic light control system using Raspberry Pi | |
| CN112633120B (en) | Model training method of intelligent roadside sensing system based on semi-supervised learning | |
| CN113281738A (en) | Multi-sensor fusion target detection system and detection method | |
| CN112810553A (en) | Data visualization system and method based on self-adaptive platform automobile open system architecture | |
| CN110136186A (en) | A kind of detection target matching method for mobile robot object ranging | |
| CN205890794U (en) | On -vehicle network terminal platform of distributing type electric automobile | |
| CN104615528B (en) | A kind of intelligent mobile phone sensor online data collection and real-time processing method | |
| CN211656247U (en) | Multi-sensor data synchronization system | |
| CN113232663B (en) | Control system applied to advanced assistant driving | |
| CN108919217A (en) | A kind of Processing Method of Point-clouds, device, controller and radar sensor | |
| CN114185059A (en) | Multi-radar fusion-based multi-person tracking system, method, medium and terminal | |
| CN110211256A (en) | Appearing method is answered in aobvious device and vehicle driving fact again for vehicle and its vehicle driving fact | |
| CN108052113A (en) | A kind of multiple no-manned plane and more intelligent vehicle composite formations communication means | |
| EP3994485A1 (en) | Time-of-flight sensing circuitry with different imaging modes and method for operating such a time-of-flight sensing circuitry | |
| WO2020142200A1 (en) | Multi-vehicle data logging device | |
| CN107809359B (en) | Method for supporting plug and play device of automobile system based on local area network UDP protocol | |
| CN112291131B (en) | CAN signal automatic analysis device and equipment | |
| CN219181531U (en) | Data synchronization device and system | |
| CN116594659B (en) | Vehicle-mounted domain controller program optimization system, method and device and vehicle | |
| CN109241815A (en) | Detection method, device and the robot of user behavior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |