CN111372111A - Data determination method and device and vehicle-mounted terminal - Google Patents
Data determination method and device and vehicle-mounted terminal Download PDFInfo
- Publication number
- CN111372111A CN111372111A CN201811594509.8A CN201811594509A CN111372111A CN 111372111 A CN111372111 A CN 111372111A CN 201811594509 A CN201811594509 A CN 201811594509A CN 111372111 A CN111372111 A CN 111372111A
- Authority
- CN
- China
- Prior art keywords
- image
- image acquisition
- processor
- main controller
- instruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the invention discloses a data determination method, a data determination device and a vehicle-mounted terminal. The main controller is connected with at least two image acquisition devices respectively, each image acquisition device is connected with a corresponding processor, and the main controller is connected with each processor. The method comprises the following steps: the main controller acquires an image acquired by each image acquisition device in response to a first image acquisition instruction, acquires a timestamp of the image sent by each processor, and associates the image with the timestamp according to a preset corresponding relationship between the image acquisition device and the processor; the first image acquisition instruction is used for instructing each image acquisition device to acquire an image at the associated moment; the time stamp is: the processor determines when it is determined that the image acquisition device acquired an image. By applying the technical scheme of the embodiment of the invention, the time synchronism among the images acquired by different cameras can be improved.
Description
Technical Field
The invention relates to the technical field of automatic driving, in particular to a data determination method and device and a vehicle-mounted terminal.
Background
In the field of machine vision, a machine may need a plurality of cameras to simultaneously acquire images at different angles, and the images acquired at the same time are processed to form complete machine vision. The machine may be a vehicle or other smart device, etc.
In order to enable a plurality of cameras on the machine to acquire a plurality of images at the same time, the main controller may simultaneously send acquisition instructions to the plurality of cameras on the machine, each camera acquiring an image upon receiving the acquisition instructions and sending the image to the main controller. When the main controller receives the images sent by each camera, the time stamp of the images is determined (namely, the images are time service). This makes it possible to obtain a plurality of images of a plurality of cameras at the same time.
The master controller, when determining the time stamp of an image, typically determines the time stamp of the image based on the time the image was received. However, the time delay from the image capture by the cameras to the transmission of the image to the main controller is large, and there is a large uncertainty, which may cause a time difference between the timestamps of the images simultaneously captured by the cameras, and thus a high time synchronism cannot be ensured.
Disclosure of Invention
The embodiment of the invention provides a data synchronization method, a data synchronization device and a vehicle-mounted terminal, which are used for improving the time synchronism among images acquired by different cameras. The specific technical scheme is as follows.
In a first aspect, an embodiment of the present invention provides a data synchronization method, where the method is applied to a main controller, where the main controller is respectively connected to at least two image acquisition devices, each image acquisition device is connected to a corresponding processor, and the main controller is connected to each processor; the method comprises the following steps:
acquiring an image acquired by each image acquisition device in response to a first image acquisition instruction; wherein the first image acquisition instruction is used for instructing each image acquisition device to acquire an image at an associated moment;
acquiring a time stamp of the image sent by each processor; wherein the time stamp is: the processor determines when it is determined that the image acquisition device acquired the image;
and associating the image with the timestamp according to a preset corresponding relation between the image acquisition equipment and the processor.
Optionally, the first image capturing instruction is sent to each image capturing device in the following manner:
and the main controller respectively sends second image acquisition instructions to each processor, so that each processor sends first image acquisition instructions to the image acquisition equipment corresponding to the processor according to the second image acquisition instructions.
Optionally, the step of sending the second image capturing instruction to each processor includes:
and respectively sending a second image acquisition instruction to each processor through a hardware trigger.
Optionally, the method further includes:
and respectively sending standard clock pulses to each processor, so that the processor adjusts the time of the processor according to the matching result of the time interval between the pulses of the standard clock pulses and the time of the processor.
In a second aspect, an embodiment of the present invention provides a data synchronization method, where the method is applied to a processor, where the processor is respectively connected to a main controller and an image acquisition device corresponding to the processor, the image acquisition device is connected to the main controller, and the main controller is connected to at least two image acquisition devices; the method comprises the following steps:
when the image acquisition equipment is determined to acquire an image according to a first image acquisition instruction, determining a time stamp of the image acquired by the image acquisition equipment; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
and sending the time stamp to the main controller so that the main controller associates the image with the time stamp according to a preset corresponding relation between the image acquisition equipment and the processor.
Optionally, the step of determining that the image capturing device captures an image according to the first image capturing instruction includes:
when a notification message sent by the image acquisition equipment is received, determining that the image acquisition equipment acquires an image according to a first image acquisition instruction; wherein the notification message is: and the image acquisition equipment sends the image acquisition instruction when acquiring the image according to the first image acquisition instruction.
Optionally, when determining the timestamp of the image acquired by the image acquisition device, the method includes:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure duration of the image acquisition equipment.
Optionally, the method further includes:
receiving a standard clock pulse sent by the main controller;
matching a time interval between pulses of the standard clock pulse to a time of the processor; and adjusting the time of the processor according to the matching result.
In a third aspect, an embodiment of the present invention provides a data synchronization apparatus, where the apparatus is applied to a main controller, where the main controller is respectively connected to at least two image acquisition devices, each image acquisition device is connected to a corresponding processor, and the main controller is connected to each processor; the device comprises:
a first acquisition module configured to acquire an image acquired by each of the image acquisition devices in response to a first image acquisition instruction; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
a second obtaining module configured to obtain a timestamp of the image sent by each of the processors; wherein the time stamp is: the processor determines when it is determined that the image acquisition device acquired the image;
and the association module is configured to associate the image with the timestamp according to a preset corresponding relation between the image acquisition equipment and the processor.
Optionally, the apparatus further comprises: a first sending module;
the first sending module is configured to send second image acquisition instructions to the processors respectively, so that each processor sends a first image acquisition instruction to the image acquisition device corresponding to the processor according to the second image acquisition instructions.
Optionally, the first sending module is specifically configured to:
and respectively sending a second image acquisition instruction to each processor through a hardware trigger.
Optionally, the apparatus further comprises: a second sending module;
the second sending module is configured to send standard clock pulses to each processor respectively, so that the processor adjusts the time of the processor according to a matching result of a time interval between pulses of the standard clock pulses and the time of the processor.
In a fourth aspect, an embodiment of the present invention provides a data synchronization apparatus, where the apparatus is applied to a processor, the processor is respectively connected to a main controller and an image acquisition device corresponding to the processor, the image acquisition device is connected to the main controller, and the main controller is connected to at least two image acquisition devices; the device comprises:
the first determination module is configured to determine a time stamp of the image acquired by the image acquisition device when the image acquisition device is determined to acquire the image according to the first image acquisition instruction; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
and the third sending module is configured to send the timestamp to the main controller, so that the main controller associates the image with the timestamp according to a preset corresponding relation between the image acquisition device and the processor.
Optionally, the apparatus further comprises: a second determination module;
the second determination module is configured to determine that the image acquisition device acquires an image according to the first image acquisition instruction;
the second determining module is specifically configured to determine that the image acquisition device acquires an image according to a first image acquisition instruction when receiving a notification message sent by the image acquisition device; wherein the notification message is: and the image acquisition equipment sends the image acquisition instruction when acquiring the image according to the first image acquisition instruction.
Optionally, the first determining module is specifically configured to:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure duration of the image acquisition equipment.
Optionally, the apparatus further comprises: an adjustment module;
the adjusting module is configured to receive a standard clock pulse sent by the main controller; matching a time interval between pulses of the standard clock pulse to a time of the processor; and adjusting the time of the processor according to the matching result.
In a fifth aspect, an embodiment of the present invention provides a vehicle-mounted terminal, where the vehicle-mounted terminal includes: the system comprises a main controller, at least two image acquisition devices and a processor connected with each image acquisition device; the main controller is connected with each processor;
the main controller sends a first image acquisition instruction to each image acquisition device; the first image acquisition instruction is used for instructing each image acquisition device to acquire an image at an associated moment;
the image acquisition equipment acquires an image when receiving a first image acquisition instruction sent by the main controller, sends a notification message for identifying the image acquisition equipment acquires the image to the processor, and sends the acquired image to the main controller;
the processor determines a time stamp of an image acquired by the image acquisition equipment when receiving a notification message sent by the image acquisition equipment, and sends the time stamp to the main controller;
the main controller receives the image sent by the image acquisition equipment, receives the timestamp of the image sent by the processor, and associates the image with the timestamp according to a preset corresponding relation between the image acquisition equipment and the processor.
Optionally, the main controller further sends a second image acquisition instruction to each processor;
the processor also receives a second image acquisition instruction sent by the main controller, and sends the first image acquisition instruction to the image acquisition equipment according to the second image acquisition instruction;
the image acquisition device specifically receives a first image acquisition instruction sent by the processor.
Optionally, when the main controller sends the second image capturing instruction to each processor, the method specifically includes:
and respectively sending a second image acquisition instruction to each processor through a hardware trigger.
Optionally, the main controller further sends standard clock pulses to each processor;
the processor also receives the standard clock pulse sent by the main controller, matches the time interval between the pulses of the standard clock pulse with the time of the processor, and adjusts the time of the processor according to the matching result.
Optionally, when determining the timestamp of the image captured by the image capturing device, the processor includes:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure duration of the image acquisition equipment.
As can be seen from the above, in the data synchronization method, the data synchronization device, and the vehicle-mounted terminal provided in the embodiments of the present invention, the processor may determine the timestamp of the image acquired by the image acquisition device, and send the timestamp to the main controller, and the main controller associates the image and the timestamp according to the correspondence between the image acquisition device and the processor. According to the embodiment of the invention, each image acquisition device acquires images according to the triggering of the same image acquisition instruction, so that the time of the acquired images has relevance, the images and the time stamps are processed separately, the time stamps of the images are determined by the processor connected with each image acquisition device, the time stamps of the images can be determined more quickly and timely, and compared with the method that the time stamps are determined by the main controller after the images are received, the accuracy of the time stamps can be improved, and further the time synchronism among the images acquired by different cameras can be improved.
The innovation points of the embodiment of the invention comprise:
1. every image acquisition equipment increases the connection treater, and the treater is connected with main control unit, and image acquisition equipment's image is sent to main control unit, and image acquisition equipment notifies the treater to come the time stamp of confirming the image, and finally main control unit is in the same place image and time stamp association. Therefore, when the image acquisition devices acquire images simultaneously, the processor determines the time stamps more quickly and accurately, and the obtained time stamps of the images are higher in synchronism.
2. The main controller sends image acquisition instructions to each processor through the hardware trigger in a unified mode, and each processor sends the image acquisition instructions to the image acquisition equipment to achieve unified triggering of images acquired by each image acquisition equipment. The triggering process can be faster by adopting a hardware trigger triggering mode, and each image acquisition device is ensured to be triggered to acquire images simultaneously as far as possible.
3. The main controller sends high-precision clock pulses to each processor respectively, so that the clock of each processor can be synchronized with the main controller, and the synchronism of the image time stamps can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data synchronization method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of another data synchronization method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of information interaction between the main controller, the image capturing device, and the processor according to this embodiment;
fig. 4 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of another data synchronization apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a vehicle-mounted terminal according to an embodiment of the present invention;
fig. 7 is another structural schematic diagram of the vehicle-mounted terminal in fig. 6.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a data synchronization method, a data synchronization device and a vehicle-mounted terminal, which can improve the time synchronism among images acquired by different cameras. The following provides a detailed description of embodiments of the invention.
Fig. 1 is a schematic flow chart of a data synchronization method according to an embodiment of the present invention. The method is applied to the main controller. The main controller is respectively connected with at least two image acquisition devices, each image acquisition device is connected with a corresponding processor, and the main controller is connected with each processor. The image acquisition device may be a camera or a device with camera functionality. The main controller and the processor may be both a CPU or a Micro Controller Unit (MCU). The schematic diagram of the structural connection between the main controller and the image acquisition device and the processor can be referred to fig. 6.
The method of this embodiment specifically includes the following steps.
S110: an image acquired by each image acquisition device in response to the first image acquisition instruction is acquired.
Wherein the first image acquisition instruction is used for instructing each image acquisition device to acquire an image at the associated moment. The associated time instants may be understood as the same time instant.
The first image capturing instruction may be transmitted to each image capturing apparatus by the other apparatus. Specifically, the other devices may send a first image capturing instruction to each image capturing device at the same time, and the image capturing device starts capturing an image when receiving the first image capturing instruction.
After the image acquisition equipment acquires the image, the image is sent to the main controller, and the main controller can receive the image acquired by the image acquisition equipment.
S120: a time stamp of the image sent by each processor is obtained.
Wherein, the timestamp is: the processor determines when the image acquisition device acquires the image. The time stamp of an image can be understood as the acquisition time information of the image.
And starting to acquire the image when the image acquisition equipment receives the first image acquisition instruction. When the image capture device captures an image, a notification message may be sent to the processor identifying that the image capture device is beginning to capture the image. And when receiving the notification message sent by the image acquisition equipment, the processor determines the time stamp of the image acquired by the image acquisition equipment and sends the time stamp to the main controller. The master controller may receive a timestamp for the image sent by each processor.
S130: and associating the image with the time stamp according to a preset corresponding relation between the image acquisition equipment and the processor.
The correspondence between the image capturing device and the processor may specifically be a correspondence between an identifier of the image capturing device and an identifier of the processor. The image acquisition device is connected with the main control through the processor, and the processor sends the determined time stamp of the image acquisition device to the processor.
When this step is implemented, see the following examples. The corresponding relation between the image acquisition equipment and the processor is as follows: the camera 1 corresponds to the processor 1, and the camera 2 corresponds to the processor 2. When the main controller receives the images a and b transmitted by the cameras 1 and 2, respectively, and the time stamps t1 and t2 transmitted by the processors 1 and 2, respectively, the images a and b can be associated with t1 and t2 according to the above correspondence.
When the image and the time stamp are associated with each other, the time stamp may be added to a designated position of the image to obtain image data including the time stamp.
As can be seen from the above, in this embodiment, the processor may determine the timestamp of the image captured by the image capturing device, and send the timestamp to the main controller, and the main controller associates the image and the timestamp according to the correspondence between the image capturing device and the processor. Each image acquisition equipment in this embodiment triggers and gathers the image according to same image acquisition instruction for the moment of gathering the image has the relevance, and separately handle image and timestamp, confirm the timestamp of image through the treater of being connected with every image acquisition equipment, can confirm the timestamp of image more fast, in time, compare in by main control unit confirm the timestamp again after receiving the image, can improve the accuracy of timestamp, and then can improve the time synchronism between the image of different camera collections.
In another embodiment of the present invention, there may be a one-to-one correspondence between the image capturing devices and the processors, for example, see the schematic structural diagram of fig. 6; or not one-to-one, for example, two (or more) image capture devices are connected to the same processor. The processor may determine a timestamp for the image captured by each image capture device, and send the timestamp to the master controller. The main controller can associate the image with the timestamp according to the corresponding relation between the timestamp identification sent by the processor and the image acquisition equipment. The scheme can reduce the number of processors and save hardware cost.
In another embodiment of the present invention, in the embodiment shown in fig. 1, in order to improve the synchronicity when each image capturing device captures an image, the first image capturing instruction may be sent to each image capturing device in the following manner:
and the main controller respectively sends second image acquisition instructions to the processors so that each processor sends first image acquisition instructions to the image acquisition equipment corresponding to the processor according to the second image acquisition instructions.
And the main controller simultaneously sends second image acquisition instructions to the processors respectively. The processing time of each processor may be uniform. Therefore, the first image acquisition instructions can be received by the image acquisition devices at the same time as far as possible.
The first image capture instruction may be the same as or different from the second image capture instruction. Specifically, when each processor receives a second image acquisition instruction sent by the main controller, the second image acquisition instruction can be directly sent to the image acquisition equipment as a first image acquisition instruction, and a first image acquisition instruction can also be determined according to the second image acquisition instruction and sent to the image acquisition equipment.
In this embodiment, the main controller sends the second image acquisition instruction to each processor in a unified manner, so that the synchronization of the trigger time for acquiring the image by the image acquisition device is higher, and the synchronization of each image acquisition device when acquiring the image is further improved.
In another embodiment of the present invention, in order to improve consistency of triggering the image capturing device to capture images by the main controller, in the above embodiment, the main controller may send the second image capturing instruction to each processor through the hardware trigger.
Specifically, it may be predetermined that the designated state of the hardware trigger represents an instruction to acquire an image, for example, the designated state may be 1 or 0. When the main controller needs the image acquisition equipment to acquire images, the state of the hardware trigger can be set to the specified state. The processor can read the state of the hardware trigger according to a preset period, and when the state is a specified state, the processor determines that a second image acquisition instruction sent by the main controller is received. Different from a software triggering mode, the hardware trigger triggering mode of the embodiment can improve the consistency of triggering.
In another embodiment of the present invention, in order to improve the time consistency of each processor, in the embodiment shown in fig. 1, the method may further include:
and respectively sending standard clock pulses to each processor, so that the processors adjust the time of the processors according to the matching result of the time intervals between the pulses of the standard clock pulses and the time of the processors.
Wherein the standard clock pulse may be a high-precision clock pulse, and the time interval between pulses is a standard time interval.
In the embodiment, when the time of each processor is consistent with that of the main controller, the time between the processors can be kept consistent as much as possible, and when the processors determine the time stamps, the consistency between the time of each image can be improved.
Fig. 2 is a schematic flow chart of another data synchronization method according to an embodiment of the present invention. The method is applied to a processor. The processor is respectively connected with the main controller and the image acquisition equipment corresponding to the processor, the image acquisition equipment is connected with the main controller, and the main controller is connected with at least two image acquisition equipment. The method comprises the following steps.
S210: and when the image acquisition device is determined to acquire the image according to the first image acquisition instruction, determining the time stamp of the image acquired by the image acquisition device.
Wherein the first image acquisition instruction is an instruction instructing each image acquisition apparatus to acquire an image at an associated time.
S220: and sending the time stamp to the main controller so that the main controller associates the image and the time stamp according to the preset corresponding relation between the image acquisition equipment and the processor.
Each image acquisition equipment in this embodiment triggers and gathers the image according to same image acquisition instruction for the moment of the image of gathering has the relevance, and separately handle image and timestamp, confirm the timestamp of image through the treater of being connected with every image acquisition equipment, can confirm the timestamp of image more fast, in time, compare in confirming the timestamp again by main control unit after receiving the image, can improve the accuracy of timestamp, and then can improve the time synchronism between the image of different camera collections.
In another embodiment of the present invention, in the embodiment shown in fig. 2, when it is determined in step S210 that the image capturing device captures an image according to the first image capturing instruction, the method specifically includes:
and when receiving a notification message sent by the image acquisition equipment, determining that the image acquisition equipment acquires an image according to the first image acquisition instruction. Wherein, the notification message is: the image capturing device transmits when capturing an image according to the first image capturing instruction. The notification message is used to identify the image capturing device to capture the image.
In particular, the image capture device may send a notification message to the processor when image capture is initiated. The notification message may specifically be in the form of an interrupt signal.
In another embodiment of the present invention, when determining the timestamp of the image acquired by the image acquisition device in step S210, the method specifically includes:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure time of the image acquisition equipment.
When the image capturing device exposes an image according to a Rolling (Rolling) mode, the exposure duration of an image may be the sum of the total Rolling duration and the total line exposure duration. When the image is exposed in the Rolling mode, the photosensitive array of each pixel row can be controlled to be exposed in turn. The Rolling duration refers to the line-to-line time delay when exposure is performed from one pixel line to the next. The total Rolling duration may be the product of the Rolling duration and (total number of rows of the image-1). The line exposure time duration is the exposure time duration of each line in the image. The total line exposure duration is the product of the line exposure duration and the total number of lines of the image.
The exposure time period of the image capturing device may be pre-stored in the processor. The exposure duration of an image acquisition device can be understood as the exposure duration of an image.
Determining the time for acquiring the central point of the image according to the time for acquiring the notification message and the exposure duration of the image acquisition device, which may specifically be the time for determining the central point of the image according to the following formula: t0+ (t1+ t 2)/2. Where t0 is the time when the notification message is acquired, that is, the time when the image starts to be captured, and t1 and t2 are the total Rolling time length and the total line exposure time length, respectively.
In the embodiment, the acquisition of the image needs to occupy a certain time length, and the exposure time of the image center point is used as the time stamp of the image, so that the time information of the image is more accurate.
In another embodiment of the present invention, in the embodiment shown in fig. 2, the method further includes the following timing process:
and receiving the standard clock pulse sent by the main controller, matching the time interval between the pulses of the standard clock pulse with the time of the processor, and adjusting the time of the processor according to the matching result.
By matching the time interval between the pulses of the standard clock pulse to the time of the processor, the difference between the time interval between the pulses of the standard clock pulse and the time of the processor can be determined, from which the time of the processor can be adjusted earlier or later by the difference so that the time of the processor coincides with the time of the standard time device.
For example, the time between adjacent pulses of the standard clock pulse is kms, when the processor receives n standard clock pulses, the time of the processor is delayed by Δ t, the difference between the two is | Δ t- (n-1) k |, and when Δ t- (n-1) k >0, the time of the processor is adjusted earlier by the difference; the processor is timed later by the difference when Δ t- (n-1) k < 0.
The timing process can be periodically executed according to a preset frequency, so that the accumulated time error of the processor can be eliminated as much as possible, and the accuracy of the data time information is improved.
In the embodiment, when the time of each processor is consistent with that of the main controller, the time between the processors can be kept consistent as much as possible, and when the processors determine the time stamps, the consistency between the time of each image can be improved.
The embodiments shown in fig. 1 and 2 are based on the same inventive concept, and the contents of the inventions can be referred to each other.
Fig. 3 is a schematic view of an information interaction flow among the main controller, the image capturing device, and the processor according to an embodiment of the present invention. The main controller sends a second image acquisition instruction to each processor at the same time, and the processors send first image acquisition instructions to the linked image acquisition equipment when receiving the second image acquisition instructions sent by the main controller. The image acquisition equipment starts to acquire images when receiving a first image acquisition instruction sent by the processor and sends a notification message to the processor. When the processor receives the notification message sent by the image acquisition equipment, the processor determines the time stamp of the image acquired by the image acquisition equipment and sends the time stamp to the main controller. The image acquisition equipment sends the image to the main controller after acquiring the image. The main controller receives the image sent by the image acquisition equipment and the time stamp sent by the processor, and associates the image with the time stamp. Therefore, the main controller can obtain the image data acquired by the plurality of image acquisition devices with synchronous acquisition time.
Fig. 4 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present invention. This embodiment corresponds to the embodiment shown in fig. 1. The device is applied to a main controller, the main controller is respectively connected with at least two image acquisition devices, each image acquisition device is connected with a corresponding processor, and the main controller is connected with each processor. The device includes:
a first acquiring module 401 configured to acquire an image acquired by each of the image acquisition devices in response to a first image acquisition instruction; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
a second obtaining module 402 configured to obtain a timestamp of the image sent by each of the processors; wherein, the time stamp is: the processor determines when the image acquisition device acquires the image;
and an associating module 403 configured to associate the image and the timestamp according to a preset correspondence between the image capturing device and the processor.
In another embodiment of the present invention, the embodiment shown in fig. 4 may further include: a first sending module;
and a first sending module (not shown in the figure) configured to send the second image acquisition instructions to the processors respectively, so that each processor sends the first image acquisition instructions to the image acquisition device corresponding to the processor according to the second image acquisition instructions.
In another embodiment of the present invention, based on the embodiment shown in fig. 4, the first sending module is specifically configured to: and respectively sending a second image acquisition instruction to each processor through a hardware trigger.
In another embodiment of the present invention, the embodiment shown in fig. 4 further comprises: a second sending module;
and a second sending module (not shown in the figure) configured to send the standard clock pulse to each processor respectively, so that the processor adjusts the time of the processor according to the matching result of the time interval between the pulses of the standard clock pulse and the time of the processor.
The above device embodiment corresponds to the method embodiment shown in fig. 1, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The embodiment of the device is obtained based on the embodiment of the method, and specific description can be referred to the embodiment of the method, which is not described herein again.
Fig. 5 is a schematic structural diagram of another data synchronization apparatus according to an embodiment of the present invention. This embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2. The device is applied to a processor, the processor is respectively connected with a main controller and image acquisition equipment corresponding to the processor, the image acquisition equipment is connected with the main controller, and the main controller is connected with at least two image acquisition equipment. The device includes:
a first determining module 501 configured to determine a timestamp of an image captured by the image capturing device when it is determined that the image capturing device captures the image according to the first image capturing instruction; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
a third sending module 502 configured to send the timestamp to the main controller, so that the main controller associates the image and the timestamp according to a preset correspondence between the image capturing device and the processor.
In another embodiment of the present invention, the embodiment shown in fig. 5 further comprises: a second determination module;
a second determination module (not shown in the drawings) configured to determine that the image capturing apparatus captures an image according to the first image capturing instruction;
the second determining module is specifically configured to determine that the image acquisition device acquires an image according to the first image acquisition instruction when receiving a notification message sent by the image acquisition device; wherein, the notification message is: the image capturing device transmits when capturing an image according to the first image capturing instruction.
In another embodiment of the present invention, in the embodiment shown in fig. 5, the first determining module 501 is specifically configured to:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure duration of the image acquisition equipment.
The above device embodiment corresponds to the method embodiment shown in fig. 2, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The embodiment of the device is obtained based on the embodiment of the method, and specific description can be referred to the embodiment of the method, which is not described herein again.
Fig. 6 is a schematic structural diagram of a vehicle-mounted terminal according to an embodiment of the present invention. The vehicle-mounted terminal includes: a main controller 601, at least two image capturing devices 602 and a processor 603 connected to each image capturing device 602, the main controller 601 being connected to each processor 603.
A main controller 601 which sends a first image capturing instruction to each image capturing device 602; the first image capturing instruction is for instructing each image capturing device 602 to capture an image at an associated time;
the image acquisition device 602 acquires an image when receiving a first image acquisition instruction sent by the main controller 601, sends a notification message identifying that the image acquisition device 602 acquires the image to the processor 603, and sends the acquired image to the main controller 601;
the processor 603, upon receiving the notification message sent by the image capturing device 602, determines a timestamp of the image captured by the image capturing device 602 and sends the timestamp to the main controller 601;
the main controller 601 receives the image sent by the image capturing device 602, receives the timestamp of the image sent by the processor 603, and associates the image and the timestamp according to a preset correspondence between the image capturing device and the processor.
In another embodiment of the present invention, as shown in fig. 6, the main controller 601 further sends a second image capturing instruction to each of the processors 603;
the processor 603 is further configured to receive a second image acquisition instruction sent by the main controller 601, and send a first image acquisition instruction to the image acquisition device 602 according to the second image acquisition instruction;
the image capturing device 602 specifically receives the first image capturing instruction sent by the processor 603.
In another embodiment of the present invention, as shown in fig. 6, when the main controller 601 sends the second image capturing instruction to each processor 603, the method specifically includes:
the second image capturing instruction is sent to each processor 603 through a hardware trigger.
In another embodiment of the present invention, as shown in fig. 6, the main controller 601 further sends standard clock pulses to the respective processors 603;
the processor 603 further receives the standard clock pulse sent by the main controller 601, matches the time interval between the pulses of the standard clock pulse with the time of the processor 603, and adjusts the time of the processor 603 according to the matching result.
In another embodiment of the present invention, the embodiment shown in fig. 6, when determining the timestamp of the image captured by the image capturing device 602, the processor 603 includes:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure duration of the image acquisition device 602.
The terminal embodiment shown in fig. 6 and the method embodiment shown in fig. 1 are based on the same inventive concept, and the description of the specific implementation mode may refer to the description of the embodiment shown in fig. 1.
Fig. 7 is another structural schematic diagram of the vehicle-mounted terminal in fig. 6. The Processor is an MCU, and an Image Signal Processor (ISP) is added between the Image acquisition equipment and the main controller. The ISP may perform signal processing on the image transmitted by the image capturing device and transmit the processed image to the main controller, so that the format of the image signal and the like can be adapted to the main controller. The image capturing device, MCU and ISP in the dashed box may be referred to as camera modules, and in practice there may be at least two camera modules connected to the master control.
The image acquisition device and the ISP can be connected through a Mobile Industry Processor Interface (MIPI). The ISP and the main controller can be connected through an MIPI/digital image transmission interface. The ISP and MCU may be connected via a two-wire serial bus (I2C). The MCU and the main controller can be connected through an I2C interface.
MCU can be connected with main control unit through serialization chip or other interface switching chips, can unify data transmission interface like this, improves the compatibility.
Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.
Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A data synchronization method is characterized in that the method is applied to a main controller, the main controller is respectively connected with at least two image acquisition devices, each image acquisition device is connected with a corresponding processor, and the main controller is connected with each processor; the method comprises the following steps:
acquiring an image acquired by each image acquisition device in response to a first image acquisition instruction; wherein the first image acquisition instruction is used for instructing each image acquisition device to acquire an image at an associated moment;
acquiring a time stamp of the image sent by each processor; wherein the time stamp is: the processor determines when it is determined that the image acquisition device acquired the image;
and associating the image with the timestamp according to a preset corresponding relation between the image acquisition equipment and the processor.
2. The method of claim 1, wherein the first image acquisition instruction is sent to each image acquisition device by:
and the main controller respectively sends second image acquisition instructions to each processor, so that each processor sends first image acquisition instructions to the image acquisition equipment corresponding to the processor according to the second image acquisition instructions.
3. The method of claim 2, wherein the step of sending second image acquisition instructions to respective processors comprises:
and respectively sending a second image acquisition instruction to each processor through a hardware trigger.
4. The method of claim 1, further comprising:
and respectively sending standard clock pulses to each processor, so that the processor adjusts the time of the processor according to the matching result of the time interval between the pulses of the standard clock pulses and the time of the processor.
5. A data synchronization method is characterized in that the method is applied to a processor, the processor is respectively connected with a main controller and image acquisition equipment corresponding to the processor, the image acquisition equipment is connected with the main controller, and the main controller is connected with at least two image acquisition equipment; the method comprises the following steps:
when the image acquisition equipment is determined to acquire an image according to a first image acquisition instruction, determining a time stamp of the image acquired by the image acquisition equipment; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
and sending the time stamp to the main controller so that the main controller associates the image with the time stamp according to a preset corresponding relation between the image acquisition equipment and the processor.
6. The method of claim 5, wherein the step of determining that the image capture device captured an image according to a first image capture instruction comprises:
when a notification message sent by the image acquisition equipment is received, determining that the image acquisition equipment acquires an image according to a first image acquisition instruction; wherein the notification message is: and the image acquisition equipment sends the image acquisition instruction when acquiring the image according to the first image acquisition instruction.
7. The method of claim 6, wherein determining the timestamp of the image captured by the image capture device comprises:
and determining the time of acquiring the central point of the image as the time stamp of the image according to the time of receiving the notification message and the pre-acquired exposure duration of the image acquisition equipment.
8. A data synchronization device is characterized in that the data synchronization device is applied to a main controller, the main controller is respectively connected with at least two image acquisition devices, each image acquisition device is connected with a corresponding processor, and the main controller is connected with each processor; the device comprises:
a first acquisition module configured to acquire an image acquired by each of the image acquisition devices in response to a first image acquisition instruction; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
a second obtaining module configured to obtain a timestamp of the image sent by each of the processors; wherein the time stamp is: the processor determines when it is determined that the image acquisition device acquired the image;
and the association module is configured to associate the image with the timestamp according to a preset corresponding relation between the image acquisition equipment and the processor.
9. A data synchronization device is characterized in that the data synchronization device is applied to a processor, the processor is respectively connected with a main controller and image acquisition equipment corresponding to the processor, the image acquisition equipment is connected with the main controller, and the main controller is connected with at least two image acquisition equipment; the device comprises:
the first determination module is configured to determine a time stamp of the image acquired by the image acquisition device when the image acquisition device is determined to acquire the image according to the first image acquisition instruction; the first image acquisition instruction is an instruction for instructing each image acquisition device to acquire an image at an associated moment;
and the third sending module is configured to send the timestamp to the main controller, so that the main controller associates the image with the timestamp according to a preset corresponding relation between the image acquisition device and the processor.
10. A vehicle-mounted terminal characterized by comprising: the system comprises a main controller, at least two image acquisition devices and a processor connected with each image acquisition device; the main controller is connected with each processor;
the main controller sends a first image acquisition instruction to each image acquisition device; the first image acquisition instruction is used for instructing each image acquisition device to acquire an image at an associated moment;
the image acquisition equipment acquires an image when receiving a first image acquisition instruction sent by the main controller, sends a notification message for identifying the image acquisition equipment acquires the image to the processor, and sends the acquired image to the main controller;
the processor determines a time stamp of an image acquired by the image acquisition equipment when receiving a notification message sent by the image acquisition equipment, and sends the time stamp to the main controller;
the main controller receives the image sent by the image acquisition equipment, receives the timestamp of the image sent by the processor, and associates the image with the timestamp according to a preset corresponding relation between the image acquisition equipment and the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811594509.8A CN111372111A (en) | 2018-12-25 | 2018-12-25 | Data determination method and device and vehicle-mounted terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811594509.8A CN111372111A (en) | 2018-12-25 | 2018-12-25 | Data determination method and device and vehicle-mounted terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111372111A true CN111372111A (en) | 2020-07-03 |
Family
ID=71210235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811594509.8A Pending CN111372111A (en) | 2018-12-25 | 2018-12-25 | Data determination method and device and vehicle-mounted terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111372111A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112751983A (en) * | 2021-04-02 | 2021-05-04 | 湖北亿咖通科技有限公司 | Image time synchronization method and device, electronic equipment and storage medium |
| CN112929578A (en) * | 2021-03-05 | 2021-06-08 | 科络克电子科技（上海）有限公司 | Image sensitization time acquisition method, device, equipment and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101321301A (en) * | 2008-07-08 | 2008-12-10 | 浙江大学 | Camera array synchronization video acquisition and processing system |
| CN103326802A (en) * | 2013-05-31 | 2013-09-25 | 燕山大学 | Wireless network system for achieving synchronous acquisition, synchronous acquisition device and method |
| CN104270567A (en) * | 2014-09-11 | 2015-01-07 | 深圳市南航电子工业有限公司 | High-precision synchronous multi-channel image acquisition system and time synchronization method thereof |
| CN104469927A (en) * | 2014-10-27 | 2015-03-25 | 北京必创科技股份有限公司 | Synchronous collection method and system for wireless sensor star network |
| CN105554594A (en) * | 2015-10-27 | 2016-05-04 | 东莞酷派软件技术有限公司 | Control processing method and user terminals |
| CN206440244U (en) * | 2016-12-27 | 2017-08-25 | 江苏量淘数据技术有限公司 | Distributed multi-view stereo vision system |
| CN108513710A (en) * | 2017-12-19 | 2018-09-07 | 深圳市大疆创新科技有限公司 | The correlating method of image and location information, device and moveable platform |
-
2018
- 2018-12-25 CN CN201811594509.8A patent/CN111372111A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101321301A (en) * | 2008-07-08 | 2008-12-10 | 浙江大学 | Camera array synchronization video acquisition and processing system |
| CN103326802A (en) * | 2013-05-31 | 2013-09-25 | 燕山大学 | Wireless network system for achieving synchronous acquisition, synchronous acquisition device and method |
| CN104270567A (en) * | 2014-09-11 | 2015-01-07 | 深圳市南航电子工业有限公司 | High-precision synchronous multi-channel image acquisition system and time synchronization method thereof |
| CN104469927A (en) * | 2014-10-27 | 2015-03-25 | 北京必创科技股份有限公司 | Synchronous collection method and system for wireless sensor star network |
| CN105554594A (en) * | 2015-10-27 | 2016-05-04 | 东莞酷派软件技术有限公司 | Control processing method and user terminals |
| CN206440244U (en) * | 2016-12-27 | 2017-08-25 | 江苏量淘数据技术有限公司 | Distributed multi-view stereo vision system |
| CN108513710A (en) * | 2017-12-19 | 2018-09-07 | 深圳市大疆创新科技有限公司 | The correlating method of image and location information, device and moveable platform |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112929578A (en) * | 2021-03-05 | 2021-06-08 | 科络克电子科技（上海）有限公司 | Image sensitization time acquisition method, device, equipment and medium |
| CN112929578B (en) * | 2021-03-05 | 2023-05-05 | 科络克电子科技（上海）有限公司 | Image sensitization moment acquisition method, device, equipment and medium |
| CN112751983A (en) * | 2021-04-02 | 2021-05-04 | 湖北亿咖通科技有限公司 | Image time synchronization method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101389789B1 (en) | Image pickup apparatus, image pickup system, image pickup method and computer readable non-transitory recording medium | |
| CN109905194A (en) | A kind of vehicle-mounted terminal system and synchronization data obtaining method, device | |
| WO2023093054A1 (en) | Data processing method, apparatus, and system, device, and storage medium | |
| CN101959022B (en) | Synchronous circuit, and image pick-up device and synchronous exposure control method thereof | |
| CN112261283B (en) | Synchronous acquisition method, device and system of high-speed camera | |
| WO2017078810A1 (en) | Synchronization image data captured from a camera array with non-image data | |
| TW201241594A (en) | A system, method, and device to distribute accurate synchronization timestamps in an expandable and timing critical system | |
| CN107948463B (en) | Camera synchronization method, device and system | |
| US20190149702A1 (en) | Imaging apparatus | |
| CN109391799B (en) | Monitoring video synchronization method and device and video acquisition equipment | |
| KR20190118548A (en) | Image sensor, and control system | |
| CN111736169A (en) | Data synchronization method, device and system | |
| CN105278413B (en) | The communication means of host computer and similar CAN device | |
| CN111107248B (en) | Multi-channel video acquisition synchronization system and method and acquisition controller | |
| CN111372111A (en) | Data determination method and device and vehicle-mounted terminal | |
| CN111556224A (en) | Multi-camera synchronous calibration method, device and system | |
| CN211308445U (en) | Multi-sensor data synchronization system and automobile | |
| CN111371982B (en) | Data synchronization method, processor and vehicle-mounted terminal | |
| CN111813716A (en) | Multi-sensor data synchronization, electronic device, and storage medium | |
| CN109302567A (en) | Camera image low latency synchronization system and image low latency synchronous method | |
| CN116347128A (en) | Data synchronization method, device, equipment and storage medium | |
| CN111193568A (en) | Time synchronization method, device, system, storage medium and vehicle | |
| JP2013048333A (en) | Image processor, image processing method and image processing system | |
| CN109842737A (en) | A kind of image exposure method and device, car-mounted terminal | |
| US11818447B2 (en) | Master image sensor, slave image sensor, imaging system, and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200703 |
|
| RJ01 | Rejection of invention patent application after publication |