CN111953958B

CN111953958B - Time delay measuring method, device, system and storage medium

Info

Publication number: CN111953958B
Application number: CN201910405481.7A
Authority: CN
Inventors: 林晨; 续立军; 赵远东; 李名杨
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-05-16
Filing date: 2019-05-16
Publication date: 2022-08-02
Anticipated expiration: 2039-05-16
Also published as: CN111953958A

Abstract

The embodiment of the application provides a time delay measuring method, equipment, a system and a storage medium. In the embodiment of the application, a control module synchronous with the clock of the image acquisition module is adopted, and the control module controls the reference object to adjust the state of the reference object to the specified state at the corresponding control time in the process of acquiring the image of the reference object by the image acquisition module; and then the time delay parameter can be calculated according to the time stamp carried by the target image of the acquired reference object in the designated state and the control time for adjusting the reference object to the designated state, so that the time delay parameter can be used for calibrating the time stamp of the subsequently acquired image subsequently, the time stamp precision of the image acquired by the image acquisition module is improved, the subsequent processing effect based on the image is improved, and the positioning precision based on the image is improved.

Description

Time delay measuring method, device, system and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a system, and a storage medium for measuring a time delay.

Background

With the continuous development of image processing technology, more and more devices are provided with image acquisition devices for acquiring images and realizing other functions according to the acquired images. For example, in the field of artificial intelligence, intelligent devices utilize captured images for positioning or navigation, etc.

However, in practical applications, the effect of performing subsequent processing by using the acquired image is poor, for example, the accuracy of positioning by using the acquired image by the smart device is low, and the like.

Disclosure of Invention

Aspects of the present application provide a time delay measurement method, device, system, and storage medium for improving the timestamp accuracy of an image acquired by an image acquisition device, and further improving the effect of subsequent processing based on the image, such as improving the positioning accuracy based on the image.

An embodiment of the present application provides a time delay measurement system, including: the device comprises a control module, a reference object and an image acquisition module; the reference object has at least two states that are perceivable by the image acquisition module; the control module is synchronous with the image acquisition module in clock; the image acquisition module is used for sending a synchronization signal to the control module in the process of acquiring the image of the reference object, adding a time stamp to the acquired image and providing the image with the time stamp to the control module; wherein the acquired image comprises a target image when the reference object is in a specified state; the control module is used for controlling the reference object to be adjusted to an appointed state at corresponding control time according to the synchronous signal, and calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module.

The embodiment of the present application further provides a time delay measuring method, including: in the process of acquiring the image of the reference object, controlling the reference object to be adjusted to a specified state at control time; adding a time stamp to a collected image, wherein the collected image comprises a target image when the reference object is in a specified state; and calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the subsequently acquired image.

The embodiment of the present application further provides a time delay measuring method, including: the image acquisition module sends a synchronization signal to the control module in the process of acquiring the image of the reference object, so that the control module controls the reference object to be adjusted to a specified state at corresponding control time based on the synchronization signal; adding a time stamp to a collected image, wherein the collected image comprises a target image when the reference object is in a specified state; providing the target image added with the timestamp to a control module so that the control module can calculate a time delay parameter according to the timestamp of the target image and the control time, wherein the time delay parameter is used for calibrating the timestamp of an image subsequently acquired by the image acquisition module; the control module is synchronous with the image acquisition module in clock.

The embodiment of the present application further provides a time delay measuring method, including: the control module receives a synchronous signal sent by the image acquisition module in the process of acquiring the image of the reference object; controlling the reference object to be adjusted to an appointed state at corresponding control time according to the synchronization signal, so that the image acquisition module adds a timestamp to the acquired target image when the reference object is in the appointed state; calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module; the control module is synchronous with the image acquisition module in clock.

The embodiment of the present application further provides a time delay measuring method, including: the processing unit acquires a target image acquired by the image acquisition module, wherein the target image is an image of a reference object in an appointed state and carries a timestamp; acquiring control time provided by a control module, wherein the control time is time for controlling the reference object to be adjusted to a specified state; calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module; wherein the control module is clock-synchronized with the image acquisition module.

An embodiment of the present application further provides a computing device, including: at least one memory, at least one processor, and a vision sensor; wherein the at least one memory is to store a computer program; the at least one processor is coupled to the at least one memory for executing the computer program for: in the process of controlling the vision sensor to acquire the image of the reference object, controlling the reference object to be adjusted to a specified state at a control time; adding a time stamp to an image acquired by the vision sensor, wherein the acquired image comprises a target image when the reference object is in a specified state; and calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the vision sensor subsequently.

An embodiment of the present application further provides an image capturing apparatus, including: a memory, a processor, a vision sensor, and a signal generation component; wherein the memory is used for storing a computer program; the vision sensor is used for acquiring an image of a reference object; the processor is coupled to the memory for executing the computer program for: in the process that the vision sensor collects the image of the reference object, a synchronous signal is sent to the control equipment through the signal generation assembly, so that the control equipment controls the reference object to be adjusted to a specified state at control time based on the synchronous signal; time stamping an image captured by the vision sensor, the captured image comprising: a target image when the reference object is in a specified state; providing the target image added with the timestamp to a control device, so that the control device can calculate a time delay parameter according to the timestamp of the target image and the control time, wherein the time delay parameter is used for calibrating the timestamp of the image provided by the image acquisition device; wherein the control device is clock synchronized with the image acquisition device.

An embodiment of the present application further provides a control device, including: a memory, a processor, and a first signal receiving component; wherein the memory is for storing a computer program; the processor is coupled to the memory for executing the computer program for: receiving a synchronous signal sent by an image acquisition device in the process of acquiring the image of the reference object through the first signal receiving assembly; controlling the reference object to be adjusted to a specified state at control time based on the synchronous signal, so that the image acquisition equipment acquires a target image when the reference object is in the specified state, adds a timestamp to the target image and provides the target image to the computing equipment; providing the control time to the computing device, so that the computing device can calculate a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition device; wherein the control device is clock synchronized with the image acquisition device.

The embodiment of the present application further provides a computing device, at least one memory and at least one processor, where the at least one memory is used for storing a computer program; the at least one processor is coupled to the at least one memory for executing the computer program for: acquiring a target image provided by image acquisition equipment, wherein the target image is an image of a reference object acquired by an image acquisition module in a specified state and carries a timestamp; acquiring control time provided by control equipment, wherein the control time is the time for controlling the reference object to be adjusted to a specified state by the control module; calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition equipment; wherein the reference object has at least two states that are perceivable by the image acquisition device, and the control device is clock synchronized with the image acquisition device.

An embodiment of the present application further provides a computing device, including: at least one memory, at least one processor, a visual sensor, and a signal generating component; the at least one memory storage for storing a computer program; the at least one processor is coupled to the at least one memory for executing the computer program for: during the process that the vision sensor acquires the image of the reference object, sending a synchronization signal to a control device through the signal generation assembly so that the control device controls the reference object to be adjusted to a specified state at a corresponding control time based on the synchronization signal, wherein the image acquired by the acquisition sensor comprises a target image when the reference object is in the specified state; calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the vision sensor; wherein the control device is clock synchronized with the computing device.

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the latency measurement methods described above.

In the embodiment of the application, a control module synchronous with the clock of the image acquisition module is adopted, and the control module controls the reference object to adjust the state of the reference object to the specified state at the corresponding control time in the process of acquiring the image of the reference object by the image acquisition module; and then the time delay parameter can be calculated according to the time stamp carried by the target image of the acquired reference object in the designated state and the control time for adjusting the reference object to the designated state, so that the time delay parameter can be used for calibrating the time stamp of the subsequently acquired image subsequently, the time stamp precision of the image acquired by the image acquisition module is improved, the subsequent processing effect based on the image is improved, and the positioning precision based on the image is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1a is a schematic structural diagram of a delay measurement system according to an embodiment of the present application;

fig. 1b is a schematic structural diagram of another delay measurement system provided in the embodiment of the present application;

fig. 1c is a schematic timing diagram of a synchronization signal, a state control signal, and an image acquired by an image acquisition module according to an embodiment of the present disclosure;

fig. 1d is a schematic structural diagram of another delay measurement system provided in the embodiment of the present application;

fig. 1e is a schematic timing diagram of a time service pulse and a time service message received by a control module and an image acquisition module according to an embodiment of the present application;

fig. 2a is a schematic flowchart of a delay measurement method according to an embodiment of the present application;

fig. 2b is a schematic structural diagram of an image capturing device according to an embodiment of the present disclosure;

fig. 3a is a schematic flowchart of another delay measurement method according to an embodiment of the present application;

fig. 3b is a schematic structural diagram of a control device according to an embodiment of the present application;

fig. 4a is a schematic flowchart of another delay measurement method according to an embodiment of the present application;

FIG. 4b is a schematic structural diagram of a computing device according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of another computing device according to an embodiment of the present application.

Fig. 6a is a schematic flowchart of another delay measurement method according to an embodiment of the present application;

fig. 6b is a schematic structural diagram of another computing device provided in the embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In some embodiments of the present application, a time delay measurement system is provided, in which an image acquisition module and a control module synchronized with a clock are combined with a reference object, the image acquisition module sends a synchronization signal to the control module when acquiring an image of the reference object, and adds a timestamp to the acquired image; the control module determines control time according to the synchronous signal, and controls the reference object to adjust the state of the reference object to an appointed state at the control time; furthermore, the system utilizes the processing module to calculate the time delay parameter according to the time stamp of the target image of the reference object in the designated state acquired by the image acquisition module and the control time corresponding to the target image, so that the time delay parameter can be utilized to calibrate the time stamp of the image acquired by the image acquisition module in the following process, the time stamp precision of the image acquired by the image acquisition module is improved, the subsequent processing effect based on the image is improved, and the positioning precision based on the image is improved.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1a is a schematic structural diagram of a delay measurement system according to an embodiment of the present application. As shown in fig. 1a, the system comprises: a control module 101, a reference object 102 and an image acquisition module 103. The control module 101 is connected to the image acquisition module 103 and the reference object 102.

In the present embodiment, the reference object 102 has at least two states that can be sensed by the image capturing module 103, and any one of the states can be used as a specified state. The designated state is a state that is selected for use in calculating the delay parameter of the image acquisition module 103. The smaller the response time of the state adjustment of the reference object 102, the better. In the case where there are a plurality of choices, an object whose response time delay of the state adjustment is relatively small or minimum may be preferentially selected as the reference object 102. The reference object 102 may have different implementation forms according to different implementation forms of the image acquisition module 103. For example, in the case where the image acquisition module 103 is a camera, a vision sensor such as a laser radar, the reference object 102 may be a controlled light source, an indicator light, a mechanical arm, or the like.

For example, the reference object 102 may be a controlled light source, which may have two states of being turned off and being turned on, and a light emitting band when the controlled light source is turned on may be sensed and collected by the pattern collection module 103. Alternatively, the lighting state may be determined as the designated state of the controlled light source, and the extinguishing state may also be determined as the designated state of the controlled light source. The controlled light source has the advantages of short response time, low cost and the like, and is favorable for improving the accuracy of the subsequently calculated time delay parameters.

For another example, the reference object 102 may be an indicator lamp having at least two lighting colors, the color of the indicator lamp may be used as the state of the indicator lamp, and one of the colors may be determined as the designated state of the indicator lamp. For example, the designation light may have three color changes of red, yellow, and green, and the indicator light has three states of red, yellow, and green. Alternatively, red may be determined as the designated state of the indicator light, or green may be determined as the designated state of the indicator light, or the like. The indicator lamp has the advantages of short response time, low cost and the like, and is beneficial to improving the accuracy of the subsequently calculated time delay parameters.

For another example, the reference object 102 may also be a robotic arm having both a lowered and a raised state. Alternatively, the lifted-up state may be determined as a specified state of the robot arm, or the put-down state may be determined as a specified state of the robot arm. The state amplitude of the mechanical arm is large, and the mechanical arm is sensed by the image acquisition module 103 very easily.

Wherein, the image acquisition module 103 can acquire an image and time stamp the acquired image. Consider that the image capture module 103 has a certain processing delay, which is a certain delay between the timestamp of the image marker and the time when the image capture module 103 actually captures the image. In this embodiment, a time delay parameter between the time when the image acquisition module 103 actually acquires the image and the time stamp which is the image marker may be measured in conjunction with the control module 101 and the reference object 102.

In the time delay measurement system provided in this embodiment, the control module 101 and the image capturing module 103 are synchronized in clock, an image capturing window of the image capturing module 103 is opposite to the reference object 102, and the control module 101 is connected to the image capturing module 103 and the reference object 102, respectively. The image acquisition module 103 is responsible for acquiring an image of the reference object 102 and sending a synchronization signal to the control module 101 during the process of acquiring the image of the reference object 102. The synchronization signal is mainly used to inform the control module 101 of the time for starting to acquire the image of the reference object 102, so that the control module 101 can control the reference object 102 to adjust its state to a specified state according to the time, so as to be acquired by the image acquisition module 103. Alternatively, the image acquisition module 103 may send a synchronization signal to the control module 101 when acquiring the image of the reference object 102.

Further, the control module 101 receives the synchronization signal sent by the image capturing module 103, and adjusts it to a specified shape at the corresponding control time reference object 102 according to the synchronization signal. Alternatively, the control module 101 may determine a control time according to the synchronization signal and control the reference object 102 to adjust it to a designated state at the control time. The image acquisition module 103 may acquire an image when the reference object 102 is in a specified state and add a time stamp to the image. In the present embodiment, for convenience of description and distinction, an image of the reference object 102 acquired by the image acquisition module 103 in a specified state is defined as a target image. On the premise of clock synchronization with the image acquisition module 103, the control module 101 may determine, according to the synchronization signal, the time at which the image acquisition module 103 acquires the target image, and control the reference object 102 to adjust to the designated state at the time, that is, control the reference object 102 to adjust to the designated state at the control time. Since the acquisition time and the control time of the target image are the same time, the time delay parameter of the image acquisition module 103 can be calculated according to the time stamp and the control time of the target image, and the time delay parameter reflects the time delay between the time when the image acquisition module 103 actually acquires the image and the time stamp marked for the image.

Further, the image acquisition module 103 provides the acquired image of the reference object to the control module 101, and the image of the reference object 102 acquired by the image acquisition module 103 includes: an image when the reference object 102 is in a specified state. In this way, the control module 101 calculates the time delay parameter of the image capturing module 103 according to the timestamp and the control time of the target image. The time delay parameter can be used for calibrating the time stamp of the image acquired by the image acquisition module 103, so that the time stamp of the image acquired by the image acquisition module 103 can be calibrated subsequently by using the time delay parameter, the time stamp precision of the image acquired by the image acquisition module is improved, and the subsequent processing effect based on the image is improved, such as the positioning precision based on the images is improved.

In some embodiments, as shown in fig. 1b, the control module 101 comprises: a control unit 101a and a processing unit 104. The image acquisition module 103 is connected to the control unit 101a and the processing unit 104, and the control unit 101a and the processing unit 104 are connected to each other. Wherein the image acquisition module 103 is clock-synchronized with the control unit 101 a. Further, the control unit 101a is configured to receive a synchronization signal sent by the image capturing module 103, and determine a control time according to the synchronization signal. Further, the control unit 101a controls the reference object 103 to adjust to a specified state at the control time, and supplies the corresponding control time to the processing unit 104. Further, the image acquisition module 103 provides the time-stamped target image to the processing unit 104. The processing unit 104 calculates a time delay parameter of the image acquisition module 103 according to the timestamp and the control time of the target image.

The control unit 101a, the image capture module 103 and the reference object 102 may be optically or electrically connected. When the control unit 101a, the image acquisition module 103 and the reference object 102 can adopt optical connection, the synchronization signal and the state control signal are optical pulse signals; when the control unit 101a can be electrically connected with the image acquisition module 103 and the reference object 102, the synchronization signal and the state control signal are electric pulse signals.

Further, the implementation manner of the image acquisition module 103 providing the target image to the processing unit 104 may be: the image capturing module 103 sends the target image to the processing unit 104, or the processing unit 104 reads the target image from the image capturing module 103. Accordingly, if the image capturing module 103 stores the target image locally, the processing unit 104 reads the target image from the image capturing module 103.

Similarly, the implementation manner of the control unit 101a providing the control time to the processing unit 104 may be: the image acquisition module 103 sends the control time to the processing unit 104, or the processing unit 104 reads the control time from the control unit 101 a. Accordingly, if the control unit 101a stores the control time locally, the processing unit 104 reads the control time from the control unit 101 a.

In an alternative embodiment, the corresponding image capturing frequency may be set according to the image capturing performance of the image capturing module 103. The image acquisition module 103 periodically acquires an image of the reference object 102 according to a set image acquisition frequency, and transmits a pulse signal as a synchronization signal to the control unit 101a every time the image of the reference object 102 is acquired. The image acquisition module 103 also adds a time stamp to each acquired image and provides the time-stamped image to the processing unit 104. Since the image acquired by the image acquisition module 103 includes the target image when the reference object 102 is in the specified state, the image acquisition module 103 supplies the image of the reference object 102 acquired each time to the processing unit 104, that is, supplies the target image to the processing unit 104.

Correspondingly, the control unit 101a performs cycle counting on the pulse signals sent by the image acquisition module 103, sends a state control signal to the reference object every time when N pulse signals are counted to control the reference object to adjust to a specified state, and provides the time counted to N pulse signals as control time to the processing unit 104; n is more than or equal to 2 and is an integer, and the specific value can be flexibly set according to the actual situation. Wherein, N is an integer greater than or equal to 2, which enables the image acquisition module 103 to delay acquiring the target image, thereby avoiding the large error of the calculated delay parameter caused by the mismatch between the target image acquired by the processing unit 104 and the control time. For example, in the example where the reference object is a controlled light source and the designated state is a lighting state, if N is 1, the image acquired by the image acquisition module 103 is always the image when the controlled light source is in the lighting state, and there is a possibility that the control time corresponding to a certain frame of image is acquired, and the acquisition is mistaken. For example, when the control time of the mth frame image is read, the control time of the (m-1) th frame image or the (m + 1) th frame image is acquired, so that the calculated delay parameter error is large. Wherein m is not less than 2 and is an integer.

Alternatively, the control unit 101a may also record a time corresponding to an edge of a pulse counted as N every time N pulse signals are counted, and take the time as a control time, and issue a state control signal to the reference object 102 at the time. Optionally, if the pulse signal is a positive pulse signal, recording a time corresponding to a rising edge of the pulse counted as N every time N pulses are counted; if the pulse signal is a negative pulse signal, every time when N pulses are counted, the time corresponding to the falling edge of the pulse counted as N is recorded. Further, the state control signal may be a pulse signal (positive pulse or negative pulse). Accordingly, when the reference object 102 receives the pulse signal, the state thereof is adjusted to the specified state.

The following takes the case that the synchronization signal sent by the image capturing module 103 and the status control signal sent by the control unit 101a are both positive pulse signals, and the above process is exemplarily described with reference to the timing chart shown in fig. 1 c.

Assuming that N is 5, as shown in fig. 1c, each time the image acquisition module 103 acquires one frame of image of the reference object, it sends a positive pulse signal to the control unit 101a, and marks a timestamp on the current frame image acquired this time. The control unit 101a receives the positive pulse signal, counts the number of the positive pulse signal, and sends a positive pulse signal (state control signal) to the reference object 102 every time 5 positive pulses are counted, and records the time (control time) of the rising edge corresponding to the positive pulse signal. Accordingly, the reference object 102 receives a positive pulse signal (state control signal) from the control unit 101a, and adjusts its own state to a specified state. The frame of image acquired by the image acquisition module 103 is the target image of the reference object 102 in the designated state, i.e. the image labeled "K" shown in fig. 1 c.

Further, the processing unit 104 may recognize at least one frame of target image from the images provided by the image capturing module 103, and determine control times provided by the control unit 101a corresponding to the at least one frame of target image respectively. Preferably, in order to improve the accuracy of the calculated time delay parameter, the processing unit 104 may identify multiple frames of target images from the images provided by the image capturing module 103, and determine the control times provided by the control unit 101a corresponding to the multiple frames of target images respectively. Further, the processing unit 104 calculates an average time delay between the time stamp of the plurality of frames of target images and the corresponding control time, and takes the average time delay as a time delay parameter. In the embodiments of the present application, a multiframe refers to 2 frames or more than 2 frames.

Alternatively, the processing unit 104 may identify a plurality of frames of target images from the images provided by the image acquisition module 103 according to the image characteristics when the reference object 102 is in the specified state.

Alternatively, if the reference object 102 has two states and N is an integer greater than or equal to 3, the processing unit 104 may identify an image different from the other (N-1) frames as the target image from every N frames of images provided by the image acquisition module 103.

Optionally, the processing unit 104 is further configured to store the time delay parameter, and subsequently calibrate the time stamp of the image acquired by the image acquisition module 103 by using the time delay parameter. In different application scenes, different applications can be performed by using the calibrated image, and different effects can be realized. In the following, an exemplary description is given in connection with different application scenarios.

Application scenario 1: the image acquired by the image acquisition module 103 and the sensing data acquired by other sensors (such as an accelerometer, a gyroscope, a laser radar, etc.) can be used for data fusion, and the fused data is used for positioning or navigating the device where the image acquisition module 103 is located. In the application scenario, the time corresponding to the image acquired by the image acquisition module 103 is aligned with the time of the data acquired by other sensors, which is helpful for improving the accuracy of positioning or navigation. The time corresponding to the image acquired by the image acquisition module 103 is aligned with the time of the data acquired by other sensors, and the time of the data acquired by each sensor can be calibrated according to the respective time calibration mode of each sensor, so that the time of the data acquired by each sensor is aligned, otherwise, the accuracy of positioning or navigating the device where the image acquisition module 103 is located is reduced. For the image acquisition module 103, the timestamp of the image acquired by the image acquisition module 103 may be time-calibrated by using the time delay parameter, so that the time of the image acquired by the image acquisition module 103 is aligned with the time of the data acquired by other sensors. For example, the time obtained by subtracting the time delay parameter from the time stamp of the image acquired by the image acquisition module 103 may be used as the actual time of the image acquired by the image acquisition module 103.

Application scenario 2: the response time of the smart device may be tested using the image captured by the image capture module 103. In this application scenario, people may send corresponding instructions to the smart device. For example, a person may send a state adjustment command to the smart device through the host computer software or a remote control associated with the smart device. The intelligent device performs corresponding actions according to the instruction, and the image of the intelligent device acquired by the image acquisition module 103 can be used for testing the response time from the instruction received by the intelligent device to the instruction completion. If the timestamp of the image acquired by the image acquisition module 103 has an error, the accuracy of the measured response time of the smart device may be affected. Based on this, in the application scenario, the time delay parameter of the image acquisition module 103 measured in the embodiment of the present application is used to calibrate the time stamp of the acquired image, which is beneficial to improving the accuracy of measuring the response time of the intelligent device. For example, the time obtained by subtracting the time delay parameter from the time stamp of the image acquired by the image acquisition module 103 may be used as the actual time of the image acquired by the image acquisition module 103. For example, assuming that the intelligent device is a mechanical arm, if the response time of lifting the mechanical arm is to be tested, a lifting instruction can be sent to the mechanical arm through upper computer software or a remote controller matched with the mechanical arm, and when the mechanical arm receives the instruction, the time of receiving the instruction is recorded; when the mechanical arm is lifted, the image acquisition module 103 is used for acquiring images of the mechanical arm and marking a timestamp on the acquired images. Further, the response time of the mechanical arm lifting action can be measured by using the time stamp of the image acquisition module 103 and the recorded time for receiving the instruction. The time delay parameter of the image acquisition module 103 measured by the embodiment of the application is used for calibrating the time stamp of the acquired image, so that the accuracy of measuring the response time of the mechanical arm lifting action is improved. Alternatively, the robot arm may send an image capturing control signal to the image capturing module 103 at the moment of lifting, and the image capturing module 103 captures an image of the robot arm.

In the embodiment of the present application, the image capturing module 103 and the control unit 101a are synchronized in clock, the clocks of the image capturing module 103 and the control unit 101a may be adjusted to be consistent manually, or the time service module 105 may be used to provide time service to the image capturing module 103 and the control unit 101a, so that the clocks of the two modules are synchronized. Correspondingly, as shown in fig. 1d, the time delay measurement system provided in the embodiment of the present application may further include: a time service module 105. The time service module 10 is connected to the image acquisition module 103 and the control unit 101a, and provides the image acquisition module 103 and the control unit 101a with time service pulses and time service times corresponding to the time service pulses, so that the image acquisition module 103 and the control unit 101a synchronize the local clock with the time service module 105, and further clock synchronization between the image acquisition module 103 and the control unit 101a is achieved. The time service module 105 may be a GPS time service device, a big dipper time service device, or a standard clock source, but is not limited thereto.

The connection mode of the lines for sending the time service pulse to the image acquisition module 103 and the control unit 101a by the time service module 10 may be optical connection or electrical connection. When the connection mode of the lines for sending the time service pulse to the image acquisition module 103 and the control unit 101a by the time service module 105 is optical connection, the time service pulse is an optical pulse; when the timing module 105 transmits the timing pulse to the image acquisition module 103 and the control unit 101a in a manner of electrical connection, the timing pulse is an electrical pulse.

Further, the connection mode of the link for sending the time service pulse to the image acquisition module 103 and the control unit 101a by the time service module 105 may be communication connection. Namely, the time service module 105 is respectively connected with the image acquisition module 103 and the control unit 101a in a communication manner, and sends time service messages to the image acquisition module 103 and the control unit 101 a. Alternatively, the timing module 105, the image capturing module 103 and the control unit 101a may be connected by wire or wirelessly. Optionally, the time service module 105 may be connected to the image acquisition module 103 and the control unit 101a through a USB interface bus, a network cable, or the like; and can also be connected with the image acquisition module 103 and the control unit 101a through a mobile network. Accordingly, the network format of the mobile network may be any one of 2G (gsm), 2.5G (gprs), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G + (LTE +), WiMax, etc. Optionally, the time service module 105 may also be in communication connection with the image capturing module 103 and the control unit 101a through bluetooth, WiFi, infrared, and the like.

Alternatively, the timing pulse may be a positive pulse or a negative pulse. The time service module 105 can carry the time service time in the time service message and send the time service message to the image acquisition module 103 and the control unit 101 a. The time service module 105 periodically sends a time service pulse and a time service message to the image acquisition module 103 and the control unit 101a according to a preset time service period, where the time service message carries time service time. The time service time is the time when the time service module 105 sends out the time service pulse. If the timing pulse can be a positive pulse, the timing time is the time of the rising edge of the timing pulse sent by the timing module 105; if the timing pulse can be a negative pulse, the timing time is the time of the falling edge of the timing pulse sent by the timing module 105. Optionally, the time service module 105 may send a time service pulse and a time service message at the same time in each time service period; in each time service period, the time service pulse may be sent first, and then the time service message may be sent, but the time for the time service message to reach the image acquisition module 103 and the control unit 101a needs to be in the current time service period. Correspondingly, the image acquisition module 103 and the control unit 101a perform clock synchronization according to the time service pulse and the time service time carried in the time service message. The time service period can be flexibly set according to the processing rates of the image acquisition module 103 and the control unit 101a, for example, the time service period can be 1s, 0.5s, 0.1s, and the like, but is not limited thereto. An embodiment in which the image capturing module 103 and the control unit 101a perform clock synchronization will be described below by taking a time transfer pulse as a positive pulse as an example.

Fig. 1e is a timing chart of the image acquisition module 103 and the control unit 101a receiving the time service pulse and the time service message. In each time service period, the image acquisition module 103 and the control unit 101a respectively perform rising edge detection on the time service pulse, and when the rising edge of the time service pulse is detected, record corresponding local time. Further, the image acquisition module 103 and the control unit 101a respectively analyze the time service messages received in the current time service period, and obtain the time service time in the time service messages. Further, the image acquisition module 103 and the control unit 101a respectively calculate a time difference between a local time when a rising edge of the time service pulse is detected in the current time service period and the time service time, and synchronize the local clock with the time service module 105 by using the time difference. Since the time service time in the time service message is the time when each time service cycle arrives, the time service module 105 sends the time service pulse, and the time difference between the local time when the rising edge of the time service pulse is detected in each time service cycle and the time service time is calculated, that is, the time difference between the local clocks of the image acquisition module 103 and the control unit 101a and the timing clock of the time service module 105 can be obtained, therefore, the image acquisition module 103 and the control unit 101a can synchronize their respective local clocks with the timing clock of the time service module 105 by using the time difference.

Alternatively, the image acquisition module 103 and the control unit 101a may be clocked by hardware or software. When hardware timing is adopted, a counter or a timer can be started for timing, and if the rising edge of the timing pulse is detected, the counter or the timer can be interrupted, so that the local time is recorded. When software timing is adopted, if the rising edge of the time service pulse is detected, the software execution can be interrupted, and then the local time is recorded.

It is worth noting that the deployment situation of each module in the time delay measurement system is not limited in the embodiment of the application, and the deployment can be flexibly configured according to the application scenario and the measurement requirement. Optionally, the modules in the latency measurement system may be deployed separately. For example, the control unit 101a, the reference object 102, the image acquisition module 103, and the processing unit 104 are all deployed in the time delay measurement system as independent devices. For another example, the control unit 101a and the reference object 102 are deployed in the time delay measurement system as independent devices, and the image acquisition module 103 and the processing unit 104 may be located in the same physical device and deployed in the time delay measurement system in the form of a physical device. For another example, the control unit 101a and the reference object 102 may be located in the same physical device and deployed in the form of a physical device in the time delay measurement system, while the image acquisition module 103 and the processing unit 104 are located in another physical device and deployed in the form of a physical device in the time delay measurement system. Optionally, the time service module 105 may also be deployed in the time delay measurement system as an independent device, or may be disposed in the same physical device as the control unit 101a and/or the reference object 102, or may be located in the same physical device as the image acquisition module 103 and the processing unit 104, and be deployed in the time delay measurement system in the form of the physical device in which it is located.

In addition, each module in the delay measurement system provided by the embodiment of the present application may also be located in the same physical device. If the modules in the delay measurement system are located in the same physical device, the reference object 102 and the image acquisition module 103 are arranged oppositely, so that the reference object 102 can be located in an image acquisition window of the image acquisition module 103, and the image acquisition module 103 can acquire an image of the reference object 102 conveniently.

Optionally, in some application scenarios, before the image acquisition module 103 leaves a factory, the delay measurement system provided in this embodiment may be built to measure the delay parameter of the image acquisition module 103. In this scenario, the image acquisition module 103 may be deployed as a stand-alone device in the time delay measurement system.

Optionally, in other application scenarios, the image capturing module 103 and the processing unit 104 are disposed in the same physical device, that is, the image capturing module 103 and the processing unit 104 are an image capturing device and a processor in the physical device, respectively. Under the condition that the image acquisition module 103 and the processing unit 104 are disposed in the same physical device, the physical device where the image acquisition module 103 and the processing unit 104 are disposed may have multiple implementation forms, for example, the physical device where the image acquisition module 103 and the processing unit 104 are disposed may be a robot, a smart phone, an unmanned aerial vehicle or an unmanned vehicle, or may be other physical devices where the image acquisition module 103 is mounted, such as a vehicle or a bus. In other words, the time delay measurement system provided by the embodiment of the present application can measure any physical device in which the image acquisition module 103 is installed, for example, a time delay parameter of the image acquisition module 103 in a robot, a smart phone, an unmanned aerial vehicle, an unmanned vehicle, an automobile, or a bus.

For example, in the use process after the physical device leaves the factory, when the time delay parameter of the image acquisition module 103 in the physical device needs to be measured, the time delay measurement system provided in the embodiment of the present application may be set up to measure the time delay parameter of the image acquisition module 103. In this scenario, the time delay measurement system includes the physical device where the image acquisition module 103 and the processing unit 104 are located, the control unit 101a, and the reference object 102. Alternatively, the control unit 101a and the reference object 102 may be deployed in the time delay measurement system as a separate device independent of the physical device in which the image acquisition module 103 and the processing unit 104 are located. In this scenario, the control unit 101a, the reference object 102, and the image acquisition module 103 and the processing unit 104 may also be located in the same physical device. Correspondingly, when the time delay parameter of the image acquisition module 103 in the physical device needs to be measured, a corresponding time delay measurement control (which may be a time delay measurement control in upper computer software matched with the physical device, or a time delay measurement key or button arranged on the physical device) may be triggered, and the time delay measurement system is started to measure the time delay parameter of the image acquisition module 103; or, a time delay measurement period may be set on the physical device, and when the time delay measurement period arrives each time, the time delay measurement system is started to measure the time delay parameter of the image acquisition module 103.

For another example, before the physical device leaves the factory, the delay measurement system provided in the embodiment of the present application may be built, the delay parameter of the image capturing module 103 is measured, and the delay parameter is stored in the physical device where the image capturing module 103 is located, so as to calibrate the timestamp of the image captured by the image capturing module 103 by subsequently calling the delay parameter. In this scenario, the time delay measurement system comprises the physical device where the image acquisition module 103 and the processing unit 104 are located, the control unit 101a and the reference object 102. Alternatively, the control unit 101a and the reference object 102 may be deployed in the time delay measurement system as a separate device independent of the physical device in which the image acquisition module 103 and the processing unit 104 are located.

Further, in the embodiment of the present application, the image capturing module 103 may be any light-sensitive sensor capable of capturing an image. For example, the image acquisition module 103 may include: the visual sensor can contain an image sensor, and can be provided with a light projector and other auxiliary equipment. Alternatively, the vision sensor may be a camera 103b (shown in fig. 1 d), a laser sensor, a line and area CCD camera, a TV camera, or a digital camera, etc., but is not limited thereto. Fig. 1d illustrates a visual sensor as a camera, and the configuration is not limited thereto.

Further, for the case where the image capturing module 103 includes a camera, the image capturing module 103 further includes: an image processing unit 103 a. The camera 103b collects an image of the reference object under the control of the image processing unit, and outputs the collected image to the image processing unit 103 a; the image processing unit 103a is configured to add a time stamp to the image captured by the camera, and provide the time-stamped image to the processing unit 104.

It should be noted that, in the embodiment of the present application, the image processing unit 103a and the processing unit 104 may share the same processor. When the image processing unit 103a and the processing unit 104 may share one processor, the processor adds a timestamp to an image acquired by the camera 103b, and calculates a delay parameter according to the timestamp of the target image and the control time, and the specific implementation process of the delay parameter may refer to relevant contents described in the foregoing embodiment for the image processing unit 103a and the processing unit 104, which is not described herein again.

In this embodiment, the control module 101 may be a logic analyzer, an oscilloscope, or a signal generator, or may be another device capable of receiving the synchronization signal sent by the processing unit 104 and performing state control on the reference object 102.

In addition to the above time delay measurement system, the embodiment of the present application also provides a corresponding time delay measurement method, and the time delay measurement method provided by the embodiment of the present application is exemplarily described below from the perspective of the image acquisition module, the control module, the processing unit, and the time delay measurement system.

Fig. 2a is a schematic flowchart of a delay measurement method according to an embodiment of the present application. As shown in fig. 2a, the method comprises:

201. the image acquisition module sends a synchronization signal to the control module in the process of acquiring the image of the reference object, so that the control module controls the reference object to be adjusted to a specified state at a corresponding control time based on the synchronization signal.

202. The image acquisition module adds a time stamp to the acquired image, wherein the acquired image comprises a target image when the reference object is in a specified state.

203. The image acquisition module provides the target image added with the timestamp to the control module so that the control module can calculate the time delay parameter according to the timestamp and the control time of the target image. The time delay parameter is used for calibrating the time stamp of the image acquired subsequently by the image acquisition module.

In this embodiment, the reference object has at least two states that can be sensed by the image capture module, and the control module is clock synchronized with the image capture module. The implementation form of the reference object and the designated state of the reference object may refer to the related contents of the above system embodiments, and are not described herein again.

In this embodiment, the control module, the image capturing module and the reference object may be optically or electrically connected, and the specific connection manner may refer to the related content of the above system embodiments, which is not described herein again.

In this embodiment, the image capture module may capture an image and timestamp the captured image. Considering that the image acquisition module has a certain processing time delay, a certain delay exists between the time stamp marked by the image and the time when the image acquisition module actually acquires the image. In this embodiment, the image capturing module may combine the control module and the reference object to measure a time delay parameter between the time when the image capturing module actually captures the image and the time stamp marked for the image. The specific implementation mode is as follows: the image acquisition module sends a synchronization signal to the control module in the process of acquiring the image of the reference object, wherein the synchronization signal is mainly used for informing the control module of the time for starting to acquire the image of the reference object, so that the control module determines the control time according to the time and controls the reference object to adjust the state of the reference object to the specified state at the control time so as to be acquired by the image acquisition module. For the implementation manner of the control module controlling the reference object to be adjusted to the designated state at the control time based on the synchronization signal, reference may be made to the related contents of the above system embodiments, and details are not repeated herein. On the premise of clock synchronization with the image acquisition module, the control module can determine the time for the image acquisition module to acquire the target image according to the synchronization signal, and control the reference object to adjust the reference object to the specified state at the time, namely control the reference object to adjust the reference object to the specified state at the control time. Because the acquisition time of the target image and the control time corresponding to each frame of target image are the same moment, the time delay parameter of the image acquisition module can be calculated according to the time stamp and the control time of the target image, and the time delay parameter reflects the time delay between the time when the image acquisition module acquires the image and the time stamp marked by the image.

Further, the image acquisition module adds a time stamp to the acquired image and provides the image with the time stamp to the processing module. Since these images include the target image when the reference object is in the specified state, the image capturing module supplies the image captured each time to the control module, and also supplies the time-stamped target image to the control module. The implementation manner of providing the target image to the control module by the image acquisition module may refer to the related content of the above system embodiment, and is not described herein again.

Further, the control module can calculate the time delay parameter according to the time stamp of the target image and the control time corresponding to the target image. The time delay parameter can be used for calibrating the time stamp of the image acquired by the image acquisition module, so that the time stamp of the image acquired by the image acquisition module can be calibrated by utilizing the time delay parameter at subsequent times, the time stamp precision of the image acquired by the image acquisition module is improved, and the subsequent processing effect based on the image is improved, such as the positioning precision based on the images is improved.

In some embodiments, the control module may include: a control unit and a processing unit. For the connection manner of the control unit, the processing unit, the image acquisition module and the reference object, reference may be made to the relevant contents of the above system embodiments, which are not described herein again. Wherein, the control unit is synchronous with the clock of the image acquisition module. Further, the image acquisition module sends a synchronization signal to the control unit. Accordingly, the control unit determines the control time according to the synchronization signal. Further, the control unit controls the reference object to be adjusted to a designated state at the control time, and provides the corresponding control time to the processing unit. Further, the image acquisition module provides the time-stamped target image to the processing unit. And the processing unit calculates the time delay parameter of the image acquisition module according to the time stamp and the control time of the target image.

In an alternative embodiment, the image acquisition frequency may be set in the image acquisition module. The image acquisition module periodically acquires images of the reference object according to the set image acquisition frequency, and sends a pulse signal to the control unit as a synchronous signal when acquiring the images of the reference object each time. The image acquisition module also adds a time stamp to each acquired image and provides the time-stamped image to the processing unit. Accordingly, the implementation manner of the control unit determining the control time according to the pulse signal sent by the image acquisition module can refer to the relevant content of the above system embodiment, and is not described herein again.

In another optional embodiment, the image acquisition module and the control unit are synchronized in clock, the clocks of the image acquisition module and the control unit can be adjusted to be consistent manually, and a time service module can be used for carrying out time service on the image acquisition module and the control unit so as to synchronize the clocks of the image acquisition module and the control unit. The time service module is respectively connected with the image acquisition module and the control unit and provides time service time corresponding to the time service pulse and the time service pulse to the image acquisition module and the control unit so that the image acquisition module and the control unit respectively synchronize a local clock with the time service module, and further realize clock synchronization of the image acquisition module and the control unit. The implementation form of the time service module, the connection mode between the time service module and the image acquisition module, the implementation mode of the time service module for sending the time service pulse and the time service message, and the implementation mode of the image acquisition module for adjusting the local clock according to the time service pulse and the time service message may all refer to the relevant contents of the above system embodiments, and are not described herein again.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps in the latency measurement method performed by the image acquisition module.

Correspondingly, the embodiment of the application also provides image acquisition equipment. Fig. 2b is a schematic structural diagram of an image capturing device according to an embodiment of the present application. As shown in fig. 2b, the image pickup apparatus includes: a memory 20a, a processor 20b, a vision sensor 20c, and a signal generating component 20 d.

The vision sensor 20c may be a camera, a laser sensor, an infrared sensor, etc., but is not limited thereto.

The memory 20a is used for storing computer programs. The processor 20b is coupled to the memory 20a for executing a computer program for: in the process of acquiring the image of the reference object by the vision sensor 20c, sending a synchronization signal to the control device through the signal generation component 20d, so that the control device controls the reference object to be adjusted to a specified state at a control time based on the synchronization signal; and adding a time stamp to the image acquired by the vision sensor 20c, wherein the image acquired by the vision sensor includes an image when the reference object is in a specified state; providing the target image added with the timestamp to a control device, so that the control device can calculate a time delay parameter according to the timestamp and the control time of the target image, wherein the time delay parameter is used for calibrating the timestamp of the image provided by an image acquisition device; wherein the reference object has at least two states that can be perceived by the vision sensor, and the control device is clock-synchronized with the image acquisition device. The signal generating component 20d may be optically or electrically connected to the control device.

In an alternative embodiment, the control device comprises: a control unit and a processing unit. The signal generating component 20d is a pulse signal generating component, and the processor 20b is specifically configured to: and controlling the vision sensor 20c to periodically acquire the image of the reference object according to the set image acquisition frequency, and sending a pulse signal to the control unit through the pulse signal generation component as a synchronous signal every time the image of the reference object is acquired, so that the control unit determines the control time according to the synchronous signal. The processor 20b is further specifically configured to: adding a time stamp to each acquired image, providing the image with the time stamp to computing equipment, identifying multi-frame target images from the images by the computing equipment, calculating the average time delay between the time stamp of the multi-frame target images and the corresponding control time, and taking the average time delay as a time delay parameter.

In another alternative embodiment, the image capturing apparatus further comprises: a communication component 20e and a signal receiving component 20 f. The processor 20b is further configured to: receiving the time service pulse provided by the time service equipment through the signal receiving component 20f, and receiving the time service time corresponding to the time service pulse provided by the time service equipment through the communication component 20 e; and synchronizing the local time and the time service equipment according to the time service pulse and the time service time. The signal receiving module 20f may be an optical pulse signal receiving module or an electrical pulse signal receiving module, depending on the implementation form of the timing pulse.

In some alternative embodiments, as shown in fig. 2b, the image capturing apparatus may further include: power components 20g, display 20h, audio components 20i, and the like. Only some of the components are schematically shown in fig. 2b, and it is not meant that the image capturing device must comprise all of the components shown in fig. 2b, nor that the image capturing device only comprises the components shown in fig. 2 b.

The image acquisition device provided by the embodiment is synchronous with the clock of the control device, and the image acquisition device sends a synchronous signal to the control device when acquiring the image of the reference object and adds a timestamp to the acquired image; the control equipment determines control time according to the synchronous signal, and controls the reference object to adjust the state of the reference object to a specified state at the control time; furthermore, the control device calculates the time delay parameter according to the time stamp of the target image of the reference object in the designated state acquired by the image acquisition device and the control time corresponding to the target image, so that the time delay parameter can be used for calibrating the time stamp of the image acquired by the image acquisition device subsequently, the time stamp precision of the image acquired by the image acquisition device is improved, the subsequent processing effect based on the image is improved, and the positioning precision based on the image is improved.

Fig. 3a is a schematic flowchart of another delay measurement method according to an embodiment of the present application. As shown in fig. 3a, the method comprises:

301. the control module receives a synchronous signal sent by the image acquisition module in the process of acquiring the image of the reference object.

302. The control module controls the reference object to be adjusted to an appointed state at corresponding control time according to the synchronous signal, so that the image acquisition module acquires a target image when the reference object is in the appointed state and adds a timestamp to the target image.

303. And calculating a time delay parameter according to the time stamp and the control time of the target image. The time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module.

In this embodiment, the control module is respectively connected with the image acquisition module and the reference object. If the control module is optically connected with the image acquisition module, the synchronous signal is an optical signal; if the control module is electrically connected with the image acquisition module, the synchronous signal is an electric signal. In this embodiment, the image capture module may capture an image and timestamp the captured image. Considering that the image acquisition module has a certain processing time delay, a certain delay exists between the time stamp marked by the image and the time when the image acquisition module actually acquires the image. In this embodiment, the image capturing module may combine the control module and the reference object to measure a time delay parameter between the time when the image capturing module actually captures the image and the time stamp marked for the image.

In this embodiment, on the premise of clock synchronization with the image capturing module, the control module may determine, according to the synchronization signal, a time at which the image capturing module captures the target image, and control the reference object to adjust to the designated state at the time, that is, control the reference object to adjust to the designated state at the control time. Because the acquisition time and the control time of the target image are the same, the time delay parameter of the image acquisition module can be calculated according to the time stamp and the control time of the target image, and the time delay parameter reflects the time delay between the time of image acquisition of the image acquisition module and the time stamp marked by the image acquisition module. Therefore, the time delay parameter can be used for calibrating the time stamp of the image acquired by the image acquisition module subsequently, so that the time stamp precision of the image acquired by the image acquisition module is improved, the subsequent processing effect based on the image is improved, and the positioning precision based on the image is improved.

In some embodiments, the control module comprises: a control unit and a processing unit. The control unit may be a logic analyzer, an oscilloscope or a signal generator, or may be other equipment capable of receiving the synchronization signal sent by the image acquisition module and performing state control on the reference object. Accordingly, the control unit may perform the operations in

steps

301 and 302 and provide the control time to the processing unit before step 303. Accordingly, the processing unit performs the operation in step 303. Wherein, the processing unit and the image acquisition module can be positioned in the same physical device.

In an optional embodiment, for the image acquisition module, the image of the reference object may be periodically acquired according to a set image acquisition frequency, and a pulse signal may be sent to the control unit as the synchronization signal each time the image of the reference object is acquired. The image acquisition module also adds a time stamp to the acquired image and provides the time-stamped image to the processing unit. Correspondingly, the control unit can carry out cycle counting on the pulse signals sent by the image acquisition module, and sends state control signals to the reference object when N pulse signals are counted, so as to control the reference object to be adjusted to a specified state, and the time counted to N pulse signals is used as control time to be supplied to the processing unit; wherein N is more than or equal to 2 and is an integer, and the specific value can be flexibly set according to the actual situation.

Alternatively, the control unit may also record a time corresponding to an edge of the pulse counted as N every time N pulse signals are counted, and take the time as a control time, and issue a state control signal to the reference object at the time. Optionally, if the pulse signal is a positive pulse signal, recording a time corresponding to a rising edge of the pulse counted as N every time when N pulses are counted; if the pulse signal is a negative pulse signal, every time when N pulses are counted, the time corresponding to the falling edge of the pulse counted as N is recorded. Further, the state control signal may be a pulse signal (positive pulse or negative pulse). Accordingly, when the reference object receives the pulse signal, the state of the reference object is adjusted to a specified state.

In another optional embodiment, the image acquisition module and the control unit are synchronized in clock, the clocks of the image acquisition module and the control unit can be adjusted to be consistent manually, and a time service module can be used for carrying out time service on the image acquisition module and the control unit so as to synchronize the clocks of the image acquisition module and the control unit. The time service unit is respectively connected with the image acquisition module and the control module and provides time service pulse and time service time corresponding to the time service pulse to the image acquisition module and the control unit so that the image acquisition module and the control unit can respectively synchronize a local clock with the time service module, and further realize clock synchronization of the image acquisition module and the control module. The implementation form of the time service module, the connection form between the time service module and the control unit, the implementation manner of the time service module sending the time service pulse and the time service message, and the implementation manner of the control unit adjusting the local clock according to the time service pulse and the time service message may all refer to the relevant contents of the above system embodiments, and are not described herein again.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the latency measurement method performed by the control module.

Correspondingly, the embodiment of the application also provides a control device. Fig. 3b is a schematic structural diagram of a control device according to an embodiment of the present application. As shown in fig. 3b, the control apparatus includes: a memory 30a, a processor 30b and a first signal receiving component 30 c.

In the present embodiment, the memory 30a is used to store a computer program. The processor 30b is coupled to the memory for executing a computer program for: receiving, by the first signal receiving assembly 30c, a synchronization signal transmitted by the image capturing apparatus in a process of capturing an image of the reference object; controlling the reference object to be adjusted to an appointed state at corresponding control time according to the synchronization signal so that the image acquisition equipment acquires a target image when the reference object is in the appointed state and adds a timestamp to the target image; calculating a time delay parameter according to the time stamp and the control time of the target image, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition equipment; wherein the reference object has at least two states that are perceivable by the image acquisition device, and the control device is clock synchronized with the image acquisition device. The first signal receiving component 30c may be an optical signal receiving component or an electrical signal receiving component, depending on the implementation form of the synchronization signal.

In an alternative embodiment, the control device further comprises: signal generating assembly 30 d. When the control time control reference object is adjusted to the designated state, the processor 30b is specifically configured to: the pulse signals sent by the image acquisition equipment are counted circularly, and when N pulse signals are counted each time, a state control signal is sent to the reference object through the signal generation assembly 30d so as to control the reference object to be adjusted to a specified state; and taking the time counted to N pulse signals as control time, wherein N is more than or equal to 2 and is an integer; the image acquisition module is used for transmitting the image of the reference object to the control equipment every time when acquiring the image of the reference object according to the set image acquisition frequency.

In another optional embodiment, the control apparatus further comprises: a communication component 30e and a second signal receiving component 30 f. The processor 30b is further configured to: receiving the time service pulse provided by the time service equipment through the second signal receiving component 30f, and receiving the time service time corresponding to the time service pulse provided by the time service equipment through the communication component 30 e; and synchronizing the local time with the time service equipment according to the time service pulse and the time service time. The second signal receiving element 30f may be an optical pulse signal receiving element or an electrical pulse signal receiving element, depending on the implementation form of the timing pulse.

In another optional embodiment, when calculating the delay parameter, the processor 30b is specifically configured to: identifying multi-frame target images from the acquired images of the reference object, and determining respective corresponding control time of the multi-frame target images; and calculating the average time delay between the time stamp of the multi-frame target image and the corresponding control time, and taking the average time delay as a time delay parameter.

In some alternative embodiments, as shown in fig. 3b, the control apparatus may further include: power components 30g, display 30h, audio components 30i, and the like. Only a part of the components is schematically shown in fig. 3b, and it is not meant that the control device must contain all the components shown in fig. 3b, nor that the control device can only contain the components shown in fig. 3 b.

The control device provided by the embodiment is synchronous with the clock of the image acquisition device, and the image acquisition device sends a synchronous signal to the control device when acquiring the image of the reference object and adds a timestamp to the acquired image; the control equipment controls the reference object to adjust the state of the reference object to a specified state at corresponding control time according to the synchronous signal; furthermore, the time delay parameter is calculated according to the time stamp of the target image of the reference object in the designated state acquired by the image acquisition equipment and the control time corresponding to the target image, so that the time delay parameter can be used for calibrating the time stamp of the image acquired by the image acquisition equipment in the subsequent process, the time stamp precision of the image acquired by the image acquisition equipment is improved, the subsequent processing effect based on the image is improved, and the positioning precision based on the image is improved.

Fig. 4a is a schematic flowchart of another delay measurement method according to an embodiment of the present application. As shown in fig. 4a, the method comprises:

401. the processing unit acquires a target image provided by the image acquisition module, wherein the target image is an image acquired by the image acquisition module when the reference object is in a specified state and carries a time stamp.

402. The processing unit acquires control time provided by the control module, wherein the control time is the time for the control module to control the reference object to be adjusted to a specified state.

403. And the processing unit calculates a time delay parameter according to the time stamp and the control time of the target image, and the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module.

In this embodiment, the image capture module may capture an image and timestamp the captured image. Considering that the image acquisition module has a certain processing time delay, a certain delay exists between the time stamp marked by the image and the time when the image acquisition module actually acquires the image. In this embodiment, the processing unit may combine the image capturing module, the control unit, and the reference object to measure a time delay parameter between the time when the image capturing module actually captures the image and the time stamp which is the image marker.

In this embodiment, on the premise of synchronizing with the clock of the image capturing module, the control unit may determine, according to the synchronization signal, the time at which the image capturing module captures the target image, and control the reference object to adjust to the designated state at the time, that is, control the reference object to adjust to the designated state at the control time. Because the acquisition time and the control time of the target image are the same time, the processing unit can calculate the time delay parameter of the image acquisition module according to the time stamp and the control time of the target image, and the time delay parameter reflects the time delay between the time of image acquisition of the image acquisition module and the time stamp marked by the image. The time delay parameter can be used for calibrating the time stamp of the image acquired by the image acquisition module, so that the time stamp of the image acquired by the image acquisition module can be calibrated by utilizing the time delay parameter at subsequent times, the time stamp precision of the image acquired by the image acquisition module is improved, and the subsequent processing effect based on the image is improved, such as the positioning precision based on the images is improved.

In an optional embodiment, the image acquisition module sends a pulse signal to the control unit as a synchronization signal every time an image of the reference object is acquired according to a set image acquisition frequency; and adding a time stamp to each acquired image and providing the time-stamped image to the processing unit. Accordingly, the processing unit may identify the multiple frames of target images from the images provided by the image acquisition module and determine the control time provided by the control unit corresponding to the distribution of the multiple frames of target images. Further, the processing unit calculates an average time delay between the time stamp of the multi-frame target image and the corresponding control time, and takes the average time delay as a time delay parameter.

Alternatively, the processing unit may identify a plurality of frames of target images from the image provided by the image acquisition module according to the image characteristics of the reference object in the specified state. Alternatively, if the reference object has two states and N is an integer greater than or equal to 3, the processing unit may identify an image different from the other (N-1) frames as the target image from every N frames of images provided by the image acquisition module.

Optionally, the processing unit is further configured to store the time delay parameter, and subsequently calibrate the timestamp of the image acquired by the image acquisition module using the time delay parameter. In different application scenarios, different applications can be performed by using the calibrated image, and different effects can be achieved, and specific implementation effects can be referred to the related contents of the application scenarios 1 and 2 in the above system embodiment, which are not described herein again.

In this embodiment of the application, the implementation manner of providing the target image to the processing unit by the image acquisition module may be: the image acquisition module sends the target image to the processing unit, or the processing unit reads the target image from the image acquisition module. Correspondingly, if the image acquisition module stores the target image locally, the processing module reads the target image from the image acquisition module.

Similarly, the implementation manner of providing the control time to the processing unit by the control unit may be: the image acquisition module sends the control time to the processing unit, or the processing unit reads the control time from the control unit. Accordingly, if the control unit stores the control time locally, the processing unit reads the control time from the control unit.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps in the latency measurement method performed by the processing unit.

Correspondingly, the embodiment of the application also provides the computing equipment. Fig. 4b is a schematic structural diagram of a computing device according to an embodiment of the present application. As shown in fig. 4b, the computing device includes: a memory 40a and a processor 40 b.

In the present embodiment, the processor 40b is coupled to the memory 40a for executing a computer program for: acquiring a target image provided by image acquisition equipment, wherein the target image is an image of a reference object acquired by an image acquisition module in a specified state and carries a timestamp; acquiring control time provided by control equipment, wherein the control time is the time for controlling a reference object to be adjusted to a specified state by a control module; calculating a time delay parameter according to the time stamp and the control time of the target image, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition equipment; wherein the reference object has at least two states that are perceivable by the image acquisition device, and the control device is clock synchronized with the image acquisition device.

In an alternative embodiment, the processor 40b, when acquiring the target image provided by the image acquisition device, is specifically configured to: and identifying the multi-frame target image from the image provided by the image acquisition module, and determining the control time corresponding to the multi-frame target image provided by the control unit.

Correspondingly, when the processor 40b calculates the delay parameter, it is specifically configured to: and calculating the average time delay between the time stamp of the multi-frame target image and the corresponding control time, and taking the average time delay as a time delay parameter.

In another alternative embodiment, the processor 40b is further configured to: storing the time delay parameter; and/or calibrating the time stamp of the image subsequently acquired by the image acquisition module by using the time delay parameter.

In some alternative embodiments, as shown in fig. 4b, the computing device may further include: communication component 40c, power component 40d, display 40e, audio component 40f, and the like. Only some of the components are shown schematically in fig. 4b, and it is not meant that the computing device must contain all of the components shown in fig. 4b, nor that the computing device can only include the components shown in fig. 4 b.

In the implementation, the control equipment and the image acquisition equipment are synchronous in clock, and the image acquisition equipment sends a synchronous signal to the control equipment when acquiring the image of the reference object and adds a timestamp to the acquired image; the control device determines a control time at which the reference object is controlled to adjust its state to a specified state, based on the synchronization signal. Further, the computing device provided in this embodiment may calculate the time delay parameter according to the time stamp of the target image in the designated state of the reference object acquired by the image acquisition device and the control time corresponding to the target image, so that the time delay parameter may be subsequently used to calibrate the time stamp of the image acquired by the image acquisition device, which is beneficial to improving the time stamp precision of the image acquired by the image acquisition device, and is further beneficial to improving the effect of subsequent processing based on the image, such as improving the positioning precision based on the images.

Fig. 5 is a schematic structural diagram of another computing device according to an embodiment of the present application. As shown in fig. 5, the computing device includes: at least one memory 50a, at least one processor 50b, a vision sensor 50c, and a signal generating component 50 d. The visual sensor 50c may be a camera, a laser sensor, an infrared sensor, etc., but is not limited thereto.

In this embodiment, the computing device may be a robot, a smart phone, an unmanned aerial vehicle, or an unmanned vehicle, or may be other physical devices installed with a vision sensor, such as a vehicle, a bus, or other transportation means.

In the present embodiment, the at least one memory 50a is used for storing computer programs.

The at least one processor 50b is coupled to the at least one memory 50a for executing computer programs for: in the process of acquiring the image of the reference object by the vision sensor 50c, sending a synchronization signal to the control device through the signal generation component 50d, so that the control device controls the reference object to adjust to a specified state at a corresponding control time based on the synchronization signal, wherein the image acquired by the acquisition sensor includes a target image when the reference object is in the specified state; calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the visual sensor; wherein the reference object has at least two states that are perceivable by the vision sensor, and the control device is clock synchronized with the computing device.

Alternatively, the signal generating component may be an optical signal generator or an electrical signal generator. If the signal generating component is an optical signal generator, the synchronous signal is an optical signal; if the signal generating component is an electric signal generator, the synchronous signal is an electric signal.

In an optional embodiment, the computing device further comprises: a signal receiving component 50d and a communication component 50 e. The at least one processor 50b is further configured to: receiving the time service pulse provided by the time service equipment through the signal receiving component 50d, and receiving the time service time corresponding to the time service pulse provided by the time service equipment through the communication component 50 e; and synchronizing the local time and the time service equipment according to the time service pulse and the time service time. The signal receiving unit 50d may be an optical pulse signal receiving unit or an electrical pulse signal receiving unit, depending on the implementation form of the timing pulse.

In another optional embodiment, the at least one processor 50b comprises: an image processing unit 50b1 and other processing units 50b2 corresponding to the vision sensors. The image processing unit 50b1 is specifically configured to: the vision sensor 50c is controlled to periodically acquire images of the reference object according to the set image acquisition frequency, and each time the vision sensor 50c acquires an image of the reference object, a pulse signal is sent to the control device as a synchronization signal and a time stamp is added to each acquired image through the signal generation component 50d, and the time-stamped image is supplied to the other processing unit 50b 2.

In yet another alternative embodiment, the other processing unit 50b2 is specifically configured to: recognizing a plurality of frame target images from the images supplied from the image processing unit 50b1 and determining a control time corresponding to the plurality of frame target images supplied from the control device; and calculating the average time delay between the time stamp of the multi-frame target image and the corresponding control time, and taking the average time delay as a time delay parameter.

In yet another alternative embodiment, the other processing unit 50b2 is further configured to: storing the time delay parameter; and/or to calibrate the time stamps of subsequently acquired images of the vision sensor 50c using the time delay parameter.

In some optional embodiments, the computing device may further comprise: the control device and the reference object (not shown in fig. 5) described above. Further, as shown in fig. 5, the computing device may further include: power components 50f, display 50g, audio components 50h, and the like. Only some of the components are shown schematically in fig. 5, and it is not meant that the computing device must include all of the components shown in fig. 5, nor that the computing device can include only the components shown in fig. 5.

The computing equipment provided by the embodiment is synchronous with the clock of the control equipment, sends a synchronous signal to the control equipment when acquiring the image of the reference object, and adds a timestamp to the acquired image; the control device determines a control time at which the reference object is controlled to adjust its state to a specified state, based on the synchronization signal. Further, the computing device provided in this embodiment may calculate the time delay parameter according to the time stamp of the target image of the acquired reference object in the designated state and the control time corresponding to the target image, so that the time delay parameter may be subsequently used to calibrate the time stamp of the image acquired by the computing device, which is beneficial to improving the time stamp accuracy of the image acquired by the computing device, and is further beneficial to improving the effect of subsequent processing based on the image, such as improving the positioning accuracy based on the images.

Fig. 6a is a schematic flowchart of another delay measurement method according to an embodiment of the present application. As shown in fig. 6a, the method comprises:

601. and controlling the reference object to be adjusted to a specified state at the control time in the process of acquiring the image of the reference object.

602. And adding a time stamp to the acquired image, wherein the acquired image comprises the target image when the reference object is in the specified state.

603. And calculating a time delay parameter according to the time stamp and the control time of the target image, wherein the time delay parameter is used for calibrating the time stamp of the subsequently acquired image.

In this embodiment, in the process of acquiring an image of a reference object, the reference object is controlled at a control time to adjust the state of the reference object to a specified state, and since the time for acquiring a target image of the reference object in the specified state and the control time corresponding to the frame image are at the same time, a time delay parameter can be calculated according to the timestamp of the target image and the control time corresponding to the target image, so that the time delay parameter can be subsequently used to calibrate the timestamp of the image acquired by the image acquisition module, which is beneficial to improving the timestamp accuracy of the image acquired by the image acquisition module, and is further beneficial to improving the effect of subsequent processing based on the image, such as improving the positioning accuracy based on the images.

In an alternative embodiment, an image acquisition frequency may be set, and images of the reference object may be periodically acquired at the set image sampling frequency, and the reference object may be controlled to be adjusted to a designated state at a plurality of control times, each control time corresponding to one image sampling time, during the acquisition of the images of the reference object at the set image acquisition frequency. Wherein, each control time corresponds to one image sampling time, which means that: one control time is determined for each acquisition of N frames of images of the reference object. Wherein N is an integer greater than or equal to 2. Alternatively, the number of frames of the acquired image may be counted, and each time N is counted, the zero clearing may be performed, and the time when N is counted each time may be used as one control time.

Accordingly, an alternative implementation of step 603 is: identifying multi-frame target images from the acquired images of the reference object, and determining respective corresponding control time of the multi-frame target images; and calculating the average time delay between the time stamp of the multi-frame target image and the corresponding control time, and taking the average time delay as a time delay parameter.

The method embodiment shown in fig. 6a may be implemented by the aforementioned delay measurement system shown in fig. 1a, fig. 1b, or fig. 1c, or may be implemented by the following computing device shown in fig. 6b, which is not limited thereto.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the method shown in fig. 6a and the alternative embodiments described above.

Correspondingly, the embodiment of the application also provides the computing equipment. Fig. 6b is a schematic structural diagram of another computing device provided in the embodiment of the present application. As shown in fig. 6b, the computing device includes: at least one memory 60a, at least one processor 60b, and a vision sensor 60 c.

The visual sensor 60c may be a camera, a laser sensor, an infrared sensor, etc., but is not limited thereto.

In the present embodiment, the at least one memory 60a is used for storing computer programs.

The at least one processor 60b is coupled to the at least one memory 60a for executing computer programs for: in the process of controlling the vision sensor 60c to acquire the image of the reference object, controlling the reference object to be adjusted to a specified state at the control time; adding a time stamp to an image acquired by the vision sensor 60c, wherein the acquired image includes a target image when the reference object is in a specified state; a time delay parameter is calculated from the time stamp and the control time of the target image, and the time delay parameter is used for calibrating the time stamp of the image subsequently acquired by the vision sensor 60 c.

In an optional embodiment, the computing device further comprises: a signal receiving component 60d and a communication component 60 e. The at least one processor 60b is further configured to: receiving the time service pulse provided by the time service equipment through the signal receiving component 50d, and receiving the time service time corresponding to the time service pulse provided by the time service equipment through the communication component 50 e; and synchronizing the local time and the time service equipment according to the time service pulse and the time service time. The signal receiving unit 60d may be an optical pulse signal receiving unit or an electrical pulse signal receiving unit, depending on the implementation form of the timing pulse.

In another optional embodiment, when the processor 60b controls the reference object to adjust to the designated state at the control time, the processor is specifically configured to: in the process of controlling the vision sensor 60b to acquire the image of the reference object at the set image acquisition frequency, the reference object is controlled to be adjusted to a specified state at a plurality of control times, each of which corresponds to one image sampling timing.

In another alternative embodiment, the at least one processor 60b includes: an image processing unit 60b1 and other processing units 60b2 corresponding to the vision sensor 60 c. The image processing unit 60b1 is specifically configured to: the vision sensor 60c is controlled to periodically acquire images of the reference object at the set image acquisition frequency, and a time stamp is added to each acquired image each time the vision sensor 60c acquires an image of the reference object, and the time-stamped image is supplied to the other processing unit 60b 2.

In yet another alternative embodiment, the other processing unit 60b2 is specifically configured to: recognizing a plurality of frame target images from the images supplied from the image processing unit 60b1, and determining a control time corresponding to the plurality of frame target images supplied from the control device; and calculating the average time delay between the time stamp of the multi-frame target image and the corresponding control time, and taking the average time delay as a time delay parameter.

In yet another alternative embodiment, the other processing unit 60b2 is further configured to: storing the time delay parameter; and/or to calibrate the time stamps of subsequently acquired images of the vision sensor 60c using the time delay parameter.

In some optional embodiments, the computing device may further comprise: the control unit, reference object or timing module (not shown in fig. 6 b) described above. Further, as shown in fig. 6b, the computing device may further include: power components 60f, display 60g, audio components 60h, and the like. Only some of the components are shown schematically in fig. 6b, and it is not meant that the computing device must include all of the components shown in fig. 6b, nor that the computing device can include only the components shown in fig. 6 b.

In the process of acquiring the image of the reference object, the computing device provided in this embodiment controls the reference object to adjust the state of the reference object to the designated state at the control time, and since the time of acquiring the target image of the reference object in the designated state and the control time corresponding to the frame image are at the same time, a time delay parameter can be calculated according to the timestamp of the image target image and the control time corresponding to the target image, so that the time delay parameter can be subsequently used to calibrate the timestamp of the image acquired by the image acquisition module, which is beneficial to improving the timestamp accuracy of the image acquired by the image acquisition module, and is further beneficial to improving the effect of subsequent processing based on the image, such as improving the positioning accuracy based on the images.

In embodiments of the present application, the memory is used to store computer programs and may be configured to store other various data to support operations on the device on which it is located. Wherein the processor may execute a computer program stored in the memory to implement the corresponding control logic. The memory may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

In embodiments of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it is located and other devices. The device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may also be implemented based on Near Field Communication (NFC) modules, Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In various embodiments of the present application, the display may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

In embodiments of the present application, a power component is configured to provide power to various components of the device in which it is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

In various embodiments of the present application, the audio component may be configured to output and/or input audio signals. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals. For example, for devices with language interaction functionality, voice interaction with a user may be enabled through an audio component, and so forth.

It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of

steps

201 and 202 may be device a; for another example, the execution subject of step 201 may be device a, and the execution subject of step 202 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 401, 402, etc., are merely used to distinguish various operations, and the sequence numbers themselves do not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data computing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data computing device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data computing device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data computing device to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media (transmyedia) such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A delay measurement system, comprising: the device comprises a control module, a reference object and an image acquisition module; the reference object has at least two states that are perceivable by the image acquisition module; the control module is synchronous with the image acquisition module in clock;

the image acquisition module is used for periodically acquiring the image of the reference object, sending a pulse signal to the control module when the image of the reference object is acquired each time, adding a time stamp to the acquired image and providing the image with the time stamp to the control module; wherein the acquired image comprises a target image when the reference object is in a specified state;

the control module is used for circularly counting the pulse signals sent by the image acquisition module, controlling the reference object to be adjusted to a specified state when N pulse signals are counted, and taking the time of counting the N pulse signals each time as control time; and calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module, and N is not less than 2 and is an integer.

2. The system of claim 1, wherein the control module comprises: a control unit and a processing unit; the control unit is synchronous with the clock of the image acquisition module;

the control unit is used for circularly counting the pulse signals, controlling the reference object to be adjusted to a specified state every time when N pulse signals are counted, and providing the time counted to N pulse signals every time as control time to the processing unit;

and the processing unit is used for calculating a time delay parameter according to the time stamp of the target image provided by the image acquisition module and the control time.

3. The system of claim 2, wherein the image acquisition module is specifically configured to:

according to the set image acquisition frequency, sending a pulse signal to the control unit as a synchronous signal when acquiring the image of the reference object each time; and adding a time stamp to each acquired image and providing the time-stamped image to the processing unit.

4. The system according to claim 3, wherein the control unit is specifically configured to:

and sending a state control signal to the reference object every time when N pulse signals are counted so as to control the reference object to be adjusted to a specified state.

5. The system of claim 3, wherein the processing unit, when calculating the delay parameter, is specifically configured to:

identifying a plurality of frames of target images from the images provided by the image acquisition module, and determining control time corresponding to each frame of target image provided by the control unit;

and calculating the average time delay between the time stamp of the multi-frame target image and the corresponding control time, and taking the average time delay as the time delay parameter.

6. The system according to claim 4, wherein the reference object is a controlled light source having an on state and an off state, and the specified state is the on state or the off state.

7. The system of claim 2, further comprising:

and the time service module is used for providing time service pulses and time service time corresponding to the time service pulses to the control unit and the image acquisition module so that the control unit and the image acquisition module can synchronize a local clock with the time service module respectively.

8. The system of claim 7, wherein the time service module is a GPS time service device, a Beidou time service device or a standard clock source.

9. The system of any of claims 2-8, wherein the processing unit is further configured to perform at least one of the following:

storing the time delay parameter;

and calibrating the time stamp of the image subsequently acquired by the image acquisition module by using the time delay parameter.

10. The system of any of claims 2-8, wherein the image acquisition module comprises: an image processing unit and a camera;

the camera is used for acquiring the image of the reference object according to a set image acquisition frequency under the control of the image processing unit and outputting the acquired image to the image processing unit each time;

the image processing unit is used for adding a time stamp to the image collected by the camera and providing the image added with the time stamp to the processing unit.

11. The system of any one of claims 2-8, wherein the control unit is a logic analyzer, an oscilloscope, or a signal generator.

12. The system of any one of claims 2-8, the control unit, the reference object, the image acquisition module, and the processing unit are deployed discretely; or, the control unit, the reference object, the image acquisition module and the processing unit are integrated in the same physical device, and the reference object and the image acquisition module are arranged oppositely.

13. The system of claim 12,

if the control unit, the reference object, the image acquisition module and the processing unit are dispersed in a plurality of physical devices, the control unit and the reference object are independently arranged, and the image acquisition module and the processing unit are positioned in the same physical device.

14. The system of claim 13, wherein the physical device on which the processing unit and the image capture module reside is a robot.

15. A method for measuring delay, comprising:

in the process of periodically acquiring the images of the reference object, taking the image time of acquiring N frames of reference objects as a control time, and controlling the reference objects to be adjusted to a specified state at the control time, wherein N is an integer greater than or equal to 2;

adding a time stamp to a collected image, wherein the collected image comprises a target image when the reference object is in a specified state;

and calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the subsequently acquired image.

16. The method of claim 15, wherein controlling the reference object to adjust to a specified state at a control time during the acquisition of the image of the reference object comprises:

in the process of acquiring the images of the reference object according to the set image acquisition frequency, controlling the reference object to be adjusted to a specified state at a plurality of control times, wherein each control time corresponds to one image sampling time and refers to counting the number of frames of the acquired images, and taking the time when the number of frames reaches N every time as one control time;

the calculating a time delay parameter according to the timestamp of the target image and the control time includes:

identifying a plurality of frames of target images from the acquired images of the reference object, and determining respective corresponding control time of the plurality of frames of target images;

17. A method for measuring delay, comprising:

the image acquisition module periodically acquires an image of a reference object, sends a pulse signal to the control module when acquiring the image of the reference object each time, so that the control module performs cycle counting on the pulse signal, controls the reference object to be adjusted to a specified state when counting N pulse signals each time, and takes the time counted to N pulse signals as control time, wherein N is an integer greater than or equal to 2;

providing the target image added with the timestamp to a control module so that the control module can calculate a time delay parameter according to the timestamp of the target image and the control time, wherein the time delay parameter is used for calibrating the timestamp of an image subsequently acquired by the image acquisition module;

the control module is synchronous with the image acquisition module in clock.

18. A method for measuring delay, comprising:

the control module receives a pulse signal sent by the image acquisition module when the image acquisition module acquires the image of the reference object each time in the process of periodically acquiring the image of the reference object;

the pulse signals sent by the image acquisition module are counted circularly, and when N pulse signals are counted, the reference object is controlled to be adjusted to a specified state, so that the image acquisition module adds a timestamp to the acquired target image when the reference object is in the specified state; and counting the time to the N pulse signals as a control time; n is not less than 2 and is an integer;

calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module; the control module is synchronous with the image acquisition module in clock.

19. The method of claim 18, wherein calculating a time delay parameter from the time stamp of the target image and the control time comprises:

and providing the control time to a processing unit in the equipment where the image acquisition module is located, so that the processing unit can calculate a time delay parameter according to the timestamp of the target image provided by the image acquisition module and the control time.

20. The method according to claim 18 or 19, wherein controlling the reference object to adjust to a specified state each time N pulse signals are counted comprises:

21. A method for measuring delay, comprising:

the processing unit acquires a target image acquired by an image acquisition module in the process of periodically acquiring an image of a reference object, wherein the target image is an image of the reference object in a specified state and carries a timestamp;

acquiring control time provided by a control module, wherein the control time is the time for the control module to circularly count pulse signals sent by an image acquisition module each time when the image acquisition module acquires an image of a reference object, and the time for counting N pulse signals each time is the time for controlling the reference object to be adjusted to a specified state;

calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition module;

the control module is synchronous with the image acquisition module in clock.

22. A computing device, comprising: at least one memory, at least one processor, and a vision sensor; wherein the at least one memory is to store a computer program;

the at least one processor is coupled to the at least one memory for executing the computer program for:

in the process of controlling the vision sensor to periodically collect the images of the reference object, taking the image time of collecting N frames of reference object as a control time, and controlling the reference object to be adjusted to a specified state at the control time, wherein N is an integer greater than or equal to 2;

adding a time stamp to an image acquired by the vision sensor, wherein the acquired image comprises a target image when the reference object is in a specified state;

and calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the vision sensor subsequently.

23. An image acquisition apparatus, characterized by comprising: a memory, a processor, a vision sensor, and a signal generation component; wherein the memory is used for storing a computer program;

the vision sensor is used for acquiring an image of a reference object;

the processor is coupled to the memory for executing the computer program for:

in the process of periodically acquiring the image of the reference object by the vision sensor, when the image of the reference object is acquired each time, a pulse signal is sent to a control device through the signal generation component, so that the control device can count the pulse signal circularly, the reference object is controlled to be adjusted to a specified state when N pulse signals are counted, the time counted to N pulse signals is taken as control time, and N is an integer greater than or equal to 2;

time stamping an image captured by the vision sensor, the captured image comprising: a target image when the reference object is in a specified state;

providing the target image added with the timestamp to a control device, so that the control device can calculate a time delay parameter according to the timestamp of the target image and the control time, wherein the time delay parameter is used for calibrating the timestamp of the image provided by the image acquisition device; wherein the control device is clock synchronized with the image acquisition device.

24. The apparatus of claim 23, further comprising: a communication component and a signal receiving component; the processor is further configured to:

receiving a time service pulse provided by time service equipment through the signal receiving assembly, and receiving time service time corresponding to the time service pulse provided by the time service equipment through the communication assembly;

and synchronizing local time and the time service equipment according to the time service pulse and the time service time.

25. A control apparatus, characterized by comprising: a memory, a processor, and a first signal receiving component;

wherein the memory is for storing a computer program; the processor is coupled to the memory for executing the computer program for:

receiving, by the first signal receiving assembly, a pulse signal sent by the image acquisition device each time an image of the reference object is acquired in a process of periodically acquiring an image of the reference object;

the pulse signals sent by the image acquisition module are counted circularly, and when N pulse signals are counted, the reference object is controlled to be adjusted to a specified state, so that the image acquisition equipment acquires a target image of the reference object in the specified state, adds a timestamp to the target image and provides the target image to the computing equipment; and counting the time to the N pulse signals as a control time; n is not less than 2 and is an integer;

providing the control time to the computing device, so that the computing device can calculate a time delay parameter according to the timestamp of the target image and the control time, wherein the time delay parameter is used for calibrating the timestamp of the image acquired by the image acquisition device;

wherein the control device is clock synchronized with the image acquisition device.

26. The device of claim 25, further comprising a signal generation component, wherein the processor, when controlling the reference object to adjust to a specified state, is specifically configured to:

and sending a state control signal to the reference object through the signal generating component every time when N pulse signals are counted so as to control the reference object to be adjusted to a specified state.

27. The apparatus of claim 25 or 26, further comprising: a communication component and a second signal receiving component; the processor is further configured to:

receiving a time service pulse provided by time service equipment through the second signal receiving component, and receiving time service time corresponding to the time service pulse provided by the time service equipment through the communication component;

and synchronizing local time with the time service equipment according to the time service pulse and the time service time.

28. A computing device, comprising: at least one memory and at least one processor, wherein the at least one memory is configured to store a computer program; the at least one processor is coupled to the at least one memory for executing the computer program for:

acquiring a target image provided by an image acquisition device in the process of periodically acquiring an image of a reference object, wherein the target image is an image of the reference object acquired by an image acquisition module in a specified state and carries a timestamp;

acquiring control time provided by control equipment, wherein the control time is the time for the control equipment to circularly count pulse signals sent by the image acquisition equipment each time when acquiring an image of a reference object, and the time for counting N pulse signals each time is the time for controlling the reference object to be adjusted to a specified state;

calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the image acquisition equipment;

wherein the reference object has at least two states that are perceivable by the image acquisition device, and the control device is clock synchronized with the image acquisition device.

29. A computing device, comprising: at least one memory, at least one processor, a visual sensor, and a signal generating component;

the at least one memory is for storing a computer program; the at least one processor is coupled to the at least one memory for executing the computer program for:

in the process that the vision sensor periodically collects the image of the reference object, the signal generating assembly sends a pulse signal to the control equipment every time the image of the reference object is collected, so that the control equipment can count the pulse signal circularly, the control reference object is adjusted to a specified state every time N pulse signals are counted, the time counted to the N pulse signals is used as control time, N is not less than 2 and is an integer, and the image collected by the vision sensor comprises a target image when the reference object is in the specified state; calculating a time delay parameter according to the time stamp of the target image and the control time, wherein the time delay parameter is used for calibrating the time stamp of the image acquired by the vision sensor; wherein the control device is clock synchronized with the computing device.

30. A computer-readable storage medium having stored thereon computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the method of any one of claims 15-21.