CN113496545B

CN113496545B - Data processing system, method, sensor, mobile acquisition backpack and equipment

Info

Publication number: CN113496545B
Application number: CN202010270486.6A
Authority: CN
Inventors: 常孟芝; 张鹏
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-04-08
Filing date: 2020-04-08
Publication date: 2022-05-27
Anticipated expiration: 2040-04-08
Also published as: CN113496545A; WO2021204144A1

Abstract

The embodiment of the application provides a data processing system, a data processing method, a sensor, a mobile acquisition backpack and equipment. According to the technical scheme, a reference pulse signal, a first trigger pulse signal and reference clock information are generated by utilizing a time signal of a clock crystal oscillator of a clock synchronization device; the reference pulse signal and the first trigger pulse signal meet the requirement of clock synchronization; sending the related reference pulse signal and reference clock information as time service information to a radar; triggering an image sensor to acquire image data according to a first trigger pulse signal; the radar can adjust the clock based on the time service information; the time stamp corresponding to the image data acquired by the image sensor can be determined by utilizing the signal relation between the reference pulse signal and the first trigger pulse signal; because the scheme is based on signals generated by the same clock crystal oscillator, when the crystal oscillator has errors, the signals can be reflected in the time stamps of the sensors, so that the synchronization on the clock can be still maintained, and the data clock synchronization is high.

Description

Data processing system, method, sensor, mobile acquisition backpack and equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data processing system, a data processing method, a sensor, a mobile collection backpack, and a device.

Background

At present, various sensors are installed in exuberant three-dimensional scanners, autopilots, unmanned planes and the like to acquire data of surrounding complex environments. However, because the data acquisition mode and the timing mode of each sensor are different, the acquired data are difficult to realize high unification on time marks, which affects the subsequent data fusion processing effect and causes the problems of inaccurate three-dimensional modeling, inaccurate positioning and the like.

Therefore, how to synchronize the clocks of the data collected by the multiple sensors is a problem that needs to be solved urgently by those skilled in the art.

Disclosure of Invention

In view of the above, the present application is proposed to provide a data processing system, a method, a sensor, a mobile acquisition backpack and a device that solve the above problems, or at least partially solve the above problems.

Thus, in one embodiment of the present application, a data processing system is provided. The system comprises:

the clock synchronization device is used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to a time signal of the clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; triggering an image sensor to acquire image data according to the first trigger pulse signal;

the radar is used for responding to the received time service information and adjusting the clock according to the time service information; collecting radar data of an object area, and obtaining a timestamp corresponding to the radar data based on the adjusted clock;

an image sensor for acquiring image data of the object region in response to the received first trigger pulse signal;

the processing device is used for receiving the radar data which is sent by the radar and carries a time stamp; receiving the image data sent by the image sensor; calculating a timestamp corresponding to the image data according to the signal relation between the reference pulse signal and the first trigger pulse signal; and performing three-dimensional modeling on the object region based on the radar data and the image data with the same timestamp.

In another embodiment of the present application, a data processing system is provided. The system comprises:

an image sensor for acquiring image data of the object region in response to the received first trigger pulse signal; obtaining a timestamp corresponding to the image data by using a signal relation between the reference pulse signal and the first trigger pulse signal;

the processing equipment is used for receiving the radar data and the corresponding timestamp sent by the radar, and the image data and the corresponding timestamp sent by the image sensor; and performing three-dimensional modeling on the object region based on the radar data and the image data with the same timestamp.

In yet another embodiment of the present application, a data processing method is provided. The method comprises the following steps:

generating a reference pulse signal, a first trigger pulse signal and reference clock information based on a time signal of a clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal satisfy a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal;

sending the reference pulse signal and the reference clock information as time service information to a first sensor so that the first sensor can adjust the clock according to the time service information, and obtaining a timestamp corresponding to first data collected by the first sensor based on the adjusted clock;

and triggering the second sensor to acquire second data according to the first trigger pulse signal so as to obtain a timestamp corresponding to the second data by using the signal relationship between the reference pulse signal and the first trigger pulse signal.

collecting data to obtain second data in response to the received trigger signal; wherein the trigger signal is generated based on a first trigger pulse signal satisfying clock synchronization with a reference pulse signal;

and obtaining a timestamp corresponding to the second data by using the signal relation between the reference pulse signal and the first trigger pulse signal.

In another embodiment of the present application, a data processing method is provided. The method comprises the following steps:

receiving second data sent by a second sensor;

acquiring locally recorded auxiliary information related to the second sensor; the auxiliary information contains a signal relation between a reference pulse signal and a first trigger pulse signal, and the reference pulse signal and the first trigger pulse signal are generated based on a time signal of the same clock crystal oscillator and meet the requirement of clock synchronization; the second sensor is triggered to acquire the second data under the action of the first trigger pulse signal;

and calculating the time stamp corresponding to the second data by using the auxiliary information.

In an embodiment of the present application, an electronic device is provided. The apparatus, comprising: a memory, a processor, and a communications component, wherein,

the memory is used for storing programs;

the processor, coupled with the memory, to execute the program stored in the memory to:

sending the reference pulse signal and the reference clock information as time service information to a first sensor through the communication assembly, so that the first sensor can adjust the clock according to the time service information, and obtaining a timestamp corresponding to first data collected by the first sensor based on the adjusted clock;

and sending the first trigger pulse signal to the second sensor through the communication assembly so as to trigger the second sensor to acquire second data, so that a timestamp corresponding to the second data is obtained by using the signal relationship between the reference pulse signal and the first trigger pulse signal.

In another embodiment of the present application, a sensor is provided. The sensor, comprising: a memory, a processor, and a sensing module, wherein,

the memory is used for storing programs;

the processor, coupled to the memory, to execute the program stored in the memory to:

responding to the received trigger signal, controlling the induction module to work so as to acquire data and obtain second data; wherein the trigger signal is generated based on a first trigger pulse signal satisfying clock synchronization with a reference pulse signal;

and obtaining the time stamp corresponding to the second data by using the signal relation between the reference pulse signal and the first trigger pulse signal.

In another embodiment of the present application, an electronic device is provided. The apparatus, comprising: a memory, a processor, and a communications component, wherein,

the memory is used for storing programs;

receiving second data sent by a second sensor through the communication component;

In one embodiment of the present application, a mobile acquisition backpack is also provided. This remove collection knapsack includes:

a backpack;

the radar is arranged on the backpack and used for adjusting the clock according to the received time service information; collecting radar data of an object area, and obtaining a timestamp corresponding to the radar data based on the adjusted clock;

the image sensor is arranged on the backpack and used for responding to the received first trigger pulse signal and acquiring image data of the object area;

the clock synchronization device is arranged on the backpack and used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to a time signal of a clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; and triggering the image sensor to acquire image data according to the first trigger pulse signal.

In yet another embodiment of the present application, an autonomous mobile acquisition device is also provided. The autonomous mobile device includes:

an apparatus body;

the radar is arranged on the equipment body and used for adjusting the clock according to the received time service information; collecting radar data of an object area, and obtaining a timestamp corresponding to the radar data based on the adjusted clock;

the image sensor is arranged on the equipment body and used for responding to the received first trigger pulse signal and acquiring image data of the object area;

the clock synchronization device is arranged on the equipment body and used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to a time signal of a clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; and triggering the image sensor to acquire image data according to the first trigger pulse signal.

In a technical scheme that this application embodiment provided, the timestamp that multiple sensor data collection corresponds is controlled by the reference pulse signal, reference clock information and the first trigger pulse signal that the time signal of same clock crystal oscillator generated completely under the condition that the crystal oscillator has the error, also the homoenergetic reflects in the timestamp of multiple sensor, still can effectively guarantee multiple sensor clock synchronization, avoid the appearance of accumulative total error, and low price, small, low power dissipation.

In another technical scheme provided by the embodiment of the application, second data is acquired in response to a first trigger pulse signal, and the calculation of the timestamp corresponding to the second data is only related to the signal relationship between a reference pulse signal and the first trigger pulse signal, so that the calculation of the timestamp is not influenced by data transmission delay and system call delay, and the accuracy of the timestamp is ensured.

In another technical solution provided in the embodiment of the present application, the locally recorded auxiliary information related to the second sensor is used to calculate the timestamp corresponding to the second data, so that the calculation of the timestamp is not affected by data transmission delay, and the real-time requirement on the system is reduced.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required to be utilized in the description of the embodiments or the prior art are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained according to the drawings without creative efforts for those skilled in the art.

FIG. 1a is a schematic diagram of a data processing system according to an embodiment of the present application;

FIG. 1b is a block diagram of a data processing system according to an embodiment of the present application;

FIG. 2a is a schematic diagram of a data processing system according to another embodiment of the present application;

FIG. 2b is a block diagram representation of a data processing system according to another embodiment of the present application;

fig. 2c is a schematic structural diagram of a mobile collecting backpack according to an embodiment of the present application;

fig. 2d is a schematic structural diagram of a first implementation form of an autonomous mobile acquisition device according to an embodiment of the present application;

fig. 2e is a schematic structural diagram of a second implementation form of the autonomous mobile acquisition device according to an embodiment of the present application;

fig. 2f is a schematic structural diagram of a third implementation form of the autonomous mobile acquisition device according to an embodiment of the present application;

fig. 3 is a schematic flowchart of a data processing method according to an embodiment of the present application;

FIG. 4 is a timing chart of a reference pulse signal and reference clock information that can be received by the laser radar;

FIG. 5 is a timing diagram illustrating synchronization of a PPS signal received by a radar, a second trigger signal received by an IMU, and a first trigger signal received by an image sensor according to an embodiment of the present disclosure;

fig. 6 is a corresponding relationship between a first trigger pulse signal and output collected data according to an embodiment of the present application;

fig. 7 is a schematic flowchart of a data processing method according to another embodiment of the present application;

FIG. 8 is a schematic flow chart diagram illustrating a data processing method according to another embodiment of the present application;

fig. 9 is a block diagram of a data processing apparatus according to an embodiment of the present application;

fig. 10 is a block diagram of a data processing apparatus according to another embodiment of the present application;

fig. 11 is a block diagram of a data processing apparatus according to another embodiment of the present application;

fig. 12 is a block diagram of an electronic device according to an embodiment of the present application;

fig. 13 is a block diagram of a sensor according to an embodiment of the present application.

Detailed Description

At present, radars (such as laser radar), image sensors (such as cameras), IMUs and the like are commonly used sensors in the technical fields of three-dimensional scanning equipment, automatic driving, unmanned aerial vehicles and the like; wherein, the radar, such as laser radar, is a radar system which detects the position, speed and other characteristic quantities of a target by emitting laser beams, and can be used for detecting the outline information of the environment and drawing a surrounding environment model; the image sensor is used for acquiring pictures of the surrounding environment; an IMU (i.e., inertial measurement unit) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object. However, when the data collected by the above sensors are fused, the data collection frequencies corresponding to different sensors are different, such as: the radar is a slow sensor, and the acquisition frequency is less than 20 Hz; the frame rate of the high-definition image sensor is related to the resolution, and the frame rate is generally less than 30 Hz; the IMU acquisition frequency is relatively fast, which is generally above 100 Hz; this will cause uncertainty in the data acquisition start time; moreover, the inconsistency of the crystal oscillation error, the transmission delay, the frequency jitter and the like of different sensors can also cause that the timestamps corresponding to the data collected by the different sensors are difficult to be consistent. Therefore, if the data collected by different sensors are directly used, the accuracy of the final result is difficult to ensure. For example: for three-dimensional scanning equipment in motion working, calculation errors can be caused due to inaccurate time of a slow sensor (such as a laser radar), so that the problems of inaccurate three-dimensional model construction, inaccurate positioning and the like can be caused.

In the prior art, a method for clock synchronization of multiple sensors mainly includes: the GPS time service is a time service provided by a GPS, and the GPS time service is that standard time information is acquired through a GPS satellite, and the time information is transmitted to various sensors needing the time information in a system through various interfaces after being decoded, so that the various sensors achieve clock synchronization. However, the GPS time service is utilized to synchronize the clock information of various sensors, so that the GPS signals are required to be ensured to exist all the time, and once the GPS signals are in an indoor signal weak environment, the clock information is lost, so that synchronization failure is caused; secondly, PTP (Precision Time Protocol), which is a Precision clock synchronization Protocol of a network measurement and control system, is mainly applied to frequency and Time synchronization between ethernet terminal devices, and a system implemented by adopting a PTP Time service mode can only reach microsecond-level synchronization Precision, which cannot meet the requirements of a positioning system; and thirdly, synchronizing other sensors based on time information corresponding to data of a certain sensor, and giving time information corresponding to the analyzed IMU data to the laser radar as unified time, but the problems exist that a hardware synchronization circuit receives the analyzed IMU data, transmission delay and analysis data delay exist, the sampling frequency of the IMU is not strict necessarily, the IMU is influenced by the precision of a crystal oscillator of the IMU, and the precision of clock synchronization is influenced seriously.

Therefore, the embodiments of the present application provide a technical solution that can effectively solve or optimize the problems of the prior art. In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

In some of the flows described in the specification, claims, and above-described figures of the present application, a number of operations are included that occur in a particular order, which operations may be performed out of order or in parallel as they occur herein. The sequence numbers of the operations, e.g., 101, 102, etc., are used merely to distinguish between the various operations, and do not represent any order of execution per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different. In addition, the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

At present, a general software developer working environment is a non-real-time operating system such as Linux or windows, and if a plurality of sensors directly connected to the non-real-time operating system perform time base synchronization by using a local clock of the system (that is, time information corresponding to sensor acquisition data acquired by the non-real-time operating system is system clock time at this moment), the problem of uncontrollable synchronization errors caused by switching between threads, call delay and the like exists, and synchronization is easy to fail. Based on this, considering that general radars all support the GPS time service function in the NMEA + PPS mode, the inventor proposes, through research, an electronic device that simulates the GPS time service function based on a real-time operating system (such as MCU, FPGA, RTOS) to control the acquisition of sensor data and timestamp synchronization, that is, the electronic device simulates and generates a NMEA + PPS reference pulse signal sent by the GPS by using the real-time operating system, and generates a synchronous trigger pulse information according to the same PPS signal to trigger other sensors such as an IMU and an image sensor, thereby implementing the acquisition of various sensor data and clock synchronization. For specific operations of the electronic device for controlling the clock synchronization of the multiple sensors, reference may be made to the following embodiments, which are not described herein again.

Fig. 1a and 1b are block diagrams illustrating a data processing system according to an embodiment of the present application. As shown in fig. 1a, the data processing system includes: a clock synchronization device 101, a radar 102, an image sensor 103, and a processing device 104. Wherein the content of the first and second substances,

a clock synchronization device 101 for generating a reference pulse signal, a first trigger pulse signal, and reference clock information from a time signal of a clock oscillator; wherein the reference pulse signal and the first trigger pulse signal satisfy a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; triggering an image sensor to acquire image data according to the first trigger pulse signal;

the radar 102 is used for responding to the received time service information and adjusting the clock according to the time service information; collecting radar data of an object area 1, and obtaining a timestamp corresponding to the radar data based on the adjusted clock;

an image sensor 103 for acquiring image data of the object region 1 in response to the received first trigger pulse signal;

the processing device 104 is configured to receive the radar data sent by the radar and carrying a timestamp; receiving the image data sent by the image sensor; calculating a timestamp corresponding to the image data according to the signal relation between the reference pulse signal and the first trigger pulse signal; the object region 1 is three-dimensionally modeled based on the radar data and the image data of the same time stamp.

The clock synchronization device may specifically be: an MCU, an FPGA, or a device equipped with a Real Time Operating System (RTO). The MCU (Micro Control Unit), also called a single-chip microcomputer or a single-chip microcomputer, is a computer with a chip level formed by integrating a CPU, an RAM, a ROM, a timing counter and various I/O interfaces of the computer on one chip; the FPGA (field Programmable Gate array) is a product which is further developed on the basis of Programmable devices such as PAL, GAL and the like, is used as a semi-custom circuit in the field of Application Specific Integrated Circuits (ASIC), not only solves the defect of a custom circuit, but also overcomes the defect of limited Gate circuit number of the original Programmable device. The real-time operating system is an operating system which can receive and process the external events or data at a sufficiently high speed when the external events or data are generated, and the processing result can control the production process or make a quick response to the processing system within a specified time, schedule all available resources to complete real-time tasks and control all real-time tasks to run in a coordinated and consistent manner; providing timely response and high reliability are main features.

If the clock synchronization device is an MCU, it is necessary to configure the same trigger source for all timers inside the MCU (the number of timers corresponds to the number of pulse signals to be generated, or the number of timers corresponds to the number of sensors), and based on the same trigger source, all timers are triggered, so that the synchronous start of all timers is realized, i.e. the synchronous timing is performed, and the purpose of aligning the rising edge of the first trigger pulse signal with the upper edge of the reference pulse signal is achieved. In the example shown in fig. 1a, the MCU needs to generate two pulse signals, i.e., a reference pulse signal and a first trigger pulse signal; therefore, the number of timers inside the MCU is two. In the schematic diagram shown in fig. 1b, there are 3 sensors requiring clock synchronization, namely, a radar, an inertial measurement unit (IMU for short), and an image sensor; therefore, the number of timers inside the MCU is 3. Here, the same trigger source of all timers may be configured as the same external interrupt signal, and once the external interrupt signal is triggered, all timers receive the external interrupt signal at the same time to operate, so that alignment of the reference pulse signal and the first trigger pulse signal is ensured.

For another example, if the clock synchronization device is an FPGA, the FPGA needs to adopt a plurality of frequency dividers, the number of which is consistent with the number of sensors to be synchronized, for generating the reference pulse signal and the first trigger pulse signal respectively, and alignment of the reference pulse signal and the first trigger pulse signal can be ensured due to parallel real-time performance of the FPGA and characteristics of the same reference clock. As shown in fig. 1b, the sensors connected to the FPGA and requiring synchronization include a radar, an IMU, and an image sensor, and at this time, the FPGA needs to use 3 frequency dividers for generating a reference pulse signal, a first trigger pulse signal, and a second trigger pulse signal, respectively. The FPGA can ensure the alignment of three signals based on the characteristics of the FPGA.

Further, the data processing system provided in this embodiment may further include: an inertial measurement unit 105, as shown in fig. 1b, the inertial measurement unit 105, communicatively connected to the clock synchronization device 101, is configured to collect inertial data in response to the received second trigger signal. The clock synchronization device 101 is further configured to generate a second trigger pulse signal meeting a clock synchronization requirement with the reference pulse signal, and trigger the inertia measurement unit to acquire inertia data according to the second trigger pulse signal; the time stamp calculation unit is further configured to receive the inertial data sent by the inertial measurement unit, and calculate a time stamp corresponding to the inertial data by using a relationship between the reference pulse signal and the second trigger pulse signal; and sending the inertial data and the corresponding time stamp to the processing equipment. The processing device 104 is further configured to model the object region in three dimensions based on the radar data, the image data, and the inertial data at the same time stamp.

In the technical scheme provided by the embodiment, the timestamps corresponding to the data collected by the various sensors are completely controlled by the reference pulse signal, the reference clock information and the first trigger pulse signal generated according to the time signal of the same clock crystal oscillator, under the condition that the crystal oscillator has errors, the clock synchronization of the various sensors can be effectively ensured, the occurrence of accumulated errors is avoided, and the scheme is simple and convenient to implement and is not influenced by the external environment.

Further, the data processing system provided by this embodiment may further include an autonomous mobile device. The autonomous mobile device may be an unmanned aerial vehicle, an autonomous mobile robot, and the like, which is not particularly limited in this embodiment. The clock synchronization apparatus 101, the radar 102, the image sensor 103, and the inertial measurement unit 105 are all disposed on the autonomous mobile device.

In a specific implementation, the processing device may be a network-side service device, such as a server, a virtual server deployed on a server cluster, a cloud service center, and the like. The radar may be a lidar and the image sensor may be a camera. Among them, the camera may be a high definition color camera or the like.

Here, it should be noted that: in this embodiment, the inertial data collected by the inertial measurement unit 105 is calculated by the clock synchronization device with a corresponding timestamp, and then the inertial data and the corresponding timestamp are sent to the processing device 104 together, so the following considerations are taken into account: the data sampling frequency of the inertial measurement unit is high (relative to the image sensor), and the time stamp of the inertial data is determined by the clock synchronization device, so that the processing load of the processing equipment can be reduced; in addition, the clock synchronization device adopts an MCU, an FPGA and the like with good real-time performance, and the clock synchronization device can deal with data with high sampling frequency, is relatively stable and has good real-time performance. For image sensors, the data sampling frequency is much lower than that of an inertial lining unit, and the processing burden of determining the time stamp for the image data by the processing device can be borne.

In the above embodiment, the timestamp corresponding to the image data acquired by the image sensor is determined by the processing device; in fact, the image sensor may also determine the corresponding timestamp by itself after acquiring the image data, and then send the image data and the corresponding timestamp to the processing device. I.e. the embodiment shown in fig. 2a and 2 b. Referring to fig. 2a and 2b, a data processing system comprises:

a clock synchronization device 101 for generating a reference pulse signal, a first trigger pulse signal, and reference clock information according to a time signal of a clock oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; triggering an image sensor to acquire image data according to the first trigger pulse signal;

an image sensor 103 for acquiring image data of the object region in response to the received first trigger pulse signal; obtaining a timestamp corresponding to the image data by using a signal relation between the reference pulse signal and the first trigger pulse signal;

the processing device 104 is configured to receive the radar data and the timestamp thereof sent by the radar, and the image data and the timestamp thereof sent by the image sensor; the object region 1 is three-dimensionally modeled based on the radar data and the image data of the same time stamp.

Similarly, the data processing system provided by the present embodiment may further include an inertial lining unit 105. The data processing system provided by this embodiment may further include an autonomous mobile device, and the clock synchronization device 101, the radar 102, the image sensor 103, and the inertia measurement unit 105 may be disposed on the autonomous mobile device. The autonomous mobile equipment can move in the object area and sample data while moving; and sending the acquired inertial data, radar data and image data to a processing device, and processing the inertial data, radar data and image data based on the same timestamp by the processing device so as to perform three-dimensional modeling on the environment where the autonomous mobile device is located.

The three-dimensional model modeled by the processing device 104 may be sent to a client, where it may be displayed.

Here, it should be noted that: the present embodiment may be referred to in part by the contents of the embodiments shown in FIGS. 1a and 1b above; for example, the contents of the inertial lining unit 105 and the autonomous mobile device can be referred to the related contents in the above embodiments, and are not described herein again.

Fig. 2c shows another embodiment of the product form of the solution presented in the present application. Specifically, as shown in fig. 2c, the mobile acquisition backpack includes: backpack 100, radar 102, image sensor 103, and clock synchronization device (not shown). Wherein the content of the first and second substances,

a radar 102, provided on the backpack 100, for performing clock adjustment according to the received time service information; collecting radar data of an object area, and obtaining a timestamp corresponding to the radar data based on the adjusted clock;

an image sensor 103, disposed on the backpack 100, for acquiring image data of the object region in response to the received first trigger pulse signal;

the clock synchronization device is arranged on the backpack 100 and used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to a time signal of a clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; and triggering the image sensor to acquire image data according to the first trigger pulse signal.

In an implementation solution, the mobile acquisition backpack has data processing capability. Namely, the mobile collecting backpack further comprises: a processing device (not shown in the figure). The processing device is arranged in the backpack 100, connected to the radar 102, the image sensor 103 and the clock synchronization device, and configured to receive the radar data with a timestamp sent by the radar 102; receiving the image data sent by the image sensor 103; calculating a timestamp corresponding to the image data according to the signal relation between the reference pulse signal and the first trigger pulse signal; and performing three-dimensional modeling on the object region based on the radar data and the image data with the same timestamp.

In another implementable technical solution, the mobile acquisition backpack only has a data acquisition capability, and needs to transmit acquired data to a target device, such as a server device and a client device, through a communication component. Namely, the mobile acquisition backpack further comprises: a communication component (not shown). The communication component is arranged on the backpack 100, connected to the radar 102 and the image sensor 103, and configured to send the radar data with a timestamp acquired by the radar 102 and the image data acquired by the image sensor 103 to a target device.

The mobile acquisition backpack is also characterized in that a storage medium and an interface (such as a USB interface and the like) are arranged on the mobile acquisition backpack, and the mobile acquisition backpack can store data acquired by the radar and the image sensor in the storage medium; the user can connect external equipment (such as a computer, a mobile phone, intelligent wearable equipment and the like) through an interface on the mobile acquisition backpack so as to read data in the storage medium. That is, the mobile collecting backpack provided by this embodiment further includes:

a storage medium (not shown) disposed in the backpack 100 and connected to the radar 102 and the image sensor 103 for storing the radar data and the image data;

an interface 106, provided on the backpack 100, for connecting an external device, so that the external device can read the data in the storage medium.

The mobile acquisition backpack provided by the embodiment further comprises an inertia lining measuring unit, and the inertia lining measuring unit is connected with the time synchronization device. Here, it should be noted that: the content of this embodiment corresponds to the content of the embodiment shown in fig. 2a and 2b, and is not described herein again.

Fig. 2d, 2e and 2f show another embodiment of the present invention. Specifically, as shown in fig. 2d, the autonomous mobile device may be a small mobile cart with autonomous movement capability, on which a radar 102, an image sensor 103, a clock synchronization device (not shown in fig. 2 d), and the like are disposed. Alternatively, the autonomous mobile device may be a data acquisition robot as shown in fig. 2 e. The robot body of the data acquisition robot is provided with a radar 102, an image sensor 103, a clock synchronization device (not shown in fig. 2 e), and the like. Still alternatively, the autonomous moving apparatus may be an unmanned aerial vehicle shown in fig. 2f, to the body of which a radar (not shown in fig. 2 f), an image sensor 103, and the like are connected, and the clock synchronization device may be provided inside the body of the unmanned aerial vehicle. In particular, the method comprises the following steps of,

the radar 102 is arranged on the equipment body 107 and used for adjusting the clock according to the received time service information; collecting radar data of an object area, and obtaining a timestamp corresponding to the radar data based on the adjusted clock; for different types of devices, the form of the device body 107 is different, for example, for an autonomous mobile cart, the device body 107 is a cart body; for a data acquisition robot, the device body 107 is a robot body; for an unmanned aerial vehicle, the device body 107 is the airframe of the unmanned aerial vehicle.

An image sensor 103 disposed on the device body 107 for acquiring image data of the object region in response to the received first trigger pulse signal;

a clock synchronization device, which is disposed on the device body 107 and is configured to generate a reference pulse signal, a first trigger pulse signal, and reference clock information according to a time signal of a clock oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; and triggering the image sensor to acquire image data according to the first trigger pulse signal.

Further, the autonomous mobile acquisition device further comprises: and (4) processing equipment. The processing device is arranged in the device body 107, connected with the radar 102, the image sensor 103 and the clock synchronization device, and configured to receive the radar data with a timestamp sent by the radar 102; receiving the image data sent by the image sensor 103; calculating a timestamp corresponding to the image data according to the signal relation between the reference pulse signal and the first trigger pulse signal; and performing three-dimensional modeling on the object region based on the radar data and the image data with the same timestamp.

Further, the autonomous mobile acquisition device may send the acquired data to the target device. That is, the autonomous mobile acquisition device provided in this embodiment further includes: a communication component. And the communication assembly is arranged on the equipment body, is connected with the radar and the image sensor, and is used for transmitting the radar data which are acquired by the radar and carry the timestamp and the image data which are acquired by the image sensor to target equipment.

Furthermore, the autonomous mobile acquisition device can also store acquired data locally, and a user can read desired data from the autonomous mobile acquisition device through an interface provided by the autonomous mobile device. Namely, the mobile acquisition backpack further comprises: a storage medium and an interface. The storage medium is arranged in the equipment body, is connected with the radar and the image sensor and is used for storing the radar data and the image data; and the interface is arranged on the equipment body and used for connecting external equipment so as to facilitate the external equipment to read the data in the storage medium.

The following embodiments will explain the technical solutions provided in the present application from the perspective of a clock synchronization device, an image sensor, an inertial measurement unit, and a processing device, respectively.

Fig. 3 is a schematic flowchart illustrating a data processing method according to an embodiment of the present application. The execution main body of the method can be a clock synchronization device with real-time processing capability, such as an MCU, an FPGA, an RTOS and the like, and the clock synchronization device can be realized by hardware and/or software. Specifically, as shown in fig. 3, the method provided in this embodiment includes:

201. generating a reference pulse signal, a first trigger pulse signal and reference clock information based on a time signal of a clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal;

202. sending the reference pulse signal and the reference clock information as time service information to a first sensor so that the first sensor can adjust the clock according to the time service information, and obtaining a timestamp corresponding to first data collected by the first sensor based on the adjusted clock;

203. and triggering the second sensor to acquire second data according to the first trigger pulse signal so as to obtain a timestamp corresponding to the second data by using the signal relationship between the reference pulse signal and the first trigger pulse signal.

In specific implementation, the time signal in 201 may be generated by a peripheral clock oscillator circuit in a clock synchronization device such as an MCU or an FPGA. The MCU (Micro Control Unit, also called a microcomputer or a single chip microcomputer) is a chip-scale computer formed by integrating a CPU, an RAM, a ROM, a timing counter, and various I/O interfaces of the computer on one chip; the FPGA (field Programmable Gate array) is a product which is further developed on the basis of Programmable devices such as PAL, GAL and the like, is used as a semi-custom circuit in the field of Application Specific Integrated Circuits (ASIC), not only solves the defect of a custom circuit, but also overcomes the defect of limited Gate circuit number of the original Programmable device. Correspondingly, step 201 "generating the reference pulse signal, the first trigger pulse signal and the reference clock information based on the time signal of the clock oscillator" may be specifically implemented by the following steps:

2011. generating the reference pulse signal and the reference clock information meeting the requirement of a preset data format based on a time signal of a clock crystal oscillator;

2012. and generating the first trigger pulse signal meeting the clock synchronization requirement with the reference pulse signal according to the working parameters of the second sensor.

In the step 2011, the reference pulse signal may be a pulse per second (1PPS) signal, the reference clock information may include year, month, day, hour, minute and second information, and a data format of the reference clock information may be the same as the NMEA command information. The rising edge of the reference pulse signal is synchronized with the second clock in the reference clock information, which is transmitted before or after the reference second pulse, and the transmission sequence is related to the type of the received sensor. For example, if the receiving sensor is a radar that does not support external triggering, the reference clock information will be sent after the reference second pulse; on the contrary, if the receiving sensor is an inertial measurement unit (hereinafter, both referred to as IMU) or an image sensor supporting an external trigger function, the reference clock information is transmitted before the reference second pulse.

In an implementable aspect, the operating parameter of the second sensor is a data acquisition frequency. Accordingly, the step 1012 "generating the first trigger pulse signal satisfying the clock synchronization requirement with the reference pulse signal according to the operating parameter of the second sensor" may specifically include:

s11, determining a parameter according to the data acquisition frequency;

and S12, generating a first trigger pulse signal with a trigger edge aligned with one pulse edge of the reference pulse signal and a trigger edge aligned with the next pulse edge of the reference pulse signal after the parameter pulse period.

The second sensor in this embodiment may be an IMU or an image sensor (e.g., a camera). Specifically, in practical applications, the data acquisition frequencies corresponding to different sensors often need to be set in combination with actual system requirements, that is, the data acquisition frequencies of different sensors often are inconsistent, for example: video is 30 frames/second, IMU is 100Hz, radar is 20 Hz. Based on this, when the first trigger pulse signal meeting the clock synchronization requirement with the reference pulse signal is generated according to the operating parameter of the sensor, the maximum number (i.e. the parameter) of the first trigger pulse signal that can be included in one period of the reference pulse signal may be determined according to the data acquisition frequency of the sensor, so as to generate the first trigger pulse signal meeting the requirement that the trigger edge is aligned with one pulse edge of the reference pulse signal, and the trigger edge is aligned with the next pulse edge of the reference pulse signal after the parameter pulse period.

It should be noted that, in the process of generating the first trigger pulse signal and the reference pulse signal, the clock synchronization apparatus needs to perform an alignment operation on the reference pulse signal and the first trigger pulse signal. The alignment may be determined according to the characteristics of the clock synchronization apparatus itself.

For example, if the clock synchronization device is an MCU, a same trigger source of all timers inside the MCU needs to be configured, and all timers are triggered based on the same trigger source, so that synchronous timing can be achieved by starting all timers simultaneously, and the purpose of aligning the rising edge of the first trigger pulse signal with the upper edge of the reference pulse signal is achieved. The number of the timers in the MCU is determined according to the number of the sensors which are in communication connection with the MCU and are to be synchronized. Here, the same trigger source of all timers may be configured as the same external interrupt signal, and once the external interrupt signal is triggered, all timers receive the external interrupt signal at the same time to operate, so that alignment of the reference pulse signal and the first trigger pulse signal is ensured.

For another example, if the clock synchronization device is an FPGA, the FPGA needs to adopt a plurality of frequency dividers, the number of which is consistent with the number of sensors to be synchronized, for generating the reference pulse signal and the first trigger pulse signal respectively, and alignment of the reference pulse signal and the first trigger pulse signal can be ensured due to parallel real-time performance of the FPGA and characteristics of the same reference clock.

It should also be noted that: in case the reference pulse signal is a pulse per second (1PPS) signal, a parameter determined according to the data sampling frequency is preferably divisible by 1s in order to ensure that the triggering edge of the first trigger pulse signal is aligned with the next pulse edge of the reference pulse signal after a number of the triggering pulse periods.

In one practical application scenario, the PPS received by the radar (i.e., the reference pulse signal) shown in fig. 5 is synchronized with the timing diagram of the first trigger pulse signal (i.e., CAM _ sync _ clk in fig. 5) and the second trigger pulse signal (i.e., IMU _ sync _ clk in fig. 5). The radar, IMU and image sensor in this figure are communicatively connected to and controlled by clock synchronization means (e.g. MCU, PFGA, RTOS) in a manner as shown in fig. 1b or 2 b. In fig. 5, the sampling frequency of the IMU is 100Hz, and the parameter determined according to the sampling frequency is 100, that is, 100 first trigger pulse signals (IMU sync _ clk) can be generated within 1PPS, and the period of the first trigger pulse is 10 ms; similarly, the sampling frequency of the image sensor is 10 frames/second, and 10 second first trigger pulse signals (CAM sync _ clk) having a period of 100ms may be generated within 1 PPS. Because the radar, the IMU and the image sensor are triggered and started simultaneously based on the same trigger source, and a parameter determined according to the data sampling frequency is divisible by 1s, it can be ensured that after the parameter is a trigger pulse period, the upper trigger edge of the first trigger pulse signal is aligned with the upper edge of the next pulse of the reference pulse signal.

In 202, the first sensor may be a radar, such as a laser radar, a millimeter wave radar, an ultrasonic radar, etc., and the first sensor has a certain requirement for receivable time service information (i.e., the reference pulse signal and the reference clock information). For example, referring to the timing chart of the timing information receivable by the laser radar shown in fig. 4, the reference pulse signal and the reference clock signal may be similar to the PPS signal and the NMEA information in the existing GPS signal, respectively. Fig. 4 is a schematic diagram of PPS signals and NMEA information in conventional GPS signals. The PPS pulse width which can be granted by the laser radar is required to be between 20ms and 200 ms; the laser radar is a sensor which does not support triggering, NMEA information is sent to the laser radar after the PPS rising edge is 430ms, the baud rate of the NMEA information is 9600bps, and the character length is within 70 ms; after receiving legal PPS signals and NMEA information, the laser radar can calibrate a local clock along with the PPS signals, so that the frequency of the local clock is consistent with that of the MCU; and after the clock is calibrated, timing by taking the time information contained in the NMEA information as a reference to obtain a time stamp corresponding to the first data. The baud rate represents the number of symbol symbols transmitted per second, and is an index for measuring the data transmission rate. It is expressed in terms of the number of times the carrier modulation state changes per unit time. In an information transmission channel, a signal unit carrying data information is called a symbol, and the number of symbols transmitted through the channel per second is called a symbol transmission rate, which is referred to as a baud rate for short.

In 203, the second sensor may be an image sensor or an IMU, and when a rising edge of the first trigger pulse signal arrives, a trigger signal is sent to the second sensor to trigger the second sensor to acquire second data; meanwhile, the time stamp corresponding to the second data is calculated by utilizing the signal relation between the reference pulse signal and the first trigger pulse signal. The signal relationship is the number of the first trigger pulse signals generated from the generation time of the reference pulse signal to the generation time of the first trigger pulse signal. That is, each time the second sensor receives a trigger signal sent by the pulse trigger signal, incremental processing is performed once to obtain corresponding parameters, and meanwhile, second data acquisition is acquired; after the parameter corresponding to the second data is obtained, a timestamp corresponding to the second data may be calculated based on a sampling period of a second sensor, reference time information in the reference clock information, and the parameter.

As an application scenario, fig. 6 shows a corresponding relationship between the first trigger signal and data acquisition, that is, a sensor supporting an external trigger function, which correspondingly outputs a latest acquired data every time a first trigger signal is received, and outputs a frame of picture every time a first trigger signal is received. Continuing to refer to fig. 3, if the second sensor is an image sensor (e.g., a camera), the acquisition frequency of the image sensor data is 10Hz, that is, the parameter clk _ count of the image sensor during the incremental processing of the received trigger pulse information may be 10, and the data acquisition period of the image sensor is 100 ms; after receiving a trigger signal, the second sensor (i.e., an image sensor) starts to acquire second data, performs incremental processing on the received trigger signal to obtain a parameter clk _ count, and calculates a timestamp corresponding to the second data according to formula (1).

T_sampleCAM＝T_NMEA+clk_count*100ms (1)

Wherein, T_NMEAReference time information acquired for the second sensor based on the received reference clock information.

Further, the method provided by this embodiment further includes:

204a, acquiring parameters related to the second sensor under the condition of the received second data fed back by the second sensor;

204b, performing incremental processing on the parameters to obtain updated parameters;

204c, calculating a timestamp corresponding to the second data according to the updated parameter;

204a, the parameter characterizes a signal relationship between the reference pulse signal and the first trigger pulse signal; the signal relationships include: the number of the first trigger pulse signals generated in total from the generation time of the reference pulse signal to the generation time of the first trigger pulse signal.

204b, the number of the currently received second data may be processed incrementally by analyzing the second data fed back by the second sensor, that is, every time a second data is newly received, 1 is added on the basis of the number of the second data originally received, so as to obtain the updated parameter.

204c "calculating the timestamp corresponding to the second data according to the updated parameter" may specifically be implemented by the following steps:

204c1, acquiring the reference clock information and the period of the first trigger pulse signal;

204c1, calculating a timestamp corresponding to the second data according to the updated parameter, the reference clock information and the period of the first trigger pulse signal.

Specifically, the corresponding relationship between the first trigger pulse signal and the data acquisition is shown in fig. 6, that is, the sensor supporting the external trigger function correspondingly outputs a latest acquired data every time a first trigger pulse signal is received, such as an IMUAnd outputting the latest measurement data correspondingly every time a first trigger pulse signal is received. Continuing to refer to fig. 2, if the second sensor is an IMU, the parameter clk _ count of the IMU when performing increment processing on the received trigger pulse information may be 100, the acquisition cycle of IMU data is 10ms, when a clock synchronization device (e.g., MCU or FPGA) receives second data fed back by the IMU, the second data is analyzed, and the number of the received second data is subjected to increment processing, so as to obtain the number of the second data currently received after updating (i.e., the number corresponding to the current parameter clk _ count), and based on the number corresponding to the updated parameter clk _ count, the reference clock information, and the cycle of the first trigger pulse signal, the timestamp T corresponding to the second data may be calculated according to formula (2)_sampleIMU。

T_sampleIMU＝T_NMEA+clk_count*10ms (2)

In the above, the parameter clk _ count may also be calculated by the IMU when performing the second data acquisition; and then sending second data containing the parameter clk _ count to the clock synchronization device, so that the clock synchronization device obtains the parameter completion timestamp calculation when resolving the second data. However, overflow occurs due to the fact that the length of the byte in the IMU, in which the parameter clk _ count is stored, is not sufficient; and the clock synchronization device counts the parameters, so that the overflow risk can be avoided.

Further, the method provided by this embodiment may further include:

205. and resetting the parameter associated with the second sensor to a set initial value when the updated parameter is equal to the parameter.

Of course, in the implementation, the parameters can be continuously accumulated, and the parameters do not need to be reset when the parameters are equal to the parameters. If the parameters are continuously accumulated, T is above_NMEAThe numerical value can be constant all the time; but only the parametersWhen the parameter is reached and needs to be reset to a set initial value, the T is_NMEAThe value is changed according to the pulse signal of the reference pulse signal.

Further, the method provided by this embodiment may further include:

206. and sending the second data and the corresponding time stamps to processing equipment so that the processing equipment can perform data fusion processing on the second data and data acquired by different sensors by using the time stamps.

According to the method provided by the embodiment of the application, the timestamps corresponding to the data collected by the various sensors are completely controlled by the reference pulse signal, the reference clock information and the first trigger pulse signal generated according to the time signal of the same clock crystal oscillator, and under the condition that the crystal oscillator has errors, the clock synchronization of the various sensors can be still effectively ensured, the occurrence of accumulated errors is avoided, and the method is low in price, small in size and low in power consumption.

Fig. 7 is a schematic flowchart illustrating a data processing method according to another embodiment of the present application. The execution subject of the data processing method is a second sensor, which may be an image sensor.

As shown in fig. 7, the data processing method includes:

301. collecting data to obtain second data in response to the received trigger signal; wherein the trigger signal is generated based on a first trigger pulse signal satisfying clock synchronization with a reference pulse signal;

302. and obtaining the time stamp corresponding to the second data by using the signal relation between the reference pulse signal and the first trigger pulse signal.

In 302, the information relationship is the number of the first trigger pulse signals generated in the period from the reference pulse signal generation time to the first trigger pulse signal generation time. Correspondingly, step 302 "obtaining the timestamp corresponding to the second data by using the signal relationship between the reference pulse signal and the first trigger pulse signal" may specifically be implemented by using the following steps:

3021. acquiring parameters, wherein the parameters embody the signal relationship by recording the receiving times of the trigger signals or the times of triggering to acquire data;

3022. after the trigger signal is received or the acquisition of the second data is completed, performing incremental processing on the parameters to obtain updated parameters;

3023. and calculating the time stamp corresponding to the second data according to the updated parameters.

The specific implementation of the steps 3021 and 3023 may refer to the corresponding content related to the second sensor in the above embodiments, and will not be described herein again.

Further, the method provided by this embodiment may further include:

3024. when the updated parameter is equal to a parameter, resetting the parameter to a preset initial value;

wherein said parameter is indicative of a maximum number of first trigger pulse signals generated during a period of said reference pulse signal.

Further, the method provided by this embodiment may further include:

303. and sending the second data and the corresponding time stamps to processing equipment so that the processing equipment can perform data fusion processing on the second data and data acquired by different sensors by using the time stamps.

According to the technical scheme, the data are collected through the trigger signal generated by responding to the first trigger pulse signal to obtain the second data, and the calculation of the timestamp corresponding to the second data is only related to the signal relation between the reference pulse signal and the first trigger pulse signal, so that the calculation of the timestamp is not influenced by data transmission delay and system call delay, and the accuracy of the timestamp is guaranteed.

According to the scheme provided by the embodiment, after the second sensor acquires second data, the time stamp corresponding to the second data is obtained by utilizing the signal relation between the reference pulse signal and the first trigger pulse signal; the second data is sent to the processing device together with its time stamp. In essence, the second sensor may not perform local timestamp calculation, and after acquiring the second data, directly send the second data to the processor, and the processor calculates a timestamp corresponding to the second data. The implementation manner of calculating the corresponding time stamp of the second data by the processor will be described in detail in the following embodiments.

Here, it should be noted that: the content of each step in the method provided by the embodiment of the present application, which is not described in detail in the foregoing embodiments, may be referred to corresponding content in the foregoing embodiments, and is not described in detail herein. In addition, the method provided in the embodiment of the present application may further include other parts or all of the steps in the embodiments in addition to the steps described above, and specific reference may be made to corresponding contents in the embodiments described above, which are not described herein again.

Fig. 8 is a schematic flow chart illustrating data processing according to another embodiment of the present application. The execution main body of the data processing method can be a processing device, such as an upper computer. As shown in fig. 8, the data processing method includes:

401. receiving second data sent by a second sensor;

402. acquiring locally recorded auxiliary information related to the second sensor; the auxiliary information contains a signal relation between a reference pulse signal and a first trigger pulse signal, and the reference pulse signal and the first trigger pulse signal are generated based on a time signal of the same clock crystal oscillator and meet the requirement of clock synchronization; the second sensor is triggered to acquire the second data under the action of the first trigger pulse signal;

403. and calculating the time stamp corresponding to the second data by using the auxiliary information.

For specific implementation of the foregoing 401 and 403, reference may be made to corresponding contents in the foregoing embodiments, and details are not described herein again. The above 402 "acquiring locally recorded auxiliary information related to the second sensor" may specifically include:

4021. acquiring parameters, wherein the parameters embody the signal relationship by recording the times of receiving the data sent by the second sensor; the signal relationships include: the number of the first trigger pulse signals generated in the period from the reference pulse signal generation time to the first trigger pulse signal generation time;

4022. reference clock information associated with the reference pulse signal is acquired.

Further, the method provided by this embodiment may further include:

404. receiving first data and a corresponding timestamp sent by a first sensor;

405. and performing data fusion processing on the first data and the second data with the same timestamp to obtain a processing result.

According to the technical scheme provided by the embodiment of the application, the timestamp corresponding to the second data is calculated by utilizing the locally recorded auxiliary information related to the second sensor, so that the calculation of the timestamp is not influenced by data transmission delay, and the real-time requirement on a system is reduced.

Here, it should be noted that: the content of each step in the method provided by the embodiment of the present application, which is not described in detail in the foregoing embodiments, may be referred to corresponding content in the foregoing embodiments, and is not described in detail herein. In addition, the method provided in the embodiment of the present application may further include, in addition to the above steps, other parts or all of the steps in the above embodiments, and specific reference may be made to corresponding contents in the above embodiments, which is not described herein again.

Fig. 9 shows a block diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 9, the data processing apparatus includes: a generating module 601, a sending module 602 and a triggering module 603. Wherein the content of the first and second substances,

the generating module 601 is configured to generate a reference pulse signal, a first trigger pulse signal, and reference clock information based on a time signal of a clock oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal;

the sending module 602 is configured to send the reference pulse signal and the reference clock information as time service information to a first sensor, so that the first sensor performs clock adjustment according to the time service information, and obtains a timestamp corresponding to first data acquired by the first sensor based on the adjusted clock;

the triggering module 603 is configured to trigger the second sensor to acquire second data according to the first trigger pulse signal, so as to obtain a timestamp corresponding to the second data by using a signal relationship between the reference pulse signal and the first trigger pulse signal.

In the technical scheme provided by this embodiment, the timestamps corresponding to the data collected by the various sensors are completely controlled by the reference pulse signal, the reference clock information and the first trigger pulse signal generated according to the time signal of the same clock crystal oscillator, and under the condition that the crystal oscillator has errors, the clock synchronization of the various sensors can be still effectively ensured, the occurrence of accumulated errors is avoided, and the clock synchronization device is low in price, small in size and low in power consumption.

Further, the generating module 601 is specifically configured to:

generating the reference pulse signal and the reference clock information meeting the requirement of a preset data format based on a time signal of a clock crystal oscillator;

and generating the first trigger pulse signal meeting the clock synchronization requirement with the reference pulse signal according to the working parameters of the second sensor.

Further, the working parameter of the second sensor is data acquisition frequency. Accordingly, the generating module 601 further has the following functions:

determining a parameter according to the data acquisition frequency;

and generating a first trigger pulse signal with a trigger edge aligned with a pulse edge of the reference pulse signal and a trigger edge aligned with a next pulse edge of the reference pulse signal after the parameter pulse period.

Further, the data processing apparatus provided in this embodiment further includes a data processing module 604, where the data processing module 604 is specifically configured to:

acquiring parameters related to the second sensor under the condition of the received second data fed back by the second sensor;

performing incremental processing on the parameters to obtain updated parameters;

calculating a timestamp corresponding to the second data according to the updated parameter;

wherein the parameter characterizes a signal relationship of the reference pulse signal and the first trigger pulse signal; the signal relationships include: the number of the first trigger pulse signals generated in total from the generation time of the reference pulse signal to the generation time of the first trigger pulse signal.

Further, the data processing module 604 is further configured to:

acquiring the reference clock information and the period of the first trigger pulse signal;

and calculating to obtain a timestamp corresponding to the second data according to the updated parameter, the reference clock information and the period of the first trigger pulse signal.

Further, the data processing apparatus provided in this embodiment further includes a resetting module 605, where the resetting module 605 is configured to reset the parameter associated with the second sensor to a set initial value when the updated parameter is equal to the parameter.

Further, the sending module 602 is further configured to send the second data and the corresponding timestamp to a processing device, so that the processing device performs data fusion processing on the second data and data acquired by different sensors by using the timestamp.

In the above, the first sensor is a radar, and the second sensor is an image sensor or an IMU.

Here, it should be noted that: the data processing apparatus provided in this embodiment may implement the technical solution described in the data processing method embodiment shown in fig. 1, and the specific implementation principle of each module or unit may refer to the corresponding content in the data processing method embodiment shown in fig. 1, and is not described herein again.

Fig. 10 shows a block diagram of a data processing apparatus according to another embodiment of the present application. As shown in fig. 10, the data processing apparatus includes: acquisition module 701, operation module 702, wherein:

the acquisition module 701 is configured to acquire data to obtain second data in response to the received trigger signal; wherein the trigger signal is generated based on a first trigger pulse signal satisfying clock synchronization with a reference pulse signal;

the operation module 702 is configured to obtain a timestamp corresponding to the second data by using a signal relationship between the reference pulse signal and the first trigger pulse signal.

The signal relationship is the number of the first trigger pulse signals generated from the generation time of the reference pulse signal to the generation time of the first trigger pulse signal.

In the technical scheme provided by this embodiment, data acquisition is performed by responding to a trigger signal generated by a first trigger pulse signal to obtain second data, and the calculation of a timestamp corresponding to the second data is only related to a signal relationship between the reference pulse signal and the first trigger pulse signal, so that the calculation of the timestamp is not affected by data transmission delay and system call delay, and the accuracy of the timestamp is ensured.

Further, the operation module 702 is further specifically configured to:

acquiring parameters, wherein the parameters embody the signal relationship by recording the receiving times of the trigger signals or the times of triggering to acquire data;

after the trigger signal is received or the acquisition of the second data is completed, performing incremental processing on the parameters to obtain updated parameters;

and calculating the time stamp corresponding to the second data according to the updated parameters.

Further, the data processing apparatus provided in this embodiment further includes: a resetting module 703, where the resetting module 703 is configured to reset the updated parameter to a preset initial value when the updated parameter is equal to a parameter;

Further, the data processing apparatus provided in this embodiment further includes: a sending module 704, where the sending module 704 is configured to send the second data and the corresponding timestamp to a processing device, so that the processing device performs data fusion processing on the second data and data acquired by different sensors by using the timestamp.

Here, it should be noted that: the data processing apparatus provided in this embodiment may implement the technical solution described in the data processing method embodiment shown in fig. 5, and the specific implementation principle of each module or unit may refer to the corresponding content in the data processing method embodiment shown in fig. 5, which is not described herein again.

Fig. 11 shows a block diagram of a data processing apparatus according to another embodiment of the present application. As shown in fig. 11, the data processing apparatus includes: a receiving module 801, an obtaining module 802 and an operation module 803; wherein, the first and the second end of the pipe are connected with each other,

the receiving module 801 is configured to receive second data sent by a second sensor;

the obtaining module 802 is configured to obtain locally recorded auxiliary information related to the second sensor; the auxiliary information contains a signal relation between a reference pulse signal and a first trigger pulse signal, and the reference pulse signal and the first trigger pulse signal are generated based on a time signal of the same clock crystal oscillator and meet the clock synchronization requirement; the second sensor is triggered to acquire the second data under the action of the first trigger pulse signal;

the operation module 803 is configured to calculate a timestamp corresponding to the second data by using the auxiliary information.

In the technical solution provided in this embodiment, the timestamp corresponding to the second data is calculated by using the locally recorded auxiliary information related to the second sensor, so that the calculation of the timestamp is not affected by data transmission delay, and the requirement on the real-time performance of the system is reduced.

Further, the obtaining module 802 is specifically configured to:

acquiring parameters, wherein the parameters embody the signal relationship by recording the times of receiving the data sent by the second sensor; the signal relationships include: the number of the first trigger pulse signals generated in the period from the reference pulse signal generation time to the first trigger pulse signal generation time;

further, the operation module 803; but also for the purpose of,

receiving first data and a corresponding timestamp sent by a first sensor;

and performing data fusion processing on the first data and the second data with the same time stamp to obtain a processing result.

In the above, the first sensor is a radar, and the second sensor is an image sensor.

Here, it should be noted that: the data processing apparatus provided in this embodiment may implement the technical solution described in the data processing method embodiment shown in fig. 6, and the specific implementation principle of each module or unit may refer to the corresponding content in the data processing method embodiment shown in fig. 6, which is not described herein again.

Fig. 12 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application. As shown in fig. 12, the electronic device includes a memory 901, a processor 902, and a communication component 903. The memory 901 may be configured to store other various data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device. The memory 901 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The processor 902, coupled to the memory 901, is configured to execute the program stored in the memory 901, so as to:

generating a reference pulse signal, a first trigger pulse signal and reference clock information based on a time signal of a clock crystal oscillator; wherein the reference pulse signal and the first trigger pulse signal meet a clock synchronization requirement, and the reference clock information is associated with the reference pulse signal;

sending the reference pulse signal and the reference clock information as time service information to a first sensor through the communication component 903, so that the first sensor can perform clock adjustment according to the time service information, and obtaining a timestamp corresponding to first data collected by the first sensor based on the adjusted clock;

the first trigger pulse signal is sent to the second sensor through the communication component 903, so as to trigger the second sensor to acquire second data, and a timestamp corresponding to the second data is obtained by using a signal relationship between the reference pulse signal and the first trigger pulse signal.

When the processor 902 executes the program in the memory 901, other functions may be implemented in addition to the above functions, which may be specifically referred to the description of the foregoing embodiments.

Further, as shown in fig. 12, the electronic apparatus further includes: power supply components 905, and the like. Only some of the components are schematically shown in fig. 12, and the electronic device is not meant to include only the components shown in fig. 12.

Fig. 13 shows a schematic structural diagram of a sensor according to another embodiment of the present application. As shown in fig. 13, the sensor includes: memory 91, processor 92 and sensing module 97. The memory 91 may be configured to store other various data to support operations on the sensors. Examples of such data include instructions for any application or method operating on the sensor. The memory 91 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The processor 92, coupled to the memory 91, is configured to execute the program stored in the memory 91 to:

When the processor 92 executes the program in the memory 91, the processor 92 may also implement other functions in addition to the above functions, which may be specifically referred to the description of the foregoing embodiments.

Further, as shown in fig. 13, the sensor further includes: communication components 93, power components 95, and the like. Only some of the components are schematically shown in fig. 13, and the sensor is not meant to include only the components shown in fig. 13.

An embodiment of the present application further provides an electronic device, which has a structure similar to that of fig. 12. Specifically, the electronic device comprises a memory, a processor and a communication component. Wherein the memory is used for storing programs; the processor, coupled with the memory, to execute the program stored in the memory to:

When the processor executes the program in the memory, the processor may implement other functions in addition to the above functions, which may be specifically referred to the description of the foregoing embodiments.

Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps or functions of the segmentation method provided in the foregoing embodiments when executed by a computer.

The technical scheme provided by each embodiment of the application can be applied to a plurality of scenes, such as three-dimensional modeling of markets and shops, and is convenient to show on an e-commerce platform; for another example, a house to be sold or rented is subjected to three-dimensional modeling, so that the house is conveniently sold or rented on a platform on line; for another example, the three-dimensional modeling is performed on a stadium and a live performance stage, so that some visual effects and the like can be produced by using the three-dimensional modeling model in a live broadcast or a relay broadcast of a match, a concert, a evening party, or the like, or some virtual field set simulation is performed based on the stadium or the stage three-dimensional modeling model. The following description will be made with reference to the following scenarios.

Scene 1, solid shop three-dimensional modeling

A camera and a radar are arranged in the entity shop; wherein, camera and radar all are connected with a clock synchronizer. The camera is used for collecting image data in the entity shop, and radar is used for measuring the internal space structure, the displayed objects and the like of the entity shop to obtain radar data. The time stamps corresponding to the image data and the radar data are controlled by a reference pulse signal generated from a time signal of a clock oscillator of the clock synchronization device, reference clock information, and a first trigger pulse signal. Under the condition that the crystal oscillator of the clock synchronization device has errors, the images and the time stamps of the radar data can be reflected, and the synchronism of the time stamps corresponding to the images and the radar data is effectively guaranteed. A processing device, such as a server or a network side server provided in the physical store, for performing three-dimensional modeling on the physical store according to the radar data and the image data with the same timestamp; sending the three-dimensional modeling result to display equipment for displaying; or sending the three modeling results to the e-commerce platform so as to display the page where the shop is located on the corresponding line on the e-commerce platform.

Scene 2, stadium three-dimensional modeling

The staff can wrap the sports stadium around the mobile acquisition back provided in the above embodiments for one circle or rotate around the center of the field standing in the sports stadium for one circle to acquire image data and radar data in all directions of the sports stadium. The mobile acquisition backpack is provided with a clock synchronization device, and timestamps corresponding to the image data and the radar data are controlled by a reference pulse signal, reference clock information and a first trigger pulse signal generated by a time signal of a clock crystal oscillator of the clock synchronization device. The mobile acquisition backpack can transmit acquired image data and radar data to the server through the communication assembly, and the server performs three-dimensional modeling on the stadium according to the radar data and the image data with the same timestamp; or storing the acquired image data and radar data in a storage medium in the backpack to wait for the data in the storage medium to be read by a processing device accessed to the mobile acquisition backpack through an interface; or transmitting the collected image data and the radar data to processing equipment in the mobile collection backpack, and performing three-dimensional modeling on the stadium by the processing equipment based on the radar data and the image data with the same timestamp.

The modeling result of three-dimensional modeling of the stadium can be used for making visual special effects in live broadcasting or rebroadcasting of events, or used for large-screen display on site, or used for virtual setting simulation of the stadium.

Scene 3, three-dimensional modeling of scene performance stage

The autonomous mobile acquisition equipment provided in the above embodiment, such as an unmanned aerial vehicle, is used to acquire image data and radar data of a live performance stage. Similarly, the time stamps corresponding to the image data and the radar data are controlled by a reference pulse signal generated from a time signal of a clock oscillator of the same clock synchronization device, reference clock information, and a first trigger pulse signal. The autonomous mobile acquisition equipment can send acquired image data and radar data to the background server, and the background server carries out three-dimensional modeling on the scene performance stage based on the radar data and the image data with the same timestamp. The modeling result of three-dimensional modeling of the on-site performance stage can be used for on-site large screen display.

Scene 4, Home decoration scene

The technical scheme that each embodiment of usable application of house ornamentation designer provided, if remove the collection knapsack on the back, after gathering the image data and the radar data of waiting to decorate the room, radar data and image data based on the same timestamp, carry out three-dimensional modeling to the house of treating to decorate. And then, the home decoration designer can complete the home decoration design on the basis of the three-dimensional modeling model of the house so as to generate a home decoration effect diagram, a drawing and the like.

Of course, the present invention can be applied to other scenarios besides the above scenarios, and this document does not necessarily refer to these scenarios.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present application.

Claims

1. A data processing system, comprising:

the clock synchronization device is used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to the time signal of the clock crystal oscillator; the reference pulse signal is generated by simulating a time service function of a global positioning system, the reference pulse signal and the first trigger pulse signal meet the requirement of clock synchronization, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; triggering an image sensor to acquire image data according to the first trigger pulse signal;

2. The system of claim 1, further comprising:

the inertia measurement unit is used for responding to the received second trigger pulse signal and acquiring inertia data;

the clock synchronization device is further configured to generate a second trigger pulse signal meeting a clock synchronization requirement with the reference pulse signal, and trigger the inertia measurement unit to acquire inertia data according to the second trigger pulse signal; the time stamp calculation unit is further configured to receive the inertial data sent by the inertial measurement unit, and calculate a time stamp corresponding to the inertial data by using a relationship between the reference pulse signal and the second trigger pulse signal; sending the inertial data and the corresponding timestamp to the processing equipment;

the processing device is further configured to perform three-dimensional modeling on the object region based on the radar data, the image data, and the inertial data of the same timestamp.

3. The system of claim 2, further comprising:

an autonomous mobile device having the capability to travel autonomously within the object region;

the clock synchronization device, the radar, the image sensor and the inertial measurement unit are all arranged on the autonomous mobile equipment.

4. The system according to any one of claims 1 to 3, wherein the processing device is a server device; the radar is a laser radar; the image sensor is a camera.

5. A data processing system, comprising:

6. A data processing method is suitable for a clock synchronization device, and is characterized by comprising the following steps:

generating a reference pulse signal, a first trigger pulse signal and reference clock information based on a time signal of a clock crystal oscillator; the reference pulse signal is generated by simulating a time service function of a global positioning system, the reference pulse signal and the first trigger pulse signal meet the requirement of clock synchronization, and the reference clock information is associated with the reference pulse signal;

sending the reference pulse signal and the reference clock information as time service information to a first sensor so that the first sensor can adjust the clock according to the time service information, and obtaining a timestamp corresponding to first data collected by the first sensor based on the adjusted clock; wherein the first sensor is a radar;

triggering a second sensor to acquire second data according to the first trigger pulse signal so as to obtain a timestamp corresponding to the second data by using the signal relationship between the reference pulse signal and the first trigger pulse signal; wherein the second sensor is an image sensor or an inertial measurement unit.

7. The method of claim 6, wherein generating the reference pulse signal, the first trigger pulse signal, and the reference clock information based on the time signal of the clock oscillator comprises:

8. The method of claim 7, wherein the operating parameter of the second sensor is a data acquisition frequency; and

the generating of the first trigger pulse signal meeting the clock synchronization requirement with the reference pulse signal by the operating parameter of the second sensor includes:

determining a parameter according to the data acquisition frequency;

9. The method of claim 8, further comprising:

10. The method of claim 9, wherein calculating the timestamp corresponding to the second data according to the updated parameter comprises:

11. The method of claim 9, further comprising:

and resetting the parameter associated with the second sensor to a set initial value when the updated parameter is equal to the parameter.

12. The method of claim 9, further comprising:

and sending the second data and the corresponding time stamps to processing equipment so that the processing equipment can perform data fusion processing on the second data and the data acquired by different sensors by using the time stamps.

13. A data processing method applied to an image sensor, comprising:

acquiring image data to obtain second data in response to the received trigger signal; wherein the trigger signal is generated based on a first trigger pulse signal satisfying clock synchronization with a reference pulse signal; the reference pulse signal and the first trigger pulse signal are generated based on a time signal of the same clock crystal oscillator;

14. The method of claim 13, wherein the signal relationship comprises:

the number of the first trigger pulse signals generated in total from the generation time of the reference pulse signal to the generation time of the first trigger pulse signal.

15. The method of claim 14, wherein obtaining the timestamp corresponding to the second data by using the signal relationship between the reference pulse signal and the first trigger pulse signal comprises:

and calculating a time stamp corresponding to the second data according to the updated parameters.

16. The method of claim 15, further comprising:

when the updated parameter is equal to a parameter, resetting the parameter to a preset initial value;

17. The method of any of claims 13 to 16, further comprising:

and sending the second data and the corresponding time stamps to processing equipment so that the processing equipment can perform data fusion processing on the second data and data acquired by different sensors by using the time stamps.

18. A data processing method, adapted to a processing device, comprising:

receiving second data sent by a second sensor; wherein the second sensor is an image sensor;

acquiring locally recorded auxiliary information related to the second sensor; the auxiliary information contains a signal relation between a reference pulse signal and a first trigger pulse signal, and the reference pulse signal and the first trigger pulse signal are generated based on a time signal of the same clock crystal oscillator and meet the requirement of clock synchronization; the reference pulse signal is generated by simulating a time service function of a global positioning system; the second sensor is triggered to acquire the second data under the action of the first trigger pulse signal;

19. The method of claim 18, wherein obtaining locally recorded auxiliary information related to the second sensor comprises:

reference clock information associated with the reference pulse signal is acquired.

20. The method of claim 18 or 19, further comprising:

receiving first data and a corresponding timestamp sent by a first sensor;

and performing data fusion processing on the first data and the second data with the same timestamp to obtain a processing result.

21. The method of claim 20, wherein the first sensor is a radar.

22. An electronic device, comprising: a memory, a processor, and a communications component, wherein,

the memory is used for storing programs;

sending the reference pulse signal and the reference clock information as time service information to a first sensor through the communication assembly, so that the first sensor can adjust the clock according to the time service information, and obtaining a timestamp corresponding to first data collected by the first sensor based on the adjusted clock; wherein the first sensor is a radar;

sending the first trigger pulse signal to a second sensor through the communication assembly to trigger the second sensor to acquire second data, so that a timestamp corresponding to the second data is obtained by using a signal relation between the reference pulse signal and the first trigger pulse signal; wherein the second sensor is an image sensor or an inertial measurement unit.

23. An image sensor, comprising: a memory, a processor, and a sensing module, wherein,

the memory is used for storing programs;

responding to the received trigger signal, controlling the induction module to work so as to acquire image data and obtain second data; wherein the trigger signal is generated based on a first trigger pulse signal satisfying clock synchronization with a reference pulse signal; the reference pulse signal and the first trigger pulse signal are generated based on a time signal of the same clock crystal oscillator;

24. An electronic device, comprising: a memory, a processor, and a communications component, wherein,

the memory is used for storing programs;

receiving second data sent by a second sensor through the communication component; wherein the second sensor is an image sensor;

25. A mobile acquisition backpack, comprising:

a backpack;

the clock synchronization device is arranged on the backpack and used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to a time signal of a clock crystal oscillator; the reference pulse signal is generated by simulating a time service function of a global positioning system, the reference pulse signal and the first trigger pulse signal meet the requirement of clock synchronization, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; and triggering the image sensor to acquire image data according to the first trigger pulse signal.

26. The mobile acquisition backpack of claim 25, further comprising:

the processing equipment is arranged in the backpack, connected with the radar, the image sensor and the clock synchronization device and used for receiving the radar data which are sent by the radar and carry timestamps; receiving the image data sent by the image sensor; calculating a timestamp corresponding to the image data according to the signal relation between the reference pulse signal and the first trigger pulse signal; and performing three-dimensional modeling on the object region based on the radar data and the image data with the same timestamp.

27. The mobile acquisition backpack of claim 25, further comprising:

and the communication assembly is arranged on the backpack, is connected with the radar and the image sensor, and is used for transmitting the radar data which are acquired by the radar and carry the timestamp and the image data which are acquired by the image sensor to target equipment.

28. The mobile acquisition backpack of claim 25, further comprising:

the storage medium is arranged in the backpack, is connected with the radar and the image sensor and is used for storing the radar data and the image data;

and the interface is arranged on the backpack and used for connecting an external device so that the external device can read the data in the storage medium.

29. An autonomous mobile acquisition device, comprising:

an apparatus body;

the clock synchronization device is arranged on the equipment body and used for generating a reference pulse signal, a first trigger pulse signal and reference clock information according to a time signal of a clock crystal oscillator; the reference pulse signal is generated by simulating a time service function of a global positioning system, the reference pulse signal and the first trigger pulse signal meet the requirement of clock synchronization, and the reference clock information is associated with the reference pulse signal; sending the reference pulse signal and the reference clock information to a radar as time service information; and triggering the image sensor to acquire image data according to the first trigger pulse signal.

30. The autonomous mobile acquisition device of claim 29 further comprising:

the processing equipment is arranged in the equipment body, connected with the radar, the image sensor and the clock synchronization device and used for receiving the radar data which are sent by the radar and carry timestamps; receiving the image data sent by the image sensor; calculating a timestamp corresponding to the image data according to the signal relation between the reference pulse signal and the first trigger pulse signal; and performing three-dimensional modeling on the object region based on the radar data and the image data with the same timestamp.

31. The autonomous mobile acquisition device of claim 29, further comprising:

and the communication assembly is arranged on the equipment body, is connected with the radar and the image sensor, and is used for transmitting the radar data which are acquired by the radar and carry the timestamp and the image data which are acquired by the image sensor to target equipment.

32. The autonomous mobile acquisition device of claim 29, further comprising:

the storage medium is arranged in the equipment body, is connected with the radar and the image sensor and is used for storing the radar data and the image data;

and the interface is arranged on the equipment body and used for connecting external equipment so as to facilitate the external equipment to read the data in the storage medium.