WO2018068771A1

WO2018068771A1 - Target tracking method and system, electronic device, and computer storage medium

Info

Publication number: WO2018068771A1
Application number: PCT/CN2017/110745
Authority: WO
Inventors: 卿明; 陈鹏
Original assignee: 纳恩博（北京）科技有限公司
Priority date: 2016-10-12
Filing date: 2017-11-13
Publication date: 2018-04-19
Also published as: CN106682572B; CN106682572A

Abstract

A target tracking method and system, an electronic device, and a computer storage medium. The method comprises: executing vision-based target tracking, and obtaining a first tracking result for a tracked target (S102); according to movement information used for describing a movement state of the tracked target, in combination with the first tracking result, performing analysis to obtain a second tracking result for the tracked target (S104), the second tracking result being the same as or different from the first tracking result, the movement information used for describing the movement state of the tracked target being measurement-based and obtained from a second electronic device (32) arranged on the tracked target.

Description

Target tracking method, system, electronic device and computer storage medium

Cross-reference to related applications

The application is based on a Chinese patent application with the application number of 201611001901.8, the application date is November 14, 2016, and the Chinese patent application with the application number of 201610891388.8 and the application date of October 12, 2016, and the Chinese patent application is required. The entire contents of this Chinese patent application are hereby incorporated herein by reference.

Technical field

The present invention relates to the field of target tracking, and in particular to a target tracking method, system, electronic device and computer storage medium.

Background technique

In the prior art, target tracking is an indispensable part of most visual systems, and has applications in video surveillance, human-computer interaction, and military fields. The current target tracking technology can detect the accuracy of the tracking system in the case of the target motion mode, the target motion scene, the sudden change of the target external features, and the sudden changes in the shooting of the target when tracking the target. Problems such as the decrease in stability, for example, the vision-based tracking technique has poor stability during long-term tracking, and is easy to lose when the target object suddenly accelerates. For the vision-based motion behavior analysis technology, it is easy to cause the tracking process. The error is accumulated in the tracking value, so that the closed loop adjustment cannot be formed.

In view of the low accuracy of the above visual tracking, there is currently no effective solution.

Summary of the invention

Embodiments of the present invention are expected to provide a target tracking method, system, electronic device, and computer Storage media to solve at least the technical problem of low tracking accuracy for long-term visual tracking.

According to a first aspect of the embodiments of the present invention, there is provided a target tracking method, applied to a first electronic device, the method comprising: performing a vision-based target tracking to obtain a first tracking result for a tracked target; Determining motion information of the tracked target motion state, and combining the first tracking result, analyzing to obtain a second tracking result for the tracked target; wherein the second tracking result is the same as the first tracking result Or different; the motion information for describing the motion state of the tracked target is obtained based on a measurement by a second electronic device disposed on the tracked target.

In an embodiment, the motion information includes motion measurement parameters for the tracked target, the motion measurement parameters are recorded in a predetermined storage space of the second electronic device, and the motion measurement parameters include at least: Acceleration and/or angular angular velocity; the first electronic device obtains motion information, including: periodically or aperiodically accessing a predetermined storage space of the second electronic device, and acquiring the second electronic device within a predetermined time period Acquiring the obtained motion measurement parameter; according to the motion information for describing the motion state of the tracked target, combining the first tracking result, analyzing and obtaining the second tracking result for the tracked target, including: Performing statistical analysis on the motion measurement parameter in the time period, obtaining motion description information of the tracked target in a predetermined time period, and correcting the first tracking result according to the motion description information to obtain the second Tracking the result; wherein the motion description information includes at least a motion speed and/or a motion direction.

In an embodiment, performing statistical analysis on the motion measurement parameter in the predetermined time period to obtain motion description information of the tracked target within a predetermined time period, including: combining the second electronic device in a predetermined The motion measurement parameter in the time period and the self-motion measurement parameter of the first electronic device in the predetermined time period, and the analysis obtains the tracked target relative to the first electronic device within the predetermined time period Motion description information.

In an embodiment, the motion information includes motion description information for the tracked target, The motion description information is obtained by the second electronic device according to the motion measurement parameter in the predetermined time period obtained by the acquisition, and the motion measurement parameter includes at least: motion acceleration and/or motion angular velocity, and the motion description information. Include at least a motion speed and/or a motion direction, the motion description information being recorded in a predetermined storage space of the second electronic device; the first electronic device obtaining motion information, including: a periodic or non-periodic access station a predetermined storage space of the second electronic device, acquiring motion description information of the second electronic device in a predetermined time period; and combining the first tracking result according to the motion information for describing the motion state of the tracked target, The obtaining the second tracking result for the tracked target includes: correcting the first tracking result according to the motion description information in the predetermined time period to obtain the second tracking result.

In an embodiment, the motion description information includes a state probability of the tracked target in different motion directions; and correcting the first tracking result according to the motion description information in the predetermined time period to obtain a second tracking a result, comprising: determining the tracked state according to a state probability of the tracked target in different moving directions, and the tracked target represented by the first tracking result is at least one candidate tracking area in a tracking image The target is in a confidence tracking area in the tracking image, and a second tracking result is generated based on the confidence tracking area; wherein the state probability is used to indicate a probability that the tracked object moves in different directions of motion.

In an embodiment, the method further includes: when the motion description information in the predetermined time period indicates that the moving direction of the tracked target in a predetermined time period is a first direction, the first time in the predetermined time period a tracking result characterizing that the moving direction of the tracked target is within a predetermined time period is a second direction, the first direction is different from the second direction, and the tracked target is in the first direction within a predetermined time period The moving distance is greater than a preset first distance threshold, and when the moving distance of the tracked target in the second direction is greater than a preset second distance threshold within a predetermined time period, an alarm prompt message is generated, the alarm prompt The information is used to prompt the first electronic device to drop the tracked target.

In an embodiment, after the generating the alert prompt information, the method further comprises: re-detecting the tracked target, and performing the vision-based target tracking again.

According to a second aspect of the embodiments of the present invention, there is also provided an electronic device, comprising: a target tracking unit configured to perform vision-based target tracking to obtain a first tracking result for a tracked target; a processing unit, Configuring, according to the motion information used to describe the motion state of the tracked target, in combination with the first tracking result, analyzing to obtain a second tracking result for the tracked target; wherein the second tracking result is The first tracking result is the same or different; the motion information for describing the motion state of the tracked target is obtained based on measurement by another electronic device disposed on the tracked target.

In an embodiment, the motion information includes motion measurement parameters for the tracked target, the motion measurement parameters being recorded in a predetermined storage space of the another electronic device, the motion measurement parameters including at least: An acceleration and/or a moving angular velocity; the electronic device further comprising a first reading unit configured to periodically or aperiodically access a predetermined storage space of the another electronic device to acquire the another electronic device at a predetermined time The motion measurement parameter is collected in the segment; the processing unit is configured to perform statistical analysis on the motion measurement parameter in the predetermined time period acquired by the first reading unit, to obtain the tracked target The motion description information in the predetermined time period, the first tracking result is corrected according to the motion description information, and the second tracking result is obtained; wherein the motion description information includes at least a motion speed and/or a motion direction .

In an embodiment, the processing unit is configured to combine the motion measurement parameter of the another electronic device acquired by the first reading unit with a predetermined time period, and the first electronic device is in the predetermined The self-motion measurement parameter in the time period is analyzed, and the motion description information of the tracked target relative to the first electronic device during the predetermined time period is obtained.

In an embodiment, the motion information includes motion description information of the tracked target, and the motion description information is used by the another electronic device according to the acquisition for a predetermined time period. The dynamic measurement parameter is obtained by statistical analysis, and the motion measurement parameter includes at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in the other a predetermined storage space of the electronic device; the electronic device further includes a second reading unit configured to periodically or aperiodically access a predetermined storage space of the another electronic device, and acquire the another electronic device at a predetermined The motion description information in the time period; the processing unit is configured to correct the first tracking result according to the motion description information in the predetermined time period acquired by the second reading unit, to obtain the first Second tracking results.

In an embodiment, the motion description information includes a state probability of the tracked target in different motion directions; the processing unit is configured to be based on a state probability of the tracked target in different motion directions, and Determining, by the first tracking result, the tracked target is at least one candidate tracking area in the tracking image, determining a trusted tracking area of the tracked target in the tracking image, and generating a second tracking based on the trusted tracking area a result; wherein the state probability is used to indicate a probability of the tracked target moving in different directions of motion.

In an embodiment, the electronic device further includes an alarm unit configured to: when the motion description information in the predetermined time period indicates that the direction of motion of the tracked target within a predetermined time period is a first direction, the predetermined The first tracking result in the time period represents that the moving direction of the tracked target in the predetermined time period is the second direction, the first direction is different from the second direction, and the tracked target is in the predetermined time period The moving distance in the first direction is greater than a preset first distance threshold, and when the moving distance of the tracked target in the second direction is greater than a preset second distance threshold in a predetermined time period, an alarm prompt message is generated. The alarm prompt information is used to prompt the first electronic device to drop the tracked target.

In an embodiment, the target tracking unit is further configured to re-detect the tracked target and re-execute the vision-based target tracking after the alarm unit generates the alarm prompt information.

According to a third aspect of the present invention, a target tracking system is further provided, the target tracking system includes: an electronic device according to an embodiment of the present invention; and another electronic device disposed on the tracked target, configured The motion information for describing the motion state of the tracked target is obtained based on the measurement.

According to a fourth aspect of the present invention, there is further provided a computer storage medium having stored therein computer executable instructions for performing target tracking according to an embodiment of the present invention method.

The object tracking method, the system, the electronic device, and the computer storage medium provided by the embodiment of the present invention, in the process of tracking the target, the first electronic device may obtain motion information of the target motion state measured by the second electronic device, according to the The motion information is combined with the first tracking result obtained by the first electronic device in real time to obtain a second tracking result of the tracked target, and the motion state of the tracked target is detected in real time. According to the technical solution of the embodiment of the present invention, the second tracking result is obtained by combining the motion information of the tracked target measured by the second electronic device, so that the obtained tracking result is more accurate, and the accuracy of the tracking target in the prior art is solved. The problem increases the robustness of visual tracking.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flow chart 1 of an optional target tracking method according to an embodiment of the present invention;

2 is a block diagram of an alternative electronic device in accordance with an embodiment of the present invention;

3 is a schematic diagram of an optional target tracking system in accordance with an embodiment of the present invention;

4 is a schematic diagram of an implementation scenario of an optional target tracking method according to an embodiment of the present invention;

5 is a flow chart 2 of an alternative target tracking method in accordance with an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is an embodiment of the invention, but not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order. It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

Glossary:

Visual tracking: refers to the detection, extraction, recognition and tracking of moving targets in the image sequence, and obtains the motion parameters of the moving target, such as position, velocity, acceleration and motion trajectory, so as to carry out the next processing and analysis to achieve the motion The behavior of the target is understood to complete a higher level of testing tasks.

The Inertial Measurement Unit (IMU) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object. Gyros and accelerometers are the main components of the IMU, and their accuracy directly affects the accuracy of the inertial system. In general, an IMU consists of three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration of an object's independent three-axis acceleration signal in the carrier coordinate system, while the gyroscope detects the angular velocity of the carrier relative to the navigational coordinate system. The signal measures the angular velocity and acceleration of the object in three dimensions and solves the attitude of the object.

In accordance with an embodiment of the present invention, an embodiment of a target tracking method is provided, it being noted that the steps illustrated in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, and, although The logical order is shown in the flowcharts, but in some cases the steps shown or described may be performed in a different order than the ones described herein.

FIG. 1 is a flowchart 1 of an optional target tracking method according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

Step S102, performing vision-based target tracking to obtain a first tracking result for the tracked target.

Step S104, according to the motion information for describing the motion state of the tracked target, combined with the first tracking result, the analysis obtains the second tracking result for the tracked target.

The method is applied to a first electronic device, the first electronic device has a target tracking unit, and the target tracking unit is configured to perform vision-based target tracking; the second tracking result is the same as or different from the first tracking result; The motion information of the target motion state is obtained based on the measurement by the second electronic device disposed on the tracked target.

In the embodiment of the present invention, in the process of tracking the target, the first electronic device may receive the motion information of the target motion state measured by the second electronic device, and combine the motion information of the target motion state acquired by the second electronic device. The first tracking result obtained by the first electronic device in real time is analyzed to obtain the second tracking result of the tracking target, and the motion state of the tracking target can be detected in real time, and the tracking target is obtained by real-time measurement by the second electronic device set on the tracking target. And the first tracking result of the tracking target acquired by the first electronic device, and the second tracking result is obtained by using the motion information to correct the first tracking result, and the second tracking result is obtained by using the foregoing solution, instead of directly The first tracking result of the first electronic device is used as the final result, and the second tracking result is obtained by analyzing the motion information of the tracking target measured by the second electronic device, and the obtained result is more accurate, and the tracking target in the prior art is solved. Low accuracy time, increasing the robustness of visual tracking .

In the above embodiment, a mobile device (ie, a second electronic device) capable of detecting a target motion state may be disposed on the tracking target without cumbersome peripheral devices, and the motion information obtained by the second electronic device is combined with the second The first tracking result obtained by the electronic device obtains the second tracking result, which not only can obtain an accurate tracking result, but also can reduce the tracking cost.

For the above embodiment, the embodiment of the present application is applied to the first electronic device, and the first electronic device may be a mobile device that detects the motion state of the tracking target, and the mobile device may be provided with a target tracking unit, and the target tracking unit may be used for Perform vision-based target tracking.

The first electronic device in the above embodiment may have a corresponding tracking system, which may be installed on the first electronic device, and is used to control the operation of each unit of the first electronic device, and the tracking system is directed to the first electronic device. Each unit issues an instruction, which can instruct each unit of the first electronic device to work, such that the first electronic device can track the target, wherein the representation of the instruction can be a data packet or code, each unit in the first electronic device After receiving the instruction, the corresponding task is executed according to the instruction information, so that the second electronic device can be interacted to achieve the function of accurately tracking the target.

In an embodiment, the first electronic device can be a robot.

For the above embodiment, the vision-based target tracking is performed by the target tracking unit, and the first tracking result for the tracking target is obtained, wherein the target tracking unit may be provided with a detecting device, and the detecting device may detect the first position of the tracking target Information and tracking the motion state of the target, which includes the direction of motion and the angle of motion. For the target tracking unit, the detection device detects the position of the tracking target, and acquires the first position of the target. After the target tracking unit acquires the first position of the tracking target, the position information of the first position forms the first data. The data may include coordinate information of the tracking target, a current time, and a distance between the tracking target and the first electronic device, where the coordinate information may be acquired according to a positioning unit in the first electronic device, and the target tracking unit And storing the acquired first data; the target tracking unit transmits the first data to the storage unit of the first electronic device, where the first electronic device acquires After the first data, the detection instruction is sent to the target tracking unit again, and the target tracking unit acquires the second location information of the tracking target again according to the received detection instruction, and the target tracking unit forms the second location information of the acquired tracking target into a second. Data, the target tracking unit sends the acquired second data to the first electronic device, and the first electronic device compares and analyzes according to the first data and the second data to obtain a moving direction, a moving speed, and a current tracking target of the tracking target. Current coordinate data. The moving direction of the tracking target may be determined according to the coordinate of the tracking target of the second data relative to the coordinate of the tracking target of the first data, and the moving speed of the tracking target may be determined according to where the first data is located. The time and the time in the second data and the moving distance of the tracking target determine the moving speed of the tracking target when moving between the first data and the second data. The first electronic device forms a first tracking result by using the analyzed moving direction, coordinate information, moving speed, moving time, and the distance between the tracking target and the first electronic device, and the first electronic device stores the first tracking result. And then send the first tracking result to the target tracking unit.

In an embodiment, the first electronic device may transmit data in a form of wireless transmission, and the system of the first electronic device sends an instruction to the target tracking unit of the first electronic device according to the first tracking result, so that the first electronic device moves, The direction and speed of the movement may be consistent with the moving direction and the moving speed of the tracking target according to the first tracking result, and the tracking speed and direction may be appropriately adjusted, for example, if the distance between the first electronic device and the tracking target is long, The first electronic device can control the tracking speed to exceed the moving speed of the tracking target in the first tracking result when starting, and the tracking speed of the first electronic device can be reduced when the distance between the first electronic device and the tracking target reaches a preset distance It is consistent with the moving speed in the first tracking result.

In an embodiment, the moving direction of the moving target includes at least one of: forward, backward, stop, left shift, right shift.

In an embodiment, the target tracking unit in the foregoing embodiment may include radio frequency identification (RFID).

In an embodiment, the tracking target in the above embodiment may be mobile, such as a moving person or item.

For the above embodiment, after obtaining the first tracking result of the tracking target, step S104 is performed, and according to the motion information for describing the motion state of the tracking target, combined with the first tracking result, the second tracking result for the tracking target is obtained by analyzing, wherein The motion information for describing the motion state of the tracking target is acquired by a second electronic device disposed on the tracking target, and the second electronic device may include a detecting unit, a processing unit, and a storage unit, and the detecting unit may detect the movement of the tracking target. a state, the detecting operation may be performed according to an instruction issued by a system of the second electronic device, after receiving the instruction, the detecting unit starts detecting the coordinate information of the current second electronic device, and simultaneously acquires information of the tracking target, and the detecting unit Transmitting the information of the tracking target to the storage unit to form first data, where the first data includes the coordinates of the current tracking target, the current time, and the moving direction of the tracking target, and the detecting unit transmits the coordinate information and the current time to form the first data to the second data. Storage unit of electronic device, second electronic device After acquiring the first data, the system sends an instruction to the detecting unit again. After receiving the instruction again, the detecting unit can detect the coordinates of the second electronic device and the information of the tracking target again, and the detecting unit transmits the information to the storage unit to form the first data. The second electronic device analyzes the moving angle and the moving speed of the tracking target according to the first data and the second data, and then the second electronic device continuously detects the moving information of the tracking target according to the above manner, and the second electronic device will track the target Information such as the direction of movement, the speed of movement, and the time of change are stored to form second motion data.

For the above embodiment, after the first electronic device obtains the second tracking data, the second electronic device may read the second motion data by using the reading unit, and after acquiring the second motion data, the first electronic device A tracking result is compared, and a second tracking result is obtained. The second tracking result may be the same as or different from the first tracking result. When the first tracking result is the same as the second tracking result, the first tracking result may not be updated. An electronic device can move according to the first tracking result, and can predict the moving direction and shift of the tracking target according to the first tracking result. The speed of the prediction may be the same as or different from the first tracking result; when the first tracking result is different from the second tracking result, the first electronic device determines the magnitude of the deviation of the result, and the deviation value of the comparison result may be given here. A threshold is set, which is determined according to the motion state of the tracking target (for example, the moving speed is set to 5 m/min), and the difference between the first tracking result and the second tracking result is large (for example, comparison) The difference of the result exceeds the threshold), the first tracking result needs to be changed, wherein the moving direction of the tracking target, the moving speed, the coordinate, and the moving change angle of the tracking target may be updated, and the updated data may be the data in the second tracking result. The data may be adjusted according to the second tracking result. If the difference between the first tracking result and the second tracking result is small, for example, when the difference of the comparison result is lower than the threshold, the first tracking result may not be Update.

When the difference between the first tracking result and the second tracking result in the foregoing embodiment is large, the result of the “following” of the first electronic device during the tracking process may be obtained, thereby updating the data and changing the first electronic device. Direction of movement, speed of movement, and angle of movement.

Specifically, after obtaining the second tracking result, the foregoing embodiment may analyze multiple conclusions, such as tracking target modeling and/or tracking target positioning. Among them, target modeling is divided into offline target modeling and online target modeling. Offline target modeling refers to the model that has already trained the tracking target before starting tracking, and whether to identify the tracking target in the real-time tracking process; online target Modeling refers to learning and updating the target model during the tracking process, and simultaneously identifying the tracking target.

In an embodiment, the second electronic device in the foregoing embodiment may be a mobile communication device (eg, a remote controller, a mobile phone) with an IMU chip built therein, and the device may communicate the IMU information to the first electronic device in real time, where The IMU chip of the second electronic device may be the detecting unit of the above embodiment.

In an embodiment, the second electronic device may be disposed on the body of the tracking target, and may be an item worn by the tracking target or a hand held by the tracking target.

In the process of transmitting data by the first electronic device and the second electronic device, the number of wires may be According to the transmission, it can also be wireless data transmission.

In an embodiment, the first electronic device may include a photographing device that can capture a tracking target, and the photographing device can transmit the photographed information to the first electronic device in real time, and the photographing device can be a camera.

In an embodiment, the tracking in the above embodiment may be visual tracking, which may be obtained by a photographing device in the first electronic device, and the photographing device captures a current environment of the first electronic device, wherein the current environment includes Tracking the walking path of the target, the obstacle in the walking direction, the tracking target, the light intensity, if there is an obstacle in the walking direction of the first electronic device, analyzing the coordinates of the obstacle and the size of the obstacle, the current tracking environment The environment between the first electronic device and the tracking target is further included, and the environment between the first electronic device and the tracking target includes an obstacle, a light intensity, and a background when the first electronic device moves toward the tracking target.

In an embodiment, the motion information includes motion measurement parameters for the tracking target, the motion measurement parameters are recorded in a predetermined storage space of the second electronic device, and the motion measurement parameters include at least: motion acceleration and/or motion angular velocity, the first electronic device Acquiring the motion information, including: periodically or aperiodically accessing a predetermined storage space of the second electronic device, and acquiring a motion measurement parameter collected by the second electronic device in a predetermined time period, where the periodicity refers to the first electronic device A time interval is set, for example, 0.5 seconds, 1 second, 5 seconds, 10 seconds, 20 seconds, etc., after a time interval, the first electronic device acquires tracking target motion change data stored in a predetermined storage space of the second electronic device. Correspondingly, according to the motion information used to describe the motion state of the tracking target, combining the first tracking result, the analyzing obtains the second tracking result for the tracking target, including: performing statistical analysis on the motion measurement parameter in the predetermined time period, and obtaining Tracking the motion description information of the target within a predetermined time period, correcting the first tracking result according to the motion description information, and obtaining a second tracking result; wherein the motion description information includes at least a motion speed and/or a motion direction, and may be combined with the second electronic The motion measurement parameter of the device in the predetermined time period and the self-motion measurement parameter of the first electronic device within a predetermined time period, and the comprehensive analysis obtains the tracking target relative to the predetermined time period. The motion description information of the first electronic device.

For the above embodiment, the first electronic device obtains the second tracking result by continuously acquiring the first tracking result and the motion data analysis of the second electronic device, so that the accuracy of the tracking can be increased by the motion data acquired in real time, thereby accurately Tracking the target also increases the stability of the tracking target.

For the above embodiment, the motion measurement parameter may be continuously updated. After the second electronic device measures the data of the new tracking target again, the data is newly stored in the predetermined storage space of the second electronic device, and the original data may be The deletion may also be temporarily retained. For example, when the predetermined storage space of the second electronic device is about to be full, the data of the original second electronic device may be deleted.

For the acceleration in the above embodiment, it can be obtained by measuring the speed change information of the tracking target, and after the tracking target changes the traveling speed, the motion acceleration of the tracking target can be calculated.

According to the above embodiment, the statistical measurement of the motion measurement parameter of the first electronic device may be performed, and the statistical analysis includes analyzing the position change, the moving direction, and the moving speed of the tracking target measured by the first electronic device, and may pass the second time. The obtained position is compared with the first obtained position to obtain a motion measurement parameter, for example, a screen with a length and a width of 10 meters is set, and the first tracking target is at the center, and the coordinates are (5, 5), and the second The tracking target is at the northernmost center, and its coordinates are (5, 10). It can be judged by analysis that the target moves in the true north direction during the time interval between the first measurement and the second measurement. For 5 meters, the moving distance and the moving time are calculated to calculate the moving speed. By calculating the change of the motion measurement parameter and analyzing the motion state of the tracking target, the first electronic device can store the motion change data, and after comprehensive analysis, finally obtain the motion description information of the tracking target within a predetermined time period, the predetermined time. The time interval between the first electronic device and the second electronic device is transmitted. The data is acquired by the first electronic device every time interval (for example, 5 seconds).

For the above embodiment, the motion information includes motion description information for the tracking target, the first power The child device obtains motion description information by periodically or aperiodically accessing a predetermined storage space of the second electronic device, and the predetermined storage space of the second electronic device is determined relative to the size of the storage device of the second electronic device. If the predetermined storage space is large, the second electronic device may store motion change information of the plurality of second electronic devices, and if the predetermined storage space is small, for example: 10, the second electronic device may store less second electronic The motion change information of the device, for example, two.

For the above embodiment, the first tracking result is corrected according to the motion description information in the predetermined time period, and finally the second tracking result is obtained. When the first tracking result is corrected, the motion change parameter may be adopted, and the motion change parameter may be adopted. Based on the first tracking result, comparing the motion information in the second electronic device, the second tracking result is obtained. For example, the value of the motion change angle may be set within 10 degrees. If the motion direction in the second tracking result changes within the value relative to the motion direction of the first tracking result, the motion direction or angle information of the first tracking result is not used. Changing, if the direction of motion in the second tracking result changes by more than the value of the first tracking result, the motion direction or angle information of the first tracking result may be corrected, and the motion direction or the motion angle in the first tracking result may be changed. Modified to a motion direction or a motion angle in the second motion data acquired according to the second electronic device.

By comparing the first tracking result with the second motion data, it can be determined whether the motion data of the first electronic device is corrected, and the tracking error can be reduced.

For the above embodiment, a large error may occur when tracking the target, for example, tracking the target in a vision-based tracking technique, and the first tracking result and the second electronic device that can be measured at the first electronic device when the time is extended When the second motion data obtained by the measurement is compared with the error, the data of the first tracking result is corrected to ensure the stability during the long-term tracking process. In an implementation manner, when the tracking target suddenly accelerates, the tracking target will be It is easy to lose, when the measured first tracking result is compared with the acquired second motion data, a large error value occurs, and the data of the first tracking result can be corrected (for example, the first electronic device also accelerates the motion, and And correspondingly change the direction of motion) Re-track the target, this will ensure that the tracking target is re-tracked to the target after losing.

For the above embodiment, the first tracking result is corrected according to the motion description information in the predetermined time period, and the second tracking result is obtained, which may be according to the state probability of the tracking target in different motion directions, and the first tracking result. Tracking the target at least one candidate tracking area in the tracking image, determining a confidence tracking area of the tracking target in the tracking image, and generating a second tracking result based on the confidence tracking area, wherein the state probability is used to indicate that the tracking target moves in different motion directions The probability.

In the above embodiment, the motion description information includes the state probability of the tracking target in different motion directions, and refers to estimating the target motion direction or speed when performing motion estimation on the tracking target, wherein the target motion direction or speed The estimation may be obtained by the first electronic device according to the second tracking result or the first tracking result, and the estimated target motion direction or speed may be the same as or different from the data of the second tracking result. After tracking the moving direction of the target, the moving area and the moving line of the corresponding tracking target in the next time interval can be obtained.

In an embodiment, the area in which the tracking target is active may be in a fixed screen, and the next candidate activity area may be sent to the first electronic device according to the second tracking result, where the first electronic device is based on the candidate active area and the tracking target. The active area and the moving route in a certain time interval in the future determine the traveling direction and the traveling speed, thereby improving the accuracy of the tracking.

In an embodiment, the method may further include: the motion description information in the predetermined time period indicates that the motion direction of the tracking target in the predetermined time period is the first direction, and the first tracking result in the predetermined time period indicates that the tracking target is in the The moving direction in the predetermined time period is the second direction, the first direction is different from the second direction, and the moving distance of the tracking target in the first direction is greater than the preset first distance threshold in the predetermined time period, and the tracking target is at the predetermined time. When the moving distance in the second direction in the segment is greater than the preset second distance threshold, an alarm prompt message is generated, and the prompt information is used. The first electronic device is prompted to drop the tracking target. In this embodiment, the alarm information is obtained according to the second tracking result, and the alarm information may be an instruction sent by the processing center of the first electronic device to the target tracking unit, and the target tracking unit may modify the first time after receiving the instruction. Motion data of an electronic device, wherein the motion data may be derived according to the second motion data, and if the second tracking result is different from the data of the first tracking result, the first electronic device adjusts its moving direction and motion The speed, for example, speeds up the movement of the first electronic device.

2 is a structural diagram of an optional electronic device according to an embodiment of the present invention. As shown in FIG. 2, the electronic device includes: a target tracking unit 21 configured to perform vision-based target tracking to obtain a tracking target. a first tracking result; the processing unit 22 is configured to: according to the motion information used to describe the motion state of the tracking target, combine the first tracking result, and obtain a second tracking result for the tracking target; wherein, the second tracking result and the first tracking The results are the same or different; the motion information for describing the motion state of the tracking target is obtained based on the measurement by another electronic device disposed on the tracking target.

In an embodiment, the motion information includes motion measurement parameters for the tracked target, the motion measurement parameters being recorded in a predetermined storage space of the another electronic device, the motion measurement parameters including at least: Acceleration and/or angular velocity of motion;

The electronic device further includes a first reading unit configured to periodically or aperiodically access a predetermined storage space of the another electronic device, and acquire the collected by the another electronic device within a predetermined time period. Motion measurement parameters;

The processing unit 22 is configured to perform statistical analysis on the motion measurement parameter in the predetermined time period acquired by the first reading unit, to obtain motion description information of the tracked target in a predetermined time period, And correcting the first tracking result according to the motion description information to obtain the second tracking result; wherein the motion description information includes at least a motion speed and/or a motion direction.

The processing unit 22 is configured to combine the other acquired by the first reading unit. Obtaining, by the electronic device, the motion measurement parameter within the predetermined time period and the self-motion measurement parameter of the first electronic device within the predetermined time period, and obtaining the tracked target for the predetermined time period relative to the The motion description information of the first electronic device.

In another embodiment, the motion information includes motion description information for the tracked target, and the motion description information is statistically analyzed by the another electronic device according to motion measurement parameters within a predetermined time period obtained by the acquisition. Obtaining that the motion measurement parameter comprises at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in a predetermined storage space of the another electronic device in;

The electronic device further includes a second reading unit configured to periodically or non-periodically access a predetermined storage space of the another electronic device, and acquire motion description information of the another electronic device within a predetermined time period;

The processing unit 22 is configured to correct the first tracking result according to the motion description information in a predetermined time period acquired by the second reading unit, to obtain the second tracking result.

Through the above embodiment, in the process of tracking the target, the electronic device may receive the motion information of the target motion state measured by the other electronic device, and combine the first electronic information according to the motion information of the target motion state acquired by the other electronic device. The first tracking result obtained by the device in real time is analyzed by the processing unit 22 to obtain a second tracking result of the tracking target, and the target tracking unit 21 is used to detect the motion state of the tracking target in real time, and real time is performed by using another electronic device set on the tracking target. Obtaining the motion information of the tracking target, combining with the first tracking result of the tracking target acquired by the first electronic device, and analyzing the second tracking result, if the first tracking result is corrected by using the motion information to obtain a second tracking result, by using the foregoing solution, The second tracking result is obtained by combining the motion information of the tracking target measured by another electronic device, and the obtained result is more accurate, which solves the problem of low accuracy when tracking the target in the prior art, and increases the robustness of the visual tracking.

The motion description information includes a state of the tracked target in different directions of motion The processing unit 22 is configured to: according to the state probability of the tracked target in different moving directions, and the at least one candidate tracking area of the tracked image represented by the first tracking result in the tracking image Determining a confidence tracking area of the tracked object in the tracking image, and generating a second tracking result based on the confidence tracking area; wherein the state probability is used to indicate that the tracked object is moving in different motion directions Probability.

The electronic device further includes an alarm unit configured to: when the motion description information in the predetermined time period indicates that the moving direction of the tracked target within a predetermined time period is a first direction, the first in the predetermined time period The tracking result characterizes that the moving direction of the tracked target in the predetermined time period is a second direction, the first direction is different from the second direction, and the tracked target is in the first direction within a predetermined time period The moving distance is greater than a preset first distance threshold, and when the moving distance of the tracked target in the second direction is greater than a preset second distance threshold within a predetermined time period, an alarm prompt message is generated, and the alarm prompt information is generated. And for prompting the first electronic device to drop the tracked target.

In an embodiment, the target tracking unit 21 is further configured to re-detect the tracking target and re-execute the vision-based target tracking after the alarm unit generates the alarm prompt information.

In this embodiment, the target tracking unit 21, the processing unit 22, and the alarm unit in the electronic device may be implemented by a central processing unit (CPU), a digital signal processor (DSP, Digital Signal Processor). a micro-control unit (MCU) or a field-programmable gate array (FPGA); the first reading unit and the second reading unit in the electronic device can be used in practical applications. Communication module (including: basic communication suite, operating system, communication module, standardized interface and protocol, etc.) and transceiver antenna implementation.

It should be noted that, when performing the target tracking, the electronic device provided by the foregoing embodiment is only illustrated by the division of each of the foregoing program modules. In actual applications, the processing allocation may be completed by different program modules as needed. The internal structure of the electronic device is divided into different Program modules to perform all or part of the processing described above. In addition, the electronic device and the target tracking method are provided in the same embodiment. For details, refer to the method embodiment, and details are not described herein.

Embodiments of the present invention also provide an electronic device including: a processor and a memory for storing a computer program executable on the processor. among them,

The memory can be implemented by any type of volatile or non-volatile storage device, or a combination thereof. The non-volatile memory may be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), or an Erasable Programmable Read (EPROM). Only Memory), Electrically Erasable Programmable Read-Only Memory (EEPROM), Ferromagnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory , CD-ROM, or Compact Disc Read-Only Memory (CD-ROM); the magnetic surface memory can be a disk storage or a tape storage. The volatile memory can be a random access memory (RAM) that acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access (SSRAM). DRAM (Dynamic Random Access Memory), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), enhancement Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory Bus Random Access Memory (DRRAM, Direct) Rambus Random Access Memory). The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

The method disclosed in the foregoing embodiments of the present invention may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in a processor or an instruction in a form of software. The above described processor may be a general purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. The processor may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present invention. A general purpose processor can be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiment of the present invention may be directly implemented as a hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, the storage medium being located in the memory, the processor reading the information in the memory, and completing the steps of the foregoing methods in combination with the hardware thereof.

In the embodiment of the present invention, when the processor is used to run the computer program, performing: performing vision-based target tracking to obtain a first tracking result for the tracked target; according to describing the motion state of the tracked target Motion information, in combination with the first tracking result, analyzing to obtain a second tracking result for the tracked target; wherein the second tracking result is the same as or different from the first tracking result; The motion information of the tracked target motion state is obtained based on the measurement by the second electronic device disposed on the tracked target.

In an embodiment, the motion information includes motion measurement parameters for the tracked target, the motion measurement parameters are recorded in a predetermined storage space of the second electronic device, and the motion measurement parameters include at least: Acceleration and/or angular velocity of movement; the processor is configured to: periodically or aperiodically access a predetermined storage space of the second electronic device, and acquire the second electronic device at a predetermined time when the computer program is executed Collecting the motion measurement parameters obtained in the segment; performing statistical analysis on the motion measurement parameters in the predetermined time period, Obtaining motion description information of the tracked target within a predetermined time period, correcting the first tracking result according to the motion description information, and obtaining the second tracking result; wherein the motion description information includes at least motion Speed and / or direction of movement.

In an embodiment, the processor is configured to: when the computer program is executed, perform motion measurement parameters in combination with the second electronic device for a predetermined period of time, and the first electronic device is in the predetermined time period The self-motion measurement parameter within the analysis obtains motion description information of the tracked target relative to the first electronic device during the predetermined time period.

In an embodiment, the motion information includes motion description information of the tracked target, and the motion description information is obtained by the second electronic device according to the motion measurement parameter in the predetermined time period obtained by the acquisition. The motion measurement parameter includes at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in a predetermined storage space of the second electronic device. The processor is configured to: periodically or non-periodically access a predetermined storage space of the second electronic device, and acquire motion description information of the second electronic device within a predetermined time period; And correcting the first tracking result according to the motion description information in the predetermined time period to obtain the second tracking result.

In an embodiment, when the processor is configured to run the computer program, performing: a state probability according to the tracked target in different motion directions, and the tracked target represented by the first tracking result Determining, in at least one candidate tracking area in the tracking image, a confidence tracking area of the tracked object in the tracking image, generating a second tracking result based on the trusted tracking area; wherein the state probability is used to represent the Track the probability that a target will move in different directions of motion.

In an embodiment, when the processor is configured to run the computer program, performing: when the motion description information in the predetermined time period indicates that the moving direction of the tracked target in a predetermined time period is a first direction, The first tracking result within the predetermined time period characterizes the tracked target The moving direction marked in the predetermined time period is a second direction, the first direction is different from the second direction, and the moving distance of the tracked target in the first direction is greater than a preset number in a predetermined time period a distance threshold, when the moving distance of the tracked target in the second direction is greater than a preset second distance threshold in a predetermined time period, generating alarm prompt information, wherein the alarm prompt information is used to prompt the first The electronic device loses the tracked target.

In an embodiment, when the processor is configured to run the computer program, performing: re-detecting the tracked target after the generating the alert prompt information, and performing the vision-based target tracking again.

In an exemplary embodiment, an embodiment of the present invention further provides a computer readable storage medium, such as a memory including a computer program, which may be executed by a processor of an electronic device to perform the steps of the foregoing method. The computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories, such as Mobile phones, computers, tablet devices, personal digital assistants, etc.

Embodiments of the present invention provide a computer storage medium, where the computer storage medium stores computer executable instructions for performing: performing vision-based target tracking to obtain a first target for a tracked target Tracking the result; according to the motion information for describing the motion state of the tracked target, combining with the first tracking result, analyzing and obtaining a second tracking result for the tracked target; wherein the second tracking result and the The first tracking result is the same or different; the motion information for describing the motion state of the tracked target is obtained based on the measurement by the second electronic device disposed on the tracked target.

In an embodiment, the motion information includes motion measurement parameters for the tracked target, the motion measurement parameters are recorded in a predetermined storage space of the second electronic device, and the motion measurement parameters include at least: Acceleration and/or angular velocity of movement; the computer executable instructions for performing: periodically or aperiodically accessing a predetermined storage space of the second electronic device Obtaining the motion measurement parameter collected by the second electronic device in a predetermined time period; performing statistical analysis on the motion measurement parameter in the predetermined time period to obtain the tracked target in a predetermined time period The motion description information is performed, and the first tracking result is corrected according to the motion description information to obtain the second tracking result; wherein the motion description information includes at least a motion speed and/or a motion direction.

In an embodiment, the computer executable instructions are configured to: combine motion measurement parameters of the second electronic device for a predetermined period of time, and self-motion of the first electronic device during the predetermined time period Measuring parameters, the analysis obtains motion description information of the tracked target relative to the first electronic device during the predetermined time period.

In an embodiment, the motion information includes motion description information of the tracked target, and the motion description information is obtained by the second electronic device according to the motion measurement parameter in the predetermined time period obtained by the acquisition. The motion measurement parameter includes at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in a predetermined storage space of the second electronic device. The computer executable instructions are configured to: periodically or non-periodically access a predetermined storage space of the second electronic device, and acquire motion description information of the second electronic device within a predetermined time period; The motion description information in a predetermined time period, the first tracking result is corrected, and the second tracking result is obtained.

In an embodiment, the computer executable instructions are configured to perform: a state probability according to the tracked target in different motion directions, and the tracked target represented by the first tracking result is in a tracking image Determining at least one candidate tracking area, determining a confidence tracking area of the tracked object in the tracking image, generating a second tracking result based on the trusted tracking area; wherein the state probability is used to indicate that the tracked target is different The probability of motion in the direction of motion.

In an embodiment, the computer executable instructions are for executing: when the predetermined time period The motion description information inside indicates that the moving direction of the tracked target in the predetermined time period is the first direction, and the first tracking result in the predetermined time period represents that the moving direction of the tracked target in the predetermined time period is a second direction, the first direction is different from the second direction, and the moving distance of the tracked target in the first direction is greater than a preset first distance threshold in a predetermined time period, and the tracked target is When the moving distance in the second direction is greater than the preset second distance threshold in the predetermined time period, the alarm prompt information is generated, and the alarm prompt information is used to prompt the first electronic device to drop the tracked target.

In an embodiment, the computer executable instructions are configured to: after the generating the alert prompt information, re-detect the tracked target and re-execute the vision-based target tracking.

FIG. 3 is a schematic diagram of an optional target tracking system according to an embodiment of the present invention. As shown in FIG. 3, the target tracking system includes: the first electronic device 31 and the second electronic device 32 in the above embodiment. The first electronic device 31 is configured to perform a vision-based target tracking, obtain a first tracking result for the tracking target, and according to the motion information for describing the tracking target motion state, combined with the first tracking result, the analysis obtains the tracking target a second tracking result of the target; the second electronic device 32, disposed on the tracking target, configured to obtain motion information for describing a motion state of the tracking target based on the measurement.

In an embodiment, the motion information includes motion measurement parameters for the tracked target, the motion measurement parameters are recorded in a predetermined storage space of the second electronic device, and the motion measurement parameters include at least: And the first electronic device 31 is configured to periodically or non-periodically access a predetermined storage space of the second electronic device, and acquire the second electronic device to acquire the data within a predetermined time period. Performing statistical analysis on the motion measurement parameter in the predetermined time period to obtain motion description information of the tracked target within a predetermined time period, according to the motion description information A tracking result is corrected to obtain the second tracking result; wherein the motion description information includes at least a motion speed and/or a motion direction.

The first electronic device 31 is configured to combine the motion measurement parameters of the second electronic device in a predetermined time period and the self-motion measurement parameters of the first electronic device in the predetermined time period, and analyze Obtaining motion description information of the tracked target relative to the first electronic device during the predetermined time period.

In an embodiment, the motion information includes motion description information of the tracked target, and the motion description information is obtained by the second electronic device according to the motion measurement parameter in the predetermined time period obtained by the acquisition. The motion measurement parameter includes at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in a predetermined storage space of the second electronic device. The first electronic device 31 is configured to periodically or non-periodically access a predetermined storage space of the second electronic device, and acquire motion description information of the second electronic device within a predetermined time period; The motion description information in a predetermined time period, the first tracking result is corrected, and the second tracking result is obtained.

In an embodiment, the motion description information includes a state probability of the tracked target in different motion directions; the first electronic device 31 is configured to be based on a state probability of the tracked target in different motion directions. And the at least one candidate tracking area in the tracking image of the tracked target represented by the first tracking result, determining a confidence tracking area of the tracked target in the tracking image, and generating a first Two tracking results; wherein the state probability is used to indicate a probability of the tracked target moving in different directions of motion.

In an embodiment, the first electronic device 31 is configured to: when the motion description information in the predetermined time period indicates that the moving direction of the tracked target in a predetermined time period is a first direction, the predetermined time The first tracking result in the segment characterizes that the moving direction of the tracked target in the predetermined time period is a second direction, the first direction is different from the second direction, and the tracked target is in the predetermined time period The moving distance in the first direction is greater than a preset first distance threshold, and the moving distance of the tracked target in the second direction is greater than the pre-predetermined time period When the second distance threshold is set, an alarm prompt message is generated, and the alarm prompt information is used to prompt the first electronic device to drop the tracked target.

In an embodiment, the first electronic device 31 is configured to re-detect the tracked target and re-execute the vision-based target tracking after the generating the alert prompt information.

Through the above embodiment, in the process of tracking the target, the first electronic device 31 can obtain the motion information of the target motion state measured by the second electronic device 32, and the motion information of the target motion state acquired according to the second electronic device 32. Combining the first tracking result obtained by the first electronic device 31 in real time, the second tracking result of the tracking target is obtained, and the motion state of the tracking target is detected in real time, and the second electronic device 32 set on the tracking target is used to measure and obtain the tracking in real time. The motion information of the target is combined with the first tracking result of the tracking target acquired by the first electronic device 31 to obtain a second tracking result. If the first tracking result is corrected by using the motion information, the second tracking result is obtained. The first tracking result of the first electronic device 31 is used as the final result, and the second tracking result is obtained by analyzing the motion information of the tracking target measured by the second electronic device 32, and the obtained result is more accurate, and the prior art is solved. The problem of low accuracy when tracking targets increases the robustness of visual tracking .

The invention is described below in an alternative embodiment.

4 is a schematic diagram of an implementation scenario of an optional target tracking method according to an embodiment of the present invention. As shown in FIG. 4, the tracking target of the solution is a person in the screen, and the measurement environment in which the device is located is a relative screen. The first electronic device is a robot, and the second electronic device is a device that uses an IMU chip. The solution completes the assistance of visual tracking by collecting feedback information sent by the IMU.

First of all, the robot continuously detects the motion state of the person in the screen by continuously scanning, and the robot has a memory therein, and the memory of the robot can store the state of the motion data of the detected person in the screen and form inside the robot storage space. The first motion data, the second electronic device including the IMU chip is placed on the person inside the screen, and the IMU chip can be checked in real time. The second electronic device includes a storage unit, and the storage unit of the IMU device can store the motion data of the detected motion state of the person to form second motion data, and the robot performs the second motion data returned by the IMU. Through a series of statistical learning analysis, the robot learns a statistical result through statistical analysis. The robot can analyze the human motion behavior in the screen through the result data, and obtain the target motion information based on the probability, for example, estimating the current movement direction of the sports person. The moving direction includes: forward, backward, stop, left shift, and right shift. The effective motion data in the motion information is integrated into the visual tracking algorithm to achieve the purpose of increasing the tracking target.

Through the above embodiment, the position motion direction and the motion speed of the person in the screen can be predicted, and the robot analyzes the motion of the second electronic device carried by the person relative to the robot according to the data returned by the IMU, and the robot can estimate by motion analysis. The movement of the target in the screen, which can generate a confidence area, thereby increasing the stability and accuracy of the tracking target.

With the above embodiment, if two pending results are returned: the first predicted moving direction Rectang1 and the second predicted moving direction Rectangle2, and the robot predicts that the pedestrian is moving to the right according to the obtained IMU feedback information, the confidence area appears on the right. On the side, the confidence of Rectangle2 is higher, Rectangle1 can be suppressed, and the middle or final filter result of visual tracking is Rectangle2, that is, the person inside the prediction screen will move to the right.

Through the above embodiments, the machine can be corrected in time when the visual tracking algorithm of the machine fails. For example, if the robot obtains that the target motion direction is seriously inconsistent with the motion direction estimated based on the visual technology, the robot can make a “follow” judgment. At this time, the robot can re-determine the moving direction and moving speed of the target in the screen through the information returned by the second electronic device of the IMU, so that the robot can correct the future moving direction of the robot through the controller.

For the above embodiment, if Rectang1 and Rectangle2 are moved to the left compared with the previous consecutive n frames, that is, the visual tracking result shows that the person has been moving to the left in the n frames of video, and the moving distance to the left is greater than the distance threshold. The IMU feedback message indicates that the target has been moving to the right. When the moving distance is greater than the distance threshold, then the visual tracking can be basically judged as “following”, the visual tracking algorithm can be re-initialized, and the moving direction of the robot is re-determined, so that the robot can continue to track the target.

The visual tracking algorithm of the above embodiment may include at least the following algorithms: a camshift algorithm, an optical flow tracking algorithm, and a particle filtering algorithm.

Through the above embodiment, in the process of tracking the person in the screen, the robot can receive the motion information of the target motion state measured by the second electronic device on the person in the screen, according to the person acquired by the second electronic device The motion information of the motion state is combined with the first tracking result obtained by the robot in real time to obtain the second tracking result of the person, and the motion state of the person is detected in real time, and the motion of the tracking target is obtained by real-time measurement using the second electronic device set on the person. The information is combined with the first tracking result of the person acquired by the robot, and the second tracking result is obtained by analyzing the first tracking result by using the motion information to obtain the second tracking result. According to the above solution, the first tracking result of the robot is not directly used. As a final result, the second tracking result is obtained by analyzing the motion information of the person measured by the second electronic device, and the obtained result is more accurate, which solves the problem of low accuracy when tracking the target in the prior art, and increases the vision. Tracking robustness.

The invention will now be described in another alternative embodiment.

FIG. 5 is a second flowchart of an optional target tracking method according to an embodiment of the present invention. A visual tracking algorithm is run on a ground mobile robot with a dynamic balance vehicle as a sports chassis, and the user's motion information is carried through the portable body. IMU's mobile devices (such as mobile phones, wristbands, remote controls, tablets, etc.) feed back to the robot to complete the tracking of the visual tracking and track the mobile device. As shown in FIG. 5, the method includes:

In step S501, the robot visual tracking algorithm and the IMU device held by the user are initialized.

Step S502, the visual tracking algorithm establishes a target model, and tracks the user online in real time.

In step S503, the IMU data buffer is periodically accessed during the tracking process, and certain statistical learning is performed on the IMU data in this stage to obtain user motion information based on the probability.

The user motion information can be divided into five states: front, back, left, right, and stop.

In step S504, an image confidence area is obtained.

In step S505, it is determined whether the confidence region and the user's final motion result are completely different or meet a certain "following" criterion.

If yes, go to step S506, if no, go to step S508.

In step S506, the visual algorithm model is no longer updated.

If the results of the two match are completely different, or meet the certain "following" criteria, then the current frame is "followed" and the visual algorithm model is no longer updated.

In step S507, the visual algorithm model is reinitialized or pedestrian detected, and the process returns to step S503.

In step S508, the intermediate or final result of the visual tracking algorithm is corrected.

The intermediate or final result of the visual tracking algorithm is corrected by using the confidence region of step S504, thereby improving the accuracy of the visual tracking algorithm.

In step S509, the visual algorithm updates the model, and returns to step S503.

Through the above embodiment, in the process of tracking the target, the traveling robot can receive the motion information of the motion state of the user measured by the IMU-containing mobile device on the user, and combine the motion information of the target motion state acquired by the mobile device. The first tracking result obtained by the robot in real time is analyzed to obtain the second tracking result of the tracking target, and the accurate information for detecting the motion state of the user in real time is obtained, and the moving information is extracted by the above-mentioned IMU-containing mobile device for object motion analysis, which can be predicted. The user's motion direction or motion angle can also find errors in the predicted target motion state that occur during the tracking process, thereby solving the technical problem of low tracking accuracy caused by long-term visual tracking, and increasing the robustness of visual tracking. . Of course, embodiments of the present invention are not limited to ground mobile robots, but are also applicable to space flight type robots (such as drones).

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

In the above embodiments of the present invention, the descriptions of the various embodiments are all focused on each other. For a part not detailed in the embodiment, reference may be made to the related description of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed technical contents may be implemented in other manners. The embodiment of the electronic device described above is only schematic. For example, the division of the unit may be a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

The above description is only a preferred embodiment of the present invention, and it should be noted that it is for the technical field. Many modifications and refinements can be made by those skilled in the art without departing from the principles of the invention, and such modifications and refinements are also considered to be within the scope of the invention.

Industrial applicability

The technical solution of the embodiment of the present invention combines the motion information of the tracked target measured by the second electronic device to obtain a second tracking result, so that the obtained tracking result is more accurate, and the problem of low accuracy when tracking the target in the prior art is solved. , increased the robustness of visual tracking.

Claims

A target tracking method is applied to a first electronic device, the method comprising:

Performing a vision-based target tracking to obtain a first tracking result for the tracked target;

Obtaining, according to the motion information for describing the motion state of the tracked target, combining the first tracking result, obtaining a second tracking result for the tracked target;

The second tracking result is the same as or different from the first tracking result; the motion information for describing the motion state of the tracked target is based on measurement by a second electronic device disposed on the tracked target obtain.
The method of claim 1, wherein the motion information comprises motion measurement parameters for the tracked target, the motion measurement parameters being recorded in a predetermined storage space of the second electronic device, the motion measurement The parameters include at least: motion acceleration and/or motion angular velocity;

Obtaining the motion information by the first electronic device, including: periodically or aperiodically accessing a predetermined storage space of the second electronic device, and acquiring the motion measurement collected by the second electronic device in a predetermined time period parameter;

According to the motion information for describing the motion state of the tracked target, combined with the first tracking result, the analysis obtains the second tracking result for the tracked target, including:

Performing statistical analysis on the motion measurement parameter in the predetermined time period, obtaining motion description information of the tracked target in a predetermined time period, and correcting the first tracking result according to the motion description information to obtain The second tracking result; wherein the motion description information includes at least a motion speed and/or a motion direction.
The method according to claim 2, wherein said performing statistical analysis on said motion measurement parameter within said predetermined time period to obtain motion description information of a tracked target within a predetermined time period comprises: combining said The motion measurement parameters of the two electronic devices in a predetermined time period and the self-motion measurement parameters of the first electronic device during the predetermined time period are analyzed and obtained And the motion description information of the tracked target relative to the first electronic device during the predetermined time period.
The method of claim 1, wherein the motion information comprises motion description information for the tracked target, the motion description information being measured by a motion of the second electronic device within a predetermined time period obtained according to the acquisition The parameter is obtained by statistical analysis. The motion measurement parameter includes at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in the second electronic device. In the predetermined storage space;

Obtaining the motion information by the first electronic device, including: periodically or non-periodicly accessing a predetermined storage space of the second electronic device, and acquiring motion description information of the second electronic device within a predetermined time period;

And obtaining, according to the motion information used to describe the motion state of the tracked target, the second tracking result obtained for the tracked target according to the first tracking result, including: according to the motion description information in the predetermined time period And correcting the first tracking result to obtain the second tracking result.
The method according to any one of claims 2 to 4, wherein the motion description information includes a state probability of the tracked target in different motion directions; the motion description information according to the predetermined time period, Correcting the first tracking result to obtain the second tracking result, including:

Determining the tracked image in the tracking image according to a state probability of the tracked target in different motion directions, and the tracked target represented by the first tracking result is at least one candidate tracking area in the tracking image a confidence tracking area in which a second tracking result is generated based on the confidence tracking area; wherein the state probability is used to indicate a probability that the tracked object moves in different directions of motion.
The method of claim 5, wherein the method further comprises: when the predetermined The motion description information in the time period indicates that the moving direction of the tracked target in the predetermined time period is the first direction, and the first tracking result in the predetermined time period represents the motion of the tracked target in the predetermined time period The direction is a second direction, the first direction is different from the second direction, and the moving distance of the tracked target in the first direction is greater than a preset first distance threshold in a predetermined time period, the tracked When the moving distance of the target in the second direction is greater than the preset second distance threshold in the predetermined time period, generating alarm prompt information, the alarm prompt information is used to prompt the first electronic device to discard the tracked aims.
The method of claim 6, wherein after the generating the alert prompt information, the method further comprises: re-detecting the tracked target, re-executing the vision-based target tracking.
An electronic device comprising:

a target tracking unit configured to perform vision-based target tracking to obtain a first tracking result for the tracked target;

a processing unit, configured to analyze, according to the motion information used to describe the motion state of the tracked target, in combination with the first tracking result, to obtain a second tracking result for the tracked target;

The second tracking result is the same as or different from the first tracking result; the motion information for describing the motion state of the tracked target is based on measurement by another electronic device disposed on the tracked target obtain.
The electronic device of claim 8, wherein the motion information comprises motion measurement parameters for the tracked target, the motion measurement parameters being recorded in a predetermined storage space of the other electronic device, the motion The measurement parameters include at least: motion acceleration and/or motion angular velocity;

The electronic device further includes a first reading unit configured to periodically or aperiodically access a predetermined storage space of the another electronic device, and acquire the collected by the another electronic device within a predetermined time period. Motion measurement parameters;

The processing unit is configured to perform statistical analysis on the motion measurement parameter in the predetermined time period acquired by the first reading unit, to obtain motion description information of the tracked target in a predetermined time period, according to The motion description information corrects the first tracking result to obtain the second tracking result; wherein the motion description information includes at least a motion speed and/or a motion direction.
The electronic device according to claim 9, wherein the processing unit is configured to combine motion measurement parameters of the another electronic device acquired by the first reading unit for a predetermined period of time, and the first The self-motion measurement parameter of the electronic device during the predetermined time period is analyzed to obtain motion description information of the tracked target relative to the first electronic device during the predetermined time period.
The electronic device of claim 8, wherein the motion information comprises motion description information for the tracked target, the motion description information being motioned by the another electronic device according to the acquisition for a predetermined period of time The measurement parameters are obtained by statistical analysis. The motion measurement parameters include at least: motion acceleration and/or motion angular velocity, the motion description information includes at least a motion speed and/or a motion direction, and the motion description information is recorded in the another electronic In the predetermined storage space of the device;

The electronic device further includes a second reading unit configured to periodically or non-periodically access a predetermined storage space of the another electronic device, and acquire motion description information of the another electronic device within a predetermined time period;

The processing unit is configured to correct the first tracking result according to the motion description information in a predetermined time period acquired by the second reading unit, to obtain the second tracking result.
The electronic device according to any one of claims 9 to 11, wherein the motion description information includes a state probability of the tracked target in different motion directions; the processing unit is configured to be based on the tracked target State probability in different directions of motion, and the first And tracking, by the tracking target, the at least one candidate tracking area in the tracking image, determining a trusted tracking area of the tracked target in the tracking image, and generating a second tracking result based on the trusted tracking area; The state probability is used to indicate the probability that the tracked target will move in different directions of motion.
The electronic device of claim 12, wherein the electronic device further comprises an alarm unit configured to: when the motion description information within the predetermined time period represents a direction of motion of the tracked target within a predetermined time period a direction, the first tracking result in the predetermined time period characterizing a moving direction of the tracked target in a predetermined time period as a second direction, the first direction being different from the second direction, and the tracked target The moving distance in the first direction is greater than a preset first distance threshold in a predetermined time period, and the moving distance of the tracked target in the second direction is greater than a preset second distance threshold in a predetermined time period When the alarm prompt information is generated, the alarm prompt information is used to prompt the first electronic device to drop the tracked target.
The electronic device according to claim 13, wherein the target tracking unit is further configured to re-detect the tracked target and re-execute the vision-based target tracking after the alarm unit generates the alarm prompt information.
A target tracking system comprising:

An electronic device according to any one of claims 8 to 14;

Another electronic device, disposed on the tracked target, is configured to obtain motion information for describing a motion state of the tracked target based on the measurement.
A computer storage medium having stored therein computer executable instructions for performing the target tracking method of any one of claims 1 to 7.