CN103345752A - Method for tracking pedestrian by cooperating robot and mobile phone - Google Patents

Abstract

The invention provides a method for tracking a pedestrian by cooperating a robot and a mobile phone. The pedestrian is tracked in the mode that the mobile robot is mutually cooperated with the mobile phone. Due to the fact that smart phones are popularized at present, the mobile phone carried by the pedestrian and the mobile robot are cooperated to complete a tracking task. According to the method, a specific person can be tracked through movement information of the person. The walking speed and the walking direction of the person can be calculated by the mobile phone according to measured data of an inertia sensor, and the information can be transmitted to the robot. The walking speeds and the walking directions of the nearby pedestrians are measured by the robot through a laser range finder of the robot or other devices, and the person meeting the movement characteristics of the target pedestrian is found. The method cannot be influenced by illumination and is wide in application range and high in accuracy.

Description

Cooperate with the mobile phone method of tracking pedestrians of a kind of robot

Technical field

The invention belongs to the Information Technology Equipment field, especially relate to cooperate with the mobile phone method of tracking pedestrians of a kind of mobile robot.

Background technology

What the present invention studied is the problem that mobile service robot is followed specific pedestrian.Current, based on pedestrian's target following of computer vision developed comparative maturity, also obtained effective application in a lot of occasions.Yet the method for tracking target based on vision sensor also exists a lot of inherent shortcomings, as algorithm be subjected to that illumination effect is big, the losing of depth information in the image imaging process, algorithm complex be high, make as block, the result of use under the situation such as night is unsatisfactory.

Because popularizing of smart mobile phone, the present invention cooperates with the mobile robot by the mobile phone that the pedestrian carries, and finishes tracing task.Tracking based on video is followed the tracks of specific people by people's characteristics of image; And the present invention follows the tracks of specific people by people's motion and spatial information, and these information are provided jointly by the sensors such as laser range finder in the inertial sensor on the mobile phone (accelerometer, electronic compass and gyroscope etc.) and the robot.

More existing literature research calculate people's walking situation by mobile phone: document [2] [3] [7] [8] has been studied by the mobile phone meter problem in step, document [1] has mainly compared the SOME METHODS of estimating pedestrian's walking direction by mobile phone, and document [2] [3] [4] [5] [6] has been studied problem how to estimate pedestrian's step-length.By these methods, can determine speed and the direction of pedestrian's walking.

In addition, in existing document, also usually detect the people of existence on every side and these people's position by the laser range finder that loads in the robot, as [9] [10] [11].By continuous range determination, just can judge these people's movement velocity and direction.

List of references:

[1]Steinhoff，U.and?Schiele，B.Dead?reckoning?from?the?pocket-An?experimental?study.In?proceedings?of?IEEE?International?Conference?on?Pervasive?Computing?and?Communications(PerCom)，Mannheim，Germany，March29-April22010，162-170.

[2]Bylemans，I.，Weyn，M.and?Klepal，M.Mobile?Phone-Based?Displacement?Estimation?for?Opportunistic?Localisation?Systems.In?Proceedings?of?Third?International?Conference?on?Mobile?Ubiquitous?Computing，Systems，Services?and?Technologies，2009(UBICOMM′09)，Sliema，Malta，11-16October2009，113-118.

[3]Shin，S.H.，Lee，M.S.，Park，C.G.and?Hong，H.S.Pedestrian?dead?reckoning?system?with?phone?location?awareness?algorithm.In?Proceedings?of?IEEE/ION?Position?Location?and?Navigation?Symposium(PLANS)，Indian?Wells/Palm?Springs，California?USA，3-6May2010，97-101.

[4]Alzantot，M.and?Youssef，M.UPTIME：Ubiquitous?pedestrian?tracking?using?mobile?phones.In?Proceedings?of?Wireless?Communications?and?Networking?Conference(WCNC)，Paris，France，1-4April2012，3204-3209.

[5]HATANO，Y.″Use?of?the?pedometer?for?promoting?daily?walking?exercise.″Int.Council?Health?Phys.Educ.Recreat，1993，29：4-8.

[6]Harvey?Weinberg，Using?the?ADXL202in?Pedometer?and?Personal?Navigation?Applications，Analog?Devices?AN-602application?Note，2002.

[7]Mladenov，M.，and?Mock，M.：‘A?step?counter?service?for?Java-enabled?devices?using?a?built-in?accelerometer’.In?Proceedings?of?the1st?International?Workshop?on?Context-Aware?Middleware?and?Services：affiliated?with?the4th?International?Conference?on?Communication?System?Software?and?Middleware(COMSWARE2009)，Dublin，Ireland，15-19June2009，1-5.

[8]Najme?Zehra?Naqvi，Ashwani?Kumar，Aanchal?Chauhan?and?Kritka?Sahni.Step?Counting?Using?Smartphone-Based?Accelerometer.International?Journal?on?Computer?Science?and?Engineering(IJCSE)，May2012，4(5)：675-681.

[9]Luo，R.C.，C.Chih?Chia，et?al.(2011).″Multisensor?Fusion?and?Integration：Theories，Applications，and?its?Perspectives.″Sensors?Journal，IEEE11(12)：3122-3138.

[10]Kristou，M.，A.Ohya，et?al.(2011).Target?person?identification?and?following?based?on?omnidirectional?camera?and?LRF?data?fusion.RO-MAN，2011IEEE.

[11]Fod，A.，A.Howard，et?al.A?laser-based?people?tracker.In?proceedings?of?IEEE?International?Conference?on?Robotics?and?Automation(ICRA′02)，2002.

Summary of the invention

In order to remedy above-mentioned mobile robot's tracking pedestrians poor reliability, defective such as calculated amount is big the invention provides cooperate with the mobile phone method of tracking pedestrians of a kind of mobile robot.

The technical scheme that realizes above-mentioned beneficial effect is, cooperate with the mobile phone method of tracking pedestrians of a kind of mobile robot, and the mode that adopts mobile robot and mobile phone to cooperatively interact is followed the tracks of the pedestrian, and described tracing process is as follows:

(1) integrated inertial sensor is measured target people motion conditions on the target people mobile phone, calculates its movement velocity.

(2) set up coordinate system, the mobile robot sets up to be positioned at conplane two-dimensional coordinate system as true origin and pedestrian certainly;

(3) detect a candidate, pedestrian position around the mobile apparatus people detection, with detected pedestrian as a candidate, and testing result showed in coordinate system, obtain a candidate's coordinate position, suppose that detected number of candidates is N people, then the probability that is the target pedestrian with candidate is set at 1/N;

(4) upgrade coordinate system, the mobile robot rebulids coordinate system behind traveling time t, still be initial point with the self-position, sets up and the middle new coordinate system that the coordinate system change in coordinate axis direction is parallel and unit is identical of step (1);

(5) detect the secondary candidate, the mobile robot detects pedestrian position on every side again, with detected pedestrian as the secondary candidate, and with its in new coordinate system respectively sign come out, obtain secondary candidate's coordinate position;

(6) upgrade candidate's coordinate, the mobile robot is updated to coordinate under the new coordinate system according to self translational speed and moving direction with detected candidate's coordinate in the step 2, and candidate and secondary candidate's coordinate position is put in the same coordinate system;

(7) calculate all pedestrian's speed and velocity reversals, the mobile robot list once candidate and secondary candidate may be the combination with a group traveling together, calculate mobile velocity magnitude and direction by the coordinate difference of this pedestrian when detecting for twice, obtain translational speed and moving direction data V ', the A ' of each combination;

(8) calculate each the combination in current and last one the moment the candidate all be the probability of the combination of target, target staff computer goes out time t internal object people's movement velocity V and direction A, and result of calculation sent to the mobile robot, the mobile robot compares according to translational speed and the direction of mobile phone result of calculation to each combination in the step (7), by formula

$\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_j},{\mathrm{Subject}}_{\mathrm{now}\_k})=(\mathrm{\α}(1-{\left(\frac{V^{\′}-V}{V\max }\right)}^2)+(1-\mathrm{\α})(1-{\left(\frac{A^{\′}-A}{360}\right)}^2))*P_{{\mathrm{subject}}_{\mathrm{last}\_j}}$

Calculate that current time and last a candidate constantly is the probability of target, wherein Subject in each combination _{Last_j}Be j candidate of the last moment, Subject _{Now_k}Be k candidate of current time, α is weight, Psubject _{Last_j}Be Subject _{Last_j}Just in time be target pedestrian's probability, Vmax is the maximal rate of pedestrian's walking:

(9) calculate the probability that detected each candidate just in time is the target people in above combination, computing formula is as follows:

$P_{{\mathrm{subject}}_{\mathrm{now}\_k}={\mathrm{\Σ}}_{i=1}^M\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_i},{\mathrm{Subject}}_{\mathrm{now}\_k})}$

M wherein is the higher candidate's of last one probability that constantly screens number.

(10) M candidate with correct probability maximum in the step (9) screens as the higher several candidates of probability, simultaneously a highest candidate of probability followed the tracks of.

The computing formula of described weight is as follows:

$\mathrm{\α}=\frac{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)}{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)+E\left({(V_{\mathrm{robot}}-V_{\mathrm{truth}})}^2{(V_{\mathrm{phone}}-V_{\mathrm{truth}})}^2\right)}$

Wherein, E represents to ask expectation value, A _Robot, V _RobotCandidate target pedestrian direction of motion and speed that the expression robot measures, A _Phone, V _PhoneTarget pedestrian's direction of motion and speed that the expression mobile phone for measuring comes out, A _Truth, V _TruthExpression candidate target pedestrian or target pedestrian's real motion direction and real motion speed.。

The present invention is directed to popularizing of present intelligent machine, the mobile phone of carrying by the pedestrian cooperates with the mobile robot, finishes tracing task, and the present invention follows the tracks of specific people by people's movable information.Mobile phone calculates speed and the direction of people's walking, and this information is sent to robot.And robot measures near pedestrian's walking speed and direction by self laser range finder or other devices, finds out the people of the motion feature that meets the target pedestrian, and the present invention is not subjected to illumination effect, applied range, accurate rate height.

Description of drawings

Fig. 1 is the schematic flow sheet of one embodiment of the present invention;

Embodiment

The present invention will be further described below in conjunction with accompanying drawing.

Cooperate with the mobile phone method of tracking pedestrians of a kind of mobile robot adopts mobile robot and the co-operating mode of mobile phone that the pedestrian is followed the tracks of, and it is characterized in that described tracing process is as follows:

(1) integrated inertial sensor is measured target people motion conditions on the target people mobile phone, calculates its movement velocity.

(2) set up coordinate system, the mobile robot sets up to be positioned at conplane two-dimensional coordinate system as true origin and pedestrian certainly;

(3) detect a candidate, pedestrian position around the mobile apparatus people detection, with detected pedestrian as a candidate, and testing result showed in coordinate system, obtain a candidate's coordinate position, suppose that detected number of candidates is N people, then the probability that is the target pedestrian with candidate is set at 1/N;

(4) upgrade coordinate system, the mobile robot rebulids coordinate system behind traveling time t, still be initial point with the self-position, sets up and the middle new coordinate system that the coordinate system change in coordinate axis direction is parallel and unit is identical of step (1);

(5) detect the secondary candidate, the mobile robot detects pedestrian position on every side again, with detected pedestrian as the secondary candidate, and with its in new coordinate system respectively sign come out, obtain secondary candidate's coordinate position;

(6) upgrade candidate's coordinate, the mobile robot is updated to coordinate under the new coordinate system according to self translational speed and moving direction with detected candidate's coordinate in the step 2, and candidate and secondary candidate's coordinate position is put in the same coordinate system;

(7) calculate all pedestrian's speed and velocity reversals, the mobile robot list once candidate and secondary candidate may be the combination with a group traveling together, calculate mobile velocity magnitude and direction by the coordinate difference of this pedestrian when detecting for twice, obtain the translational speed of each combination and moving direction data V ', A ' (V ' be unit with " meter per second ", direction A ' is unit with " angle ", is the number of degrees that turn clockwise from the earth magnetism direct north);

(8) calculate each the combination in current and last one the moment the candidate all be the probability of the combination of target, target staff computer goes out time t internal object people's movement velocity V and direction A, and result of calculation sent to the mobile robot, the mobile robot compares according to translational speed and the direction of mobile phone result of calculation to each combination in the step (7), by formula

$\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_j},{\mathrm{Subject}}_{\mathrm{now}\_k})=(\mathrm{\α}(1-{\left(\frac{V^{\′}-V}{V\max }\right)}^2)+(1-\mathrm{\α})(1-{\left(\frac{A^{\′}-A}{360}\right)}^2))*P_{{\mathrm{subject}}_{\mathrm{last}\_j}}$

Calculate that current time and last a candidate constantly is the probability of target, wherein Subject in each combination _{Last_j}Be j candidate of the last moment, Subject _Now-kBe k candidate of current time, α is weight, Psubject _{Last_j}Be Subject _{Last_j}Just in time be target pedestrian's probability, Vmax is the maximal rate of pedestrian's walking:

(9) calculate the probability that detected each candidate just in time is the target people in above combination, computing formula is as follows:

$P_{{\mathrm{subject}}_{\mathrm{now}\_k}={\mathrm{\Σ}}_{i=1}^M\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_i},{\mathrm{Subject}}_{\mathrm{now}\_k})}$

M wherein is the higher candidate's of last one probability that constantly screens number.

(10) M candidate with correct probability maximum in the step (9) screens as the higher several candidates of probability, simultaneously a highest candidate of probability followed the tracks of.

The computing formula of described weight is as follows:

$\mathrm{\α}=\frac{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)}{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)+E\left({(V_{\mathrm{robot}}-V_{\mathrm{truth}})}^2{(V_{\mathrm{phone}}-V_{\mathrm{truth}})}^2\right)}$

Wherein, E represents to ask expectation value, A _Robot, V _RobotCandidate target pedestrian direction of motion and speed that the expression robot measures, A _Phone, V _PhoneTarget pedestrian's direction of motion and speed that the expression mobile phone for measuring comes out, A _Truth, V _TruthExpression candidate target pedestrian or target pedestrian's real motion direction and real motion speed, among the present invention, expectation value E specifically calculates according to mobile phone and the measurement pedestrian's of robot accuracy situation, depends on the error of equipment and the quality of measuring method.

Below the present invention is carried out concrete illustrating.In the present embodiment, suppose that initial pedestrian has four, as initial point, selectively the arctic, magnetic field is set up coordinate system with the vertical direction of Y-axis positive dirction as X-axis as the Y-axis positive dirction to the mobile robot with certainly; Set up behind the coordinate system respectively these four pedestrians as a candidate, its coordinate of record in coordinate system (x1, y1), (x2, y2), (x3, y3), (x4, y4), the probability that all is the target pedestrian with four pedestrians is set at 1/4 simultaneously.Behind the elapsed time t, suppose that mobile robot self displacement is S unit of X-axis positive dirction, the mobile robot still selectively sets up new coordinate system with the vertical direction of Y-axis positive dirction as X-axis as the Y-axis positive dirction in the arctic, magnetic field with this position, and new coordinate system is consistent with old coordinate system unit; After new coordinate system is set up, detect pedestrian under the current coordinate system position coordinates (xt1, yt1), (xt2, yt2), (xt3, yt3), (xt4, yt4), and with it as candidate's second time.First candidate's that the mobile robot will record coordinate is updated to the coordinate under the new coordinate system, update mode is the coordinate displacement that a candidate's coordinate figure deducts the mobile robot, namely (x1, y1), (x2, y2), (x3, y3), (x4, y4) be updated to (x1-s, y1), (x2-S, y2), (x3-s, y3), (x4-S, y4).After detecting secondary candidate and the coordinate position of a candidate in the same coordinate system, list once candidate and the secondary candidate combination that may overlap.Changes in coordinates according to these combinations calculates its translational speed V ' and moving direction A ' in time t respectively then.Suppose that pedestrian 1 is pedestrian's reference of this embodiment among four pedestrians, then the entrained mobile phone of pedestrian 1 sends to the mobile robot with time t one skilled in the art 1 movement velocity V and direction of motion A, the mobile robot compares V, A and V ', A ', calculate that former and later two candidates constantly are the probability of target in each combination, the probability that to add up detected each candidate of current time after having calculated again just in time be the target people.The mobile robot keeps the individual candidate of M (M in the present embodiment≤4) of the probability maximum that calculates, and the candidate with probability maximum among this M candidate follows the trail of as the target pedestrian simultaneously.

The above; be preferable case study on implementation of the present invention; be not that the present invention is imposed any restrictions, every any simple modification, change and equivalent structure of above embodiment being done according to the technology of the present invention essence changes, and all still belongs in the protection domain of technical solution of the present invention.

Claims (2)

1. the mobile robot method of tracking pedestrians that cooperates with mobile phone adopts mobile robot and the co-operating mode of mobile phone that the pedestrian is followed the tracks of, and it is characterized in that described tracing process is as follows:

(1) integrated inertial sensor is measured target people motion conditions on the target people mobile phone, calculates its movement velocity.

(2) set up coordinate system, the mobile robot sets up to be positioned at conplane two-dimensional coordinate system as true origin and pedestrian certainly;

(3) detect a candidate, pedestrian position around the mobile apparatus people detection, with detected pedestrian as a candidate, and testing result showed in coordinate system, obtain a candidate's coordinate position, suppose that detected number of candidates is N people, then the probability that is the target pedestrian with candidate is set at 1/N;

(4) upgrade coordinate system, the mobile robot rebulids coordinate system behind traveling time t, still be initial point with the self-position, sets up and the middle new coordinate system that the coordinate system change in coordinate axis direction is parallel and unit is identical of step (1);

(5) detect the secondary candidate, the mobile robot detects pedestrian position on every side again, with detected pedestrian as the secondary candidate, and with its in new coordinate system respectively sign come out, obtain secondary candidate's coordinate position;

(6) upgrade candidate's coordinate, the mobile robot is updated to coordinate under the new coordinate system according to self translational speed and moving direction with detected candidate's coordinate in the step 2, and candidate and secondary candidate's coordinate position is put in the same coordinate system;

(7) calculate all pedestrian's speed and velocity reversals, the mobile robot list once candidate and secondary candidate may be the combination with a group traveling together, calculate mobile velocity magnitude and direction by the coordinate difference of this pedestrian when detecting for twice, obtain translational speed and moving direction data V ', the A ' of each combination;

(8) calculate each the combination in current and last one the moment the candidate all be the probability of the combination of target, target staff computer goes out time t internal object people's movement velocity V and direction A, and result of calculation sent to the mobile robot, the mobile robot compares according to translational speed and the direction of mobile phone result of calculation to each combination in the step (7), by formula

$\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_j},{\mathrm{Subject}}_{\mathrm{now}\_k})=(\mathrm{\α}(1-{\left(\frac{V^{\′}-V}{V\max }\right)}^2)+(1-\mathrm{\α})(1-{\left(\frac{A^{\′}-A}{360}\right)}^2))*P_{{\mathrm{subject}}_{\mathrm{last}\_j}}$

Calculate that current time and last a candidate constantly is the probability of target, wherein Subject in each combination _{Last_j}Be j candidate of the last moment, Subject _{Now_k}Be k candidate of current time, α is weight, Psubject _{Last_j}Be Subject _{Last_j}Just in time be target pedestrian's probability, Vmax is the maximal rate of pedestrian's walking:

(9) calculate the probability that detected each candidate just in time is the target people in above combination, computing formula is as follows:

$P_{{\mathrm{subject}}_{\mathrm{now}\_k}={\mathrm{\Σ}}_{i=1}^M\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_i},{\mathrm{Subject}}_{\mathrm{now}\_k})}$

M wherein is the higher candidate's of last one probability that constantly screens number.

(10) M candidate with correct probability maximum in the step (9) screens as the higher several candidates of probability, simultaneously a highest candidate of probability followed the tracks of.

2. the mobile robot according to claim 1 method of tracking pedestrians that cooperates with mobile phone is characterized in that the computing formula of described weight is as follows:

$\mathrm{\α}=\frac{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)}{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)+E\left({(V_{\mathrm{robot}}-V_{\mathrm{truth}})}^2{(V_{\mathrm{phone}}-V_{\mathrm{truth}})}^2\right)}$

Wherein, E represents to ask expectation value, A _Robot, V _RobotCandidate target pedestrian direction of motion and speed that the expression robot measures, A _Phone, V _PhoneTarget pedestrian's direction of motion and speed that the expression mobile phone for measuring comes out, A _Truth, V _TruthExpression candidate target pedestrian or target pedestrian's real motion direction and real motion speed.

Images