JP2006048666A - Method and device for accompanying object person of mobile object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and device for accompanying an object person of a mobile object by which walk is easily predicted irrespective of an age or motion of the object person and without equipping any special tool, etc. and the walk of the object person is tracked. <P>SOLUTION: In the method and device for accompanying the object person where the mobile object 301 automatically moves with the object person 101, control is performed so that the mobile object 301 moves along a walk path of the object person 101 by detecting a walk state of the object person 101, detecting walk speed and walk path of the object person 101 and predicting a walk prediction area of the object person 101 based on information about the detected walk state, walk speed and path. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving subject subject accompaniment method and a moving subject subject accompaniment device for automatically tracking a subject using a companion moving body for tracking the subject, specifically a walking tracking method, And a gait tracking device.

  As a conventional tracking method and apparatus, there is an apparatus that uses an image input device to track the movement of a person (see Patent Document 1).

  The tracking device (tracking robot for elderly people) described in Patent Document 1 will be described with reference to FIG.

  In FIG. 30, the tracking robot 1 for elderly people includes a GPS receiver 2, a map database 3, a position detection means 4, a notification means 5, a TV camera 6, an image compression means 7, an identification means 8, An abnormal behavior detecting means 9, an obstacle detecting sensor 10, an avoidance movement control means 11, a motor 12, and a theft countermeasure section 13 are provided.

An outline of the operation of the robot 1 will be described. The robot 1 stores image information of an elderly person who is a subject to be tracked by the robot 1 in an image memory of the identification means 8 in advance. The subject is identified by identifying the subject based on the height of the subject's head, the degree of bending of the waist, the way of walking, the clothes, etc. Based on the signal, the shooting direction of the TV camera 6 is controlled, and automatic tracking is performed to continue shooting. Alternatively, a small transmitter is attached to the subject, and the robot 1 tracks the subject by receiving this outgoing radio wave.
JP-A-11-73582

  However, in the prior art, the subject is an elderly person with a slow moving speed, and the identification or tracking ability is limited. In addition, since the movement of the target person cannot be predicted, the target person may not be identified. Especially when the target person suddenly changes the direction of movement, the target person cannot be identified, and the target is not covered. Therefore, it is necessary to attach a small transmitter to the person. For this reason, when the radio wave condition is bad, there is a problem that the target person is lost.

  Furthermore, since tracking is performed by image processing of the entire person, there is a problem that the processing and algorithm become complicated.

  Accordingly, the present invention solves the above-described problems of the prior art, and easily predicts walking and tracks the walking of the subject regardless of the age and movement of the subject and without providing special tools. It is an object of the present invention to provide a mobile object accompanying method and a mobile object accompanying apparatus that can perform the above.

  In order to achieve the above object, the present invention is configured as follows.

According to the first aspect of the present invention, there is provided a method for accompanied by a moving object in which the moving object moves according to the movement of the object,
Detecting the landing state of the subject's feet,
Detecting the walking direction of the subject,
Based on the detected landing state of the foot and the information on the direction, the predicted walking range of the target person is predicted, and the landing state of the foot due to the walking of the target person is detected within the predicted walking predicted range. Provided is a method for moving a moving object along with a subject who controls the moving body so as to move along the walking direction of the subject.

  According to the second aspect of the present invention, the predicted walking range of the subject is centered on the coordinates of the foot of the subject landing and has a radius determined from the stride of the subject. The method according to claim 1, wherein the method is a fan-shaped range directed in the walking direction of the person.

  According to the third aspect of the present invention, when the landing state of the foot is detected, the subject is identified, and the subject of the moving body according to the first aspect is configured to detect the position coordinates of both feet of the subject. Provide an accompanying method.

  According to a fourth aspect of the present invention, when detecting the landing state of the foot, the moving body according to the first aspect, which detects both feet of the subject and detects the position coordinates of the feet of the subject. Provide a method to accompany the target audience.

  According to the fifth aspect of the present invention, when the position coordinates of both feet are detected, it is determined that the foot of the subject is in the same position coordinates for a certain time or more, and the landing of the subject is made. A moving object subject accompanying method according to a fourth aspect of storing the position coordinates of a moving foot is provided.

  According to the sixth aspect of the present invention, when determining the foot of the subject who is at the same position coordinate for the predetermined time or longer, the time that the foot is at the same position coordinate is set to be greater than 0.1 seconds. A moving subject accompanying method according to the fifth aspect is provided.

  According to the seventh aspect of the present invention, when detecting the position coordinates of the both feet, the position coordinates of the feet of the subject landing on the ground are detected, and the position coordinates of the feet of the subject are stored. A moving subject accompanying method according to the fourth aspect is provided.

  According to the eighth aspect of the present invention, when the moving body is controlled to move along the walking direction of the subject, the position of the subject is determined based on the position information of the both feet of the subject already detected. A moving direction of the moving body is generated by connecting a midpoint of a line segment connecting both feet and a midpoint of a line segment connecting both feet of the subject in the next step, and the generated moving direction of the moving body The moving object subject accompanying method according to the first aspect of controlling the moving body to move is provided.

  According to a ninth aspect of the present invention, the object of the moving body according to the first aspect, wherein when the moving body is controlled to move, the moving body is moved in the walking direction with the moving body preceding the subject. Provide a method for accompanies.

  According to a tenth aspect of the present invention, when the moving body is controlled to move, the moving body according to the first aspect is configured to move the walking direction by moving the moving body behind the subject. Provide a method to accompany the subject.

According to an eleventh aspect of the present invention, there is provided a moving subject accompanying apparatus for moving a moving body according to the movement of the subject,
Walking state detection means for detecting a walking state that is a landing state of the subject's feet;
Walking direction detecting means for detecting the walking direction of the subject,
Based on the detected landing state of the foot and the information on the direction, the predicted walking range of the target person is predicted, and the landing state of the foot due to the walking of the target person is detected within the predicted walking predicted range. There is provided a moving object subject accompanying apparatus having control means for controlling the moving body to move along the walking direction of the object person.

  According to a twelfth aspect of the present invention, the predicted walking range of the target person is centered on the coordinates of the foot on which the target person is landing, and has a radius determined from the stride of the target person. An object accompaniment device for a moving body according to an eleventh aspect, which is a fan-shaped range directed in the walking direction of a person.

  According to a thirteenth aspect of the present invention, an eleventh aspect of the present invention is the eleventh aspect, wherein the walking state detection means includes an object person determination unit for determining the object person and both foot position detection means for detecting the position coordinates of both feet of the object person. Provided is a moving object subject accompanying apparatus according to an aspect.

  According to a fourteenth aspect of the present invention, in the eleventh aspect, the walking state detecting means includes both foot position detecting means for detecting position coordinates of both feet of the subject. provide.

  According to the fifteenth aspect of the present invention, the both foot position detecting means determines that the foot of the subject is in the same position coordinates for a predetermined time or more, and determines the foot of the subject. A moving object subject companion apparatus according to a fourteenth aspect having a storage unit for storing position coordinates is provided.

  According to the sixteenth aspect of the present invention, in the means for determining the foot of the subject who is at the same position coordinate for the predetermined time or more, the time that the foot is at the same position coordinate is set to be greater than 0.1 second. The moving object subject accompanying apparatus according to the fifteenth aspect is provided.

  According to a seventeenth aspect of the present invention, the both foot position detecting means includes means for judging the foot of the subject person who has landed on the ground, and a storage unit for storing the position coordinates of the foot of the subject person. A moving object subject accompanying apparatus according to the fifteenth aspect is provided.

  According to an eighteenth aspect of the present invention, the walking direction detecting means includes a midpoint of a line segment connecting both feet of the subject from the position coordinates of the feet of the subject already detected, and the next step. A moving object subject accompanying apparatus according to a twelfth aspect, comprising means for generating a moving direction of the moving body by connecting a midpoint of a line segment connecting both feet of the object person.

  According to a nineteenth aspect of the present invention, the control means has the means for moving the moving object in the twelfth aspect having means for moving the walking direction in advance of the moving object. I will provide a.

  According to a twentieth aspect of the present invention, the control means accompanies the moving object subject according to the twelfth aspect, wherein the controlling means has means for moving the moving body behind the subject and moving the walking direction. Providing equipment.

  With this configuration, it is possible to obtain information on the walking state, walking speed, and route of the target person, identify and predict the target person's moving path, and perform preceding tracking or trailing tracking on the target person's moving path. It can be carried out.

  According to the mobile subject accompanying method and the mobile subject accompanying device of the present invention, the predicted walking range of the target person is predicted, and it is confirmed that the target person continues walking in the predicted range. Since the moving body is tracked along the walking path of the target person, the moving body can be easily and reliably moved along the moving path of the target person regardless of the sudden movement of the follow-up movement. Will be able to.

  The following is a walking object type tracking method that is an example of a moving object subject accompaniment method and moving object object accompaniment apparatus according to an embodiment of the present invention, and an object person walking tracking method using the moving object. The apparatus will be described with reference to FIGS. In the drawings, the same components are denoted by the same reference numerals.

  FIG. 1A is a block diagram showing a schematic configuration of a walking tracking device according to the present embodiment and a moving body equipped with the walking tracking device, FIG. 1B is a coordinate system used in the present embodiment, and 101 is a subject to be tracked 102 is an arrow indicating the walking state of the subject 101, and 301 is a moving body. FIG. 2 is a flowchart relating to processing steps of the target person walking tracking method by a moving body on which the walking tracking device according to the present embodiment is mounted, FIGS. 3 to 7 are detailed flowcharts for each step of FIG. , FIG. 9 is a flowchart about the start of walking and prediction of walking, FIGS. 3, 10 to 27 are explanatory diagrams for explaining a specific example of automatic walking tracking in this embodiment, and FIGS. 28 and 29 are diagrams of the present invention. It is explanatory drawing of the determination method of the walking speed and direction of the subject in other embodiment.

  First, based on FIG. 1A, the structure of the tracking apparatus which tracks the subject (an elderly person etc.) which is an example of a walk tracking type mobile body is demonstrated.

Reference numeral 103 denotes a subject measurement device, which includes an image capturing device 103a such as a camera that detects the walking state 102 at time t = t ₀ (for example, the present) and an image processing unit 103b that processes the captured image. Or a sensor part as a signal receiver that receives a signal from a signal transmitter that transmits information on the right foot and the left foot attached to both feet of the subject 101, and identifies or recognizes the subject 101 Information, walking state information, etc. are measured.

105 is a walking state acquiring unit, still images and signals of the subject data (e.g., the subject 101 acquired by the camera 103a o'clock t = _{t 0} at time t = _{t 0} which is input from the subject measuring device 103 Signal acquired by the receiver) 106 and walking state data at time t = t ₀ (for example, less than or equal to that of the subject 101 detected by processing such as matching from data related to the image in the subject data 106) (Position data obtained by processing such as triangulation from the barycentric coordinates of the foot or the signal acquired by the signal receiver) 108 is calculated. The walking state data 108 may include walking characteristics such as the stride and the walking cycle of the target person 101.

Reference numeral 107 denotes a walking state storage unit. The walking state data 108 at time t = t ₀ and time t = t ₋₁ (for example, the past) calculated by the walking state acquisition unit 105 and before time t = t _0. Is memorized.

Reference numeral 109 denotes a walking state determination unit, which is the walking state data 108 at time t = t ₀ calculated by the walking state acquisition unit 105 and the time t before the time t = t ₀ stored in the walking state storage unit 107. = Walking state data 110 at t ₋₁ , and the walking state determination unit 109 obtains the walking state data 108 at time t = t ₀ and the time t = t ₋₁ (t ₀ > t _−1. ) To calculate a walking vector (speed and angle) (moving body moving vector or moving direction of the moving body) 111 at time t = t ₀ of the subject 101.

Reference numeral 112 denotes a subject storage unit, which is subject data 106 at time t = t ₀ from the subject measurement device 103 or foot data of the subject 101 at time t = t ₀ calculated by the walking state acquisition unit 105 ( For example, at least one of the still image of the eyelid of the subject 101 detected by processing such as matching from the subject data 106 and the signal 113 acquired by the signal receiver) is stored.

Reference numeral 114 denotes a target person determination unit, which includes target person data 106 at time t = t ₀ from the target person measuring device 103 and foot data 113 of the target person 101 at time t = t ₀ calculated by the walking state acquisition unit 105. And the target person storage data 115 at the time t = t ₋₁ stored in the target person storage unit 112 and the target person at the time t = t ₀ from the target person storage data 115. Confirmation data 116 (for example, correct / incorrect data of the subject data 106 or the foot data 113 determined from the subject data 106 or the foot data 113 by a process such as pattern matching) is calculated. Note that the subject determination unit 114 also has a function of performing processing such as recognition or determination of information to be identified or identified by the subject 101 during subject identification processing described later.

Reference numeral 117 denotes a movement command unit, which is based on the walking vector 111 at the time t = t ₀ from the walking state determination unit 109 and the target person confirmation data 116 at the time t = t ₀ from the target person determination unit 114. 118 is calculated.

  Reference numeral 901 denotes an input device such as a key for inputting subject identification information or walking state information, a mouse, or voice input.

  Reference numeral 902 denotes a display device such as a liquid crystal display device that functions as an example of an output device and displays target person identification information, walking state information, and the like.

  The subject measurement device 103, the camera 103a, the image processing unit 103b, the walking state acquisition unit 105, the walking state determination unit 109, the walking state storage unit 107, the movement command unit 117, the target person storage unit 112, the target person determination unit 114, The input device 901 and the display device 902 constitute a tracking device 1000 that performs the operations described below.

  Reference numeral 119 denotes a drive unit of the walking tracking type mobile unit 301 on which the tracking device 1000 is mounted, and executes the movement command 118 acquired from the movement command unit 117 of the tracking device 1000 to perform the operation described below. Thus, for example, as will be described later, the walking tracking mobile body 301 for tracking is driven. Specifically, the moving body 301 moves a pair of driving wheels 119a, a pair of driving devices 119b such as motors for driving the pair of traveling wheels 119a to rotate forward and backward, and a pair of driving devices 119b. A pair of drivers 119c controlled based on the command 118 is included. Note that the traveling wheels 119a are not limited to a pair, and may be an arbitrary number such as three or four, and the driving device 119b and the driver 119c may be appropriately provided according to the number. .

  Note that the walking state detection means includes a target person determination unit 114 that determines the target person 101, and both foot position detection means (such as the target person measurement apparatus 103) that detect the positions of both feet 302 and 303 of the target person 101. It consists of more. As an example, the walking speed / route detection means includes a walking state determination unit 109. As an example, the control means includes a walking state acquisition unit 105, a movement command unit 117, and the like.

Next, it is assumed that the coordinates of the target person 101 and the moving body 301 are indicated by the coordinate system shown in FIG. Here, the subject 101 and the moving body 301 move on the XY plane, the position coordinate Or (0, 0) is set as the position of the moving body 301, and the position coordinate Oh _R0 (X _R0 , Y _R0 ) has a time t = coordinates of the right foot 302 of subject 101 of t _0, the position coordinates _{Oh L0 (X L0, Y L0} ) is the coordinates of the left foot 303 of the subject 101 in time t = _{t 0.}

Further, in the following description, it is assumed that the subject 101 is in a state where the left foot 303 of the subject 101 has come out one step ahead of the right foot 302 at time t = t ₀ as shown in FIG. Therefore, at time t = t ₁ after time t ₀ , when the right foot 302 is normally stepped forward one step while the position coordinate Oh _L0 (X _L0 , Y _L0 ) of the left foot 303 of the subject 101 remains unchanged After that, the position coordinates of the right foot 302 are assumed to be Oh _R1 (X _R1 , Y _R1 ). Furthermore, at time t = t ₂ after time t _1, the position coordinates Oh _R1 (X _R1 , Y _R1 ) of the right foot 302 of the subject 101 remain unchanged, and the step is taken when the left foot 303 is normally stepped one step forward. After that, the position coordinates of the left foot 303 are assumed to be Oh _L1 (X _L1 , Y _L1 ).

  Furthermore, after explaining the outline of the operation steps according to the tracking method in the present embodiment based on the flowchart of FIG. 2, the details of each operation step will be explained. As a general rule, this operation step includes subject identification processing: step S100, subject walking state data acquisition processing: step S200, subject confirmation processing: step S300, subject walking speed and angle calculation processing: step S400. , Subject tracking process: performed in the order of step S500.

(Target person identification process: step S100)
The target person 101 in FIG. 1A is recognized and specified by the tracking apparatus 1000 mounted on the mobile body 301 and configured as shown in FIG. 1A, and the specified target person data 106 is stored in the target person storage unit 112. The information is input to the walking state acquisition unit 105 and the target person determination unit 114, respectively. Here, as a method for recognizing the target person 101, features such as the shoes of the target person 101 may be input and stored in the tracking device 1000 in advance using the camera 103 a or the like.

(Subject's walking state data acquisition process: step S200)
Information on the walking state 102 of the target person 101 is measured by the target person measuring device 103 and input to the walking state acquisition unit 105 as part of the target person data 106, and the walking state acquisition unit 105 calculates the walking state data 108. Are stored in the subject storage unit 112 and input to the walking state determination unit 109. Similarly, the walking state data 108 of the subject person 101 is also input to the walking state storage unit 107 as part of the subject person data 106.

(Subject confirmation process: step S300)
The subject determining section 114, and stores the subject data 106 of the time t = _{t 0,} at least one of the foot data 113 of the subject 101 at the time calculated by the walking state acquiring unit 105 t = _{t 0} The stored data is compared with the target person stored data 115 at the time t = t ₋₁ stored in the target person storage unit 112, and the target person confirmation data 116 at the time t = t ₀ is output. Based on the output target person confirmation data 116, it is confirmed whether or not the target person 101 currently recognized is the target person 101 previously measured. However, the subject stored data 115 is always the same type as the subject data 106 at the time t = t ₀ to be compared by the subject judgment unit 114, the foot data 113 of the subject at the time t = t ₀ , or both data. Data.

(Subject's walking speed and angle calculation processing: step S400)
The walking state determination unit 109, a walking state data 108 at time t = _{t 0,} compares the walking state data 110 at time stored t _{= t -1} walking state storage unit 107, the walking vector 111 (speed and angle ) Here, the stride included in the walking state data 108 may be stored in advance in the tracking device 1000 as an average value of the stride of the target person 101 or a general person. Alternatively, the stride may be calculated from the height information of the target person 101 when the target person 101 is recognized (step S100). The height information of the target person 101 may be calculated from image information or directly taught to the tracking device 1000.

(Subject tracking process: step S500)
Based on the subject confirmation data 116 and the walking vector 111 at time t = t _0, the movement command 118 is transmitted from the movement command unit 117 of the tracking device 1000 to the drive unit 119 of the moving body 301, and the moving body 301, as will be described later, While maintaining a predetermined distance (minimum distance) from the subject 101, the moving body 301 is controlled to track the subject 101. The position that is the movement target of the moving body 301 is, for example, the position coordinates of the midpoint between the position coordinates of the right foot 302 and the position coordinates of the left foot 303 of the subject 101.

  Below, each of step S100-S500 mentioned above is demonstrated in detail.

  Here, FIG. 3 is a process for identifying the target person: a flowchart of step S100, FIG. 4 is a process for acquiring the walking state data of the target person: a flowchart of step S200, FIG. 5 is a process for checking the target person: a flowchart of step S300, FIG. FIG. 7 is a flowchart of the target person's walking speed and angle calculation process: Step S400. FIG. 7 is a target person's tracking process: Step S500.

In the following processing, it is assumed that the subject 101 walks in a normal state as follows. That is, the left foot 303 of the subject 101 is a walk-starting state state exiting before step than the right foot 302, and starts walking from this state, the pre-walking start and time t = t _0. Then, the subject 101 advances the right foot 302 one step forward with the left foot 303 as an axis, and lands at time t = t ₁ . At this time, the position of the right foot 302 at time t = _{t 0,} the time 'by raising the right foot 302 at time _{t = t 0' t = t} 0 issues a right foot 302 towards the front when the ' The right foot 302 lands at time t = t ₁ . Then, the subject 101 is a left foot 303 and right foot 302 in the axial advance one step before, lands at time t = _{t 2.}

FIG. 10 shows a two-dimensional view of the subject 101 in a walking start state at a time t = t ₀ (a state in which the left foot 303 of the subject 101 is one step ahead of the right foot 302) and both feet 302 and 303 at that time. It is explanatory drawing which matches and shows a position coordinate. However, in the figure, the triangle mark indicates the position where the left foot 303 has landed, and the circle mark indicates the position where the right foot 302 has landed. The same applies to other figures. FIG. 13 is an explanatory diagram showing the predicted walking range in the normal walking state at the time t = t ₁ when the time t = t ₀ is superimposed on FIG. FIG. 17 shows the two-dimensional position coordinates of the subject 101 in the normal walking state at the time t = t ₁ (the right foot 302 of the subject 101 is one step ahead of the left foot 303) and the both feet 302, 303 at that time. FIG. FIG. 20 shows a two-dimensional view of the subject 101 in a normal walking state at a time t = t ₂ (a state in which the left foot 303 of the subject 101 is one step ahead of the right foot 302) and both feet 302 and 303 at that time. It is explanatory drawing which matches and shows a position coordinate.

Further, FIG. 11 shows that the moving body 301 is trying to track the subject 101 in the walking start state at the time t = t ₀ (the state shown in FIG. 10) (the moving body 301 is still in the movement stopped state). It is explanatory drawing which shows an example. FIG. 19 shows an example in which the moving body 301 tries to track the target person 101 in the normal walking state at the time t = t ₁ (the state shown in FIG. 17) (the moving body 301 is still in the movement stopped state). It is explanatory drawing which shows. Figure 21 is an explanatory diagram showing an example of the moving body 301 is moved to start tracking the subject 101 in a normal walking state at time t = t ₂ (the state of FIG. 20).

FIG. 12 shows a walking start state at time t = t ₀ (the state shown in FIG. 10, that is, both feet are landing), and attention is paid to the image 880 acquired by the camera 103a and the image 880. It is a figure which shows the image area | region 890 to perform. However, a dotted rectangular frame 881 in the image 880 is added to the image after the measurement in order to measure the distance between the moving body 301 and the subject 101. The same applies to other figures. FIGS. 14 and 15 show the predicted walking range in the normal walking state at time t = t ₁ when time t = t ₀ ′ (the right foot 302 of the subject 101 is floating above the ground). FIG. 13 is a diagram overlapping with FIG. 12. However, FIG. 15 is the same state as FIG. 14, but is a diagram when the entire walking expected range is clearly shown. FIG. 16 is a diagram showing the predicted walking range in the normal walking state at time t = t ₁ when time t = t ₀ ″ (the right foot is floating before landing), superimposed on FIG. is there. FIG. 18 shows an image 880 acquired by the camera 103a in the normal walking state at time t = t ₁ (the state shown in FIG. 17, that is, the state where the right foot has landed), and an image region 890 of interest in the image 880. FIG. Here, it is assumed that t ₀ <t ₀ ′ <t ₀ ″ <t ₁ . Further, Sr ₀ is a movement target of the moving body 301, and may be a position coordinate of a midpoint of the position coordinate Oh _R0 of the right foot 302 and the position coordinate Oh _L0 of the left foot 303 here.

(Subject identification process: step S100)
Step S100 of the identification process for the subject person 101 will be described in detail with reference to the flowchart of FIG. 3 and FIGS.

  First, the data of the target person 101 who follows the moving body 301 is input to the target person storage unit 112, the walking state acquisition unit 105, and the target person determination unit 114 of the tracking device 1000, or the target person measurement device 103 or a key, mouse, or voice input. From the input device 901 (step S101). At this time, if the subject measurement device 103 of the tracking device 1000 mounted on the moving body 301 is, for example, a camera 103a for image input, the camera 103a can recognize the subject 101 (camera). In order to position the target person 101 within the reference range on the initially set image 103a, it is necessary that the target person 101 is positioned in front of the moving body 301 as shown in FIG. .

  For this reason, when the target person 101 is located at a position other than the front of the moving body 301, the target person 101 moves to the front of the moving body 301 or the moving body 301 moves to move the target person 101. It may be positioned in front of the body 301. Here, the reference range is a range (for example, a focused image range 890 in FIG. 12) that becomes a predetermined coordinate when pixel coordinates of an image are converted into two-dimensional coordinates.

  As an example of the data of the target person 101 input from the camera 103a of the target person measuring apparatus 103, as shown in FIG. 12, image data of the whole body or lower body of the target person 101 is obtained. In addition, as an example of the data of the target person 101 input from the input device 901, information that can be identified by the person and physical characteristics of the target person 101 (face, body shape, shoe size, etc.) , Characteristics of shoes and clothes (colors, sizes, and shapes of shoes, etc., colors, sizes, and shapes of clothes), and walking state data (steps, etc.). These data may be stored in advance in a database outside the subject person storage unit 112 or outside the subject person storage unit 112 of the tracking apparatus 1000, and these data may be read out from the database and used.

  The target person determination unit 114 determines whether or not the target person 101 can be identified from the input data (step S102). In other words, if the subject person 101 cannot be identified due to lack of data, the display device 902 notifies that the subject person cannot be identified, for example, by displaying or warning sound (step S105), and then to step S101. Return.

  When the target person 101 can be specified in step S102, the specified target person 101 is displayed on the display device 902 (step S103).

  Next, the target person determination unit 114 determines whether or not the information displayed on the display device 902 has been confirmed by the target person 101 itself (step S104). When the target person 101 confirms the information displayed on the display device 902 (for example, when input from the input device 901 such as voice input or key input indicating confirmation is performed within a predetermined time), Proceed to the walking state data acquisition process of the subject person 101 (step S200). When the target person 101 does not confirm the displayed information, the process returns to step S101 and data input of the target person 101 is performed again. At the time of this confirmation work, the target person 101 may correct the information displayed on the display device 902 from the input device 901.

(Subject's walking state data acquisition process: step S200)
Step S200 of the walking state data acquisition process of the subject person 101 will be described in detail based on the flowchart of FIG. 4 and FIG.

  Information on the walking state 102 of the subject person 101 is measured by the subject person measuring device 103 (step S201).

  Next, the information on the walking state 102 measured in step S201 is input to the walking state acquisition unit 105, the target person storage unit 112, and the target person determination unit 114 as part of the target person data 106 (step S202). Here, examples of input data include image data of the walking state of the target person 101, position information of both feet (below the heel) of the target person 101, stride length data, and the like.

  Next, based on the data input to the walking state acquisition unit 105 in step S202, the walking state acquisition unit 105 calculates the walking state data 108, and the calculated walking state data 108 is stored in the subject storage unit 112. And input to the walking state determination unit 109 (step S203). Here, as an example of the walking state data 108, the coordinates of the center of gravity of the foot below the heel of the subject person 101 may be included, and other features of the walking such as the stride and the walking cycle of the subject person 101 may be included.

Next, based on the stride data of the target person 101 (if there is no stride data of the target person 101, the normal person's stride data), the walking state acquisition unit 105 predicts the walking of the target person 101. A range 601 is calculated (step S204). The predicted walking range 601 is calculated based on the stride data set in the tracking device 1000 in advance from the stride centered on the coordinates of the axis foot of the subject 101 (step stride × 1.2 ≦ radius R ≦ (step stride + shoe size) × 1.2) and has a radius R calculated by the equation of 1.2), and passes through the shaft foot Oh _L0 and is perpendicular to a straight line Oh _R0 Oh _R1 having a length R in a direction opposite to the shaft foot side, as shown in FIG. A fan-shaped range having an angle in the range of 100 degrees to 120 degrees with respect to a straight line segment (the line segment Oh _L0 Fs in FIG. 13) may be used.

  If it is in such a range, when a normal walking motion is performed, the foot almost lands within that range. In other words, the predicted walking range here is a range where the foot lands within the range when the subject 101 performs a normal walking motion. In addition, even if the target person 101 is trying to perform normal walking motion, if the tendency is always to stagger, taking into account that, the fan-shaped range is taken wider or the radius is set larger. do it.

The predicted walking range 601 in FIGS. 13, 14, 15, 16, and 17 is the time t = t ₁ predicted from the position coordinates Oh _R0 and Oh _L0 of the feet 302 and 303 at time t = t ₀ . Is the predicted range of the position coordinates Oh _R1 of the right foot 302, and the predicted walking range 602 of FIGS. 19, 20, and 21 is predicted from the position coordinates Oh _R1 and Oh _L0 of both feet 302 and 303 at time t = t _1. the expected range of the position coordinates _{Oh L1} of the left foot 303 when that the time t = _{t 2.} The predicted walking ranges 601 and 602 are positions of the current (latest time: for example, time t = t ₀ ) position of the axial foot and the past (for example, time t = t ₋₁ ) before the walking of the axial foot. As a sector of a radius R that is a stride of the subject 101 that is 180 degrees or less on the basis of a half straight line that is substantially perpendicular to the straight line that connects to the inner side of the subject 101 from the position of the axial foot. Yes.

  Note that, for example, when an unexpected movement of the target person 101 can be predicted, such as when an obstacle is detected on the image acquired from the camera, the angle of the predicted walking range at that time is temporarily Specifically, it may be changed to 180 degrees. Here, as a case where a sudden movement of the target person 101 can be predicted, a case where the obstacle is a moving object or a case where the target person 101 is not aware of the obstacle can be considered. In general, since it is difficult to predict the sudden movement of the subject 101, it is preferable to increase the angle of the predicted walking range. Increasing the predicted walking range in the radial direction without changing the angle is not preferable because the moving body 301 may accelerate rapidly and collide with the subject 101.

  In addition, when there is a difference in walking state from a healthy person, such as a physically handicapped person, such as a disabled person, if it is frequently recognized that the foot will land outside the expected walking range due to frequent sudden movements, etc. In addition to the mode for the healthy person that performs the tracking operation based on the predicted walking range described above, the non-tracking operation is performed based on the predicted walking range that is larger than the predicted walking range in the mode for the healthy person or a large range of walking. The mode for healthy persons is stored in the walking state storage unit 107 or the like, and based on the history information that the number of times that the foot has landed outside the expected walking range by input from an input device or the like exceeds the predetermined number of times. The mode can be switched, and a more suitable walking prediction range can be set for the subject 101.

Note that the fan-shaped angle is calculated by the equation (90 ≦ ∠FsOh _L0 Fe ≦ 180 (degrees)), and the stride setting method is, for example, extracting the height from the acquired data of the subject 101 and (height × 0 .37 ≦ step length ≦ height × 0.45). In FIG. 13, 610 is positioned on the front side of the subject 101 with the left foot 303 of the subject 101 as the center, and is orthogonal to a straight line Oh _R0 Oh _R1 that connects the position coordinates of the right foot 302 before and after the move. And a semicircular maximum predicted walking range including a straight line passing through the position coordinates of the left foot 303 (a straight line along the line segment Oh _L0 Fs). The fan-shaped walking predicted range 601 is included in the maximum walking predicted range 610.

  The stride of height × 0.37 is the stride when walking normally, and the stride of height × 0.45 is the stride when walking earlier than usual.

  Next, as will be described later with reference to FIG. 8, it is determined whether or not the subject 101 has taken a step within a predetermined time (step S205). Specifically, information on the walking state 102 of the subject person 101 is measured by the subject person measuring device 103, and whether or not the position of the heel of any one of the subject person 101 is changed in the walking state acquisition unit 105. You just have to judge. If it is determined in step S205 that the subject 101 has not taken one step within the predetermined time, it is assumed that there is no intention to walk, and the process returns to step S201 or step S203. That is, when redoing from the measurement, the process returns to step S201, and when redoing the calculation without changing the measurement, the process returns to step S203.

  If it is determined in step S205 that the subject 101 has taken one step within the predetermined time, it is determined whether or not the position of the stepped foot is within the predicted walking range 601 (step S206). When it is determined that the position of the foot that has been stepped on in step S206 is not within the predicted walking range 601, after correcting the step length or the angle by the walking state acquisition unit 105 (step S212), step S201 or step S203. Return to. That is, when redoing from the measurement, the process returns to step S201, and when redoing the calculation without changing the measurement, the process returns to step S203. However, instead of performing the correction of the stride or the angle by the walking state acquisition unit 105, the target person 101 is regarded as having lost the balance as in Step S809 and Step S810 of FIG. You may make it stop.

  If it is determined in step S206 that the position of the stepped foot is within the expected walking range 601, the distance between the target person 101 and the moving body 301 is measured by the target person measuring device 103, and the distance is determined. The movement command unit 117 determines whether or not the follow start distance (for example, 1 m) has been exceeded (step S207).

  If it is determined in step S207 that the distance does not exceed the tracking start distance, it is determined whether the tracking has already started (for example, whether the walking vector 111 whose speed is not 0 has already been output) or not. The determination is made by the unit 117 (step S210). If it is determined in step S210 that tracking has already started, it is determined that the moving body 301 has approached the subject 101 too much, and the movement (following operation) of the moving body 301 is stopped (step S211). Terminate the process.

  If it is determined in step S210 that tracking has not been started, the process returns to step S201 or step S203. That is, when redoing from the measurement, the process returns to step S201, and when redoing the calculation without changing the measurement, the process returns to step S203.

  If it is determined in step S207 that the distance exceeds the following start distance, whether or not the subject 101 has walked two or more steps from the start of walking based on the walking state data 108 stored in the walking state storage unit 107. Is determined (step S208). If it is determined in step S208 that the user has not walked more than two steps from the start of walking, the series of processing ends.

  If it is determined in step S208 that the user has walked two or more steps from the start of walking, the walking state acquisition unit 105 corrects the stride based on the stride data stored so far (step S209). Then, the process proceeds to the confirmation process (step S300) of the target person 101. This is because the stride can be averaged if two or more steps have been taken since the start of walking.

  Here, the process of determining whether or not the target person 101 in step S205 has taken a step within a predetermined time, in other words, the process of recognizing the start of walking will be described in detail with reference to FIG.

  For example, a method for recognizing the start of walking when the sensor portion is the camera 103a for image input will be described with reference to FIG.

  First, in the subject measurement apparatus 103, the walking state obtaining unit 105 performs matching processing on both feet of the subject 101 from an image obtained by the camera 103a and image-processed by the image processing unit 103b (step S701). Images of both feet of the subject 101 serving as a reference for matching are acquired in advance by the camera 103a.

  Next, the centroid coordinates on the images of both feet tracked by matching are acquired by the walking state acquisition unit 105 (step S702).

  Further, the center-of-gravity coordinates are converted by the walking state acquisition unit 105 onto the two-dimensional coordinates based on the moving body 301 described in FIG. 1B (step S703).

Next, the walking state acquisition unit 105 _sets Oh _R0 (X _R0 , Y _R0 ) as the center-of-gravity coordinates of the right foot of the subject 101 out of both feet of the subject 101 on the two-dimensional coordinates, and the center-of-gravity coordinates of the left foot. _{Is set} to Oh _L0 (X _L0 , Y _L0 ) (step S704). However, since the left and right are opposite depending on the orientation of the subject 101, the subject 101 is facing in the same direction as the moving body 301 (the front of the moving body 301 is facing the back of the subject 101). In addition, the state of the subject person 101 who can recognize both feet and the positional relationship between the moving body 301 are assumed to be a relation in which the subject identification process of FIG. 3 can be performed.

  Next, the walking state acquisition unit 105 determines whether the coordinates of either one of both feet change within 0.1 seconds (step S705).

  When it is determined in step S705 that the coordinates of either one of both feet have changed within 0.1 seconds, the walking state acquisition unit 105 determines that the subject 101 has started walking (step S706), and the process ends. To do.

  If it is determined in step S705 that the coordinates of either one of both feet have not changed within 0.1 seconds, the walking state acquisition unit 105 determines that the subject 101 has stopped without walking, It returns to step S705 and waits for the subject person 101 to start walking.

  In addition, the process of determining whether or not the position of the stepped foot in the step S206 is within the predicted walking range, in other words, the walking prediction process after starting the walking will be described with reference to FIG.

  The walking state acquisition unit 105 determines that the foot whose coordinates do not change for 0.1 second or more among the barycentric coordinates of both feet of the subject 101 is an axial foot (step S801).

  The walking state acquisition unit 105 determines whether or not the barycentric coordinates of the moved foot exist (step S802).

  If it is determined in step S802 that the center of gravity coordinates of the moved foot exist, the walking state acquisition unit 105 determines whether the position of the center of gravity coordinate of the moved foot is within the predicted walking range 601 (step S802). S803).

  If it is determined in step S803 that the position of the center of gravity coordinates of the moved foot is within the predicted walking range 601, the walking state acquisition unit 105 determines that the walking state is normal (step S804).

  Next, the process proceeds from step S207 to step S208, and the predicted walking range of the next moving foot is predicted by the walking state acquisition unit 105 (step S805), and the process ends.

  If it is determined in step S802 that the coordinates of the moved foot do not exist on the acquired image, the walking state acquisition unit 105 determines that the foot is overlapped (step S806). For example, as shown in FIGS. 22 to 23, when the right foot is landed so as to overlap the front of the left foot, the state shown in FIG. 23 is obtained on the image 880 of the camera 103a.

  Following step S806, for example, if the left foot 303 is the current axial foot (A) and the right foot 302 is moved to the next axial foot (B), the position of the right foot 302 that is the next axial foot (B) is moved. The walking state acquisition unit 105 estimates that the body 301 is located at a position advanced by a stride from the left foot 303 of the current axial foot (A) on a half line passing through the left foot 303 of the current axial foot (A) from the body 301 (step S807). ).

  Following step S807, when the left foot 303 of the axial foot (A) moves, the walking state acquisition unit 105 determines whether or not the right foot 302 of the axial foot (B) is at the estimated position (step S808).

  When it is determined that the right foot 302 of the axial foot (B) is located at the position estimated in step S808, the process proceeds to step S805, and the walking state acquisition unit 105 predicts the predicted walking range of the left foot 303 to move next.

  If it is determined that the right foot 302 of the axial foot (B) is not present at the position estimated in step S808, the walking state obtaining unit 105 outputs a walking vector 111 having a speed of 0 and an angle of 0 degrees, and a movement command The moving body 301 can be stopped via the unit 117 and the driving unit 119 of the moving body 301 (step S810).

  If it is determined in step S803 that the position of the moved foot is outside the expected walking range, the walking state acquisition unit 105 determines that the subject 101 is out of balance (step S809). Proceeding to step S810, as described above, the walking state obtaining unit 105 outputs the walking vector 111 having a speed of 0 and an angle of 0 degrees, via the movement command unit 117 and the driving unit 119 of the moving body 301, The moving body 301 is stopped (step S810).

In this case, as shown in FIG. 24, stumbling with his left foot 303 is something at time t = _{t 2,} starting tumbled the subject 101 at time t = _{t 3,} the target at time t = _{t 4} Suppose that person 101 falls and puts both hands on the ground. In FIG. 24, two subjects 101 at time t = t ₂ are the same, two subjects 101 at time t = t ₃ are the same, and time t = t ₂ and time t = t ₃ If the movement of the subject 101 is superimposed on the subject 101 at time t = t ₁ , it will be difficult to understand, so this is shown in a shifted manner. Thus, when the subject 101 trips over something, the position of the tripped foot or the position of the foot that has moved after tripping is outside the expected walking range 602.

  Therefore, when the walking state acquisition unit 105 determines that the position of the next moved foot is outside the predicted walking range 602, as described above, the walking state acquisition unit 105 has a speed of 0 and an angle of The walking vector 111 of 0 degree is output, and the moving body 301 is stopped via the movement command unit 117 and the driving unit 119 of the moving body 301. As a result, when the subject 101 loses balance and cannot normally walk, the movement of the moving body 301 can be stopped immediately.

  Note that after stopping, there is a high possibility that normal walking will be resumed, so it is preferable to return to step S200 and wait for the start of walking. In this way, even if the subject 101 stumbles and falls temporarily, when the person 101 stands up immediately and starts normal walking, the tracking operation by the moving body 301 can be smoothly resumed.

  Further, when walking has not started for a predetermined time or longer, an abnormality is displayed on the display device 902, a warning sound is emitted from the warning device, or in some cases, when a communication device is installed, communication is performed. A warning signal may be output to a predetermined device using the device.

Also, as shown in FIG. 25, the subject 101 loses balance, and the left foot 303 greatly steps sideways at time t = t ₂ , so that the subject 101 comes out to a position Oh _L11 outside the expected walking range 602. Sometimes, the position of the moved foot is outside the expected walking range 602. Therefore, when the walking state acquisition unit 105 determines that the position of the next moved foot is outside the predicted walking range 602, as described above, the walking state acquisition unit 105 has a speed of 0 and an angle of 0. The walking vector 111 is output, and the moving body 301 is stopped via the movement command section 117 and the driving section 119 of the moving body 301. As a result, if the target person 101 loses balance and cannot normally walk, the movement of the moving body 301 can be stopped immediately.

  Note that after stopping, there is a high possibility that normal walking will be resumed, so it is preferable to return to step S200 and wait for the start of walking. In this way, even if the subject 101 temporarily loses his balance, the tracking operation by the moving body 301 can be smoothly resumed when normal walking is immediately started.

  Further, when walking has not started for a predetermined time or longer, an abnormality is displayed on the display device 902, a warning sound is emitted from the warning device, or in some cases, when a communication device is installed, communication is performed. A warning signal may be output to a predetermined device using the device.

(Subject confirmation process: step S300)
Step S300 of the confirmation process of the subject person 101 will be described in detail based on the flowchart of FIG.

  First, it is determined whether or not the subject 101 has walked two or more steps from the start of walking (step S301). If it is determined in step S301 that the user has not walked two or more steps since the start of walking, the process waits until the user walks two or more steps from the start of walking.

If it is determined in step S301 that the user has walked two or more steps from the start of walking, the subject determination unit 114 determines whether or not the subject data 106 at the time t = t ₀ (for example, current) from the subject measurement device 103. Alternatively, the foot data 113 of the subject person 101 at the time t = t ₀ (for example, current) calculated by the walking state acquisition unit 105 is input. Further, the target person storage data 115 at the time t = t ₋₁ (for example, the past) stored in the target person storage unit 112 is also input to the target person determination unit 114 (step S302).

Next, the target person determination unit 114 compares the target person data 106 or the foot data 113 of the target person 101 with the target person storage data 115 to determine whether or not they match (step S303). However, the subject's foot subjects the stored data 115 always has time to compare subject determining section 114 t = t 0 _(e.g., current) Target data 106 in, or time t = t 0 _(e.g., current) It is assumed that the data 113 is the same type of data as the data 113 or both.

Note that, when past data does not exist at the start of processing, the target person stored data 115 when t = t ₀ may be used.

If it is determined in step S303 that the two match, the target person confirmation data 116 at time t = t ₀ (for example, current) is output from the target person determination unit 114 to the movement command unit 117 (step S304). The process proceeds to the walking speed and angle calculation process (step S400) of the subject person 101. By outputting the target person confirmation data 116, it can be confirmed that the target person 101 currently recognized is definitely the target person 101 measured previously.

  If it is determined in step S303 that the two are different, a data acquisition error is displayed on the display device 902 (step S305), and the process returns to the walking state data acquisition process of the subject 101 (step S200).

(Subject's walking speed and angle calculation processing: step S400)
Step S400 of the walking speed and angle calculation process of the subject person 101 will be described in detail based on the flowchart of FIG.

  First, it is determined whether or not the subject 101 has walked two or more steps from the start of walking (step S401). If it is determined in step S401 that the user has not walked two or more steps since the start of walking, the process waits until the user walks two or more steps from the start of walking.

  If it is determined in step S401 that the user has walked two or more steps from the start of walking, the walking state acquisition unit 105 outputs the walking state data 108 to the walking state storage unit 107 for storage (step S402).

Next, the walking state determination unit 109 inputs the walking state data 108 at the time t = t ₀ (for example, current) input from the walking state acquisition unit 105 and the time t = t ₋₁ (stored in the walking state storage unit 107). For example, the walking state data 110 in the past) is compared and the difference between the two is calculated (step S403). Here, the stride included in the walking state data 108 may be stored in advance in the tracking device 1000 as an average value of the stride of the target person 101 or a general person. Alternatively, when the target person 101 is identified (step S100), the stride may be calculated from the height information of the target person 101 (for example, the equation (height × 0.37 ≦ step length ≦ height × 0.45) is used. May be used). The height information of the subject person 101 may be calculated from the image information, or may be directly input to the tracking device 1000 from the input device 901 and taught.

If no past data exists at the start of processing, the walking state data at t = t ₀ may be used. Furthermore, regarding the calculation method of the walking vector 111 when there is no past data, when t = t ₀ , the coordinates of the middle point of the coordinate data of both feet of the walking state data and the coordinate data of both feet are close to the moving body 301. The walking vector 111 may be calculated from the coordinates.

  Next, the walking state determination unit 109 calculates a walking vector 111 (speed and angle) from the difference calculated from the walking state data 108 and the walking state data 110 (step S404), and the target person 101 tracking process is performed. The process proceeds to (Step S500).

(Subject tracking process: step S500)
Step S500 of the tracking process of the subject person 101 will be described in detail based on the flowchart of FIG.

First, the movement command unit 117 determines whether or not the subject confirmation data 116 and the walking vector 111 at the time t = t ₀ (for example, the present) are input to the movement command unit 117 (step S501). If it is determined in step S501 that the target person confirmation data 116 and the walking vector 111 at time t = t ₀ (for example, current) are not input, the walking state data acquisition process for the target person 101 (step S200). Return to.

If it is determined in step S501 that the target person confirmation data 116 and the walking vector 111 at time t = t ₀ (for example, the current time) are input, the movement command unit 117 uses the walking vector 111 to move the moving object 301. Are calculated (step S502). Specifically, the position that is the movement target of the moving body 301 is, for example, the position coordinates of the midpoint between the position coordinates of the right foot 302 and the position coordinates of the left foot 303 of the subject 101. Then, the speed and angle relating to the movement from the current position of the moving body 301 to the position coordinates of the midpoint are calculated.

  Next, the movement command unit 117 displays on the display device 902 that the following operation is to be started (step S503). As an example of display on the display device 902, in addition to displaying the start of the follow-up operation on the liquid crystal display device, the start of the follow-up operation may be notified by sound from a speaker as another example of the display device 902.

  Next, the movement command 118 is transmitted from the movement command unit 117 of the tracking device 1000 to the drive unit 119 of the moving body 301 (step S504).

  Next, based on the movement command 118, the driving unit 119 of the moving body 301 starts driving so that the moving body 301 tracks the subject 101 while maintaining a predetermined distance (minimum distance) from the subject 101. The moving body 301 is moved or braked (step S505).

  Next, the movement command unit 117 determines whether or not a command to end the tracking of the subject person 101 has been input from the input device 901 or the like (step S506). If it is determined in step S506 that no tracking end command has been input, the process returns to the walking state data acquisition process (step S200) of the subject 101. If it is determined in step S506 that a tracking end command has been input, the process returns to the process for identifying the subject 101 (step S100).

  Next, an example of a system using these tracking methods and tracking devices will be described with reference to FIGS.

  In the system shown in FIG. 1 and FIG. 26, an example in which it is assumed that the moving body 301 tracks the target person 101 is shown. Here, using the information of the walking state 102 of the target person 101, the movement control is performed by the tracking device 1000 so that the moving body 301 tracks the back of the target person 101.

  The tracking device 1000 first performs a process for identifying the target person 101 (step S100). That is, an image of both feet of the subject 101 (the right foot of the subject 101 is denoted by 302 and the left foot is denoted by 303) is acquired by the camera 103a and the image processing unit 103b as the subject measuring device 103, and both the feet 302 and 303 are obtained. Are measured two-dimensionally.

  The subject measuring apparatus 103 acquires signals of both feet 302 and 303 transmitted from signal transmitters separately provided on both feet 302 and 303 of the subject 101 by a signal receiver, and both feet 302 of the subject 101 are obtained. , 303 may be used to measure the respective coordinates.

  In addition, the image or information of the right foot 302 and the left foot 303 of the subject person 101 is displayed on a display device 902 such as a display, so that it can be confirmed whether the subject person 101 is correctly determined. At this time, in some cases, correction of the positions of the right foot 302 and the left foot 303 of the subject 101 may be input from the input device 901.

Then, after the target person 101 specifying process (step S100) is completed and the target person 101 is specified, a walking state data acquisition process (step S200) for acquiring the walking state data of the target person 101 is performed. . In other words, information regarding the right foot 302 and the left foot 303 of the subject 101 is stored in the storage device 304 as the subject storage data 115. And the information of the acquired position coordinates of both feet 302 and 303 (as the reference of the coordinates, for example, the position of the moving body 301 or the walking state data at time t = t ₀ may be used as a reference). Time before one step, for example, if the current time is t = t ₀ , the walking state determination unit 109 calculates the predicted walking range 601 based on the walking state data 110 at t = t ₋₁ , and Waiting for the start of walking, and starting walking, the predicted walking range 601 is performed based on previously stored information until two or more steps are taken.

  Next, when the target person 101 advances two steps or more and the walking state data acquisition process (step S200) of the target person 101 is completed, the confirmation process (step S300) of the target person 101 is started. That is, it is determined whether or not the target person 101 to be tracked is the same as the target person 101 identified first. If the target person 101 is the same, the walking speed and angle calculation process of the target person 101 is started to start the tracking operation. (Step S400) is performed. If it is not the same as the target person 101 identified first, the tracking start operation is stopped. This is a process for preventing the mobile body 301 from tracking the other person by mistake when another person enters between the target person 101 and the mobile body 301 after performing the specific process. is there.

  Next, in the walking speed and angle calculation process (step S400) of the subject person 101, the walking vector 111 is obtained by an arithmetic processing unit as an example of the walking state acquisition unit 105. Information about the walking vector 111 is stored in the walking state storage unit 107.

Next, when the walking speed and angle calculation process (step S400) of the subject is completed, the tracking process (step S500) of the subject is started. Target tracking processing (step S500), the subject's confirmation data 116 and walking vector 111 at time t = _{t 0,} a movement command 118 from the movement command unit 117 of the tracking unit 1000 to the driving unit 119 of the mobile unit 301 As described later, the moving body 301 tracks the subject 101 while maintaining a predetermined distance (minimum distance) from the subject 101.

  At this time, the position that is the movement target of the moving body 301 is, for example, the position coordinates of the midpoint between the position coordinates of the right foot 302 and the position coordinates of the left foot 303 of the subject 101. Then, the speed and angle relating to the movement from the current position of the moving body 301 to the position coordinates of the middle point are calculated, and the drive unit 119 of the moving body 301 is driven and controlled based on the calculated speed and angle.

  After that, each time the target person 101 takes one step, a predicted walking range is created, the foot landing is confirmed within the predicted walking range, the predicted walking range is created again, and the landing of the foot is confirmed within the predicted walking range. On the other hand, the driving unit 119 of the moving body 301 is driven and controlled to perform the tracking operation toward the position coordinates of the midpoint between the position coordinates of the right foot 302 and the position coordinates of the left foot 303 of the subject 101. As a result, as shown in FIGS. 26 and 27, the moving body 301 can smoothly track the walking target person 101.

  According to the embodiment, the information on the walking state of the target person 101 is detected, and the walking vector 111 of the target person 101 (in other words, the walking speed and the walking angle, and thus the walking route) is detected and detected. Based on the walking state, the walking speed, and the route information, by predicting the predicted walking range of the target person 101, while confirming that the walking of the target person 101 is performed within the predicted walking range, The moving body 301 can be controlled to move along the walking route of the subject 101.

  In this way, even if the walking route of the subject 101 changes variously, the moving body 301 can be moved following the change. Accordingly, the moving body 301 can follow the movement path of the subject 101 easily and reliably regardless of the sudden movement of the follow-up movement of the subject 101.

  For example, when an obstacle suddenly appears in front of the subject 101, or when an obstacle approaches the subject 101, the subject 101 suddenly falls back or suddenly moves sideways. Can be considered. In addition, there may be a case where the target person 101 suddenly stumbles over something or sways greatly. Even if such a sudden movement of the target person 101 occurs, when the predicted walking range of the target person 101 is predicted and the prediction is not satisfied, the predicted walking range is predicted again and the target person 101 is re-estimated. The moving body 301 can be moved along the walking route.

  Further, when the position coordinates of both feet of the subject 101 are detected in the walking state acquisition unit 105, the feet of the position coordinates of the feet of the subject 101 at the same position for a certain time or longer (for example, longer than 0.1 seconds). Can be reliably detected when the foot is in contact with the ground, and the position coordinates of the moving target of the moving body 301 can be calculated. The position coordinates of the midpoint can be obtained with certainty, and the moving speed and moving path of the moving body 301 can be reliably calculated from such information on the walking state of the subject 101, so that the moving body 301 can be stably It becomes possible to control movement.

  Further, as another example of the sudden movement of the target person 101, when the target person 101 moves backward and the target person 101 approaches the moving body 301 below the minimum distance, the moving of the moving body 301 is immediately performed. If it is made to stop, it can prevent reliably that the mobile body 301 collides with the subject 101. FIG.

  Further, when it is determined that the coordinates of the moved foot of the subject 101 do not exist on the acquired image 880, the walking state acquisition unit 105 determines that the foot is overlapped, and the position coordinate of the foot is determined. It can also be estimated, and even when the feet overlap, the target person 101 can be reliably predicted to walk.

  As an example of a moving object subject accompaniment method and a moving object object accompaniment apparatus according to another embodiment of the present invention, there is an object preceding (moving moving object preceding an object person) method and apparatus. . In the subject preceding method and apparatus of FIG. 28, an example in which it is assumed that the subject 101 is preceded by the moving body 301 is shown.

  Here, it is assumed that the moving body 301 precedes using the information of the walking state 102 of the subject person 101. In other words, the moving object 301 is a case where the object person 101 always moves in front of the object person 101 along a route on which the object person 101 is going to walk. This is to give the subject 101 a sense of security that 301 is always present. Note that the subject preceding method and device basically perform the same structure and operation as the subject walking tracking method and device, except that the moving body 301 precedes the subject 101. .

  In this case, the face of the subject 101 is denoted by 401, the right foot is denoted by 302, and the left foot is denoted by 303. The moving body 301 acquires images of the face 400 and both feet 302 and 303 of the target person 101 by the camera 103a and the image processing unit 103b as the target person measuring apparatus 103, and coordinates of the both feet 302 and 303 are two-dimensionally obtained. Measurement is performed, and the face 401 is used as information for identifying (identifying) the target person.

  The subject measuring apparatus 103 acquires signals of both feet 302 and 303 transmitted from signal transmitters separately provided on both feet 302 and 303 of the subject 101 by a signal receiver, and both feet 302 of the subject 101 are obtained. , 303 may be used to measure the respective coordinates.

  Further, in the same manner as the target person specifying process (step S100), the image or information of the face 401, the right foot 302, and the left foot 303 of the target person 101 is displayed on the display device 902 such as a display to confirm the target person 101. May be performed. At this time, the position correction of the right foot 302 and the left foot 303 of the subject 101 may be performed depending on circumstances.

  And the information about the right foot 302 and the left foot 303 of the subject 101 is stored in the walking state storage unit 107 as the walking state storage data, similarly to the walking state data acquisition process (step S200) of the subject. Information about the face 401 of the subject 101 is stored in the subject storage unit 112 as subject storage data 115.

Then, similar to the confirmation process (step S300) of the subject person, the walking speed and angle calculation process (step S400) of the subject person, and the tracking process (step S500) of the subject person, Based on the information, the walking vector 111 is obtained by an arithmetic processing device as the walking state acquisition unit 105 provided in the tracking device 1000. Information about the walking vector 111 is stored in the walking state storage unit 107. The walking state determination unit 109 creates the next predicted walking range based on the walking state data 110 at time t = t ₋₁ immediately before walking.

Here, in the walking speed and angle calculation process of the subject's step 400, the method for calculating the walk vector 111 in the case where the past data is not present, when t = t _0, the midpoint of the coordinate data of both feet of the walking state data The walking vector 111 may be calculated from the coordinates farther from the moving body 301 among the coordinates and the coordinate data of both feet.

  Next, a method for determining the walking state data 108 used in these control methods and the walking vector 111 will be described with reference to FIG.

In Figure 29, the position information of the footprint both feet at time t = _{t 0} of the subject 101 (the right foot 302, left foot 303) becomes the walking state data 108. Also, a line segment connecting the midpoint 501 of both feet when the time t = t ₀ and the midpoint 502 of both feet when the time t = t ₁ is defined as the walking vector 111.

29, as the walking state data 108, the right foot 302 at time t = _{t 0,} the position coordinates of the respective positions of the left foot 303 and _Oh R0, _{Oh L0,} at time t = _{t 1,} the position coordinates _{Oh L0} A position where the right foot 302 is moved in a state where the left foot 303 located on the ground is grounded with the shaft foot as an axial foot is referred to as Oh _R1 . Also, the right foot 302 from the position Oh _R1 to the left foot position coordinate Oh _{L1, the} left foot position coordinate Oh _L1 to the right foot position coordinate Oh _R2 , and the right foot position coordinate Oh _R2 to the left foot position coordinate Oh _L2 are timed by the same walking. Suppose that it will be determined as time passes.

Here, the position coordinates _{Oh R0} and the time t = _t walk vector 111 position line segment connecting the coordinates _{Oh R1} position of the right foot 302 when the ₁ position of the right foot 302 at time t = _{t 0.} Note that the walking vector 111 may be similarly obtained using the left foot 303.

  In this other embodiment, the next expected walking range may be predicted from the direction of the toe of the footprint.

  By using the walking state data 108, the expected walking range can be limited. Note that if the position of both feet is outside the expected walking range, it may be determined that the subject 101 has made an irregular movement, and the moving body 301 may urge the warning by a temporary stop or sound or light.

  When detecting the position of both feet in the walking state acquisition unit 105, means for detecting the position coordinate of the foot of the subject 101 who is in the same position for a certain period of time and means for storing the position coordinate data of the detected foot I have. Further, in the determination of the position coordinates of the foot, it is desirable to set that the time that the foot is in the same position is considered to be in the same position when the time is longer than 0.1 seconds (S> 0.1 seconds), By setting in this way, the midpoint point is obtained when the foot is surely in contact with the ground, and speed and direction prediction can be stably performed based on the locus point from the walking state data 108. It becomes possible.

In addition, the detection of the landing state of each foot can be regarded as the position coordinates of the foot being the same or a predetermined allowable range longer than a predetermined time (for example, 0.1 seconds) (stepping instead of walking, etc., in practice, a range that can be determined that has not started walking, for example, the range of the circle of radius R _t from the position coordinates of the detected feet, as a numerical example of the allowable range, 0 <a radius R _t ≦ stride × tolerance 130%), and the image processing unit does not land when the sole of the foot is visible, and the state where the sole of the foot is almost invisible is predetermined. If the time continues, it may be possible to detect the landing.

  Moreover, the apparatus which transmits position information, such as GPS, to the subject person 101 may be provided and used in combination with the present invention.

  In addition to an inverted two-wheel vehicle and a four-wheel vehicle, the moving body can also include an inverted two-wheel or biped robot.

  It is to be noted that, by appropriately combining any of the various embodiments, the effects possessed by them can be produced.

  While the present invention has been described in connection with preferred embodiments with reference to the drawings, various changes and modifications will be apparent to those skilled in the art. Such changes and modifications are to be understood as included within the scope of the inventive concept according to the appended claims.

  The mobile object accompaniment method and mobile object accompaniment device of the present invention are applied to a tracking method and tracking apparatus for automatically tracking an object using various tracking mobile objects of the object, In particular, the present invention is effective when applied to a walking tracking apparatus that is required to easily and surely track the movement path of the subject in response to the sudden movement of the subject.

1 is a block diagram showing a schematic configuration of a gait tracking device according to an embodiment of the present invention and a mobile object equipped with the gait tracking device. Explanatory drawing which shows the mobile body by which the walk tracking apparatus which concerns on this embodiment is mounted, and the target person of tracking object The flowchart which concerns on the processing step of the automatic walk tracking method by the moving body by which the walk tracking apparatus which concerns on this embodiment is mounted. The flowchart which shows the specific process of the subject of the walk tracking method by the walk tracking apparatus which concerns on this embodiment. The flowchart which shows the subject's walk state data acquisition process of the walk tracking method by the walk tracking apparatus which concerns on this embodiment. The flowchart which shows the confirmation process of the subject of the walk tracking method by the walk tracking apparatus which concerns on this embodiment. The flowchart which shows the walk speed and angle calculation process of the subject of the walk tracking method by the walk tracking apparatus which concerns on this embodiment. The flowchart which shows the tracking process of the subject of the walk tracking method by the walk tracking apparatus which concerns on this embodiment. The flowchart which shows the recognition method of the walk start of the subject of the walk tracking method by the walk tracking apparatus which concerns on this embodiment. The flowchart which shows the prediction method of the walk after the subject of the walk tracking method by the walk tracking apparatus which concerns on this embodiment starts a walk. In the walking tracking method by the walking tracking apparatus according to the present embodiment, the target person in the walking start state at the time t = t ₀ (the state in which the subject's left foot is one step ahead of the right foot) and the both feet at that time Explanatory drawing showing two-dimensional coordinates in correspondence In walking tracking method according to the walking tracking device according to the present embodiment, explanation diagram showing an example in which the mobile is attempting to track the subject walking start state at time t = t ₀ (the state of FIG. 10) In the walking tracking method by the walking tracking device according to the present embodiment, an image acquired by the camera in the walking start state at the time t = t ₀ (the state of FIG. 10, that is, both feet are landing) and the image Figure showing the image area of interest In the walk tracking method by the walk tracking device concerning this embodiment, when time t = t ₀ , explanatory drawing which shows the walk expected range of the normal walking state at time t = t ₁ on FIG. In the walking tracking method by the walking tracking apparatus according to the present embodiment, when time t = t ₀ ′ (the right foot is floating above the ground), the walking prediction in the normal walking state at time t = t ₁ Diagram showing range overlaid on FIG. In the walking tracking method by the walking tracking apparatus according to the present embodiment, when time t = t ₀ ′ (the right foot is floating above the ground), the walking prediction in the normal walking state at time t = t ₁ FIG. 13 is a diagram showing the range overlaid on FIG. 12 when the entire expected walking range is clearly shown. In the walking tracking method by the walking tracking apparatus according to the present embodiment, when time t = t ₀ ″ (the right foot is floating before landing), the walking prediction in the normal walking state at time t = t ₁ Diagram showing range overlaid on FIG. In the walking tracking method by the walking tracking apparatus according to the present embodiment, the subject in the normal walking state (the state where the subject's right foot has come out one step ahead of the left foot) at time t = t ₁ and the both feet at that time Explanatory drawing showing two-dimensional coordinates in correspondence In the walking tracking method by the walking tracking device according to the present embodiment, the image acquired by the camera in the normal walking state at time t = t ₁ (the state of FIG. 17, that is, the state where the right foot has landed) and the image Diagram showing image area of interest In walking tracking method according to the walking tracking device according to the present embodiment, explanation diagram showing an example in which the mobile is attempting to track the subject in a normal walking state (state of FIG. 17) at time t = t ₁ In the walking tracking method by the walking tracking device according to the present embodiment, the target person in the normal walking state at the time t = t ₂ (the state in which the subject's left foot is one step ahead of the right foot) and the both feet at that time Explanatory drawing showing two-dimensional coordinates in correspondence In the walking tracking method by the walking tracking apparatus which concerns on this embodiment, explanatory drawing which shows an example in which the mobile body has started tracking a subject in the normal walking state (state of FIG. 20) at time t = t ₂ . Explanatory drawing which shows the state where it is about to land so that the subject's right foot overlaps the front of the left foot Explanatory drawing which shows the state which landed so that a subject's right foot may overlap with the front of a left foot At time t = t ₂ , the subject's left foot trips over something, at time t = t ₃ , the subject starts to roll, and at time t = t ₄ , the subject rolls and holds both hands on the ground Explanatory diagram showing when connected And subjects such imbalance, explanatory view showing a time left foot is leaving the greater will be taken next, walking expected range of positions at time t = t ₂ Explanatory drawing for demonstrating the specific example of the automatic walk tracking method by the moving body by which the walk tracking apparatus which concerns on this embodiment is mounted. Explanatory drawing for demonstrating the other specific example of the automatic walk tracking method by the moving body by which the walk tracking apparatus which concerns on this embodiment is mounted. Explanatory drawing which shows the example of the determination method of the subject's walking speed and walking direction in the automatic walking tracking method by the moving body by which the walking tracking apparatus which concerns on other embodiment of this invention is mounted. Explanatory drawing which shows the example of the walk state of the subject in the automatic walk tracking method by the moving body by which the walk tracking apparatus which concerns on embodiment of FIG. 28 is mounted. Block diagram for explaining a conventional method of tracking a moving object

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Target person 103 Target person measuring device 105 Walking state acquisition part 106 Target person data 107 Walking state memory | storage part 108 Walking state data 109 Walking state judgment part 110 Walking state data 111 Walking vector 112 Target person memory part 113 Foot data 114 Target person judgment Unit 115 Target person stored data 116 Target person confirmation data 117 Movement command unit 118 Movement command 119 Drive unit 301 Moving object 1000 Tracking device

Claims (20)

A method for accompaniing a moving object in which the moving object moves according to the movement of the object,
Detecting the landing state of the subject's feet,
Detecting the walking direction of the subject,
Based on the detected landing state of the foot and the information on the direction, the predicted walking range of the target person is predicted, and the landing state of the foot due to the walking of the target person is detected within the predicted walking predicted range. The method of accompanying a moving object with a subject, wherein the moving object is controlled to move along the walking direction of the subject.   The expected walking range of the subject is a sector oriented in the walking direction of the subject with the radius determined from the stride of the subject and centered on the coordinates of the foot of the subject landing The method according to claim 1, wherein the moving object is accompanied by a subject.   The method for accompanying a moving object subject according to claim 1, wherein when the landing state of the foot is detected, the subject is discriminated and position coordinates of both feet of the subject are detected.   2. The method according to claim 1, wherein when the landing state of the foot is detected, both feet of the subject are detected and position coordinates of the both feet of the subject are detected.   When detecting the position coordinates of both feet, it is determined that the foot of the subject is in the same position coordinates for a certain time or more, and the position coordinates of the landing foot of the subject is stored. The method according to claim 4, wherein the subject is accompanied by a moving object.   6. The movement according to claim 5, wherein when the foot of the subject who is at the same position coordinate for the predetermined time or more is determined, the time for which the foot is at the same position coordinate is set to be greater than 0.1 seconds. How to accompany the body.   5. The position coordinates of the feet of the subject landing on the ground are detected when the position coordinates of the feet are detected, and the position coordinates of the feet of the subject are stored. How to accompany an object of a moving object.   When controlling the moving body to move along the walking direction of the subject person, the midpoint of a line segment connecting both feet of the subject person from the position information of the both feet of the subject person already detected, Control is made so as to generate the moving direction of the moving body by connecting the midpoint of the line segment connecting both feet of the subject in the next step, and to move the moving body according to the generated moving direction of the moving body The method according to claim 1, wherein the subject is accompanied by a moving object.   The method of moving a moving object with a subject according to claim 1, wherein when the moving body is controlled to move, the moving body is moved in the walking direction with the moving body preceding the subject.   2. The method according to claim 1, wherein when the moving body is controlled to move, the moving body is moved behind the target person to move the walking direction. A moving object subject accompaniment device in which a moving object moves according to the movement of the object person,
Walking state detection means for detecting a walking state that is a landing state of the subject's feet;
Walking direction detecting means for detecting the walking direction of the subject,
Based on the detected landing state of the foot and the information on the direction, the predicted walking range of the target person is predicted, and the landing state of the foot due to the walking of the target person is detected within the predicted walking predicted range. And a moving object target person accompaniment apparatus, characterized by comprising control means for controlling the moving body to move along the walking direction of the target person.   The expected walking range of the subject is a sector oriented in the walking direction of the subject with the radius determined from the stride of the subject and centered on the coordinates of the foot of the subject landing It is the range of these, The object person accompaniment apparatus for moving bodies of Claim 11 characterized by the above-mentioned.   12. The mobile object according to claim 11, wherein the walking state detection means includes a target person determination means for determining the target person and a both foot position detection means for detecting the position coordinates of both feet of the target person. Subject companion device.   The moving object accompanying device according to claim 11, wherein the walking state detecting means includes both foot position detecting means for detecting position coordinates of both feet of the subject.   The both foot position detecting means includes means for determining that the foot of the subject is in the same position coordinates for a predetermined time or more, and a storage unit for storing the position coordinates of the foot of the subject. 15. The moving object subject accompanying apparatus according to claim 14.   16. The means for judging the foot of the subject who is at the same position coordinate for the predetermined time or longer sets the time for which the foot is at the same position coordinate to be greater than 0.1 seconds. Mobile object companion device.   16. The both foot position detecting means includes means for judging the foot of the subject landing on the ground, and a storage unit for storing the position coordinates of the foot of the subject. The target person accompaniment apparatus for moving bodies described.   The walking direction detection means includes a midpoint of a line connecting both feet of the subject from the position coordinates of the feet of the subject already detected and a line connecting the feet of the subject in the next step. 13. The moving object subject accompanying apparatus according to claim 12, further comprising means for generating a moving direction of the moving object by connecting a midpoint.   13. The mobile object accompanying person apparatus according to claim 12, wherein the control means includes means for moving the walking direction in advance of the object before the object. 13. The moving object accompanying person apparatus according to claim 12, wherein the control means includes means for moving the moving body behind the subject and moving the walking direction.

Images