JP2011238139A - Moving object detecting program and moving object detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a flow of movement of plural persons.SOLUTION: A mobile object detecting device 50 comprises a storage unit 51, a moving locus calculating unit 52, a moving locus specifying unit 53, and a moving transition detecting unit 54. The moving locus calculating unit 52 calculates a moving locus of plural moving objects for each moving object based on position information and time information. In plural moving locus calculated by the moving locus calculating unit 52, the moving locus specifying unit 53 specifies a reference moving locus which shows a longer length of moving locus compared to lengths of other moving locus. The moving transition detecting unit 54 detects a moving transition of plural moving objects by integrating other moving objects similar to features of the reference moving locus specified by the moving locus specifying unit 53.

Description

  The present invention relates to a moving object detection program and a moving object detection device.

  The service robot performs various types of guidance and luggage transport operations in the service area. For example, this service area corresponds to a public space such as a commercial facility, a museum, or a station. In recent years, in order to improve the efficiency of service robot behavior, the density of people in the service area and the flow of people moving within a certain period of time can be detected to determine the optimal location and orientation for service robots to perform services. There is a need for technology to make adjustments. There is also a need for a technique for detecting the location of an event within a service area that occurs temporarily.

  In addition, as a technique for detecting the movement of a person, image data including movement information of a person is divided into a predetermined size, and the movement speed is aggregated and displayed for each area divided into the predetermined size. There is one that makes it possible to easily grasp the movement characteristics of a person relative to an object.

JP 2006-85366 A

  In general, when a service robot performs a service, it is more convenient to detect the flow of movement of a plurality of people in an integrated manner than to detect the flow of movement of an individual. However, there has been a technique for individually tracking a person's movement, but there is no technique for integrating and detecting the flow of movement of a plurality of persons.

  Further, when the movement speed is totaled for each area divided into a predetermined size as in the above-described prior art, there is a problem that it is impossible to accurately detect the movement characteristics of a person. 24-27 is a figure for demonstrating the problem of a prior art.

  As shown in FIG. 24, when the divided area is too large, a plurality of velocity vectors are included in the same area. In the example shown in FIG. 24, the region 1 includes velocity vectors 1a to 1c. Thus, when a plurality of velocity vectors are included in the same region, the velocity vectors 1a to 1c are averaged to become a velocity vector 1d. As described above, when the speed vectors are averaged, the accuracy of the moving speed corresponding to the region is lowered.

  In addition, when the direction of each speed vector in the same region is the opposite direction, the accuracy of the moving speed is significantly lowered by averaging the speed vectors. As shown in FIG. 25, the velocity vectors when two people are moving in the opposite direction in the same region 1 are velocity vectors 1e and 1f, respectively. It is assumed that the moving speed of each person is high, and the moving speed of one person is slightly faster than the moving speed of the other person. When the velocity vectors 1e and 1f are averaged, the velocity vector 1g is obtained. In this case, although the speed vectors 1e and 1f are large, the average speed vector 1g is small, so it is determined that the movement speed of the person in the region 1 is slow. The accuracy of the moving speed is significantly reduced.

  On the other hand, if the divided area is too small, an area that does not include the velocity vector is generated, and continuity is lost. In the example shown in FIG. 26, the velocity vectors do not exist in the regions 2b, 2d to 2i, and continuity is lost.

  As described above, in order to accurately detect a person's movement, the size to be divided must be adjusted in consideration of the movement speed of the person, but no such countermeasure has been taken. Note that it may be difficult to estimate the movement locus of a person even if the size to be divided is adjusted in consideration of the movement speed of the person.

  As shown in FIG. 27, when each region includes velocity vectors 3a to 3d, there are a plurality of patterns of connection of velocity vectors in adjacent regions, and it is not possible to uniquely identify the movement locus. . For example, when the speed vectors 3a and 3b are connected and the speed vectors 3c and 3d are connected, the movement trajectory of the person becomes the movement trajectories 3e and 3f, respectively. On the other hand, when the speed vectors 3a and 3d are connected and the speed vectors 3b and 3c are connected, the human movement trajectories are the movement trajectories 3g and 3h, respectively. In order to determine which movement trajectory is correct, it is difficult to say that the movement trajectory can be detected with high accuracy without relying on empirical rules.

  The disclosed technology has been made in view of the above, and an object thereof is to provide a moving body detection program and a moving body detection apparatus capable of detecting a movement transition of a plurality of persons. Note that the above-mentioned problem is a problem that occurs not only in service robots but also in monitoring systems that monitor public spaces.

  A moving object detection program disclosed in the present application stores in a storage unit, in a computer, position information indicating a position of a moving object and time information indicating a time when the moving object is present at the position, in association with each other. Based on the position information and the time information, a movement trajectory calculation procedure for calculating a movement trajectory of a plurality of moving bodies for each moving body, and among the plurality of movement trajectories calculated by the movement trajectory calculation procedure, By integrating a movement trajectory identification procedure for identifying a reference movement trajectory that indicates a movement trajectory that is longer than the length of other movement trajectories, and other movement trajectories similar to the characteristics of the reference movement trajectory, It is a requirement to execute a movement transition detection procedure for detecting a movement transition of a moving body.

  According to one aspect of the moving body detection program disclosed in the present application, it is possible to detect the movement transition of a plurality of persons.

FIG. 1 is a diagram illustrating the configuration of the moving object detection device according to the first embodiment. FIG. 2 is a diagram illustrating the configuration of the service robot according to the second embodiment. FIG. 3 is a diagram for explaining LRF processing. FIG. 4 is a functional block diagram illustrating configurations of the storage unit and the control unit. FIG. 5 shows the data structure of the source data. FIG. 6 is a diagram for explaining the processing of the movement trajectory integration unit. FIG. 7 is a diagram illustrating an example of a movement trajectory spline. FIG. 8 is a diagram illustrating a data structure of the movement trajectory spline list. FIG. 9 is a diagram for explaining the average value of the shortest distances. FIG. 10 is a diagram illustrating a data structure of the flow list. FIG. 11 is a diagram for explaining calculation of the entire flow. FIG. 12 is a diagram illustrating an overall flow calculated by the movement trajectory integration unit. FIG. 13 is a diagram for explaining processing of the attention location detection unit. FIG. 14 is a diagram illustrating a data structure of the deceleration coordinate point list. FIG. 15 is a diagram for explaining a place with a high degree of attention. FIG. 16 is a diagram for explaining other processing of the attention location detection unit. FIG. 17 is a diagram (1) for explaining the processing of the real-time action recognition unit. FIG. 18 is a diagram (2) for explaining the processing of the real-time action recognition unit. FIG. 19 is a flowchart of the process procedure of the control unit according to the second embodiment. FIG. 20 is a flowchart illustrating a processing procedure of the movement trajectory integration unit. FIG. 21 is a flowchart illustrating a processing procedure of the attention location detection unit. FIG. 22 is a flowchart illustrating the processing procedure of the real-time action recognition unit. FIG. 23 is a diagram illustrating a hardware configuration of a computer constituting the control unit according to the present embodiment. FIG. 24 is a diagram (1) for explaining the problems of the prior art. FIG. 25 is a diagram (2) for explaining the problem of the prior art. FIG. 26 is a diagram (3) for explaining the problem of the prior art. FIG. 27 is a diagram (4) for explaining the problem of the prior art.

  Embodiments of a moving object detection program and a moving object detection device disclosed in the present application will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a diagram illustrating the configuration of the moving object detection device according to the first embodiment. As illustrated in FIG. 1, the moving body detection device 50 includes a storage unit 51, a movement locus calculation unit 52, a movement locus identification unit 53, and a movement transition detection unit 54.

  The storage unit 51 stores position information indicating the position of the moving object and time information indicating the time when the moving object has existed at the position in association with each other. The movement trajectory calculation unit 52 calculates the movement trajectories of the plurality of moving bodies for each moving body based on the position information and the time information.

  The movement trajectory identification unit 53 identifies a reference movement trajectory that indicates a movement trajectory that is longer than the length of other movement trajectories among the plurality of movement trajectories calculated by the movement trajectory calculation unit 52. . The movement transition detection unit 54 detects movement transitions of a plurality of moving bodies by integrating other movement tracks similar to the characteristics of the reference movement track specified by the movement track specifying unit 53.

  The moving body detection device 50 identifies a reference movement locus based on the length of each movement locus among movement loci of a plurality of moving bodies, and integrates other movement loci similar to the characteristics of the reference movement locus. . For this reason, the moving body detection apparatus 50 according to the first embodiment can detect the movement transition of the movement of the entire moving body.

  Note that the movement trajectory calculation unit 52, the movement trajectory identification unit 53, and the movement transition detection unit 54 illustrated in FIG. 1 correspond to the movement trajectory integration unit 172 of FIG. 4 described later. The storage unit 51 corresponds to a storage unit 160 of FIG.

  FIG. 2 is a diagram illustrating the configuration of the service robot according to the second embodiment. As shown in FIG. 2, the service robot includes a real-time clock 110, a camera 120a, an LRF 120b, a human detection unit 130, a drive unit 140, a notification unit 150, a storage unit 160, and a control unit 170.

  The real time clock 110 is a device that sequentially generates current time data and outputs the generated time data to the human detection unit 130 and the control unit 170. The camera 120 a is a device that sequentially captures images and outputs the captured image data to the human detection unit 130.

  An LRF (Laser Range Finder) 120b is a device that detects the coordinates of an object within a search range. The LRF 120b outputs the coordinate data of the detected object to the human detection unit 130. Hereinafter, an example of processing in which the LRF 120b detects the coordinates of an object will be described. FIG. 3 is a diagram for explaining LRF processing.

  The Xg axis and the Yg axis in FIG. 3 are coordinate axes with a predetermined position as a reference, and this coordinate axis system is a global coordinate axis system. The Xr axis and the Yr axis in FIG. 3 are coordinate axes based on the service robot 100, and this coordinate axis is a local coordinate axis system. The search range 5 in FIG. 3 corresponds to the laser irradiation range of the LRF 120b.

  The LRF 120b measures the distance between the service robot 100 and the object and the angle between the service robot 100 and the object by irradiating the laser in the horizontal direction and measuring the time that the irradiated laser hits and reflects the object. The LRF 120b calculates the coordinates of the object on the local coordinate axis from the measured distance and angle. Then, the LRF 120b calculates the coordinates of the object on the global coordinate axis by adding the coordinates of the service robot 100 on the global coordinate axis and the coordinates of the object on the local coordinate axis. The LRF 120b outputs the coordinate data of the object on the global coordinate axis to the human detection unit 130.

  The human detection unit 130 is a processing unit that acquires coordinate data of each object from the LRF 120b and detects human coordinate data among the acquired coordinate data. For example, the person detection unit 130 holds a template including the characteristics of a person's image, and compares the template with image data from the camera 120a to determine whether a person is present in front of the service robot 100. Determine whether.

  For example, the human detection unit 130 detects the coordinate data of the object acquired from the LRF 120b as the human coordinate data while determining that there is a person in front of the service robot 100, and ID (Identification) Is granted. Then, the human detection unit 130 outputs data that associates the ID, the coordinate data, and the time data obtained from the coordinate data to the control unit 170.

  The drive unit 140 is a device that drives the service robot 100 in response to a control command from the control unit 170. For example, wheels or the like (not shown) are attached to the drive unit 140, and the drive unit 140 moves the service robot 100 by driving the wheels. The notification unit 150 is a processing unit that notifies various applications output from the control unit 170 to the upper application.

  The storage unit 160 is a storage device that stores data used by the control unit 170 to execute various processes. The storage unit 160 corresponds to, for example, a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), or a flash memory, or a storage device such as a hard disk or an optical disk.

  The control unit 170 is a control device that detects a flow of movement of a plurality of persons in a service area, a position that is noticed by the plurality of persons, and a position of an event that temporarily affects the plurality of persons. The control unit 170 corresponds to an integrated device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Or this control part 170 respond | corresponds to electronic circuits, such as CPU (Central Processing Unit) and MPU (Micro Processing Unit).

  Next, specific configurations of the storage unit 160 and the control unit 170 illustrated in FIG. 2 will be described. FIG. 4 is a functional block diagram illustrating configurations of the storage unit and the control unit. As shown in FIG. 4, the storage unit 160 stores original data 161, a movement trajectory spline list 162, a flow list 163, a deceleration coordinate point list 164, and an intersection point list 165. The control unit 170 includes a source data registration unit 171, a movement locus integration unit 172, an attention place detection unit 173, a real-time action recognition unit 174, and a drive control unit 175.

  The source data registration unit 171 is a processing unit that acquires a person ID, person coordinate data, and time data from the person detection unit 130 and generates the source data 161 based on the acquired information. FIG. 5 shows the data structure of the source data. As shown in FIG. 5, the original data 161 includes a time stamp, ID (Identification), x coordinate, y coordinate, speed in the x direction, and speed in the y direction.

  Among these, ID is information that uniquely identifies a person. The x coordinate indicates a person's x coordinate on the global coordinate axis. The y coordinate indicates the y coordinate of the person on the global coordinate axis. The time stamp indicates the time when a person is present at the x and y coordinates. The speed in the x direction indicates the speed of the person in the x direction at the time indicated by the time stamp. The speed in the y direction indicates the speed of the person in the y direction at the time indicated by the time stamp.

  For example, the source data registration unit 171 calculates the velocity in the x direction by dividing the difference value between the preceding and following x coordinates by the difference value between the preceding and succeeding time stamps. The source data registration unit 171 calculates the speed in the y direction by dividing the difference value between the preceding and following y coordinates by the difference value between the preceding and succeeding time stamps.

  The movement trajectory integration unit 172 is a processing unit that extracts a movement flow of the plurality of persons by integrating the movement trajectories of the plurality of persons. The overall movement flow of a plurality of people has the same meaning as the movement movement of the whole of the plurality of people. FIG. 6 is a diagram for explaining the processing of the movement trajectory integration unit. As shown in FIG. 6, the movement trajectory integration unit 172 calculates a movement trajectory spline for each ID based on the x-coordinate and y-coordinate of the original data 161 (step S10). Here, the movement trajectory spline corresponds to a human movement trajectory.

  After calculating the movement locus spline, the movement locus integration unit 172 classifies the movement locus spline for each similar movement locus spline (step S11). Then, the movement trajectory integration unit 172 performs averaging for each classified movement trajectory spline, and detects the trajectory after the averaging as a flow of a plurality of persons (step S12).

  Here, the process of the movement trajectory integration unit 172 shown in step S10 of FIG. 6 will be described. The movement trajectory integration unit 172 calculates the movement trajectory spline data by calculating an approximate curve of the x and y coordinates using the least square method. FIG. 7 is a diagram illustrating an example of a movement trajectory spline. Curve 1A in FIG. 7 is a movement trajectory spline corresponding to ID “XXX”, and curve 2A is a movement trajectory spline corresponding to ID “XX”.

  The movement locus integration unit 172 registers the movement locus splines in the movement locus spline list 162 after calculating the movement locus splines for all the IDs stored in the original data 161. FIG. 8 is a diagram illustrating a data structure of the movement trajectory spline list. As illustrated in FIG. 8, the movement trajectory spline list 162 stores a movement trajectory spline for each person ID. The movement trajectory spline list 162 also stores the coordinates of the start point and the end point. Note that the movement trajectory spline stored in the movement trajectory spline list 162 may be continuous x and y coordinate data, or may be an approximate expression obtained by the least square method.

  Next, the process of the movement locus integration unit 172 shown in step S11 of FIG. 6 will be described. First, the movement trajectory integration unit 172 sorts the movement trajectory splines stored in the movement trajectory spline list 162 in order from the longest one, and takes out the longest movement trajectory spline. The movement locus integration unit 172 deletes the extracted movement locus spline from the movement locus spline list 162. In the following description, the longest movement locus spline among the plurality of movement locus splines stored in the movement locus spline list 162 is referred to as a reference spline.

  The movement trajectory integration unit 172 sequentially compares the reference spline and the remaining movement trajectory splines stored in the movement trajectory spline list 162 to obtain the direction difference, the average value of the shortest distance, and the variance, and moves similar to the reference spline. Determine the trajectory spline.

  First, the direction difference will be described. The movement trajectory integration unit 172 calculates the direction difference by obtaining a difference between the direction from the start point to the end point of the reference spline and the direction from the start point to the end point of the movement trajectory spline.

  Next, the average value of the shortest distance will be described. The movement trajectory integration unit 172 samples the movement trajectory spline by a predetermined number of samples. Then, the movement trajectory integration unit 172 calculates the shortest distance from the nth coordinate of the sampled movement trajectory spline to the reference spline. For example, when the number of samples is Si, n is an integer from 1 to Si. The movement locus integration unit 172 calculates the average value of the shortest distances by dividing the total value of the shortest distances by Si.

  FIG. 9 is a diagram for explaining the average value of the shortest distances. In FIG. 9, description will be made using the reference spline 1B and the movement trajectory splines 2B and 3B. The number of samples is 3. The movement trajectory integration unit 172 divides the value obtained by adding the distances 1C, 2C, and 3C by 3 when obtaining the average value of the shortest distances between the reference spline 1B and the movement trajectory spline 2B. The movement trajectory integration unit 172 divides the value obtained by adding the distances 1D, 2D, and 3D by 3 when determining the average value of the shortest distances between the reference spline 1B and the movement trajectory spline 3B.

  Subsequently, dispersion will be described. First, the movement trajectory integration unit 172 calculates the average value in the same manner as the method for calculating the average value of the shortest distances. Then, the movement trajectory integration unit 172 squares the values obtained by dividing the average value from each shortest distance, and sums the squared values. The movement trajectory integration unit 172 calculates the variance by dividing the total value by the square of the average value.

  The movement trajectory integration unit 172 detects a movement trajectory spline in which the direction difference is less than the first threshold, the average value of the shortest distances is less than the second threshold, and the variance is less than the third threshold for each reference spline. The movement trajectory spline that satisfies the above conditions is similar to the reference spline. The movement trajectory integration unit 172 stores the reference spline and the movement trajectory spline similar to the reference spline in association with each other in the flow list 163.

  FIG. 10 is a diagram illustrating a data structure of the flow list. As shown in FIG. 10, the flow list 163 includes a reference spline ID, a reference spline, and a movement trajectory spline that uniquely identify the reference spline.

  Next, the process of the movement trajectory integration unit 172 shown in step S12 of FIG. 6 will be described. The movement trajectory integration unit 172 refers to the flow list 163 and calculates the overall flow by averaging the reference spline and each movement trajectory spline for each reference spline ID. FIG. 11 is a diagram for explaining calculation of the entire flow. For example, assume that a reference spline ID includes a reference spline 4A and a movement trajectory spline 4B. In this case, the movement trajectory integration unit 172 calculates the entire flow 4C by applying the least square method to the values of the coordinates x and y of the splines 4A and 4B.

  FIG. 12 is a diagram illustrating an overall flow calculated by the movement trajectory integration unit. In the example shown in FIG. 12, the movement trajectory splines 5B and 5C and the reference spline 5A are averaged to obtain the entire flow 5D. In addition, the movement trajectory spline 6A and the reference spline 6B are averaged to obtain the entire flow 6C. The movement locus integration unit 172 outputs the entire flow data to the drive control unit 175.

  Note that the movement trajectory integration unit 172 may adjust the number of samples based on the length of the reference spline and the length of other movement trajectory splines when performing averaging. For example, the movement trajectory integration unit 172 determines the sample number Smax of the reference spline. Then, the movement trajectory integration unit 172 calculates the number of samples Si of the other movement trajectory splines by the formula (1), where SPi is the length of the other movement trajectory spline and SPmax is the length of the reference spline.

Si = SPmax / SPi × Smax (1)
As shown in equation (1), the longer the reference spline, the greater the number of samples of other movement trajectory splines, which greatly affects the calculation when calculating the overall flow.

  Returning to the description of FIG. The attention location detection unit 173 is a processing unit that detects a location with a high degree of attention by utilizing the fact that the movement speed of the person is reduced when a person passes through a location with a high degree of attention. FIG. 13 is a diagram for explaining processing of the attention location detection unit. In FIG. 13, the vertical axis indicates speed, and the horizontal axis indicates time. A curve 7A shows the relationship between the time and speed of the person a, a curve 7B shows a relationship between the time and speed of the person b, and a curve 7C shows the time and speed of the person c. It shows the relationship. Each of the curves 7A to 7C can be obtained from the time stamp of the original data 161 and the velocities in the x and y directions. It is assumed that a predetermined ID is assigned to each person ac.

  First, the attention location detection unit 173 extracts a time zone during which each person decelerates. A person's deceleration is equivalent to a negative person's acceleration. For this reason, the attention location detection unit 173 calculates the acceleration for each person based on the original data 161, and extracts the time zone in which the calculated acceleration is negative. In the example shown in FIG. 13, the time zone in which the person a decelerates is 8A, the time zone in which the person b decelerates is 8B, and the time zone in which the person c decelerates is 8C.

  The attention location detection unit 173 acquires a movement locus of each person corresponding to the decelerated time zone from the movement locus spline list 162 after extracting the time zone in which each person decelerated. Then, the attention location detection unit 173 samples the movement trajectory of the decelerated time zone at equal intervals, and registers the sampled x, y coordinate points and movement directions in the deceleration coordinate point list 164 in association with each other. Note that the attention location detection unit 173 calculates the movement direction from the relationship between the start point and the end point of the movement locus.

  FIG. 14 is a diagram illustrating a data structure of the deceleration coordinate point list. As shown in FIG. 14, the deceleration coordinate point list 164 stores IDs, movement directions, and deceleration coordinate points in association with each other. The ID uniquely identifies a person, and the deceleration coordinate point corresponds to the x and y coordinates when the movement trajectory of the decelerated time zone is sampled at equal intervals.

  The attention location detection unit 173 creates a 2D histogram with the classes DX and DY using the deceleration coordinate point for each ID. Then, the attention location detection unit 173 extracts locations in descending order of the frequency of the histogram, and extracts a region where the frequency is equal to or higher than the predetermined frequency as a location with a high attention level. FIG. 15 is a diagram for explaining a place with a high degree of attention.

  In FIG. 15, a place 9A is an area where the person a has moved to a time zone 8A where the person a has decelerated, a place 9B is an area where the person b has moved to a time zone 8B where the person b has decelerated, and a place 9C This is the area moved to the time zone 8C. For example, if a histogram is created based on the x and y coordinates that each person a to c has moved, the frequency of the area corresponding to the place 9D becomes equal to or greater than a predetermined frequency. In this case, the attention place detection unit 173 determines the place 9D as a place with a high degree of attention. For example, there is a high possibility that an interesting exhibit 9E exists near the place 9D.

  Further, the attention location detection unit 173 creates a histogram with the class DA using the moving direction for each ID. And the attention location detection part 173 determines the moving direction with the highest frequency of a histogram as a moving direction with respect to a location with high attention degree. The attention location detection unit 173 outputs, to the drive control unit 175, a location with a high degree of attention and data on the movement direction for the location with a high degree of attention.

  Even if the degree of attention is not high, it is considered that the moving speed of a person decreases when an obstacle is present. For this reason, the attention location detection unit 173 holds the location information where the obstacle exists, removes the location with high attention included in the location information, and then drives the data of the remaining locations with high attention. You may output to the control part 175. FIG.

  By the way, in the processing of the attention location detection unit 173 described above, a location with a high degree of attention is detected using a histogram, but is not limited thereto. The attention location detection unit 173 may detect a location with a high degree of attention by using a kernel density estimation method. FIG. 16 is a diagram for explaining other processing of the attention location detection unit.

  The horizontal axis in FIG. 16 indicates time. The vertical axis indicates different values depending on each curve. Specifically, for the curve 10A, the vertical axis corresponds to the position. For curve 10B, the vertical axis corresponds to speed. For curve 10C, the vertical axis corresponds to acceleration. The attention location detection unit 173 creates the curves 10A and 10B based on the original data 161. In addition, the attention location detection unit 173 calculates the acceleration for each hour based on the front-rear speed and the time difference between the front-rear and creates the curve 10C.

  The location-of-interest detection unit 173 focuses on the transition of the acceleration of the curve 10C, and specifies the time when the acceleration decreases as the deceleration start time Tni. The attention location detection unit 173 focuses on the transition of the acceleration of the curve 10C, and specifies the time during which the acceleration increases after the acceleration decreases as the deceleration completion time Tnj.

  Then, the attention location detection unit 173 specifies a position between the deceleration start time Tni and the deceleration completion time Tnj. In the example shown in FIG. 16, the position range between the deceleration start time Tni and the deceleration completion time Tnj is the position range 10D. The location-of-interest detection unit 173 samples the position range 10D at the interval DT, and registers the x and y coordinates that are the sampling results in the deceleration coordinate point list 164. The attention location detection unit 173 calculates the position range where the person has decelerated by executing the above-described process for other people as well. Then, the attention location detection unit 173 samples each position range and registers the sampling result in the deceleration coordinate point list 164.

  Note that the method of specifying the deceleration start time Tni and the deceleration completion time Tnj by the attention location detection unit 173 is as follows. The attention location detection unit 173 differentiates the curve 10C at a time interval of dt, where Tn0 is the disclosure time of the curve 10C. The differentiated value is Jerk. The attention location detection unit 173 sets the time zone in which Jerk is negative as the deceleration time zone, sets the start time of this deceleration time zone as the deceleration start time Tni, and sets the end time of the deceleration time zone as the deceleration completion time Tnj.

  The resolution when the attention location detection unit 173 performs sampling is appropriately switched depending on the operation. For example, when high accuracy is required for detection of a target location, the sampling interval is set to a short interval. Thus, the detection accuracy of the attention place is improved by shortening the sampling interval. On the other hand, when high accuracy is not required for detecting the location of interest, the sampling interval is set to a long interval. Thus, the calculation cost can be reduced by increasing the sampling interval.

  Returning to the description of FIG. The real-time action recognition unit 174 is a processing unit that detects the position of an event that occurs instantaneously at a certain time Tn. The real-time action recognition unit 174 acquires the person ID, the person coordinate data, and the time data from the person detection unit 130, and calculates the movement speed of the person for each ID. For example, the real-time action recognition unit 174 sequentially calculates the movement speed by dividing the difference between the preceding and following coordinates by the difference between the preceding and following times.

  The real-time action recognition unit 174 calculates a straight line that passes through the position of the person at time Tn and has the same direction as the direction of the moving speed. The real-time action recognition unit 174 calculates the straight line for each person. Then, the real-time action recognition unit 174 calculates the intersection of each straight line, and detects the coordinates of the calculated intersection as the position of the event that occurs instantaneously. The real-time action recognition unit 174 registers the intersection of each straight line in the intersection list 165. Further, the real-time action recognition unit 174 outputs event position data to the drive control unit 175.

  FIG. 17 is a diagram (1) for explaining the processing of the real-time action recognition unit. In FIG. 17, 11A, 11B, and 11C indicate the positions of the persons a, b, and c at time Tn. 12A, 12B, and 12C indicate the directions of the moving speeds of the persons a, b, and c at time Tn. The straight line 13A passes through the position 11A and has the same direction as the direction 12A. The straight line 13B passes through the position 11B and has the same direction as the direction 12B. The straight line 13C passes through the position 11C and has the direction 12C. It is a straight line in the same direction. The intersection of the straight lines 13A, 13B, and 13C becomes the intersection 14A. In this case, the real-time action recognition unit 174 detects the intersection 14A as the position of the event that occurs instantaneously.

  In the example illustrated in FIG. 17, the case where each person is approaching the intersection has been described. However, even when the person leaves the intersection, it is considered that some event has occurred at a position corresponding to the intersection. FIG. 18 is a diagram (2) for explaining the processing of the real-time action recognition unit. In FIG. 18, 11D, 11E, and 11F indicate the positions of the persons d, e, and f at the time Tn. 12D, 12E, and 12F indicate the directions of the moving speeds of the persons d, e, and f at time Tn. The straight line 13D passes through the position 11D and has the same direction as the direction 12D. The straight line 13E passes through the position 11E and has the same direction as the direction 12E. The straight line 13F passes through the position 11F and passes through the position 12F. It is a straight line in the same direction. The intersection of the straight lines 13D, 13B, and 13C is the intersection 14B. In this case, the real-time action recognition unit 174 detects the intersection 14B as the position of the event that occurs instantaneously.

  In addition, after calculating the intersection of each straight line, the real-time action recognition unit 174 determines the position within the certain range of the event that occurs instantaneously when the number of intersections existing within the certain range is equal to or greater than the threshold. You may detect as a position. The intersection may be an intersection of two straight lines or an intersection of three or more straight lines.

  Returning to the description of FIG. The drive control unit 175 indicates the flow of a plurality of people acquired from the movement trajectory integration unit 172, the location with a high degree of attention acquired from the attention location detection unit 173, and the position of the event that occurs instantaneously acquired from the real-time action recognition unit 174. Based on this, the processing unit drives the driving unit 140. For example, the drive control unit 175 outputs a control command to the drive unit 140 so that the service robot 100 moves so as to follow the flow of a plurality of people. Further, the drive control unit 175 outputs a control command to the drive unit 140 so that the service robot 100 moves to a position with a high degree of attention. Further, the drive control unit 175 outputs a control command to the drive unit 140 so that the service robot 100 moves to the position of the event that occurs instantaneously.

  Next, a processing procedure of the control unit 170 illustrated in FIG. 2 will be described. FIG. 19 is a flowchart of the process procedure of the control unit according to the second embodiment. The process illustrated in FIG. 19 is executed, for example, when the service robot 100 is activated. As shown in FIG. 19, the control unit 170 sets the detection cycle to Ci (step S101). The value of Ci is, for example, 100 ms.

  The control unit 170 stores the person's ID, coordinates, and speed in the storage unit 160 with a time stamp (step S102). In addition, the control unit 170 creates a movement trajectory spline and stores it in the storage unit 160 (step S103).

  When the data within a certain period of time is statistically determined (step S104, Yes), the control unit 170 detects a flow of a plurality of people within a certain period of time and a place with a high degree of attention, and outputs a detection result (step S105). The process proceeds to step S106. Here, the fixed time is, for example, about 10 minutes.

  On the other hand, the control unit 170 determines whether or not to perform real-time action recognition (step S106) when the data within a certain period of time is not statistics (step S104, No). When performing real-time action recognition (step S106, Yes), the controller 170 executes real-time action recognition, outputs the execution result (step S107), and proceeds to step S108.

  On the other hand, when the real time action recognition is not performed (No at Step S106), the control unit 170 determines whether to end the process (Step S108). If the control unit 170 ends the process (step S108, Yes), the control unit 170 ends the process. When the process is continued (No at Step S108), the control unit 170 adds 1 to Ci (Step S109), and proceeds to Step S101 again.

  Next, the processing procedure of the movement trajectory integration unit 172 shown in FIG. 4 will be described. FIG. 20 is a flowchart illustrating a processing procedure of the movement trajectory integration unit. The process illustrated in FIG. 20 is executed by the source data registration unit 171 when the source data 161 is generated, for example.

  As illustrated in FIG. 20, the movement trajectory integration unit 172 creates a movement trajectory spline list 162, and sets the movement trajectory spline list 162 to be empty (step S201). Further, the movement trajectory integration unit 172 adds the movement trajectory splines of all persons within a predetermined time to the movement trajectory spline list 162 (step S202).

  The movement trajectory integration unit 172 sorts the movement trajectory spline list 162 based on the length of the movement trajectory spline (step S203). The movement trajectory integration unit 172 calculates a direction at the start point and end point of each movement trajectory spline, and stores the calculation result (step S204).

  The movement locus integration unit 172 acquires the longest movement locus spline i among the movement locus splines included in the movement locus spline list 162 (step S205). Then, the movement trajectory integration unit 172 creates a flow list 163 using the movement trajectory spline i as a reference spline (step S206).

  The movement trajectory integration unit 172 acquires a movement trajectory spline j whose azimuth difference from the movement trajectory spline i is less than the first threshold (step S207). The movement trajectory integration unit 172 samples the movement trajectory spline j at N points at equal intervals, and calculates the shortest distance from the movement trajectory spline i (step S208).

  When the average value and variance of the shortest distances are equal to or greater than the threshold (No at Step S209), the movement trajectory integration unit 172 adds 1 to j (Step S210), and proceeds to Step S207. On the other hand, when the average value and the variance of the shortest distances are less than the threshold (Yes in step S209), the movement trajectory integration unit 172 adds the movement trajectory spline j to the flow list 163 of the movement trajectory spline i. In addition, the movement locus integration unit 172 deletes the movement locus spline i from the movement locus spline list 162 (step S211). As described above, the threshold value compared with the average value of the shortest distance is the second threshold value, and the threshold value compared with the variance is the third threshold value.

  If there is a movement trajectory spline whose azimuth difference from the movement trajectory spline i is less than the first threshold (Yes in step S212), the movement trajectory integration unit 172 proceeds to step S210. On the other hand, if there is no movement locus spline whose azimuth difference from the movement locus spline i is less than the first threshold (No in step S212), the movement locus integration unit 172 determines whether the movement locus spline list 162 is empty. It is determined whether or not (step S213).

  If the movement trajectory spline list 162 is not empty (step S213, No), the movement trajectory integration unit 172 adds 1 to i (step S214), and moves to step S205.

  On the other hand, when the movement trajectory spline list 162 is empty (step S213, Yes), the movement trajectory integration unit 172 averages all the movement trajectory splines included in the flow list 163 in consideration of the weight (step). S215). Then, the movement trajectory integration unit 172 sets the averaged movement trajectory spline as a corresponding flow model and outputs it to the drive control unit 175 (step S216).

  Next, a processing procedure of the attention location detection unit 173 illustrated in FIG. 4 will be described. FIG. 21 is a flowchart illustrating a processing procedure of the attention location detection unit. The process illustrated in FIG. 21 is executed by the source data registration unit 171 when the source data 161 is generated, for example.

  The attention location detection unit 173 acquires the trajectory of the movement position, the trajectory of the speed, and the trajectory of the acceleration of all persons within a predetermined time (step S301). Further, the attention location detection unit 173 generates the deceleration coordinate point list 164 (step S302).

  The attention location detection unit 173 extracts a time zone [Tni, Tnj] in which the Nth person decelerates (step S303), acquires a movement trajectory in the time zone [Tni, Tnj], and moves at a distance DT. Is resampled (step S304).

  The attention location detection unit 173 adds the resampled X and Y coordinate points and the movement direction to the deceleration coordinate point list 164 (step S305). When the registration is not completed (No at Step S306), the attention location detection unit 173 proceeds to Step S303 again.

  On the other hand, when the registration is completed (Yes in step S306), the attention location detection unit 173 creates a 2D histogram with classes DX and DY using the X and Y coordinate points of the deceleration coordinate point list 164 ( Step S307).

  The attention location detection unit 173 determines a location where the frequency of the histogram is N or more (step S308), and creates a moving direction histogram at the determined location (step S309). And the attention place detection part 173 determines the angle of the highest frequency of a histogram as a moving direction of an applicable place (step S310).

  Next, the processing procedure of the real-time action recognition unit 174 shown in FIG. 4 will be described. FIG. 22 is a flowchart illustrating the processing procedure of the real-time action recognition unit. The process illustrated in FIG. 22 is executed, for example, when the person ID, coordinate data, and time data are acquired from the person detection unit 130.

  The real-time action recognition unit 174 acquires the positions and velocities of all persons detected in the detection cycle (step S401), and creates a straight line that passes through each person's position and has the same direction as the speed (step S402).

  The real-time action recognition unit 174 calculates the X and Y coordinates of the intersection between the straight line i and a straight line other than the straight line i (step S403) and registers it in the intersection list 165 (step S404). If the registration is not completed (step S405, No), the real-time action recognition unit 174 adds 1 to i (step S406), and proceeds to step S403.

  On the other hand, when the registration is completed (Yes in step S405), the real-time action recognition unit 174 creates a 2D histogram with the class DR using the X and Y coordinates of the intersection list 165 (step S407). When the frequency of the histogram exceeds the threshold value N, the real-time action recognition unit 174 detects the average value of the class as the event occurrence position (step S408).

  As described above, the control unit 170 included in the service robot 100 identifies the longest movement trajectory spline among a plurality of human movement trajectory splines as a reference spline. And the control part 170 detects the flow of several persons by integrating the other movement spline similar to the characteristic of a reference spline. As described above, since the flow of a plurality of people can be detected, for example, the service robot 100 can move to an optimal position when performing a service without being influenced by the movement of a single person.

  In addition, the control unit 170 calculates the speed of the person and detects the position of the person when the speed is reduced. Generally, when a person approaches a place with a high degree of attention, it is considered that the moving speed is reduced. For this reason, since the control part 170 can detect a place with a high degree of attention, the service robot 100 can move to a place with a high degree of attention and provide a service.

  Further, each time a person moves, the control unit 170 calculates a temporary movement direction of each person, and a position that affects the movement of each person based on a straight line in the same direction as each calculated movement direction. Is detected. For this reason, the control unit 170 can detect a place that is noticed by each person in real time, and the service robot 100 can appropriately execute the service by moving to the place that is noticed.

  By the way, in the second embodiment, the control unit 170 detects a flow of movement of a plurality of persons in the service area, a position that is noticed by a plurality of persons, and a position of an event that is temporarily given to the plurality of persons. The case where 100 is moved was demonstrated. However, the function of the control unit 170 can be applied to devices other than the service robot 100. For example, when the function of the control unit 170 is applied to a monitoring system that controls a camera and a flow of movement of a plurality of people, a position that is noticed by a plurality of persons, and a position of an event that is temporarily given to a plurality of persons are detected. You may make it monitor by directing a camera in the direction of a place.

  In the second embodiment, the case where the LRF 120b is mounted on the service robot 100 has been described. However, the present invention is not limited to this. For example, the LRF 120b may be installed at a predetermined position other than the service robot 100.

  In addition, the drive control unit 175 compares the movement transition of the plurality of persons acquired from the movement trajectory integration unit 172 with the place of high attention acquired from the attention place detection unit 173, and the movement transition of the plurality of persons and the degree of attention are compared. A region overlapping with a high place may be detected. For example, the drive control unit 175 outputs a control command to the drive unit 140 so that the service robot 100 moves toward the detected area.

  The control unit 170 described above can be realized by mounting each function of the control unit 170 or the like on an information processing apparatus such as a known personal computer or workstation.

  FIG. 23 is a diagram illustrating a hardware configuration of a computer constituting the control unit according to the present embodiment. The computer 200 includes a CPU 201 that executes various arithmetic processes, an input device 202 that receives data input from a user, and a network interface device 203 that exchanges data with other computers via a network. . The computer 200 also includes a camera 204, an LRF 205, and a real time clock 206. The computer 200 also includes a RAM (Random Access Memory) 207 that temporarily stores various types of information and a hard disk device 208. Each device 201 to 208 is connected to a bus 209.

  The hard disk device 208 stores a detection program 208a having functions similar to those of the human detection unit 130 and the control unit 170 shown in FIG. Also, the hard disk device 208 stores various data 208b corresponding to the data stored in the storage unit 160 of FIG.

  The CPU 201 reads the detection program 208 a and develops it in the RAM 207. Then, the detection program 208a functions as the detection process 207a. The detection process 207a corresponds to the human detection unit 130 and the control unit 170 in FIG. The CPU 201 reads various data 208 b stored in the hard disk device 208 and stores it in the RAM 207.

  The detection process 207a uses various data 207b and data from the camera 204, the LRF 205, and the real-time clock 206 to detect the flow of movement of multiple people, a place of high attention, and the position of an event that occurs temporarily. To do.

  Note that the detection program 208 a is not necessarily stored in the hard disk device 208. The computer 200 may read and execute the detection program 208a stored in a storage medium such as a CD-ROM. Alternatively, the detection program 208a may be stored in a public line, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or the like, and the computer 200 may read and execute the detection program 208a.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1)
A storage procedure for storing, in the storage unit, position information indicating the position of the moving body and time information indicating the time when the moving body has existed at the position;
Based on the position information and the time information, a movement trajectory calculation procedure for calculating a movement trajectory of a plurality of moving bodies for each moving body;
A movement trajectory identification procedure for identifying a reference movement trajectory that indicates a movement trajectory that is longer than a length of other movement trajectories among a plurality of movement trajectories calculated by the movement trajectory calculation procedure;
A moving body detection program for executing a movement transition detection procedure for detecting movement transitions of a plurality of moving bodies by integrating other movement paths similar to the characteristics of the reference movement path.

(Supplementary Note 2) A position detection procedure for calculating the acceleration of the moving body based on the position information and the time information stored in the storage unit and detecting the position of the moving body when the acceleration decreases. The moving object detection program according to appendix 1, which is further executed by a computer.

(Additional remark 3) The overlapping position detection procedure which detects the position where the movement locus | trajectory of the several mobile body detected in the said movement transition detection procedure and the position of the mobile body detected in the said position detection procedure overlap is further performed to a computer The moving body detection program according to appendix 2, characterized in that:

(Additional remark 4) At every fixed time interval, the temporary moving direction of each moving body is calculated, respectively, and the straight line from the position of each moving body to the moving direction overlaps each other or from the position of each moving body. 4. The moving body detection program according to appendix 1, 2, or 3, further causing a computer to execute an overlapping area detection procedure for detecting an area in which straight lines in directions opposite to the moving direction overlap each other.

(Additional remark 5) The memory | storage part which matches and memorize | stores the positional information which shows the position of a mobile body, and the time information which shows the time when this mobile body existed in the said position,
Based on the position information and the time information, a movement trajectory calculation unit that calculates a movement trajectory of a plurality of moving bodies for each moving body;
Among the plurality of movement trajectories calculated by the movement trajectory calculation unit, a movement trajectory specifying unit that specifies a reference movement trajectory indicating a movement trajectory that is longer than the length of the other movement trajectories,
A moving body detection apparatus comprising: a movement transition detection unit that detects movement transitions of a plurality of moving bodies by integrating other movement paths similar to the characteristics of the reference movement path.

(Supplementary Note 6) A position detection unit that calculates the acceleration of the moving body based on the position information and the time information stored in the storage unit, and detects the position of the moving body when the acceleration decreases. The moving body detection device according to appendix 5, further comprising:

(Additional remark 7) It further provided the overlap position detection part which detects the position where the movement locus | trajectory of the several mobile body detected in the said movement transition detection part and the position of the mobile body detected in the said position detection part overlap. The moving body detection device according to appendix 6, characterized by:

(Additional remark 8) At every fixed time interval, the temporary moving direction of each moving body is calculated, respectively, and the straight line from the position of each moving body to the moving direction overlaps each other or from the position of each moving body. The moving body detection device according to appendix 5, 6 or 7, further comprising an overlapping area detection unit for detecting an area where straight lines in directions opposite to the moving direction overlap each other.

(Supplementary note 9) The mobile object detection device is
A storage step of associating the position information indicating the position of the moving object with the time information indicating the time when the moving object was present at the position;
Based on the position information and the time information, a movement trajectory calculating step for calculating a movement trajectory of a plurality of moving bodies for each moving body;
A movement trajectory specifying step for specifying a reference movement trajectory that indicates a movement trajectory that is longer than the length of the other movement trajectories among the plurality of movement trajectories calculated in the movement trajectory calculation step;
A moving body detection method comprising: a movement transition detection step of detecting movement transitions of a plurality of moving bodies by integrating other movement paths similar to the characteristics of the reference movement path.

(Additional remark 10) The position detection step which calculates the acceleration of a moving body based on the position information and the time information stored in the storage unit, and detects the position of the moving body when the acceleration is reduced. The moving object detection method according to appendix 9, further comprising:

(Additional remark 11) It further included the overlap position detection step which detects the position where the movement locus | trajectory of the some mobile body detected in the said movement transition detection step and the position of the mobile body detected in the said position detection step overlap. The moving body detection method according to appendix 10, characterized by:

(Additional remark 12) At every fixed time interval, the movement direction of each temporary moving body is calculated, respectively, and the straight line from the position of each moving body to the moving direction respectively overlaps or the position of each moving body 12. The moving object detection method according to appendix 9, 10 or 11, further comprising an overlapping area detecting step for detecting an area where straight lines in directions opposite to the moving direction overlap each other.

DESCRIPTION OF SYMBOLS 50 Moving body detection apparatus 51 Memory | storage part 52 Movement locus calculation part 53 Movement locus identification part 54 Movement transition detection part

Claims (6)

On the computer,
A storage procedure for storing, in the storage unit, position information indicating the position of the moving body and time information indicating the time when the moving body has existed at the position;
Based on the position information and the time information, a movement trajectory calculation procedure for calculating a movement trajectory of a plurality of moving bodies for each moving body;
A movement trajectory identification procedure for identifying a reference movement trajectory that indicates a movement trajectory that is longer than a length of other movement trajectories among a plurality of movement trajectories calculated by the movement trajectory calculation procedure;
A moving body detection program for executing a movement transition detection procedure for detecting movement transitions of a plurality of moving bodies by integrating other movement paths similar to the characteristics of the reference movement path.   Based on the position information and the time information stored in the storage unit, an acceleration of the moving body is calculated, and a position detection procedure for detecting the position of the moving body at the time when the acceleration decreases is further executed on the computer. The moving body detection program according to claim 1, wherein:   The computer further executes an overlapping position detection procedure for detecting a position where the movement trajectories of the plurality of moving bodies detected in the movement transition detection procedure overlap with the positions of the moving bodies detected in the position detection procedure. The moving body detection program according to claim 2.   At each fixed time interval, the temporary movement direction of each moving body is calculated, and the region where the straight line from the position of each moving body to the moving direction overlaps or the position of each moving body is the moving direction. 4. The moving object detection program according to claim 1, further comprising causing the computer to execute an overlapping area detection procedure for detecting an area where straight lines in opposite directions overlap each other. A storage unit that associates and stores position information indicating the position of the moving object and time information indicating the time when the moving object has existed at the position;
Based on the position information and the time information, a movement trajectory calculation unit that calculates a movement trajectory of a plurality of moving bodies for each moving body;
Among the plurality of movement trajectories calculated by the movement trajectory calculation unit, a movement trajectory specifying unit that specifies a reference movement trajectory indicating a movement trajectory that is longer than the length of the other movement trajectories,
A moving body detection apparatus comprising: a movement transition detection unit that detects movement transitions of a plurality of moving bodies by integrating other movement paths similar to the characteristics of the reference movement path.   The apparatus further includes a position detection unit that calculates the acceleration of the moving body based on the position information and the time information stored in the storage unit and detects the position of the moving body when the acceleration decreases. The moving body detection apparatus according to claim 5.

Images