JP7339485B2 - measurement system - Google Patents

Description

TECHNICAL FIELD The present invention relates to a measuring device and a measuring system for measuring a movement trajectory of a moving body.

Along with the development of sensor technology, next-generation systems based on human behavior data are attracting a lot of attention. In particular, the location information of people walking around indoor and outdoor spaces such as large-scale indoor and outdoor events, amusement parks and theme parks, offices and commercial facilities can be used for marketing research based on crowd movement history information, and for energy management systems such as smart homes and smart buildings. It is expected to be applied to many services such as personal navigation in theme parks, commercial facilities, museums and art galleries. GPS (Global Positioning System) is widely used as a positioning technology for mobile terminals such as smartphones, but it is impossible to receive radio waves from satellites underground or indoors, and people who do not carry smartphones. It is not possible to obtain the position or movement trajectory of For indoor use, methods using cameras, RFID (Radio Frequency IDentification) tags, and acceleration sensors and gyration sensors installed in mobile terminals have been proposed. There is a problem with points.

Pedestrian measurement using a laser ranging scanner (LIDAR: Light Detection and Ranging) is attracting attention. Since LIDAR can accurately measure the distance to surrounding objects, it is possible to scan a wide range at high speed (for example, the detection range of commercially available LIDAR is about 30 m and the scanning range is 270 degrees). It is possible. Since the LIDAR measurement data is only the position information of the measurement target represented by the azimuth angle and distance seen from the LIDAR, there is little risk of infringing on the privacy of the pedestrian who is the measurement target, and there is little resistance to being recorded. The feeling is also small.

Non-Patent Document 1 describes a system for pedestrian tracking in an environment using multiple LIDARs that detect around waist height. In this system, all point cloud data obtained from each LIDAR is first sent to the server. Thereafter, the server calculates the body center point of the pedestrian, and then groups the body center points obtained from each LIDAR to determine whether it is a single (identical) pedestrian. , for a single pedestrian, compute the centroids of their body center points to obtain the location of the same pedestrian. Finally, by connecting the body center points of the same pedestrian in time series, the movement trajectory of the pedestrian is estimated (tracked).

Takumi Takafuji, Kazuhisa Fujita, Yuta Higuchi, Satohito Hiromori, Hirozumi Yamaguchi, Teruo Higashino, Shinji Shimojo, Proposal of High Accuracy Indoor Position Estimation Method Using Tracking Scanner and Motion Sensor, Transactions of Information Processing Society of Japan Vol.57 No. .1 353-365 (Jan. 2016)

In the measuring device described in Non-Patent Document 1, all measurement data is received for each periodic measurement with each LIDAR, and the server calculates the body center point of each pedestrian with all LIDAR, and from them It is necessary to search for the same pedestrian, extract its body center point, and repeat the calculation process and extraction process to perform tracking processing that integrates the body center point in time series, which increases the data processing burden on the server. is large.

The present invention has been made in view of the above, and provides a measurement system equipped with a measurement device that detects and tracks a moving object such as a pedestrian for each sensor, and a server that integrates the tracking sensors. is.

A measuring device according to the present invention comprises a sensor that periodically transmits a laser beam while repeatedly scanning a scanning range, and receives a laser beam reflected by each part of a moving object within a detection area; a first calculation means for calculating the position of a moving object point group corresponding to each part of the moving object, extracting an elliptical body from the calculated moving object point group, and sequentially calculating the position of the center of gravity of the extracted elliptical body; second calculating means for determining whether a distance between two of the calculated center-of-gravity positions of the torsos is equal to or less than a threshold, and assigning a common ID to the torsos determined to be equal to or less than the threshold; and movement trajectory generating means for generating a movement trajectory by connecting the positions of the center of gravity of the trunk to which the common ID is assigned in time series.

According to the present invention, a laser beam is periodically transmitted from the sensor while repeatedly scanning the scanning range, and the laser beam is received by being reflected by each part of a moving object existing within the detection area. Then, the position of the moving object point group corresponding to each part of the moving object is calculated from the received laser beam by the first calculating means, the elliptical body is extracted from the calculated moving object point group, and further extracted. The position of the center of gravity of the elliptical body is sequentially calculated. Further, the second calculating means determines whether or not the distance between the center-of-gravity positions of the two torsos is equal to or less than a threshold, and assigns a common ID to the torso determined to be equal to or less than the threshold. be done. Then, the position of the center of gravity of the trunk to which the common ID is assigned is connected in time series by the movement trajectory generating means. Therefore, the measuring device calculates movement track information from the position information acquired from its own sensor, thereby reducing the data processing load on the server side. Also, when the distance between the center of gravity positions of the two torsos is determined to be less than the threshold, it is determined that they are the same moving object, so that even if it temporarily stands still (stops), it is properly detected as the same moving object. .

Further, the measurement system according to the present invention includes the measurement devices arranged so that the detection ranges partially overlap, and a server for acquiring information on each of the movement trajectories generated by each of the measurement devices, wherein the server is provided with movement trajectory integration means for extracting pairs of movement trajectories between the measuring devices and performing matching processing of the movement trajectories for each pair.

According to the present invention, the movement trajectory integration means of the server extracts pairs of movement trajectories between the measuring devices, and only performs matching processing of the movement trajectories for each pair. is reduced.

According to the present invention, it is possible to provide a measuring device on the sensor side that serves to reduce the data processing load on the server, and to provide a measuring system composed of a server to which a plurality of such measuring devices are connected.

It is a figure explaining the laser scanning operation|movement by a sensor, (A) is a top view, (B) is a front view, (C) is a top view which shows the example of arrangement|positioning of a detection area and a sensor. It is a figure explaining the method of the person detection by a sensor, (A) is a top view explaining the detection of the human body surface by a laser scan, (B) is a figure of the planar view explaining the person determination method. 1 is an overall schematic diagram showing one embodiment of a measurement system according to the present invention; FIG. 1 is a functional configuration diagram showing an embodiment of a measuring device including a sensor; FIG. It is a functional block diagram which shows one Embodiment of a server. It is a figure which shows an example of the calibrator for calibration of a sensor, (A) is a structure drawing, (B) is a figure of planar view for description which calculates the position and direction of a calibrator. FIG. 4 is a diagram for explaining the procedure of calibration processing using a common calibrator when two sensors are arranged, (A) is a diagram showing the positional relationship between the first sensor and the calibrator; FIG. 11C is a diagram showing the positional relationship between the second sensor and the calibrator, and (C) is a diagram explaining a method of synthesizing the results of (A) and (B) to calibrate the positions of the two sensors. 4 is a flow chart showing an example of a procedure of sensor processing executed by a processor of the measuring device; 3 is a flowchart showing an example of a procedure of merge processing executed by a processor of a server, (A) showing the procedure of merge processing I, and (B) showing the procedure of merge processing II. 4 is a flow chart showing an example of a procedure of calibration processing executed by a processor of the measuring device; 4 is a flowchart showing an example of a procedure of processing for calculating absolute position information of each sensor, which is executed by a processor of a server; FIG. 10 is a plan view illustrating a method of detecting entry and exit associated with movement of a person within a detection area; 4 is a flow chart showing an example of the procedure of people counting processing executed by the processor of the server; FIG. 10 illustrates an embodiment of an arrangement of sensors that enables detection including attributes of a person; Flowchart showing data transmission processing performed by a mobile terminal to be carried, (A) showing installation processing of a combination application in a mobile terminal, a flowchart of data transmission processing I to a server, and (B) a mobile terminal after combination 10 is a flowchart of data transmission processing I to the server, showing an example of utilization processing of data received from. FIG. 10 is a flowchart showing an example of a process of combining a movement of a portable mobile terminal and a movement trajectory, which is executed by a server; FIG. It is a figure of the example of a screen displayed on the display part of a mobile terminal. FIG. 10 is a diagram showing an application example of counting the number of passengers ascending and descending for transportation.

FIGS. 1A and 1B are diagrams for explaining the laser scanning operation by the sensor, in which (A) is a plan view, (B) is a front view, and (C) is a plan view showing an example of arrangement of detection areas and sensors. As shown in FIG. 3, the measuring system 1 according to the present invention includes at least a measuring device 10 having a sensor 11 and a server 20 for executing various data processing using data transmitted from each measuring device 10. ing.

As the sensor 11 employed in the present invention, this embodiment employs a laser ranging scanner (LIDAR: Light Detection and Ranging). The sensor 11 is cylindrical and has a predetermined height, for example, about 120 cm. The sensor 11 horizontally transmits pulsed laser beams b1, . . . bj, b(j+1) . The sensor 11 includes a transmitter 111 (see FIG. 4) that periodically transmits a laser beam b while scanning in the circumferential direction, and a receiver 112 (see FIG. 4) that receives laser reflected pulses from obstacles. configured with. As the sensor 11, one having general performance is adopted. For example, the detection distance is 30 m, the scanning range is 270 degrees, the resolution is 0.25 degrees, the scanning speed is 25 ms, and the range measurement error is ±5 cm. small. A power source such as a secondary battery (not shown) is mounted, preferably built-in, on the sensor 11 to facilitate placement at an appropriate position. By setting the height of the sensor 11 to about 120 cm, when the laser beam b is radiated in the horizontal direction, it is possible to detect the waist area of a general person, making it possible to identify the person through the shape of the torso. ing. Note that the irradiation direction of the laser beam b from the sensor 11 is not limited to the horizontal plane, and a three-dimensional mode in which the laser beam b is emitted with an inclination can be adopted according to the application and the object to be detected.

As shown in FIG. 1C, a plurality of sensors 11 are installed in, for example, a venue 50, which is a detection area. The venue 50 may be circular, rectangular, or of various other shapes, such as an uneven area, and is surrounded by a wall 51 . A part of the venue 50 has a doorway 52 . A required number of sensors 11 are arranged at appropriate positions in consideration of the size and shape of the hall, the number of people to be targeted, the density, and the like. The plurality of sensors 11 are preferably arranged at such positions and orientations that a plurality of sensors can partially overlap and measure the same area.

2A and 2B are diagrams for explaining the method of human detection by the sensor 11. FIG. 2A is a plan view for explaining the detection of the surface of the human body by laser scanning, and FIG. 2B is a plan view for explaining the method for human determination. be. In FIG. 2(A), the laser beam b from the sensor 11 is transmitted in a certain detection direction to the reflection point (this is called the detection point p) on the surface of the nearest object (pedestrian 6) existing in that direction. is accurately detected based on the radio wave propagation time from the time of transmission to the time of reception. A set of produced detection points p is hereinafter referred to as point cloud data.

Therefore, each sensor 11 measures the distance d and the angle θ and as a set of three-dimensional vectors. That is, when the detection distance in the unit direction vector v at the time t of the i-th sensor 11 is d, the detection point information (i, t, v, d) is output to the measuring device 10 . FIG. 2A illustrates such a relationship in two-dimensional space. Note that FIG. 2B is for explaining the conditions for detecting a pedestrian, and detailed explanation will be given later.

FIG. 3 is an overall schematic diagram showing one embodiment of the measurement system according to the present invention. The measurement system 1 includes a plurality of measurement devices 10, a server 20, and a mobile terminal 30 employed as necessary. The measuring device 10 includes a sensor 11 that acquires point cloud data by scanning a laser beam b, and a controller 120 that performs certain data processing on the point cloud data detected by the sensor 11 . The server 20 includes a processing unit 21 that performs data processing, and a database 22 that temporarily stores data necessary for processing, transmission data from each measuring device 10, and data in process. The server 20 is connected to an access point 201 for performing wireless LAN communication (for example, Wi-Fi) with the mobile terminal 30, a signage display 202, and the Internet 40. FIG. The ground station 41 transmits and receives e-mail data including SNS (Social Net System) between the Internet 40 and the mobile terminal 30 .

The mobile terminal 30 is typically a smart phone, and has a processor for processing data inside the main body 31 and a display unit 311 on the surface. In addition, the body 31 generally incorporates a motion sensor 32 such as a GPS (Global Positioning System) receiver, a gyro sensor, an acceleration sensor, etc., and further includes a temperature sensor, a humidity sensor, an atmospheric pressure sensor, and other environmental sensors. ing. Among the motion sensors 32, the gyro sensor and the acceleration sensor are capable of relatively detecting movement of the pedestrian carrying the mobile terminal 30, such as movement, direction change, and jumping according to walking. The communication unit 33 performs data communication with the access point 201 and the ground station 41 on condition that the "HitoNavi" application described in FIG. 15A is installed. The input unit 34 includes a group of mechanical keys or character keys composed of button images, and in addition to various instructions, composes a text message for e-mail or the like through input operations.

FIG. 4 is a functional configuration diagram showing an embodiment of the measuring device 10 including the sensor 11. As shown in FIG. The measuring device 10 has a sensor 11 and a measuring section 12 . The measurement unit 12 may be installed inside the support body of the sensor 11 . The measurement unit 12 includes a control unit 120 having a processor, and the sensor 11, program storage unit 1201 and data storage unit 1202 are connected to the control unit 120. FIG. The program storage unit 1201 stores processing programs executed by the control unit 120 . The data storage unit 1202 temporarily stores measurement data received by the sensor 11, and temporarily stores data during processing.

By executing the processing program, the control unit 120 functions as a sensor drive unit 121, a measurement data acquisition unit 122, a calibration unit 123, a background processing unit 124, a human detection unit 125, a movement trajectory generation unit 126, and a communication processing unit 127. Function.

The sensor driving unit 121 receives an activation instruction to the sensor 11, drives the transmitting unit 111 and the receiving unit 112 of the sensor 11, and emits a laser beam b as shown in FIGS. It periodically transmits while scanning, and receives the reflected laser beam.

As shown in FIG. 1A, the measurement data acquisition unit 122 detects the distance to an obstacle by repeatedly transmitting the laser beam b while scanning it in the circumferential direction. Measured data relating to detection direction and distance is acquired in processing, human detection to moving trajectory generation processing, and calibration processing.

The calibration unit 123 is executed as an initial process, and is a pre-process in the process of acquiring accurate absolute position information of the plurality of sensors 11 installed in the detection area, that is, the calibrator 8 ( 6) is executed.

The background processing unit 124 is executed as a preliminary process, and detects the wall 51 of the hall 50 shown in FIG. 1(C). When the detection distance is the same for a predetermined long time in the same detection direction, the detection point p is regarded as a stationary body that does not move, for example, the wall 51, and the point cloud data thereof is used as the background point cloud. Register in advance. Note that the stationary body can include installation objects and pillars installed in the venue 50 .

The human detection unit 125 identifies the body of the pedestrian and detects its position from the point cloud data. The human detection unit 125 performs (I) derivation of a moving object point group based on the difference of the background point clouds, (II) detection of a human body from the moving object point group, and (III) estimation of the same person from the preceding and succeeding frames. I do. The point cloud data may use the received signal detected by the sensor 11 each time, but in consideration of noise suppression, processing speed, etc., the received signal of multiple times (for multiple scans) is averaged. It can be a thing. For example, it may be processed with point cloud data obtained by averaging received signals for eight consecutive scans from the same direction, that is, with a period of 200 ms (=Δt: 25 ms×8).

(I) Derivation of moving object point clouds by subtraction of background point clouds For each time t and a sufficiently short time Δt (for example, 200 ms), for the moving object point clouds detected at [t-Δt, t], their vertical coordinates XY coordinates are obtained by removing (Z coordinates). On the other hand, the point group acquired by the background processing unit 124 and regarded as the registered background points is compared with the point group from which the Z coordinate is deleted, and the point group whose XY coordinates are regarded as matching is deleted. A point group remaining after this deletion process is called a moving object point group.

(II) Detection of Human Body from Moving Object Point Group The human detection unit 125 first extracts an ellipse from the moving object point group. That is, assuming that the pedestrian is a cylinder, as shown in FIG. Group as candidates for (torso).

Next, the human detection unit 125 performs filtering according to the size of the ellipse. That is, from the moving object point cloud, as shown in FIG. 2(B), (a) a constant α or less with the maximum distance within the group, (b) a constant β or more with the minimum distance from the detection point outside the group, (c) Extract a group of points satisfying a certain constant γ or more in the number of detected points in the group. Note that FIG. 2B is a simplified diagram for explanation, and illustrates the group 61 and the group 62 as separate groups adjacent to each other. If the above conditions are not met, so-called filtering processing is performed, such as excluding it from detection targets. Also, an elliptical shape is calculated from the moving object point cloud data in the group, and its size, for example, the elliptical diameter (either side of the long or short axis) is calculated. When the calculated ellipse diameter exceeds a preset maximum human body diameter R (generally R=60 cm), the human detection unit 125 does not determine that the human body is a human and excludes it from the detection target. Then, a so-called filtering process is performed.

In filtering, the following contents are reflected in the conditions. That is, when the maximum diameter of the human body is R (generally about R=60 cm) and the general walking speed of a person is V, α=R+V*Δt. In addition, each person has a personal space (approximately 30 cm), and walks while maintaining a certain distance from surrounding people for ease of walking. If the distance is S (S is empirically about 50 cm), β=S (see FIG. 2B). Also, the number of detection points for a person depends on the distance between the sensor 11 and the person (since the sensor 11 performs radial detection, the closer the distance to the person is, the more detection points are). In general, if a person present at a distance d from the sensor 11 with a detection step angle θ is regarded as a cylinder with a diameter R, the number of detection points for that person can be obtained by |2*arc cos (d+R)/θ|. Assuming that the maximum error in the detected distance caused by the positional coordinate error of the sensor 11 and the distance measurement error is ε, α=R+V*Δt+2ε and β=S−2ε.

Next, the human detection unit 125 performs a process of using the elliptical shape detected by the filtering as a human body. That is, the coordinates of the center of gravity of the body of the person in each group are calculated at each time [t-Δt, t] to obtain the person coordinate point. The human coordinate point may be calculated by a method of calculating the center of gravity of the ellipse (body center point). For example, it is geometrically calculated from the detection direction in which the detection distance d from the sensor 11 is the minimum and the detection direction at both left and right ends. method can be adopted.

(III) Estimating the Same Person from Previous and Next Frames s) is used to determine if there are two person coordinate points at a distance less than or equal to V*Δt. Then, when the determination is affirmative, the person detection unit 125 regards them as a single person moving by walking, and assigns the same person ID to those person coordinate points. In addition, if the human detection unit 125 stores information on human coordinate points for a plurality of periods (frames) in the time series, it is possible to prevent another person from blocking the sensor 11 during that time. Even if the measurement is temporarily interrupted for some reason such as passing, the interrupted human coordinate points can be generated using a general interpolation method by using the stored information.

The movement trajectory generation unit 126 derives a movement trajectory for each person ID by connecting the person coordinate points acquired by the person detection unit 125 in chronological order for each person ID. Various data regarding the coordinate positions calculated by the control unit 120 are represented by relative coordinates with the sensor 11 as a reference.

The communication processing unit 127 exchanges data with the server 20, and in this embodiment, information obtained by the calibration unit 123 and the moving trajectory generation unit 126 is transmitted to the server.

FIG. 5 is a functional configuration diagram showing an embodiment of the server 20. As shown in FIG. The server 20 includes a processing unit 21 having a processor, and the processing unit 21 is connected with a database 22, a program storage unit 23, and a display device 202. FIG. The database 22 stores various data acquired by the measuring device 10 and temporarily stores data during processing. The program storage unit 23 stores processing programs executed by the processing unit 21 .

By executing the processing program, the processing unit 21 performs a movement trajectory integration unit 211, a calibration unit 212, a combination processing unit 213, a signage image creation unit 214, a mobile terminal image creation unit 215, a communication processing unit 216, and a movement monitoring unit. 217.

The movement trajectory integration unit 211 performs a process of merging the movement trajectory data transmitted from each measuring device 10 for each single person. The movement trajectory data from each measuring device 10 may be slightly deviated even for the same person. There is a possibility that the movement trajectory of each person will be recognized as the movement trajectory of the same person. Therefore, the movement trajectory integration unit 211 determines whether or not the movement trajectory between the measuring devices 10 belongs to the same person, and performs integration processing based on the result.

Here, the integration (merging) process includes (i) obtaining movement trajectory data of a person detected by each measuring device 10, (ii) transforming the movement trajectory data of a person into absolute coordinates, and (iii) each trajectory pair. , the degree of matching is calculated, and (iv) if it is equal to or greater than the threshold value, it is determined that the trajectory is that of the same person.

More specifically, after the process (i), the movement trajectory integration unit 211 acquires the movement trajectory data obtained by each measuring device 10 and considers the XY coordinates of the sensor 11 as in (ii). to convert to the absolute coordinate system (XY plane). Next, the position information and time information of each measuring device 10 are converted into absolute coordinates and a common time reference to standardize the coordinate system. Further, the moving trajectory integration unit 211 calculates the degree of matching for each trajectory pair as in (iii). That is, the movement trajectory integration unit 211 sequentially selects pairs of movement trajectories to be compared in the absolute coordinate system, and for each pair, within a certain time range, calculates the difference (distance) of the position information between the pairs at the same time. are counted, and the degree of matching for the counted result is obtained. In (iv), the degree of matching is compared with a threshold. and integrate. The integration is performed, for example, by deleting one of the trajectory information. On the other hand, if the degree of matching is less than the threshold, it is determined that the two trajectories belong to different persons.

The calibration unit 212 is a process for matching accurate relative positions between the two sensors 11 . A plurality of sensors 11 are preferably arranged in the detection area so that the detection ranges partially overlap each other. In this case, when determining whether the movement trajectory detected by each sensor 11 is that of the same person or that of a different person, the position within the detection area of each sensor 11 and, if necessary, the same direction that is common to the direction. Must be set in a coordinate system.

The calibration unit 212 performs calibration based on the position information of the calibrator 8 acquired by the calibration unit 123 in each measuring device 10 .

Conventionally, the relative positional relationship was grasped by directly matching the background points detected by each sensor 11 manually or mechanically. There is a problem that a common object (for example, a wall or a pillar) cannot always be captured depending on the positional relationship. Therefore, in this embodiment, as shown in FIG. 6, a method of accurately specifying the position using a calibrator 8 having two poles 81 and 82 having a specific size and positional relationship is adopted.

6A and 6B are diagrams showing an example of the calibrator 8, where (A) is a structural diagram, and (B) is a plan view for explaining calculation of the position and orientation of the calibrator. The calibrator 8 has two long columnar poles 81 and 82 erected in a predetermined positional relationship on the upper surface of a flat plate-shaped base 80, and can be moved integrally to any position and installed. is. The poles 81 and 82 reflect the laser. The poles 81 and 82 function as two types of reflectors, and differ in shape, here radius, from each other so as to be identifiable. It should be noted that the calibrator 8 is not limited to a cylindrical shape, and various shapes can be adopted, and it is not necessary to be divided into two, and it is sufficient if it has shape parts that can be distinguished from each other.

First, pre-processing is performed by the calibration unit 123 (see FIG. 4) of the measurement device 10 . That is, after obtaining the background point group as a preliminary preparation, with the sensor 11 installed at an appropriate place in the venue 50, measurement is performed in the same manner as the moving object detection, and the background point group is removed, for example, FIG. We obtain a moving object point cloud such as

Next, the positional information of the calibrator 8 is detected from the moving object point group using the same processing as the body detection method described above. First, using the diameter information of the poles 81 and 82, the point group of the D1 circle (181) and the point group of the D2 circle (182) are detected from the moving object point group. Furthermore, for each set of point cloud set assumed to correspond to the D1 circle and point cloud set assumed to correspond to the D2 circle, their center-to-center distance L is equal to the center-to-center distance of the poles 81-82 of the calibrator 8. If so, consider them D1-circle, D2-circle. Furthermore, using the position information of the D1 and D2 circles, the distance between the middle position Pc of the D1 and D2 circles and the sensor 11, and the angle (vector) Di of the direction connecting the center of the D2 circle from the sensor 11 Calculate The calibration unit 123 then transmits the calculated distance and angle (vector) Di to the server 20 .

Here, the calibration processing by the two sensors 11 at the venue 50 will be described with reference to FIG. FIG. 7 is a diagram for explaining the procedure of calibration processing using a common calibrator 8 when two sensors 11 and 11' are arranged. (B) is a diagram showing the positional relationship between the second sensor 11' and the calibrator 8. FIG.

FIG. 7C shows processing executed by the calibration unit 212 of the server 20, in which the results of (A) and (B) are combined to calibrate the positions of the two sensors 11 and 11'.

The calibration unit 212 calculates the relative position and absolute position of the sensor 11 . First, the calibration unit 212 acquires the position information of the calibrator 8 detected by each sensor 11 at a certain time, and then, starting from the position of the calibrator 8, using the distance and angle information from the calibrator 8, a plurality of specifies the relative position of the sensor 11 (see FIG. 7(C)).

Next, the calibration operator inputs and sets the positional information of the absolute coordinates of any one of the sensors 11 through an input operation unit (not shown). The absolute coordinate system may be world coordinates or local coordinates that commonly cover the entire detection area. The calibration unit 212 calculates the remaining absolute position information of the sensor 11' based on the absolute position information of the sensor 11 whose absolute coordinates are set and the relative positional relationship of the other sensors 11'.

FIG. 8 is a flow chart showing an example of the procedure of sensor processing executed by the processor of the measuring device 10. As shown in FIG. In FIG. 8, first, as a preliminary process, a background point group acquisition process from a stationary object is executed (step S1). Next, the measuring device 10 is activated, and periodic measurement processing is started while scanning the laser beam b in the circumferential direction (step S3). Also in the measurement process, stationary objects are detected together with pedestrians. Therefore, the background point group is removed from the point group data obtained in the measurement process, and the moving object point group is detected (step S5).

Next, grouping processing is performed from the moving object point cloud to extract an ellipse for each group (step S7), and filtering processing is performed to determine whether or not the ellipse is the size of a human (step S9). Note that if it is determined that the size of the ellipse is not the size of a human ellipse, the group is excluded from detection targets. Subsequently, a human torso detection process, that is, a process of calculating the barycentric coordinates of the human torso at each time [t-Δt, t] and detecting the human coordinate points is executed (step S11). Further, the same person is estimated between the previous and subsequent frames from the positional relationship of the human coordinate points detected in the previous (t−Δt) frame and the current frame t, and a person ID is assigned (step S13). . If it is the same person, a movement trajectory is derived, that is, processing is performed to connect the human coordinate points between frames in time series (step S15), and the derivation result is transmitted to the server 20 together with the person ID (step S17). . Next, it is determined whether or not the measurement should be continued (step S19). If the measurement is continued, the process returns to step S3 and the same processing is repeated. If not, the flow ends.

FIG. 9 is a flowchart showing an example of the procedure of merge processing executed by the processor of the server, (A) showing the procedure of merge processing I, and (B) showing the procedure of merge processing II. The merge process I is a process of receiving transmission data from the measuring device 10 and the mobile terminal 30 as will be described later. First, transmission data is received from each measuring device 10 (step #1), and then transmission data is received from the mobile terminal 30 (step #3).

Next, in the merge process II, first, movement trajectory data of each person detected by each sensor 11 within a certain time range is extracted (step #11), and the extracted movement trajectory data of each person is converted to absolute coordinates. It is converted (step #13).

Next, each pair is sequentially extracted from the combination of pairs of movement trajectories, and the degree of matching is calculated based on the difference in position for each detection time between the extracted pairs (step #15). Then, it is determined whether or not the calculated degree of matching for the extracted pair is equal to or greater than a threshold value (step #17). are integrated (step #19). On the other hand, if there is no matching, step #19 is skipped and the trajectory is treated as another person's movement trajectory. Next, the presence or absence of the next pair is determined (step #21), and if the next pair remains, the process returns to step #15, the pair is extracted from the remaining pairs, and similar matching processing is executed. be. When the last pair ends in step #21, this flow ends.

FIG. 10 is a flow chart showing an example of a procedure of calibration processing executed by the processor of the measuring device 10. As shown in FIG. First, background point group acquisition processing is executed as preprocessing (step S31). Next, the calibrator 8 is placed in a detection area, for example, at an appropriate location in the venue 50 (see FIG. 7A), and in that state, the measuring device 10 is activated to periodically scan the laser beam b in the circumferential direction. Measurement processing is started (step S33). A stationary object is also detected together with the calibrator 8 in the measurement process. Therefore, the background point group is removed from the point group data obtained in the measurement process, and the moving object point group for the poles 81 and 82 of the calibrator 8 is extracted (step S35).

Next, positional information of the centers of D1 and D2 circles corresponding to the poles 81 and 82 of the calibrator 8 is calculated from the moving object point group (step S37). For example, calculation of the position information for the poles 81 and 82 is performed as shown in FIG. 6(B). In this way, using the position information of the D1 and D2 circles, the difference between the center distance L of the D1 and D2 circles and the center distance between the poles 81 and 82 is calculated. , the calculated position information is regarded as corresponding to the poles 181 and 182 of the calibrator 8 (step S39).

Next, the distance between the intermediate position Pc of the D1 and D2 circles and the sensor 11 and the angle (vector) Di in the direction connecting the center of the D2 circle from the sensor 11 are calculated (step S41). It is transmitted to the server 20 (step S43).

FIG. 11 is a flow chart showing an example of a procedure of processing for calculating the absolute position information of each sensor 11, which is executed by the processor of the server 20. As shown in FIG. First, a calculation result (step S41) is received from each measuring device 10 (step #31).

Next, read out the calculation results detected by each measuring device 10 at a certain time, and from these calculation results, two (or a plurality of) calibrators 8, . 11 relative positional relationship is calculated (step #33). Next, the absolute position information of one sensor 11 in the absolute coordinate system is manually set in the server 20 (or input setting via the measuring device 10 including the sensor 11 may be set) ( step #35).

Next, the calibration unit 212 calculates (converts) the absolute position information of the remaining sensors 11 from the relative positional relationship between the absolute position information of the sensors 11 for which the absolute position information has been set and the other sensors 11 (step #37). The calculated absolute position information of each sensor 11 is stored in the database 22 and transmitted to each measuring device 10 as necessary (step #39). As a result, the position of each sensor 11 is defined by common coordinate information, which facilitates processing. Note that the absolute position information of each sensor 11 may include direction information.

FIG. 12 is a plan view illustrating a method of detecting passage through a certain area, for example, entering/exiting, as people move within the venue 50, which is the detection area. In FIG. 12, a venue 50 has sensors 11 installed at the upper left and lower right of the figure, and a doorway 52 provided near the center of a wall 51 on the upper part of the figure. FIG. 12 illustrates a state in which Tr1 and Tr2 are detected as human movement trajectories. The movement monitoring unit 217 detects whether or not the integrated movement trajectory of a person passes through a certain area, for example, crosses a doorway, and further manages the number of persons.

A trajectory Tr1 indicates a mode of movement from left to right (t4, t3, t2, t1) near the center of the hall 50. FIG. Trajectory Tr2 detects the state (t4, t3, t2) of passing through the area of doorway 52 from hall 50 . Since it has moved outside the detection area at time t1, it cannot be detected after time t2. In this example, the pedestrian on the trajectory Tr2 can be detected at (t4, t3, t2) and has passed through (crossed) the entrance 52 between time t2 and time t1, so it is assumed that the pedestrian is exiting. can be found, and the number of people should be minus 1. On the other hand, if it is assumed that time passes in the reverse direction of the trajectory Tr2, it can be detected that the person has entered through (crossed) the doorway 52 from the outside, and the number of people can be increased by one. If the initial number of people in the hall is known, it is possible to monitor the number of people in the hall by counting the number of people. The sensor 11 detects the entrance/exit 52 as an entrance/exit area (passage area) that is simply formed in a straight line, thereby monitoring pedestrians moving to intersect (traverse) a specific area in the venue 50. .

FIG. 13 is a flow chart showing an example of the procedure of the people counting process executed by the processor of the server 20. As shown in FIG. First, the initial number i is input (step #51). If the initial number of people is 0, no particular input is required. Next, measurement is performed, a movement trajectory is generated, and monitoring processing is performed (step #53). In the monitoring process, it is determined whether or not the movement locus has crossed the entrance/exit 52 (step #55). If the doorway 52 is traversed "inside to outside" (Yes at step #57), the number of people is subtracted as i=i-1 (step #59) and vice versa. If not (i.e., crossing from "outside to inside"), the number of people is added as i=i+1 (step #61), and then the process returns to step #53 to repeat the same process.

FIG. 14 is a diagram showing an embodiment of the arrangement of sensors 11 that enable detection including human attributes. In this example, in addition to one or a plurality of sensors 11 installed on the floor of the hall 50 and having the detection surface in the horizontal direction, another sensor 11' is installed at a higher position on the wall 51, for example. there is The sensor 11' has a downwardly slanted detection surface to enable three-dimensional detection. It should be noted that the sensor 11' may have a shape having a longer pillar, and the detection surface may be tilted downward. According to this configuration, the sensor 11 enables the identification of the person and the detection of the movement trajectory through the detection of the torso, while the sensor 11' detects the head (highest height position) of the moving person. , it becomes possible to detect an attribute of height, which can be used, for example, to distinguish between an adult and a child. In addition, by using the sensor 11', it is possible to detect various parts of the body in the vertical direction as the person walks, so that various attributes (feature information) relating to the human body can be acquired, and the sensor 11' can be applied to appropriate applications.

As described above, by applying the measurement system 1, it is possible to acquire the accurate two-dimensional positions of people in the detection area. On the other hand, assuming that these people carry the mobile terminal 30, the combination processing unit 213 uses the motion sensor 32 built into the mobile terminal 30 to acquire the movements of those people, The trajectory of movement and the mobile terminal 30 are combined (an ID for linking is set). Furthermore, by visualizing the combination results, it becomes possible to expand the scope of use and obtain new added value.

FIG. 15 is a flowchart showing data transmission processing performed by the mobile terminal 30 to be carried, (A) is a flowchart of data transmission processing I to the server showing installation processing of the combination application in the mobile terminal 30, and (B). 4 is a flowchart of data transmission processing II to the server, showing an example of processing for utilizing data received from the mobile terminal 30 after combination.

In FIG. 15A, first, prior to entering the venue 50, for example, a QR code (registered trademark), which is a two-dimensional bar code installed near the doorway 52, is imaged by the digital camera of the mobile terminal 30, The processing unit in the terminal analyzes the QR code (registered trademark), accesses the corresponding URL (Uniform Resource Locator) site via the ground station 41 and the Internet 40, and installs the "Hitonavi" application. (step ST1). Alternatively, the "Hitonavi" application may be installed in advance from a specific site on the Internet or by communicating with the access point 201 placed at the doorway 52. FIG. When the "HitoNavi" application is installed, the address of the user's own mobile terminal 30 is registered in the server 20, and access from the server 20 side is enabled thereafter.

The "HitoNavi" application continuously reads detection information from the motion sensor 32 built into the mobile terminal 30 and transmits the information to the server 20 via short-range communication such as Wi-Fi (step ST3).

In FIG. 15B, first, it is determined whether or not a message has been input from the input unit 34 (step ST11). If there is a message input, the "HitoNavi" application transmits the message to the server 20 (step ST13), and if there is no message input, step ST13 is skipped.

Furthermore, the "hitonavi" application determines whether or not a message has been received (step ST15). If a message has been received, the received message is displayed in balloon graphics on the associated mobile terminal 30 in the bird's-eye area face map (step ST17), and if no message has been received, step ST17 is skipped. .

FIG. 16 is a flowchart showing an example of a process of combining the movement of the portable mobile terminal 30 and the movement trajectory, which is executed by the combination processing unit 213 of the server 20 . Motion information is periodically received from the mobile terminal 30 (step #71), and then commonality between the movement trajectory and the motion information is evaluated (step #73). The combination processing for commonality evaluation evaluates the match at the same time between the motion obtained from the motion sensor 32 and the movement trajectory of the person detected within the detection area, and evaluates whether it can be said that the person is the same person. It is something to do. A determination is made as to whether or not there is commonality (step #75), and if there is no commonality, this flow ends. On the other hand, if there is commonality, a linking process is executed (step #77).

Next, it is determined whether or not SNS information has been received from the associated mobile terminal 30 (step #79). If the SNS information is not received, this flow ends. On the other hand, when the SNS information is received, the information received from the mobile terminal 30 is processed by the signage image creating unit 214 and displayed on the signage display 202 in association with the person's image (step #81). Next, the information displayed on the signage display 202 is transmitted to the associated mobile terminals 30 (step #83).

FIG. 17 is a diagram of an example of a screen displayed on the display unit 311 of the mobile terminal 30 by the mobile terminal image creation unit 215. As shown in FIG. A venue image G50 is displayed in a bird's-eye view on the screen. Furthermore, information (temperature, humidity, noise, air pressure, surrounding congestion, etc.) obtained from other sensors etc. built in the mobile terminal 30 is linked with the ID and sent via the access point 201 to tweets ( "Mobile terminal data" in which time stamps are attached to information that the owner of the mobile terminal 30 is interested in or reports, such as tweets, etc., is transmitted to the server 20 via the ground station 41 and reflected on the screen. be done.

On the screen of the display device 202 for signage or the screen of the display unit 311 of the mobile terminal 30, a person mark such as an avatar image (see image G6 in FIG. 17) is displayed corresponding to the position of each person, and "Mobile terminal data" is displayed using a balloon (see image G71 in FIG. 17). As a result, the correct location information of each mobile terminal 30 and the "mobile terminal data" captured by the mobile terminal 30 can be collectively displayed on the venue image G50 displayed on those screens. Further, as shown in FIG. 17, an image with a mark (see image G72) representing the person in question may be attached to the display unit 311 of the mobile terminal 30 of the person in question. In addition, the movement trajectory of each person may be displayed on the display device 202 and the display unit 311 for a certain period of time.

FIG. 18 is a diagram showing an application example of counting the number of passengers ascending/descending from a transportation system, which is executed by the movement monitoring unit 217. In FIG. This configuration is similar to the mode of counting the number of people entering and exiting the venue 50 shown in FIG. That is, FIG. 18 shows a mode of counting the number of passengers on a bus. In the bus, a seat SEAT is appropriately installed along a side portion B51 of the vehicle interior BU indicating the vicinity of the hatch on the rear side, and a door B52 is provided at the hatch so that it can be opened and closed. The door opening/closing sensor 9 is, for example, a mechanical switch that detects opening/closing of a door B52 that slides as indicated by an arrow in the figure by a drive source (not shown). Preferably, a plurality of sensors 11 in which the measurement unit 12 is installed are installed, and one may be arranged near the door B52.

The server 20' has the configuration shown in FIG. 3 and executes the same processing as the flowchart shown in FIG. This makes it possible to detect the number of people ascending and descending with respect to the BU inside the vehicle, thereby making it possible to grasp the number of passengers at each point in time. In addition, by associating the opening/closing time and the opening time of the door B52 at each stop detected by the door opening/closing detection unit 29 of the server 20' with the number of people ascending/descending, it is possible to grasp or manage the operation status.

In addition, in the present invention, the following method can also be adopted as a combination method with a mobile terminal. That is, the sensor 11 detects the position of a pedestrian who has entered the detection area, for example, the hall 50, and tracks the pedestrian. The server 20 issues a location information inquiry request to the mobile terminal 30 via the access point 201 on condition that the "Hitonavi" application is installed. If the pedestrian who has entered is carrying a mobile terminal 30, it returns its position information to the server 20 via the access point 201 in response to a request. The movement trajectory and time of the pedestrian who has entered are compared with the returned position information and time, and if they can be regarded as the same position, it is determined that the mobile terminal 30 of the pedestrian has entered. Therefore, while the pedestrian is in the detection area, the mobile terminal 30 can acquire an accurate locus of movement obtained by the sensor 11 via the server 20 . As a method for acquiring the position of the mobile terminal 30, a method using differences in radio wave intensity from Wi-Fi base stations, a beacon positioning (BLE: Bluetooth Low Energy) method, and others can be adopted.

In addition, in this embodiment, as the structure of the calibrator 8, as shown in FIG. A mode in which one pole 81 is employed as a separate body may be employed. In this embodiment, the positions on the base 80 where the poles 81 are to be erected may be set in advance by marking a plurality of positions, for example, two positions. Then, in the calibration process, the poles 81 are sequentially replaced at the first point and the second point, and the measurement is performed respectively, so that the measurement process at the two points is executed in the same manner as in FIG. A pole 82 may be used in place of the pole 81, or another type of pole may be used.

Also, in the present embodiment, the locus of movement of a person is detected, but the object to be detected may be something other than a person. For example, by monitoring the movement trajectory of a transport robot that autonomously runs in a factory, it becomes possible to detect and prevent collisions, abnormal running, and runaway.

As described above, the measuring apparatus according to the present invention includes a sensor that periodically transmits a laser beam while repeatedly scanning a scanning range and receives a laser beam reflected by each part of a moving object within a detection area; calculating the position of a moving body point group corresponding to each part of the moving body from the received laser beam, extracting an elliptical body from the calculated moving body point group, and sequentially calculating the position of the center of gravity of the extracted elliptical body; A second calculating means for determining whether or not the distance between two of the calculated center-of-gravity positions of the torso is equal to or less than a threshold, and assigning a common ID to the torso determined to be equal to or less than the threshold. and movement trajectory generation means for generating a movement trajectory by linking the positions of the center of gravity of the trunk to which the common ID is assigned in time series.

According to the present invention, a laser beam is periodically transmitted from the sensor while repeatedly scanning the scanning range, and the laser beam is received by being reflected by each part of a moving object existing within the detection area. Then, the position of the moving object point group corresponding to each part of the moving object is calculated from the received laser beam by the first calculating means, the elliptical body is extracted from the calculated moving object point group, and further extracted. The position of the center of gravity of the elliptical body is sequentially calculated. Further, the second calculating means determines whether or not the distance between the center-of-gravity positions of the two torsos is equal to or less than a threshold, and assigns a common ID to the torso determined to be equal to or less than the threshold. be done. Then, the position of the center of gravity of the trunk to which the common ID is assigned is connected in time series by the movement trajectory generating means. Therefore, the measuring device calculates movement track information from the position information acquired from its own sensor, thereby reducing the data processing load on the server side. Also, when the distance between the center of gravity positions of the two torsos is determined to be less than the threshold, it is determined that they are the same moving object, so that even if it temporarily stands still (stops), it is properly detected as the same moving object. .

In addition, background processing means for calculating in advance the position of a stationary object existing in the detection area as a background point group is provided, and the first calculating means calculates the difference from the background point group to derive the moving object point group. It is something to do. This configuration eliminates the influence of stationary objects in the background.

Further, the first calculation means performs grouping from the moving object point group using the following conditions α, β, and γ, and uses each group as the body of the ellipse. However, the maximum distance within the group is less than or equal to a constant α, the minimum distance to the moving object point cloud outside the group is greater than or equal to a constant β, and the number of moving object point clouds within the group is greater than or equal to a certain constant γ. According to this configuration, it is possible to accurately detect each person.

Also, the constant α is preferably α=R+V*Δt. However, R is the assumed maximum diameter of the trunk, V is the average walking speed, and Δt is the time interval in the time series. According to this configuration, the detection range for each person is set in consideration of the general walking speed of a person.

Further, the second calculating means interpolates the interrupted center-of-gravity position of the trunk using the center-of-gravity position of the trunk for a plurality of cycles in the time series. According to this configuration, even if another moving object crosses between the sensors, the position of the center of gravity can be obtained by interpolating between them.

Further, the measurement system according to the present invention includes the measurement devices arranged so that the detection ranges partially overlap, and a server for acquiring information on each of the movement trajectories generated by each of the measurement devices, wherein the server is provided with movement trajectory integration means for extracting pairs of movement trajectories between the measuring devices and performing matching processing of the movement trajectories for each pair.

According to the present invention, the movement trajectory integration means of the server extracts pairs of movement trajectories between the measuring devices, and only performs matching processing of the movement trajectories for each pair. is reduced.

Further, the measurement system according to the present invention includes a calibrator including a reflector, and the measurement device detects the reflector erected to at least a first position and a second position of the calibrator by the sensor. and the server superimposes the positions and orientations of the calibrators obtained by a plurality of measuring devices on a common coordinate system, It has server-side calibration means for calculating the installation position of the sensor. According to this configuration, since the arrangement positions among the plurality of sensors are calibrated, the accuracy of the arrangement position information of each sensor is maintained.

Further, the server evaluates commonality between motion information from a motion sensor of the mobile terminal and the movement trajectory of the moving body, and if there is commonality, combination processing of combining the movement trajectory of the moving body and the mobile terminal. It has means. According to this configuration, since the movement trajectory of the person detected by the sensor and the mobile terminal carried by the person are combined, it is possible to associate the information from the mobile terminal with the movement trajectory, and the information can be effectively utilized. can be achieved.

Further, the server is characterized by comprising movement monitoring means for detecting whether or not the moving locus of the moving object has passed through a specific area within the detection area. According to this configuration, if a specific area is defined within the detection area so as to be detectable, it becomes possible to monitor the passing situation of the moving object in the specific area.

Also, the movement monitoring means monitors the number of passengers boarding and disembarking across the elevating area of the bus. According to this configuration, when the elevating area of the bus is set as the specific area, it is possible to monitor the number of passengers getting on and off the bus as well as the number of passengers.

1 measurement system 8 calibrator 10 measurement device 11, 11' sensor 120 control unit 123 calibration unit (apparatus-side calibration means)
124 background processing unit (background processing means)
125 human detection unit (first calculation means, second calculation means)
126 movement trajectory generation unit (movement trajectory generation means)
20 server 21 processing unit 211 movement trajectory integration unit (movement trajectory integration means)
212 calibration unit (server-side calibration means)
213 combination processing unit (combination processing means)
217 movement monitoring unit (movement monitoring means)
30 Mobile terminal 52 Doorway BU Inside car

Claims (9)

A plurality of measuring devices arranged so that the detection ranges partially overlap each other, and a server capable of communicating with each of the measuring devices,
The measuring device is
a sensor that periodically transmits a laser beam while repeatedly scanning a scanning range and receives a laser beam that is reflected by each part of a moving object within the detection area;
calculating the position of a moving body point group corresponding to each part of the moving body from the received laser beam, extracting an elliptical body from the calculated moving body point group, and sequentially calculating the position of the center of gravity of the extracted elliptical body; 1 calculation means;
a second calculating means for determining whether the distance between two of the calculated center-of-gravity positions of the trunks is equal to or less than a threshold, and assigning a common ID to the trunks determined to be equal to or less than the threshold;
movement trajectory generating means for generating a movement trajectory by connecting the center of gravity positions of the torso to which the common ID is assigned in time series ;
The server acquires information on the trajectory generated by each of the measuring devices, extracts pairs of the trajectory between the measuring devices, and performs matching processing of the trajectory for each pair. A measurement system with moving trajectory integration means . The measuring device comprises background processing means for calculating in advance the position of a stationary object existing in the detection area as a background point group,
2. The measurement system according to claim 1, wherein said first calculation means derives said moving object point group by taking a difference from said background point group. 2. The measurement system according to claim 1, wherein said first calculation means performs grouping from said moving object point group using the following conditions α, β, and γ, and defines each group as said elliptical trunk.
However, the maximum distance within the group must be equal to or less than a constant α, the minimum distance to the moving object point cloud outside the group must be equal to or greater than a constant β, and the number of moving object point clouds within the group must be equal to or greater than a certain constant γ. The measurement system according to claim 1, wherein the constant α is α=R+V*Δt.
However, R is the assumed maximum diameter of the trunk, V is the average walking speed, and Δt is the time interval in the time series. 2. The measurement system according to claim 1, wherein the second calculation means interpolates the interrupted center-of-gravity position of the trunk using the center-of-gravity position of the trunk for a plurality of cycles in the time series. Equipped with a calibrator with a reflector,
The measuring device performs device-side calibration for acquiring information on the position and orientation of the calibrator by detecting, by the sensor, the reflector that has been erected to at least a first position and a second position of the calibrator. have the means to
6. The server according to any one of claims 1 to 5, wherein the server includes server-side calibration means for calculating the installation position of each sensor by superimposing the positions and orientations of the calibrators obtained by a plurality of measuring devices in a common coordinate system. The measurement system described in . The server evaluates motion information from the motion sensor of the mobile terminal for commonality with the movement trajectory of the moving object, and if there is commonality, combination processing means for combining the movement trajectory of the moving object and the mobile terminal. The measurement system according to any one of claims 1 to 6 . 8. The measurement system according to any one of claims 1 to 7, wherein said server comprises movement monitoring means for detecting whether or not the movement trajectory of said moving body has passed through a specific area within said detection area. . 9. A measurement system according to claim 8, wherein said movement monitoring means monitors the number of passengers boarding and disembarking across the elevating area of the bus.

Images