JP7205180B2 - Monitoring system and processing equipment - Google Patents

Description

The present disclosure relates to monitoring systems and processing devices.

2. Description of the Related Art Traffic information monitoring systems have been developed to collect information on vehicles and pedestrians in a monitoring area such as an intersection and provide the information to users in order to support vehicle driving and pedestrian safety. For example, Patent Literature 1 describes a laser radar that acquires information on a plurality of measurement points by scanning an intersection with a laser beam, and detects vehicles existing in the intersection based on the information on the plurality of measurement points. .

JP 2010-197341 A

A laser radar cannot irradiate a laser beam in a range (blind spot) that is shaded by an obstacle, so there is a possibility that an object cannot be detected in such a range. For example, when a large truck or the like crosses an intersection, it may not be possible to detect a motorcycle or a pedestrian in the shadow of the truck. This phenomenon is called occlusion. To address such a problem, a plurality of laser radars may be installed in order to improve detection accuracy, and object detection may be performed using measurement point information acquired by each laser radar. However, since the amount of data of information on a plurality of measurement points acquired by a laser radar is generally large, communication may take time, and it may take time to obtain detection results in the monitoring area.

The present disclosure describes a monitoring system and a processing device capable of suppressing an increase in the time required for detection while improving detection accuracy.

The monitoring system according to one aspect of the present disclosure is installed on the ground and irradiates a laser beam toward an irradiation possible area including at least a part of the monitoring area and receives the reflected light of the laser beam. This system monitors a monitoring area using a first laser sensor and a second laser sensor that generate measurement point information including the position coordinates of each measurement point within the possible area. This monitoring system includes a first processing device that generates first partial information by processing first point group information including information on a plurality of measurement points generated by a first laser sensor; a second processing device that generates second partial information by processing second point cloud information including a plurality of measurement point information; and a detection device for generating. The first processing device clusters the first point group information, and when one or more first clusters are obtained by clustering, a parameter capable of specifying a geometric shape approximating each of the one or more first clusters generates first partial information including

According to the present disclosure, it is possible to suppress an increase in the time required for detection while improving detection accuracy.

FIG. 1 is a diagram schematically showing the configuration of a monitoring system according to one embodiment. FIG. 2 is a diagram showing an arrangement example of the master laser radar device and the slave laser radar device shown in FIG. 3 is a perspective view of FIG. 2. FIG. FIG. 4 is a diagram showing an example of point cloud information. FIG. 5 is a diagram showing the hardware configuration of the processing device shown in FIG. FIG. 6(a) is a diagram showing point group information projected onto the xy plane. (b) of FIG. 6 is a diagram for explaining the removal of the measurement point information. FIG. 6(c) is a diagram for explaining the approximation of a cluster to a geometric shape. FIG. 7 is a diagram for explaining parameters that can specify a geometric shape. FIG. 8 is a diagram for explaining the synthesizing process. FIG. 9 is a flowchart showing partial information generation processing performed by the slave laser radar device. FIG. 10 is a flowchart showing details of the data amount reduction process shown in FIG. FIG. 11 is a flowchart showing partial information generation processing performed by the master laser radar device. FIG. 12 is a flowchart showing detection processing performed by the master laser radar device.

[1] Outline of Embodiment A monitoring system according to one aspect of the present disclosure is installed on the ground and irradiates a laser beam toward an irradiable area including at least a part of the monitoring area, and the laser beam is reflected. This is a system that monitors a monitoring area using a first laser sensor and a second laser sensor that generate measurement point information including position coordinates of each measurement point within an irradiable area by receiving light. This monitoring system includes a first processing device that generates first partial information by processing first point group information including information on a plurality of measurement points generated by a first laser sensor; a second processing device that generates second partial information by processing second point cloud information including a plurality of measurement point information; and a detection device for generating. The first processing device clusters the first point group information, and when one or more first clusters are obtained by clustering, a parameter capable of specifying a geometric shape approximating each of the one or more first clusters generates first partial information including

In this monitoring system, a monitoring area is monitored using a first laser sensor and a second laser sensor installed on the ground. As a result, blind spots in the monitoring area are reduced, so detection accuracy can be improved. Further, when the first processing device obtains one or more first clusters by clustering the first point group information including a plurality of measurement point information generated by the first laser sensor, one or more First partial information is generated that includes parameters that can identify a geometric shape that approximates each of the first clusters. Therefore, the data amount of the first partial information can be smaller than the data amount of the first point group information. As a result, the time required for transmitting the first partial information from the first processing device to the detecting device can be shortened compared to the case where the first point cloud information is transmitted from the first processing device to the detecting device. . As a result, it is possible to suppress an increase in the time required for detection while improving the detection accuracy.

The second processing device may cluster the second point group information, and when one or more second clusters are obtained by clustering, identify geometric shapes that approximate each of the one or more second clusters. A second piece of information including possible parameters may be generated. In this case, the data amount of the second partial information can be smaller than the data amount of the second point cloud information. As a result, the time required for transmitting the second partial information from the second processing device to the detection device can be shortened compared to the case where the second point cloud information is transmitted from the second processing device to the detection device. . As a result, it is possible to suppress an increase in the time required for detection while improving the detection accuracy.

When the first geometric shape identified by the parameters included in the first partial information and the second geometric shape identified by the parameters included in the second partial information overlap each other, the detection device detects the One geometry and a second geometry may be detected as the same object. When the first geometric shape identified by the parameters included in the first partial information and the second geometric shape identified by the parameters included in the second partial information overlap each other, the first geometric shape and the second geometry are likely to be the same object as each other. Therefore, detection accuracy can be further improved by detecting the first geometric shape and the second geometric shape as the same object.

The first processing device may exclude measurement point information of measurement points included in a preset detection exclusion range from the first point group information, and cluster the remaining measurement point information. In this case, the number of pieces of measurement point information to be clustered can be reduced, thereby reducing the calculation load of the first processing device.

The monitoring system described above may further comprise a first laser sensor and a second laser sensor. In this case, since the first point group information and the second point group information are generated within the monitoring system, there is no need to obtain the first point group information and the second point group information from outside the monitoring system.

A processing apparatus according to another aspect of the present disclosure is installed on the ground and can be irradiated by irradiating a laser beam toward an irradiation possible area including at least a part of a monitoring area and receiving the reflected light of the laser beam. an acquisition unit that acquires point cloud information including a plurality of measurement point information from a laser sensor that generates measurement point information including the position coordinates of each measurement point in an area; is obtained, a processing unit for generating partial information including parameters capable of specifying a geometric shape approximating each of the one or more clusters, and an output unit for outputting the partial information to the outside.

In this processing device, when one or more clusters are obtained by clustering point group information including a plurality of measurement point information generated by a laser sensor, a geometric shape approximating each of the one or more clusters is generated, and the partial information is output to the outside. Therefore, the data amount of the partial information can be smaller than the data amount of the point cloud information. As a result, the time required for transmitting the partial information from the processing device to the outside can be shortened compared to the case where the point group information is transmitted from the processing device to the outside. As a result, it becomes possible to suppress an increase in the time required for detection.

[2] Exemplification of Embodiments Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and overlapping descriptions are omitted.

FIG. 1 is a diagram schematically showing the configuration of a monitoring system according to one embodiment. FIG. 2 is a diagram showing an arrangement example of the master laser radar device and the slave laser radar device shown in FIG. 3 is a perspective view of FIG. 2. FIG. FIG. 4 is a diagram showing an example of point cloud information. FIG. 5 is a diagram showing the hardware configuration of the processing device shown in FIG.

A monitoring system 1 shown in FIG. 1 is a system that monitors a monitoring region Rd using laser sensors 4 and 6, which will be described later. More specifically, the monitoring system 1 detects an object (moving object) existing in the monitoring area Rd. The monitoring area Rd is an area to be monitored. The monitoring area Rd can be set anywhere on the road. For example, an intersection, a confluence, and the middle of the road are selected as the setting location of the monitoring region Rd. As shown in FIGS. 2 and 3, the monitoring area Rd is set at the intersection C in this embodiment. Objects to be monitored include vehicles and pedestrians (people).

A monitoring system 1 includes a slave laser radar device 2 and a master laser radar device 3 . The slave laser radar device 2 and the master laser radar device 3 are installed near the monitoring area Rd. In this embodiment, the slave laser radar device 2 and the master laser radar device 3 are provided on the diagonal line of the intersection C. As shown in FIG. An intersection C is a point where four roads Tr1 to Tr4 converge. The road Tr1 and the road Tr2 extend along one direction and are connected to each other via an intersection C. The roads Tr3 and Tr4 extend in a direction crossing the roads Tr1 and Tr2, and are connected to each other via an intersection C. The slave laser radar device 2 and the master laser radar device 3 are fixed to a support member P (see FIG. 3) installed on the ground. The support member P is, for example, a columnar structure provided on the roadside near the intersection C. As shown in FIG. Support members P may be utility poles and warehouse walls.

The slave laser radar device 2 and the master laser radar device 3 are connected by a communication line N so as to be able to communicate with each other. The communication line N may be wired or wireless. The communication line N may be a non-dedicated line such as an Internet line or a mobile communication network, or a dedicated line. At the intersection C or the like, since the distance between the slave laser radar device 2 and the master laser radar device 3 is long, it may be difficult to wire a dedicated line as the communication line N. Therefore, a wireless communication line can be used as the communication line N in this embodiment.

The slave laser radar device 2 has a laser sensor 4 and a processing device 5 . The laser sensor 4 is fixed to the support member P. As shown in FIG. That is, the laser sensor 4 is installed on the ground. The laser sensor 4 generates measurement point information of each measurement point within the irradiation area Ra by irradiating the irradiation area Ra with a laser beam and receiving the reflected light of the irradiated laser beam. The irradiable area Ra is an area where the laser sensor 4 can irradiate a laser beam, and has a range of about 150 m, for example. Irradiable area Ra includes at least part of monitoring area Rd.

As shown in FIG. 4, the measurement point information includes time information and position information. The time information is information indicating the time when the measuring point information of the measuring point indicated by the position information was generated (when the reflected light was received). The position information is information indicating the position coordinates of the measurement point. The position coordinates may use a polar coordinate system represented by yaw angle, pitch angle, and depth, or may use a three-dimensional coordinate system represented by x-coordinates, y-coordinates, and z-coordinates. The coordinate system CS used in this embodiment is a three-dimensional coordinate system. The x-axis of the coordinate system CS extends along the road Tr1 and the road Tr2, and is set so that the direction from the road Tr1 to the road Tr2 is positive. The y-coordinate of the coordinate system CS extends along the roads Tr3 and Tr4, and is set so that the direction from the road Tr3 to the road Tr4 is positive. The z-axis of the coordinate system CS is set with the ground surface as a reference (z=0), and is set so that the area above the ground surface is positive. The measurement point information may further include reflection intensity information. The reflected intensity information is information indicating the intensity of reflected light received from the measurement point indicated by the position information at the time indicated by the time information.

The laser sensor 4 scans the irradiable area Ra in the main scanning direction and the sub-scanning direction by changing the irradiation direction of the laser light. As a result, the plurality of measurement points included in the irradiable area Ra are sequentially irradiated with the laser beam. One round of laser light irradiation to all the measurement points included in the irradiation-enabled area Ra may be referred to as one frame. Irradiation of the laser beam to the irradiable region Ra is repeated at predetermined time intervals. The laser sensor 4 outputs point group information Dm1 (first point group information) including a plurality of measurement point information for one frame to the processing device 5 for each frame. The laser sensor 4 outputs the point cloud information Dm1 to the processing device 5 in the form of a table shown in FIG. 4, for example. The point cloud information Dm1 may further include a frame ID (Identifier). A frame ID is identification information that can uniquely identify a frame. As the frame ID, for example, a frame number indicating the order of frames can be used.

The processing device 5 is a device that generates partial information Dp1 (first partial information) by processing point group information Dm1 including information on a plurality of measurement points generated by the laser sensor 4 . The processing device 5 is configured by, for example, an information processing device such as a computer.

As shown in FIG. 5, the processing device 5 physically includes one or more processors 101, a main storage device 102 such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and an auxiliary storage device such as a hard disk device. It can be configured as a computer having hardware such as a storage device 103 and a communication device 104 that is a data transmission/reception device. Each functional unit of the processing device 5 shown in FIG. 1 is realized by executing a program module for realizing each function in a computer constituting the processing device 5 . A program including these program modules is provided by a computer-readable recording medium such as a ROM or a semiconductor memory, for example. Also, the program may be provided via a network as a data signal.

The processing device 5 functionally includes an acquisition unit 51 , a storage unit 52 , a processing unit 53 , and an output unit 54 .

The acquisition unit 51 acquires the point group information Dm1 from the laser sensor 4 . The acquisition unit 51 outputs the acquired point cloud information Dm1 to the processing unit 53 .

The storage unit 52 stores various setting information. Various setting information includes exclusion information indicating a detection exclusion range. The detection exclusion range is a range that is excluded from monitoring. The detection exclusion range includes a range outside the monitoring region Rd and a monitoring unnecessary range within the monitoring region Rd. The detection exclusion range is set in advance. For example, the user sets the detection exclusion range using an input device (not shown). For example, a three-dimensional space simulating the intersection C is displayed on an output device (not shown) such as a display, and the user sets a detection exclusion range using a frame or the like.

For example, below the ground and above the road may not be monitored. Therefore, in order to exclude the height below the ground, a range lower than a predetermined height with reference to the ground surface may be set as the height of the detection exclusion range. For example, the range of z<20 cm is set as the detection exclusion range. In order to exclude the upper sky, a range higher than a predetermined height with reference to the ground surface may be set as the height of the detection exclusion range. For example, the range of z>500 cm is set as the detection exclusion range. In addition, in and around the intersection C, there are fixed objects (stationary objects) such as traffic lights, poles, utility poles, roadside trees, and elevated structures. Since these fixed objects do not need to be monitored, a fixed object range may be set as a detection exclusion range to exclude these fixed objects. The range of the fixed object may be set as coordinates (x, y) indicating the boundary of the fixed object on the xy plane.

The processing unit 53 generates partial information Dp1 by processing the point cloud information Dm1. Specifically, when the processing unit 53 receives the point cloud information Dm1 from the acquisition unit 51, the processing unit 53 acquires the measurement point information of the measurement points included in the detection exclusion range indicated by the exclusion information read from the storage unit 52 as the point cloud information. Exclude (delete) from Dm1. As a result, measurement points not to be monitored are excluded, and abnormal measurement point information caused by noise or the like is excluded. As shown in (a) of FIG. 6, the point cloud information Dm1 includes measurement point information on the ground, but the ground does not need to be monitored. Therefore, as shown in FIG. 6B, the measurement point information on the ground is removed from the point group information Dm1.

The processing unit 53 clusters the point group information Dm1. Here, the processing unit 53 clusters the remaining measurement point information obtained by excluding the measurement point information of the measurement points included in the detection exclusion range from the point group information Dm1. The processing unit 53 performs clustering using a known algorithm. Examples of clustering algorithms include algorithms using Euclidean distance. In this algorithm, two measurement points are classified into the same cluster if the distance between them is less than or equal to a predetermined threshold. In this way, the processing unit 53 joins neighboring measurement points among the plurality of measurement points in the monitoring area Rd and divides them into clusters.

When one or more clusters are obtained by clustering, the processing unit 53 approximates each cluster to a geometric shape. As geometric shapes, simple shapes that can be expressed by several parameters are used. The processing unit 53 performs approximation to a geometric shape using a known algorithm. In this embodiment, a rectangle in the xy plane is used as the geometric shape. As shown in (c) of FIG. 6, the processing unit 53 projects the plurality of measurement points included in the cluster onto the xy plane, and approximates the plurality of measurement points projected onto the xy plane to a rectangle. The processing unit 53 creates, for example, a minimum area rectangle that circumscribes the plurality of measurement points included in the cluster. For example, the Rotating Calipers method is used to create this rectangle. Note that the geometric shape may include all the measurement points included in the cluster, or may not include some of the measurement points.

The processing unit 53 calculates parameters that can specify a geometric shape (rectangle) that approximates each cluster, and generates partial information Dp1 including these parameters. As shown in FIG. 7, examples of parameters that can specify a rectangle considering rotation on the xy plane include the width W, the height L, the center coordinates (x, y), and the rotation angle θ. The rotation angle θ is the rotation angle with respect to the x-axis positive direction on the xy plane. The width W is the length of the rectangle in the y-axis direction when the rectangle is rotated so that the rotation angle θ of the rectangle is 0 degrees. The vertical width L is the length of the rectangle in the x-axis direction when the rectangle is rotated so that the rotation angle θ of the rectangle is 0 degrees. Partial information Dp1 may further include a frame ID. The processing unit 53 may generate partial information Dp1 including a frame ID, the number of clusters, and parameters that can specify the geometric shape of each cluster. The processing unit 53 outputs the partial information Dp1 to the output unit 54 .

It should be noted that there may be no objects to be monitored, such as pedestrians and vehicles, at the intersection C because the traffic volume is small at midnight or the like. In such a case, the processing unit 53 cannot obtain clusters even if the point group information Dm1 is clustered. In this case, the processing unit 53 generates, for example, partial information Dp1 in which the number of clusters is 0 and no parameter is included.

The output unit 54 outputs the partial information Dp1 to the outside of the processing device 5 . Specifically, upon receiving the partial information Dp1 from the processing unit 53 , the output unit 54 transmits the partial information Dp1 to the master laser radar device 3 via the communication line N.

In this way, the processing device 5 clusters the point group information Dm1, and when one or more clusters are obtained by clustering, includes parameters that can identify geometric shapes approximating each of the one or more clusters. Partial information Dp1 is generated and the partial information Dp1 is transmitted to the master laser radar device 3 . Since the partial information Dp1 does not include the measurement point information, the data amount of the partial information Dp1 is greatly reduced compared to the data amount of the point cloud information Dm1.

The master laser radar device 3 has a laser sensor 6 , a processing device 7 and a detection device 8 . The laser sensor 6 is fixed to the support member P. As shown in FIG. That is, the laser sensor 6 is installed on the ground. Like the laser sensor 4, the laser sensor 6 irradiates a laser beam toward the irradiable region Rb and receives the reflected light of the radiated laser beam, so that each measurement point in the irradiable region Rb is detected. Generate information. The irradiable region Rb is a region in which the laser sensor 6 can irradiate laser light, and has a range of, for example, about 150 m. Irradiable region Rb includes at least part of monitoring region Rd. The same coordinate system CS as that of the laser sensor 4 is used for the laser sensor 6 as well. The laser sensor 6 outputs point group information Dm2 (second point group information) including a plurality of measurement point information for one frame to the processing device 7 for each frame. The point cloud information Dm2 may further include a frame ID.

Note that the time of the laser sensor 6 is synchronized with the time of the laser sensor 4 . The time of the laser sensor 4 and the time of the laser sensor 6 are synchronized using, for example, NTP (Network Time Protocol). Therefore, the start time and end time of one frame of the laser sensor 6 coincide with the start time and end time of one frame of the laser sensor 4 . The start time of one frame is the generation (acquisition) time of the first measurement point information of that frame. The end time of one frame is the generation (acquisition) time of the last measurement point information of that frame.

The processing device 7 is a device that generates partial information Dp2 (second partial information) by processing the point cloud information Dm2 including information on a plurality of measurement points generated by the laser sensor 6 . The processing device 7 is configured by, for example, an information processing device such as a computer. The physical configuration of the processing device 7 is similar to that of the processing device 5 . Each functional unit of the processing device 7 shown in FIG. 1 is realized by executing a program module for realizing each function in a computer constituting the processing device 7 . A program including these program modules is provided by a computer-readable recording medium such as a ROM or a semiconductor memory, for example. Also, the program may be provided via a network as a data signal. The processing device 7 functionally includes an acquisition unit 71 , a storage unit 72 , a processing unit 73 , and an output unit 74 .

The acquisition unit 71 acquires the point cloud information Dm2 from the laser sensor 6 . The acquisition unit 71 outputs the acquired point cloud information Dm2 to the processing unit 73 .

The storage unit 72 stores various setting information. Various setting information includes exclusion information indicating a detection exclusion range. The detection exclusion range is set in advance. For example, similarly to the slave laser radar device 2, the user sets the detection exclusion range using an input device. The detection exclusion range set in the processing device 7 may be the same as or different from the detection exclusion range set in the processing device 5 . When the same detection exclusion range is set in the processing device 5 and the processing device 7 , the processing device 7 may transmit exclusion information indicating the detection exclusion range set in the processing device 7 to the processing device 5 .

The processing unit 73 generates partial information Dp2 by processing the point cloud information Dm2. Specifically, when the processing unit 73 receives the point cloud information Dm2 from the acquisition unit 71, the processing unit 73 acquires the measurement point information of the measurement points included in the detection exclusion range indicated by the exclusion information read from the storage unit 72 as the point cloud information Dm2. Exclude from As a result, measurement points not to be monitored are excluded, and abnormal measurement point information caused by noise or the like is excluded.

The processing unit 73 clusters the point group information Dm2. Here, the processing unit 73 clusters the remaining measurement point information obtained by excluding the measurement point information of the measurement points included in the detection exclusion range from the point group information Dm2. The processing unit 73 performs clustering using a known algorithm, similarly to the processing unit 53 . The processing unit 73 connects neighboring measurement points among the plurality of measurement points in the monitoring area Rd and divides them into clusters.

When one or more clusters are obtained by clustering, the processing unit 73 approximates each cluster to a geometric shape. As geometric shapes, simple shapes that can be expressed by several parameters are used. The processing unit 73, like the processing unit 53, performs approximation to a geometric shape using a known algorithm. In this embodiment, the processing unit 73 projects the plurality of measurement points included in the cluster onto the xy plane, and approximates the plurality of measurement points projected onto the xy plane to a rectangle. The processing unit 73 calculates parameters that can specify a geometric shape (rectangle) that approximates each cluster, and generates partial information Dp2 including these parameters. Partial information Dp2 may further include a frame ID. The processing unit 73 may generate partial information Dp2 including a frame ID, the number of clusters, and parameters that can specify the geometric shape of each cluster. If no clusters are obtained by clustering, the processing unit 73 generates partial information Dp2 with the number of clusters being 0 and no parameters, for example. The processing section 73 outputs the partial information Dp2 to the output section 74 .

The output unit 74 outputs the partial information Dp2 to the outside of the processing device 7 . Specifically, the output unit 74 outputs the partial information Dp2 to the detection device 8 upon receiving the partial information Dp2 from the processing unit 73 .

In this way, the processing device 7 clusters the point group information Dm2, and when one or more clusters are obtained by clustering, includes parameters that can identify geometric shapes that approximate each of the one or more clusters. Partial information Dp2 is generated and the partial information Dp2 is output to the detection device 8 . Since the partial information Dp2 does not include the measurement point information, the data amount of the partial information Dp2 is significantly reduced compared to the data amount of the point group information Dm2.

The detection device 8 is a device that generates a detection result in the monitoring region Rd based on the partial information Dp1 and the partial information Dp2. The detection device 8 is configured by, for example, an information processing device such as a computer. The physical configuration of the detection device 8 is similar to that of the processing device 5 . Each functional unit of the detection device 8 shown in FIG. 1 is realized by executing a program module for realizing each function in a computer that constitutes the detection device 8 . A program including these program modules is provided by a computer-readable recording medium such as a ROM or a semiconductor memory, for example. Also, the program may be provided via a network as a data signal. Note that the processing device 7 and the detection device 8 may be configured by one information processing device. In this case, a program including program modules for realizing the functions of the functional units of the processing device 7 and the detection device 8 may be provided.

The detection device 8 functionally includes an acquisition unit 81 , a storage unit 82 , a synthesis unit 83 , a detection unit 84 and an output unit 85 .

Acquisition unit 81 acquires partial information Dp1 from processing device 5 and acquires partial information Dp2 from processing device 7 . Due to a transmission delay or the like, the timing at which the acquiring unit 81 receives the partial information Dp1 and the timing at which the acquiring unit 81 receives the partial information Dp2 may differ from each other even in the same frame. In this embodiment, the partial information Dp1 is transmitted via the communication line N, whereas the partial information Dp2 is transmitted via the internal communication line of the master laser radar device 3. After receiving Dp2, it receives partial information Dp1. Therefore, the acquisition unit 81 outputs the partial information Dp2 to the storage unit 82 and stores it in the storage unit 82 . Acquisition unit 81 outputs partial information Dp1 to synthesis unit 83 .

The storage unit 82 stores the partial information Dp2 for each frame. The storage unit 82 may store the partial information Dp2 in a different file for each frame.

The combining unit 83 generates combined information by combining (merging) the partial information Dp1 and the partial information Dp2 of the same frame as the frame of the partial information Dp1. More specifically, first, when receiving the partial information Dp1 from the acquisition unit 81, the synthesizing unit 83 receives the partial information Dp2 of the plurality of frames stored in the storage unit 82, and obtains the frame ID that is the same as the frame ID of the partial information Dp1. acquires partial information Dp2 including the frame ID of .

The synthesizer 83 synthesizes the partial information Dp1 and the obtained partial information Dp2. More specifically, the synthesizing unit 83 selects one geometric shape from one or more geometric shapes specified by the parameters included in the partial information Dp1 and the geometric shape specified by the parameters included in the partial information Dp2. and a geometry selected from one or more geometries. If the two geometric shapes overlap each other, the synthesizing unit 83 detects these two geometric shapes as the same object.

Note that the distance between two objects may be close, such as a vehicle stopped at an intersection or the like and a pedestrian walking beside it. In such cases, the geometries of the clusters corresponding to these two objects may partially overlap. Therefore, when the area of the overlapping portion of the two geometric shapes exceeds a predetermined ratio of the area of one of the geometric shapes, the synthesizing unit 83 determines that the two geometric shapes are the same object. You may The predetermined ratio is set in advance, and is set to about 30%, for example. If a given geometric shape does not overlap with another geometric shape, and if the area of a portion where a given geometric shape A shape may be determined to be an independent single object.

The synthesizing unit 83 creates a geometric shape that includes the two geometric shapes determined to be the same object. Synthesis unit 83 maintains the geometry determined as a single object. Synthesis unit 83 compares all geometric shape combinations of the geometric shapes specified by the parameters included in partial information Dp1 and the geometric shapes specified by the parameters included in partial information Dp2, and then Let each of the remaining geometric shapes be a detected object.

As shown in FIG. 8, the rectangular geometric shapes G11 to G15 are specified by the parameters included in the partial information Dp1, and the rectangular geometric shapes G21 to G24 are specified by the parameters included in the partial information Dp2. Assume that In this case, the geometric shape G11 and the geometric shape G21 overlap each other, and the overlapping area exceeds 30% of the area of the geometric shapes G11 and G21. G21 is determined to be the same object. Therefore, a geometric shape G31 containing the geometric shape G11 and the geometric shape G21 is created. Similarly, the geometric shape G12 and the geometric shape G22 are determined to be the same object, and a geometric shape G32 containing the geometric shape G12 and the geometric shape G22 is created. On the other hand, the geometric shapes G13 to G15 and the geometric shapes G23 and G24 do not overlap with any geometric shapes, so their shapes are maintained. Geometric shapes G13 to G15, G23, G24, G31, and G32 are detected as detection objects, respectively.

One of the partial information Dp1 and the partial information Dp2 may not contain a parameter. In this case, since the geometric shapes do not overlap each other, the synthesizing unit 83 uses the geometric shape specified by the parameters included in the partial information as the detected object.

The synthesizing unit 83 calculates parameters specifying each detected object. The parameters include width W, length L, center coordinates (x, y), and rotation angle θ. Instead of the center coordinates, the coordinates of the four corners (the front right edge, the front left edge, the rear right edge, and the rear left edge) of the detected object may be used, or the average position of the measurement point information included in the cluster may be used. , the center-of-gravity position of the detected object may be used. The synthesizing unit 83 generates synthesized information including parameters of each detected object and detection time information indicating the detection time when the detected object was detected. The detection time is, for example, an intermediate time between the start time and the end time of the frame. If neither the partial information Dp1 nor the partial information Dp2 includes a parameter, the synthesizing unit 83 may generate synthesized information indicating that no detected object has been detected. Synthesis section 83 outputs synthesis information to detection section 84 .

The detector 84 tracks the detected object. That is, the detection unit 84 associates objects detected in different frames (different times) with object IDs. The object ID is identification information that can uniquely identify the detected object. Specifically, when the detection unit 84 receives the combined information from the combining unit 83, based on the parameters of each detected object included in the combined information, the velocity and angular velocity estimated from the past observation results, and the like, the current It is determined whether the detected object detected in the previous frame corresponds to any of the detected objects detected in the past frames.

When the detection unit 84 determines that the detected object detected in the current frame does not correspond to any of the detected objects detected in the past frames, the detection unit 84 assigns a new object ID to the detected object as a new detected object. do. When the detection unit 84 determines that the detected object detected in the current frame corresponds to the detected object detected in the past frame, the detection unit 84 detects the object ID assigned to the corresponding detected object in the current frame. given to the detected object. The detection unit 84 deletes the object ID of a detected object that has not been detected for a long time among the detected objects assigned with the object ID.

The problem of tracking (IDing) multiple detected objects is called a multi-target tracking problem. The detector 84 tracks each detected object using a known algorithm. Known algorithms include SNN (Suboptimal Nearest Neighbor), GNN (Global Nearest Neighbor), and JPDAF (Joint Probabilistic Data Association Filter). The detection unit 84 outputs the object ID, object position information, detection time information, and the like to the output unit 85 as detection results. The detection unit 84 may output the combined information to the output unit 85 as the detection result.

The output unit 85 outputs the detection result to the outside of the detection device 8 (master laser radar device 3). The output unit 85, for example, transmits the detection result to an external device (not shown). Examples of external devices include higher management systems and traffic control systems. Transmission of the detection result may be performed by wireless communication or by wired communication. The output unit 85 may convert the data format of the detection result into a data format that can be easily processed by the external device, and transmit the converted detection result to the external device.

Next, processing performed by the slave laser radar device 2 will be described. FIG. 9 is a flowchart showing partial information generation processing performed by the slave laser radar device. FIG. 10 is a flowchart showing details of the data amount reduction process shown in FIG. The partial information generation process shown in FIG. 9 is started, for example, when the slave laser radar device 2 is activated.

First, the laser sensor 4 synchronizes the time so that the time of the laser sensor 4 and the time of the laser sensor 6 match (step S11). For example, the laser sensor 4 sets the time using NTP.

After the time is synchronized, the laser sensor 4 sequentially irradiates a plurality of measurement points included in the irradiable area Ra with laser light, and receives the reflected light of the radiated laser light, so that the irradiable area Ra Generate measuring point information for each measuring point in Note that the laser sensor 4 starts emitting laser light at the same timing as the laser sensor 6 starts emitting laser light. Therefore, the start time and end time of each frame of the laser sensor 4 coincide with the start time and end time of each frame of the laser sensor 6 . Then, the laser sensor 4 outputs point group information Dm1 including a plurality of measurement point information for one frame to the processing device 5 .

Subsequently, the acquiring unit 51 acquires the point group information Dm1 from the laser sensor 4 (step S12). Then, the acquisition unit 51 outputs the acquired point cloud information Dm1 to the processing unit 53 .

Subsequently, the processing unit 53 reduces the data amount of the point cloud information Dm1 (step S13). As shown in FIG. 10, in step S13, first, when the processing unit 53 receives the point cloud information Dm1 from the acquisition unit 51, the processing unit 53 determines the measurement points included in the detection exclusion range indicated by the exclusion information read from the storage unit 52. is excluded from the point group information Dm1 (step S21). Then, the processing unit 53 clusters the remaining measurement point information (step S22).

Subsequently, when one or more clusters are obtained by clustering, the processing unit 53 approximates each cluster to a geometric shape (step S23). Then, the processing unit 53 calculates parameters that can specify a geometric shape approximating each cluster, and generates partial information Dp1 including the number of clusters obtained by clustering and these parameters (step S24). . Note that if no clusters are obtained by clustering, the number of clusters indicates 0, and partial information Dp1 that does not include parameters is generated. The processing unit 53 then outputs the partial information Dp1 to the output unit 54 .

Subsequently, when receiving the partial information Dp1 from the processing unit 53, the output unit 54 transmits the partial information Dp1 to the master laser radar device 3 via the communication line N (step S14). Then, the slave laser radar device 2 determines whether or not to end the processing (step S15). When the slave laser radar device 2 determines not to end the processing (step S15; NO), the irradiation by the laser sensor 4 is performed again, and the processing of steps S12 to S15 is performed. On the other hand, when the slave laser radar device 2 determines to end the processing (step S15; YES), the partial information generation processing performed by the slave laser radar device 2 ends. Note that the slave laser radar device 2 determines to end the process when the user performs an operation to stop monitoring.

Next, processing performed by the master laser radar device 3 will be described. FIG. 11 is a flowchart showing partial information generation processing performed by the master laser radar device. The partial information generation process shown in FIG. 11 is started, for example, when the master laser radar device 3 is activated.

First, the laser sensor 6 synchronizes the time in order to match the time of the laser sensor 4 with the time of the laser sensor 6 (step S31). For example, the laser sensor 6 sets the time using NTP.

After the time is synchronized, the laser sensor 6 sequentially irradiates a plurality of measurement points included in the irradiable region Rb with laser light, and receives the reflected light of the radiated laser light, thereby irradiating the irradiable region Rb. Generate measuring point information for each measuring point in Then, the laser sensor 6 outputs to the processing device 7 point cloud information Dm2 including a plurality of measurement point information for one frame.

Subsequently, the acquiring unit 71 acquires the point group information Dm2 from the laser sensor 6 (step S32). The acquisition unit 71 then outputs the acquired point cloud information Dm2 to the processing unit 73 .

Subsequently, the processing unit 73 reduces the data amount of the point group information Dm2 (step S33). Since the process of step S33 is the same as the process of step S13, detailed description thereof will be omitted. In step S<b>33 , the processing unit 73 generates partial information Dp<b>2 and outputs the partial information Dp<b>2 to the output unit 54 .

Subsequently, upon receiving the partial information Dp2 from the processing unit 73 , the output unit 74 outputs the partial information Dp2 to the detection device 8 . Upon receiving the partial information Dp2 from the processing device 7 (output unit 74), the acquisition unit 81 of the detection device 8 outputs the partial information Dp2 to the storage unit 82 and stores it in the storage unit 82 (step S34).

Subsequently, the master laser radar device 3 determines whether or not to end the processing (step S35). If the master laser radar device 3 determines not to end the processing (step S35; NO), the laser sensor 6 performs irradiation again, and the processing of steps S32 to S35 is performed. On the other hand, when the master laser radar device 3 determines to end the processing (step S35; YES), the partial information generation processing performed by the master laser radar device 3 ends. Note that the master laser radar device 3 determines to end the process when the user performs an operation to stop monitoring.

Next, detection processing performed by the master laser radar device 3 will be described. FIG. 12 is a flowchart showing detection processing performed by the master laser radar device. The detection process shown in FIG. 12 is started by, for example, activating the master laser radar device 3 .

First, the acquiring unit 81 determines whether or not the partial information Dp1 has been received from the slave laser radar device 2 via the communication line N (step S41). When determining that the partial information Dp1 has not been received (step S41; NO), the obtaining unit 81 repeats the determination of step S41 until the partial information Dp1 is received. On the other hand, when acquiring portion 81 determines that partial information Dp1 has been received (step S41; YES), acquiring portion 81 outputs received partial information Dp1 to synthesizing portion 83 .

Subsequently, upon receiving the partial information Dp1 from the acquisition unit 81, the synthesizing unit 83 synthesizes the partial information Dp1 and the partial information Dp2 of the same frame as the partial information Dp1 (step S42). Specifically, first, the synthesizing unit 83 acquires partial information Dp2 including the same frame ID as the frame ID of the partial information Dp1 from among the partial information Dp2 of the plurality of frames stored in the storage unit 82 . Then, combining unit 83 combines one geometric shape selected from the one or more geometric shapes specified by the parameters included in partial information Dp1 and the one or more geometric shapes specified by the parameters included in partial information Dp2. and a geometric shape selected from the shapes.

More specifically, the synthesizing unit 83 determines whether the two geometric shapes are the same object. For example, if the area of the overlapping portion of the two geometric shapes exceeds a predetermined ratio of the area of one of the geometric shapes, the synthesizing unit 83 determines that these two geometric shapes are the same object. do. If the synthesizing unit 83 determines that the two geometric shapes are the same object, it creates a geometric shape that includes these two geometric shapes. On the other hand, if the synthesizing unit 83 determines that the two geometric shapes are not the same object, it maintains the respective geometric shapes. The synthesizing unit 83 performs the above comparison processing for all combinations of two geometric shapes. Then, the synthesizing unit 83 uses each of the remaining geometric shapes as a detected object, and calculates parameters that can identify each detected object. Then, the synthesizing unit 83 generates synthesized information including parameters of each detected object and detection time information indicating the detection time when the detected object was detected. The synthesizing unit 83 then outputs the synthesizing information to the detecting unit 84 .

Subsequently, the detection unit 84 tracks the object based on the combined information (step S43). The detection unit 84 then outputs the detection result to the output unit 85 . Subsequently, the output unit 85 transmits the detection result to the external device (step S44). The output unit 85 may convert the data format of the detection result into a data format that can be easily processed by the external device, and transmit the converted detection result to the external device. Thus, the detection processing of the master laser radar device 3 is completed.

As described above, in the monitoring system 1, the monitoring region Rd is monitored using the laser sensors 4 and 6 installed on the ground. For example, the laser sensor 6 can be used to supplement the area of the monitoring area Rd where the laser sensor 4 cannot acquire the measurement point information. As a result, blind spots in the monitoring region Rd are reduced, so detection accuracy can be improved.

Also, since the data amount of the point cloud information Dm1 acquired (generated) by the laser sensor 4 and the data amount of the point cloud information Dm2 acquired (generated) by the laser sensor 6 are generally large, Real-time processing becomes more difficult as the number increases and as the communication speed of the communication line N decreases. On the other hand, in the monitoring system 1, the processing device 5 clusters the point cloud information Dm1, and when one or more clusters are obtained by the clustering, a geometric shape approximating each of the one or more clusters is generated. Partial information Dp1 including identifiable parameters is generated. Therefore, the data amount of the partial information Dp1 can be smaller than the data amount of the point cloud information Dm1. As a result, the time required for transmitting the partial information Dp1 from the processing device 5 to the detection device 8 can be shortened compared to the case where the point cloud information Dm1 is transmitted from the processing device 5 to the detection device 8 . As a result, it is possible to suppress an increase in the time required for detection while improving the detection accuracy.

Thus, in the monitoring system 1, the time required for transmitting the partial information Dp1 is shortened, so it is possible to monitor the wide intersection C in real time while reducing the number of laser sensors. In addition, since the amount of communication data between the slave laser radar device 2 and the master laser radar device 3 is reduced, even if the communication speed of the communication line N is slow (the communication band is narrow), the increase in the time required for detection can be avoided. can be suppressed.

Similarly, when the processing device 7 clusters the point group information Dm2, and one or more clusters are obtained by the clustering, a portion containing parameters capable of specifying a geometric shape approximating each of the one or more clusters Generate information Dp2. Therefore, the data amount of the partial information Dp2 can be smaller than the data amount of the point cloud information Dm2. As a result, the time required for transmitting the partial information Dp2 from the processing device 7 to the detection device 8 can be shortened compared to the case where the point cloud information Dm2 is transmitted from the processing device 7 to the detection device 8 . As a result, it is possible to suppress an increase in the time required for detection while improving the detection accuracy.

Also, in a configuration in which the slave laser radar device 2 transmits point cloud information to the master laser radar device 3, the master laser radar device 3 (detection device 8) needs to cluster all the point cloud information. Therefore, as the number of installed slave laser radar devices 2 increases, the calculation load on the master laser radar device 3 (detection device 8) increases. On the other hand, in the monitoring system 1, the processing devices 5 and 7 perform object detection processing, so the detection device 8 does not need to perform clustering. Therefore, the calculation load of the detection device 8 can be reduced. That is, even if the number of installed slave laser radar devices 2 increases, it is possible to suppress an increase in the calculation load of the master laser radar device 3 (detection device 8).

The laser sensor 4 and the laser sensor 6 may irradiate the same object with laser light and generate measurement point information for the same object. Therefore, when the geometric shape specified by the parameters included in the partial information Dp1 and the geometric shape specified by the parameters included in the partial information Dp2 overlap each other, these two geometric shapes It is highly likely that they are the same object. In such a case, if these two geometric shapes are detected as separate objects, there is a risk of erroneous detection. Therefore, by detecting these two geometric shapes as the same object, it is possible to further improve the detection accuracy.

The processing device 5 excludes the measurement point information of the measurement points included in the detection exclusion range from the point cloud information Dm1, and clusters the remaining measurement point information. Therefore, the number of pieces of measurement point information to be clustered can be reduced, so that the calculation load of the processing device 5 can be reduced. Similarly, the processing device 7 excludes the measurement point information of the measurement points included in the detection exclusion range from the point cloud information Dm2, and clusters the remaining measurement point information. Therefore, the number of pieces of measurement point information to be clustered can be reduced, so that the calculation load of the processing device 7 can be reduced.

The monitoring system 1 has a laser sensor 4 and a laser sensor 6 . Therefore, since the point cloud information Dm1 and Dm2 are generated within the monitoring system 1, there is no need to acquire the point cloud information Dm1 and Dm2 from outside the monitoring system 1. FIG.

Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments.

For example, intersection C may be a T-junction. The place where the monitoring region Rd is set is not limited to an intersection, but may be a confluence point where two or more roads meet, a confluence point between an exit of a facility such as a building and a road, or a point in the middle of a road. The monitoring area Rd may be set on a general road or on an expressway. Also, the monitoring area Rd may be set at a railroad crossing, a factory, a work site, or the like.

The arrangement of the slave laser radar device 2 and the master laser radar device 3 is not limited to the arrangement shown in FIGS. The surveillance system 1 may comprise two or more slave laser radar devices 2 . Each slave laser radar device 2 may comprise a detection device 8 . For example, when the master laser radar device 3 fails, the detection process may be continued by stopping the master laser radar device 3 and allowing one of the slave laser radar devices 2 to operate as the master laser radar device.

In the above embodiment, each of the slave laser radar device 2 and the master laser radar device 3 is configured as one device, but may be configured as two or more devices. For example, the processing device 5 may be separate from the slave laser radar device 2 . The processing device 7 and the detection device 8 may be separate from the master laser radar device 3 .

Alternatively, the detection device 8 may be composed of a separate computer (information processing device) that does not have the function of a laser radar. In this case, a slave laser radar device 2 is provided instead of the master laser radar device 3 .

Although the slave laser radar device 2 and the master laser radar device 3 are fixed to the support member P in the above embodiment, the processing devices 5 and 7 and the detection device 8 may not be fixed to the support member P. .

The monitoring system 1 (slave laser radar device 2) does not have to include the laser sensor 4. FIG. In this case, the processing device 5 acquires the point cloud information Dm1 from the external laser sensor 4 . Similarly, the monitoring system 1 (master laser radar device 3) may not have the laser sensor 6. In this case, the processing device 7 acquires the point cloud information Dm2 from the external laser sensor 6 .

The processing device 5 may comprise a laser sensor 4 . The processing device 7 may comprise a laser sensor 6 .

As the laser sensors 4 and 6, a three-dimensional laser radar that can irradiate laser light in all directions may be used.

In the above embodiment, the laser sensor 4 outputs a plurality of pieces of measurement point information for one frame to the processing device 5 for each frame. may be output. The laser sensor 4 may sequentially output the measurement point information to the processing device 5 one by one. Similarly, the laser sensor 6 may output a plurality of measurement point information to the processing device 7 in units other than one frame. The laser sensor 6 may sequentially output the measurement point information to the processing device 7 one by one.

Although the processing device 5 transmits the partial information Dp1 for one frame to the detection device 8 in the above embodiment, the partial information Dp1 for two or more frames may be collectively transmitted to the detection device 8 . Similarly, in the above embodiment, the processing device 7 transmits the partial information Dp2 for one frame to the detection device 8, but the partial information Dp2 for two or more frames is collectively transmitted to the detection device 8. good too.

The master laser radar device 3 may use the detection results for further processing. For example, the master laser radar device 3 may use the detection results to monitor vehicles violating traffic.

For example, when the distance between two intersections is short, a master laser radar device 3 may be installed at each of the two intersections, and one slave laser radar device 2 may be installed at the midpoint of the road connecting the two intersections. . At this time, one slave laser radar device 2 may belong to two monitoring systems and transmit partial information Dp1 to the master laser radar device 3 of each monitoring system. In this example, the monitored regions Rd in the two monitoring systems are different from each other.

In the above embodiments, the processing devices 5 and 7 (processing units 53 and 73) use rectangles as geometric shapes for approximating clusters, but other figures may be used. For example, the processing devices 5 and 7 (processing units 53 and 73) may approximate clusters to ellipsoids, polygons, and the like. In addition, the processing devices 5 and 7 (processing units 53 and 73) project each measurement point information included in the cluster onto the xy plane, and use the three-dimensional measurement point information to obtain a three-dimensional geometric shape. (for example, a rectangular parallelepiped). In this case, examples of parameters that can specify the geometric shape include width W, length L, height H (length in the z-axis direction), center coordinates (x, y, z), and rotation in the xy plane. Angle θ can be mentioned.

In the above embodiment, the processing device 5 (processing unit 53) clusters the remaining measurement point information after excluding the measurement point information of the measurement points included in the detection exclusion range from the point cloud information Dm1. Dm1 may be clustered. Similarly, in the above embodiment, the processing device 7 (processing unit 73) clusters the remaining measurement point information after excluding the measurement point information of the measurement points included in the detection exclusion range from the point cloud information Dm2. The point group information Dm2 may be clustered.

1 monitoring system 4 laser sensor (first laser sensor)
5 processing device (first processing device)
6 laser sensor (second laser sensor)
7 processing device (second processing device)
8 detection devices 51, 71 acquisition units 53, 73 processing units 54, 74 output unit Dm1 point group information (first point group information)
Dm2 point cloud information (second point cloud information)
Dp1 partial information (first partial information)
Dp2 partial information (second partial information)
Ra Irradiable area Rb Irradiable area Rd Monitoring area

Claims (3)

Each measurement point is installed on the ground and irradiates a laser beam toward an irradiable area including at least a part of a monitoring area, and receives the reflected light of the laser beam to determine the position of each measurement point within the irradiable area. A monitoring system that monitors the monitoring area using a first laser sensor and a second laser sensor that generate measurement point information including coordinates,
a first processing device that generates first partial information by processing first point group information including a plurality of measurement point information generated by the first laser sensor;
a second processing device that generates second partial information by processing second point group information including a plurality of measurement point information generated by the second laser sensor;
a detection device that generates a detection result in the monitoring area based on the first partial information and the second partial information;
The first processing device clusters the first point group information, and when one or more first clusters are obtained by the clustering, identifies a geometric shape that approximates each of the one or more first clusters. generating the first part information including possible parameters ;
The second processing device clusters the second point group information, and when one or more second clusters are obtained by the clustering, identifies a geometric shape that approximates each of the one or more second clusters. generating the second part information including possible parameters;
When the first geometric shape specified by the parameters included in the first partial information and the second geometric shape specified by the parameters included in the second partial information overlap each other, the detection device detects the first geometric shape and the second geometric shape as the same object . 2. The first processing device according to claim 1 , wherein, from the first point group information, the measuring point information of measuring points included in a preset detection exclusion range is excluded, and the remaining measuring point information is clustered. Monitoring system. the first laser sensor;
the second laser sensor;
3. The monitoring system of claim 1 or claim 2 , further comprising:

Images