JP6139178B2 - Sensor integration apparatus and sensor integration method - Google Patents

Description

  The present invention relates to a sensor integration device.

  As a background art in this technical field, there is JP 2012-163495 A (Patent Document 1). In this publication, “a sensor integrated system comprising a processor, a storage device connected to the processor, and an output device connected to the processor, wherein the storage device was observed by a plurality of sensors. Holding observation data, wherein the observation data includes items selectively described among predetermined items, and the processor combines the items described by selection of a plurality of the observation data, and includes at least one target The movement trajectory of the object is estimated, and the output device outputs information indicating the movement trajectory. "

JP 2012-163495 A

  There is a need to monitor a wide area using sensors, such as monitoring important facilities, the environment or disaster areas, or search / rescue assistance.

  In wide-area monitoring, it is often necessary to use a combination of multiple sensors because a single sensor cannot cover the area of interest. As sensors, not only fixed radars and surveillance cameras installed on the ground, but also sensors mounted on various mobile sensor platforms such as vehicles, flying objects, and water and underwater moving objects are becoming available. If these different types of sensors (heterogeneous sensors) can be used in combination, a wide area can be monitored efficiently.

  For example, Patent Document 1 discloses a target tracking method using a plurality of different types of sensors. However, a method for estimating a movement trajectory when the observation ranges of the sensors do not overlap is not disclosed. In addition, although a method for generating and tracking wake hypotheses using multiple target models is disclosed, characteristics (attribute information) as a group other than position, such as the number of targets or the type of target included in the group, are considered. This tracking method is not disclosed.

  The present invention has been made in view of the above-described problems, and realizes estimation of the movement trajectory of a target that goes inside and outside the sensor observation range by combining observation data and map information in a plurality of sensor observation ranges. For the purpose.

In order to solve the above-described problem, the present invention provides a sensor integration device including a processor, a storage device connected to the processor, and an output device connected to the processor, and the storage device includes: It holds sensor information, target information, and map information, and the sensor information includes information indicating observation ranges of a plurality of sensors, and the target information is a plurality of observation ranges observed by the plurality of sensors. Position information for each time of a plurality of individual targets, information for each time of a group target composed of the plurality of individual targets, and information indicating attributes other than the position information of each group target And the information indicating the attribute of each group target includes information indicating one or more types of the attribute and information indicating the attribute value of each type, and each group target One Or more types of attributes, the at least one of said individual target number and positional relationships in each population target, and the type of at least one of said individual target included in each population target, size, The map information includes at least one of a color and a shape, and the map information includes road network information of a region included in each observation range and a region not included in any of the observation ranges adjacent to each observation range. The processor is a point where each collective target that has passed through each observation range and moved to an area not included in any of the observation ranges based on the target information has exited from each observation range. Based on the first procedure for specifying the passage point, the passage time and the passage speed of the passage point, and the target information, any one of the observation ranges from an area not included in any of the observation ranges. Point invasion population target penetrated into a second procedure for specifying the penetration time and penetration rate, based on said road network information, entry of a population target mentioned above penetrate the passing point of each population target that the passage A third procedure for specifying a route distance through the road to the point, at least one of a passing speed of each of the group targets that have passed and an intrusion speed of the group target that has entered, and each of the group targets that have passed Based on the path distance from the passing point to the intruding point of the invading collective target, the travel time of the path distance from the passing point of each passing group target to the intruding point of the intruding collective target is calculated. The fourth procedure to be estimated, the length of the travel time estimated based on the speed and the route distance, and the passing time of each passing group target specified based on the target information A fifth procedure for evaluating the identity of each of the group targets that have passed and the group target that has entered by comparing the time from the time to the time of entry of the group target that has entered , And the output device executes a sixth procedure for outputting the result of the identity evaluation, and the processor, in the fifth procedure, has intruded with the attribute of each group target passed through. When the similarity with the attribute of the collective target does not satisfy a predetermined condition, it is determined that the passed collective target and the intruded collective target are not the same, and in the fifth procedure, the target information The length of time from the passage time specified based on the time to the intrusion time is included in a range of a predetermined time length including the length of travel time estimated based on the speed and path distance. If not, said passing And determining that each of the collective targets is not the same as the intruded collective target, and wherein the output device determines, in the sixth procedure, that the at least one of the intruded collective targets is the same as the intruded collective target. Output identification information of the collective target that has passed, and the storage device includes end point information about an end point where a boundary between the plurality of observation ranges and a road intersect, and end-point network information about a road network between the end points. The end point information includes information indicating a position of each end point, and the end point network information is generated based on the end point information and the map information, and the end point information and any of the observations. Information relating to other end points connected via roads in areas not included in the range, and not included in any observation range between the related end points Including information indicating a route distance passing through a road in the area, wherein the processor refers to the network information between the end points, and the processor is associated with one or more end points corresponding to the passing points of the collective targets that have passed. A tenth step of identifying end points, and adding information that associates the identified one or more end points with each of the group objects that have passed to the end point information; and referring to the end point information, the intruded population An eleventh procedure for identifying each passed collective target associated with an endpoint corresponding to an entry point of the target, and each passed collective target identified in the eleventh procedure; The third procedure to the fifth procedure are executed for the invading collective target, and in the third procedure, the processor corresponds to a passing point of each collective target that has passed. The path distance between the end point to be intruded and the end point corresponding to the intrusion point of the invading collective target is acquired from the network information between end points .

  According to an embodiment of the present invention, it is possible to track a target in a wide area that extends in and out of the sensor observation range, so that it is possible to efficiently and inexpensively grasp a wide-area situation with a small number of sensors.

It is a functional block diagram which shows the structure of the sensor integrated system of embodiment of this invention. It is a block diagram which shows the hardware constitutions of the sensor integrated system of embodiment of this invention. It is a block diagram which shows the modification of the hardware constitutions of the sensor integrated system of embodiment of this invention. It is explanatory drawing which shows the outline | summary of the sensor integration process of the sensor integration system of embodiment of this invention. It is explanatory drawing of the separate target object information table contained in target object information DB of the sensor integrated apparatus of embodiment of this invention. It is explanatory drawing of the group target object information table contained in target object information DB of the sensor integrated apparatus of embodiment of this invention. It is explanatory drawing of the sensor information table contained in sensor information DB of the sensor integrated apparatus of embodiment of this invention. It is explanatory drawing of the endpoint information table contained in the endpoint information DB of the sensor integrated apparatus of embodiment of this invention. It is explanatory drawing of the network table between end points contained in the network DB between end points of the sensor integrated apparatus of embodiment of this invention. It is a flowchart which shows the sensor pre-processing which the sensor integration apparatus of embodiment of this invention performs. It is a flowchart which shows the sensor integration pre-processing which the sensor integration apparatus of embodiment of this invention performs. It is a flowchart which shows the sensor observation range target passage process which the sensor integrated apparatus of embodiment of this invention performs. It is a flowchart which shows the sensor observation range target penetration | invasion process which the sensor integrated apparatus of embodiment of this invention performs. It is a flowchart which shows the target collation process which the sensor integrated apparatus of embodiment of this invention performs. It is a figure which shows the modification of the sensor integration process which the sensor integration apparatus of embodiment of this invention performs. It is explanatory drawing of the information output by the sensor integration apparatus of embodiment of this invention, and a user interface. It is explanatory drawing of the information output by the sensor integrated apparatus of embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  In this embodiment, the movement trajectory of the target is expressed as “wake”, but the target targeted by the present invention is any object such as a vehicle, a person, or a ship as long as it is a moving object. Also good. The sensors targeted by the present invention include, for example, radar, camera (fixed type for fixed point monitoring, mobile platform mounted type, portable type, etc.), acoustic sensor, vibration sensor, counter, person (visual report, social network service) Any sensor that can capture the presence or movement of a target, such as registered information, etc., may be used.

  FIG. 1 is a functional block diagram showing a configuration of a sensor integrated system according to an embodiment of the present invention. The sensor integration system according to the present embodiment includes a sensor integration device 100, a plurality of sensors, and a network 160 that connects the sensor integration device 100 and the plurality of sensors. The sensor integration system according to the present embodiment may further include one or more other sensor integration devices 180 connected to the network 160. The sensor integration device 100 includes an input / output device 190.

  The plurality of sensors may be any sensor that can acquire information on the target. Here, the information regarding the target is, for example, information indicating the position of the target or information indicating the attribute. Although FIG. 1 shows two sensors, that is, a sensor 1_151 and a sensor 2_152, the sensor integrated system of this embodiment may include more sensors. The plurality of sensors may include only one type of sensor, but may include a plurality of types of sensors. For example, the sensor 1_151 may be a radar and the sensor 2_152 may be a camera.

  The sensors 1_151 and 2_152 and the like transmit the sensor information 171 and the sensor information 172 including the observation data acquired by sensing to the sensor integration device 100 via the network 160. The sensor information 171 and the like of the present embodiment includes sensor attributes, data, and data attributes. The sensor attribute may be included in the type of sensor that transmitted the sensor information 171 or the like (for example, information indicating whether it is a radar or a camera), an identifier (ID), and observation data of the sensor. It includes sensor specifications such as certain items (time, position, speed, acceleration, and track). Furthermore, information such as sensor observation range and acquirable attributes may be included. The data includes observation data acquired by each sensor, the time when it was acquired, and the like. The data attribute may include information indicating the attribute of the observation data, for example, the accuracy of the observation data. The sensor integration device 100 may acquire the sensor information 171 and the like for each step or may acquire it by streaming.

  Each sensor may be fixedly connected to the sensor integration apparatus 100, but addition of a new sensor and deletion of an existing sensor may be permitted. A new sensor may be registered in the sensor integration device 100 based on a request from the added sensor (that is, PUSH registration), or the sensor integration device 100 may search for a sensor and register the searched sensor. Good (ie PULL registration).

  In the following description, the range in which each sensor can observe the target is described as the observation range of each sensor. In addition, as will be described later with reference to FIG. 4, in this embodiment, the observation ranges of a plurality of sensors may not overlap or be adjacent, and thus there may be a region that is not included in any of the observation ranges. . Thus, a region not included in any observation range is described as a region outside the observation range.

  The sensor integration device 100 includes an input / output unit 101, a target information processing unit 102, a sensor information processing unit 103, a map information processing unit 104, a geographic information system 105, and a database 106.

  The sensor information processing unit 103 performs processing for handling attributes of various sensors such as the sensor 1_151 and sensor observation data obtained by the sensors. Specifically, the sensor information processing unit 103 is a wake extraction processing unit 103A that estimates the movement trajectory of the target from the sensor observation data by correlation processing or the like, and if the observation range is not included in the sensor attribute, The observation range specifying processing unit 103B to be specified is provided.

  The target information processing unit 102 manages a target that spans a plurality of sensor observation ranges, the target management unit 102A, “passing” the target out of the sensor observation range, and the target entering the sensor observation range “ An intrusion / passage determination unit 102B that determines “intrusion” determines attributes of a group target obtained by grouping individual targets, and information on a plurality of group targets observed in different observation ranges is related to the same group. A group attribute matching unit 102C that identifies whether or not the information is information, a track integration processing unit 102D that integrates a plurality of tracks associated with a plurality of group targets determined to be the same group, and the like.

  The map information processing unit 104 uses the geographic information system 105 to manage terrain and local conditions, manage route information such as road maps, and end point information (including a passing / intruding target list associated with end points) described later. Realize management.

  The geographic information system 105 includes a map information management function and search function, a spatial analysis function (network analysis, perspective analysis, etc.), and an editing function. Since the geographic information system 105 can be realized by a known technique, a detailed description thereof will be omitted.

  The database 106 includes a sensor information database (DB) 106A (including sensor attributes and sensor observation data), a target information DB 106B (including a track), a map information DB 106C, a processing parameter DB 106D, an end point information DB 106E, an end point network DB 106F, and the like. Including. The sensor information DB 106A includes information on sensor attributes, sensor observation data, and the like. The target information DB 106B includes information related to the attributes of the target, track data of the target, and the like. The target information DB 106B includes a DB related to individual targets and a DB related to a group of targets each including a plurality of individual targets. The end point information DB 106E includes information on the position of an end point (described later) of the observation range of each sensor and a target that enters and exits the observation range from the end point. The inter-endpoint network DB 106F includes information on a route from the end point to another end point via a region outside the observation range. Details of these DBs will be described later (see FIGS. 5A to 5E and the like). The map information DB 106C includes map information of a region including a plurality of observation ranges and at least a region outside the observation range connecting between the observation ranges. Specifically, the map information DB 106C stores road network data indicating, for example, road attributes such as connection relations between nodes corresponding to intersections and links corresponding to roads connecting the intersections, and whether or not roads are allowed to pass. May be included. Using this road network data, it is possible to search for a moving route of a target consisting of a road that can be passed. Alternatively, the map information DB 106C may include terrain information regarding altitude, rivers, coastlines, and the like instead of (or in addition to) the road network data. Since a well-known map information DB 106C can be used, detailed description is omitted. The processing parameter DB 106D holds various parameters used for processing executed by the target information processing unit 102, the sensor information processing unit 103, and the like. For example, parameters such as α and β described later with reference to FIG. 6E may be held in the processing parameter DB 106D.

  The input / output unit 101 includes, for example, sensor information 171 received from the sensor 1_151 or the like via the network 160 by the sensor integration device 100, target information 181 received from the other sensor integration device 180 via the network 160 by the sensor integration device 100, And the information etc. which the user of the sensor integration apparatus input using the input / output device 190 are stored in the database 106. Further, the input / output unit 101 includes, for example, GUI (Graphical User Interface) image information for assisting the user in inputting information and image information for presenting the processing result by the sensor integration device 100 to the user. 190 is output.

  Since the other sensor integration device 180 may be the same as the sensor integration device 100, a detailed description thereof will be omitted. Although omitted in FIG. 1, the other sensor integration device 180 can also acquire sensor information from one or more sensors and perform the same processing as the sensor integration device 100 based on the sensor information. For example, the other sensor integration device 180 may generate the same information as the target information DB 106B by the same processing as the sensor integration device 100 and transmit the same information as the target information 181 to the sensor integration device 100. Further, the target information and sensor information may be integrated.

  FIG. 2 is a block diagram showing a hardware configuration of the sensor integrated system according to the embodiment of the present invention. The sensor integration device 100 of this embodiment includes an interface (I / F) 201, a processor 202, a storage device 203, an input device 204, and an output device 205. These are connected to each other via a communication line (for example, a bus) 206.

  The I / F 201 is connected to the network 160, receives the sensor information 171 and the like from the sensor 1_151 and the like, and receives the target information 181 from the other sensor integration device 180 via the network 160. The network 160 may be anything, such as an IP network or other types of networks. 1 shows an example in which sensor information and the like (that is, sensor information 171 and 172 and target object information 181) are input to the sensor integration apparatus 100 via the network 160. It may be input to the sensor integration device 100 without going through. For example, the sensor 1_151 or the like may be connected to the sensor integration device 100 via a cable such as a USB (Universal Serial Bus), and sensor information or the like may be input to the sensor integration device 100 via the cable. In this case, the I / F 201 is a USB interface. Alternatively, the sensor 1_151 or the like may store sensor information or the like in a portable storage medium (for example, a portable flash memory or storage disk), and sensor information or the like may be input to the sensor integration device 100 from the storage medium. In this case, the I / F 201 is a storage medium reading device. Alternatively, the sensor 1_151 or the like may be directly connected to the sensor integration device 100.

  The processor 202 realizes various functions of the sensor integration device 100 by controlling each unit of the sensor integration device 100. For example, the processor 202 controls the I / F 201, the input device 204, and the output device 205 in accordance with a program stored in the storage device 203, whereby the input / output unit 101, target information processing unit 102, sensor information in FIG. The functions of the processing unit 103, the map information processing unit 104, and the geographic information system 105 may be realized. In that case, a general PC (Personal Computer) may be used as the hardware shown in FIG. Alternatively, the processor 202 may not be a general-purpose processor as described above, but may be one or more dedicated processors that realize the functions of the above-described units.

  The storage device 203 stores a program executed by the processor 202 and data referred to by the processor 202. For example, the database 106 illustrated in FIG. 1 is stored in the storage device 203. The storage device 203 is, for example, a volatile semiconductor memory such as a DRAM (Dynamic Random Access Memory), a nonvolatile semiconductor memory such as a flash memory, a disk device such as a hard disk drive, or a combination thereof. May be.

  The input device 204 is used for receiving an instruction from the user, and may include, for example, a keyboard and a mouse. If the sensor integration device 100 does not need to receive an instruction from the user, the input device 204 may be omitted. The output device 205 outputs a processing result of the sensor integration device 100 (for example, an extracted wake). Further, the output device 205 may output GUI image information or the like for assisting the user in inputting information. For example, the output device 205 may be an image display device such as a liquid crystal display device. The input device 204 and the output device 205 in FIG. 2 correspond to the input / output device 190 in FIG. Note that the input / output unit 101 illustrated in FIG. 1 may output a wake or the like via an API (Application Program Interface), a Web service, or a storage medium. In that case, the output device 205 may be a network interface or a writing device to a storage medium.

  FIG. 3 is a block diagram showing a modification of the hardware configuration of the sensor integrated system according to the embodiment of the present invention. The sensor integration system includes a plurality of sensor integration devices 100 connected to a network. Here, sensor integration apparatuses 100 </ b> A to 100 </ b> C are shown as examples of the plurality of sensor integration apparatuses 100. The hardware configuration of each of the sensor integration devices 100A to 100C is the same as the configuration of the sensor integration device 100 illustrated in FIG. Also, each of the sensor integration devices 100A to 100C illustrated in FIG. 3 corresponds to the sensor integration device 100 or the other sensor integration device 180 illustrated in FIG.

  For example, the sensor integration device 100A extracts the track of the target within the observation range of the sensor 1_151 based on the sensor information 171 acquired from the sensor 1_151, and uses the target information 181 including the extracted track data as the sensor integration device 100C. Send to. Similarly, the sensor integration device 100 extracts the track of the target within the observation range of the sensor 2_152 based on the sensor information 172 acquired from the sensor 2_152, and uses the target information 181 including the extracted track data as the sensor integration device. Send to 100C. Although omitted in FIG. 3, another sensor integration device 100 may similarly transmit the target information to the sensor integration device 100 </ b> C. The sensor integration device 100C integrates wakes within a plurality of observation ranges based on the target information 181 received from the sensor integration devices 100A and 100B.

  As described above, when extraction of wakes in each observation range and integration of wakes in a plurality of observation ranges are performed by different sensor integration devices 100, each sensor integration device 100 is shown in FIG. It may not be necessary to have all the functions shown. For example, the sensor integration apparatuses 100A and 100B do not perform wake integration, and therefore do not need to include at least the wake integration processing unit 102D. The sensor integrated device 100C does not normally extract a wake within each observation range, and therefore does not need to include at least the wake extraction processing unit 103A. However, the sensor integrated device 100C may include the wake extraction processing unit 103A because there is a possibility that the wake extraction in each observation range may actually be performed again.

  Further, the sensor integration system shown in FIG. 3 includes a group of sensor integration devices 100 similar to the above-described sensor integration devices 100A to 100C (that is, a plurality of sensor integration devices each extracting a wake within the observation range of one sensor. 100 and one sensor integration device 100 that integrates tracks in a plurality of observation ranges based on target information from the plurality of sensor integration devices 100. In that case, the sensor integration system collects target information from a plurality of sensor integration devices 100 that integrate wakes in a plurality of observation ranges, and integrates the sensor integration device 100 having a hierarchical structure that integrates wakes in all observation ranges. Also have.

  For example, a large number of sensors are installed in a wide area, and each sensor is provided with one sensor integration device 100 that extracts a wake in the observation range of the sensor, and integrates wakes in a plurality of observation ranges in each partial region. A sensor integration device 100 may be provided, and further, a sensor integration device 100 that integrates wakes within the entire observation range of all partial areas in a wide area may be provided.

  Further, in FIG. 3, the sensor integration devices 100 </ b> A and 100 </ b> B each process sensor information acquired from one sensor, and the sensor integration device 100 </ b> C processes target information acquired from the plurality of sensor integration devices 100. Although the embodiment is shown, the sensor integration devices 100A and 100B and the like extract the wakes in the plurality of observation ranges based on the sensor information acquired from the plurality of sensors, respectively, and the targets acquired from the plurality of sensor integration devices 100 Based on the object information, the sensor integration device 100C may integrate the wake.

  By adopting the above-described hierarchical structure or autonomous distributed type functional configuration, the sensor integration system of the present embodiment is a centralized management in which one sensor integration device 100 processes information from all sensors (see FIG. 2). The present invention can be realized by adopting any form of distributed management (see FIG. 3) in which each of the plurality of sensor integration devices 100 processes information from one or more sensors.

  Hereinafter, the information held by the sensor integration device 100 shown in FIG. 1 and FIG. 2 and the processing to be executed will be described. However, the information held by the plurality of sensor integration devices 100 shown in FIG. It is the same as that described below, except that it is distributed and held in the sensor integration device 100 and processing is executed in a distributed manner by a plurality of sensor integration devices 100.

  FIG. 4 is an explanatory diagram illustrating an overview of sensor integration processing of the sensor integration system according to the embodiment of this invention. As an example, the observation range 401 of the sensor 1_151 and the observation range 402 of the sensor 2_152 are shown. These observation ranges are neither adjacent nor overlapping, and a target that moves from one observation range to the other observation range always passes the region outside the observation range. The targets 411A to 411C observed in the observation range 401 by the sensor 1_151 and the targets 412A to 412C observed in the observation range 402 by the sensor 2_152 are shown. In the example of FIG. 4, these targets are vehicles that move along the road 420. Hereinafter, in the description of the present embodiment, a moving body that travels or walks on the ground like a vehicle or a person is cited as an example of the target. However, the present embodiment can also be applied to an arbitrary moving body other than a vehicle or a person (for example, a ship or an aircraft).

  The track extraction processing unit 103A of the sensor integrated device 100 extracts tracks in the observation range 401 of the targets 411A to 411C. Further, the wake extraction processing unit 103 </ b> A may recognize a group including the targets 411 </ b> A to 411 </ b> C as one target 411 and extract a wake in the observation range 401 of the target 411. Hereinafter, a target including a plurality of targets may be referred to as a collective target, and a target included in the collective target may be referred to as an individual target. For example, the individual target is a single vehicle, and the collective target is a train of vehicles.

  Similarly, the wake extraction processing unit 103A extracts a wake in the observation range 402 of the individual targets 412A to 412C, and further extracts a wake in the observation range 402 of the collective target 412 including the individual targets 412A to 412C. May be.

  The wake extraction processing unit 103A can extract the wake of the target within each observation range using any known method (for example, the method described in Patent Document 1). Description is omitted.

  If there is a path through which the target can move between the observation ranges 401 and 402, for example, the target observed by the sensor 1_151 in the observation range 401 will move to the observation range 402 and a different time from that. It may be observed in the other observation range. In the example of FIG. 4, since there is a road 420 connecting the observation ranges 401 and 402, for example, the target 411A moves from the observation range 401 to the observation range 402 through the road 420 and is observed as the target 412A or the like. there is a possibility. In such a case, the wake in the observation range 401 such as the target 411A and the wake in the observation range 402 of the target 412A are integrated and handled as the wake of the same target, thereby providing a plurality of sensors. It is possible to track a target that moves in a region that extends within and outside the observation range.

  However, the track of the target in the region outside the observation range between the observation ranges 401 and 402 cannot be extracted with high accuracy even if the same method as the method for extracting the track in each observation range is used. In this embodiment, a plurality of observation ranges are associated with each other based on map information (stored in the map information DB 106C) and out of the observation range from the observation range of a certain sensor (hereinafter referred to as “pass the observation range”). The target is also referred to as a “target” and a target that has entered the observation range of another sensor. This makes it possible to estimate a wake obtained by integrating the wakes of the target in different observation ranges.

  In particular, when the distance between the sensor observation ranges is large and the area outside the observation range is wide, it is often difficult to integrate the tracks of the individual targets. On the other hand, a plurality of individual targets are treated as one collective target, and a large flow as a collective target, and attribute information of the individual target included in the collective target (number of individual targets, individual targets of characteristic types) It is possible to integrate wakes by determining the presence or absence of an object.

  Here, the map information may be network data such as a road network, vector data such as road shapes or contour lines, and raster data such as land use or topography. The road network data may be information related to a node corresponding to an intersection and a link corresponding to a road connecting between the intersections, as is generally used. In addition to the distance information that can be used for estimating the traffic, it may include traffic regulation information that can be used to determine ease of travel.

  In the following description, a point on the boundary of each observation range that passes when the target enters and exits each observation range is referred to as an “end point”. Not only the point where the target is actually estimated to pass but also the point where the target is expected to pass may be treated as the end point. Specifically, for example, when road network information is included in the map information DB 106C, a point where a road intersects with the boundary of each observation range is an end point.

  In the example of FIG. 4, end points 430A and 430B are points where the road 420 and the boundary of the observation range 401 intersect, and end points 430C and 430D are points where the road 420 and the boundary of the observation range 402 intersect. For example, a target observed as the target 411 in the observation range 401 travels on the road 420 and passes through the end point 430B, and further travels on the road 420 and enters the observation range 402 from the end point 430C to be observed as the target 412. May be. Since actual roads have a width, the end point where the road and the observation range cross each other is generally not a “point” according to a strict geometric definition, but a range with a certain size. is there. For example, when the target travels on a portion other than the road, or when the map information DB 106C does not include the road network information of the area, the position of the end point is estimated based on the track of the observed target. May be.

  The generation of one collective target from a plurality of individual targets (in other words, determining which multiple individual targets are handled as one collective target) is an arbitrary method including a known method. Therefore, detailed description is omitted. For example, a plurality of individual targets classified into one cluster by performing known clustering may be handled as one collective target, or a predetermined template and a plurality of individual targets as described in Patent Document 1. A collective target may be generated based on a result of comparison with an object, or any other method of managing other individual targets and collective targets in association with each other may be used.

  Next, the contents of the database 106 held by the sensor integration device 100 will be described.

  The target object information DB 106B of the present embodiment includes an individual target information table 500 that holds information about the track and attributes of individual targets observed by each sensor, and a group target that holds information about the tracks and attributes of the group target. An object information table 510. The former will be described with reference to FIG. 5A, and the latter will be described later with reference to FIG. 5B.

  FIG. 5A is an explanatory diagram of the individual target information table 500 included in the target information DB 106B of the sensor integration device 100 according to the embodiment of this invention. The individual target information table 500 includes information regarding one or more attributes other than the individual target ID 501, the position 502, and the position of each individual target. Information on one or more attributes of each individual target is information that associates one or more attribute types with attribute values of those types. In the example of FIG. 5A, attribute 1 (full length) 503, attribute 2 ( Type) 504 and attribute 3 (color) 505.

  The individual target ID 501 is an identifier of each individual target. For example, as shown in FIG. 4, when the individual targets 411A to 411C are observed in the observation range 401 and the individual targets 412A to 412C are observed in the observation range 402, unique identifiers are assigned to these six individual targets. Given and held as an individual target ID 501. The individual target ID may be an ID system associated with the observed sensor.

A position 502 is a coordinate value indicating the position of each individual target observed by each sensor at each time. For example, as a position 502, a coordinate value p _{t at} a time _t , a coordinate value p _{t-1 at a} time t- _{1, a} coordinate value p _{t-2 at} a time _t-2 , and so on at another time. Coordinate values are retained. These coordinate values for each time correspond to the track of each individual target.

  The attribute 1 (full length) 503 is an example of information indicating the size of each individual target, and may be a value such as “aa meters” or a width such as “bb meters to cc meters”. It may be a value with Alternatively, it may be a value having accuracy information such as “aa meter (standard deviation dd)”.

  The attribute 2 (type) 504 is information indicating the type of each individual target, for example, which type of each individual target is a moving object such as a vehicle, a ship, or an aircraft. In the present embodiment, as an example, a target traveling on land such as a vehicle is handled. Therefore, attribute 2 (type) 504 indicates whether each individual target is a vehicle or a pedestrian, or a vehicle. It may be information indicating whether the vehicle is of a kind.

  The attribute 3 (color) 505 is information indicating the color of each individual target.

  The individual target information table 500 may further include information regarding attributes other than those described above (for example, the shape of each individual target).

  As will be described later, the type of attribute information that can be acquired may differ depending on the type of the sensor 1_151 that observes the target. For example, when the sensor 1_151 or the like is a camera that can capture a color image, the color of the individual target can be specified from the captured image, but when the sensor 1_151 or the like is a radar, the individual target is obtained from the acquired information. Unable to determine the color of the. In such a case, the corresponding item (for example, attribute 3 (color) 505) in the individual target information table 500 is empty. Alternatively, the attribute item portion may be made into a sub-table, or a combination of the attribute item name and its value may be managed.

  FIG. 5B is an explanatory diagram of the collective target information table 510 included in the target information DB 106B of the sensor integration device 100 according to the embodiment of this invention. The collective target information table 510 includes collective target ID 511, position 512, individual target list 513, and information regarding one or more attributes other than the position of each collective target. Information on one or more attributes of each individual target is information that associates one or more attribute types with attribute values of those types. In the example of FIG. 5B, attribute 1 (number) 514 and attribute 2 ( Position pattern) 515.

  The collective target ID 511 is an identifier of each collective target. For example, as shown in FIG. 4, when a collective target 411 is observed in the observation range 401 and a collective target 412 is observed in the observation range 402, a unique identifier is given to these two individual targets, It is held as the target ID 511.

  The position 512 is a coordinate value indicating the position of each collective target at each time observed by each sensor, and may be held as data in the same format as the position 502 related to the individual target. These coordinate values for each time correspond to the track of each collective target.

  The individual target list 513 is a list of identifiers of individual targets included in each collective target. The identifier held here corresponds to the identifier held as the individual target ID 501.

  An attribute 1 (number) 514 indicates the number of individual targets included in each collective target. For example, the attribute 1 (number) 514 may be a value such as “cc” or a value having a width such as “aa to bb”. Alternatively, it may be a value having accuracy information such as “aa (standard deviation cc)”.

  An attribute 2 (position pattern) 515 indicates a relative positional relationship (that is, arrangement) of a plurality of individual targets included in each collective target.

  The collective target information table 510 may further include information on attributes other than those described above for each collective target. Alternatively, the attribute item portion may be made into a sub-table, or a combination of the attribute item name and its value may be managed.

  By searching the individual target information table 500 using the value of the individual target list 513 of each group target as a key, attributes such as the total length and type of the individual target included in each group target can be acquired. it can. The attributes of the individual target included in each group target can be handled as part of the attributes of each group target and used for collation determination of the group target.

  FIG. 5C is an explanatory diagram of the sensor information table 520 included in the sensor information DB 106A of the sensor integration device 100 according to the embodiment of this invention. The sensor information table 520 includes information regarding each sensor (for example, sensor 1_151 and sensor 2_152). In the example of FIG. 5C, the sensor information table 520 includes a sensor ID 521, a sensor type 522, an observation range 523, an end point list 524, an acquisition attribute 525, and the like.

  The sensor ID 521 is an identifier of each sensor. The sensor type 522 is information indicating the type of each sensor, for example, whether each sensor is a radar, a camera, or another type of sensor. The sensor type 522 is used for switching various processes and adjusting parameters.

  The observation range 523 is information indicating the position, size, shape, and the like of the observation range of each sensor, and may include, for example, the coordinate value of the boundary line (that is, the contour line) of the observation range of each sensor. When the information regarding the observation range is not included in the sensor attribute information, the observation range identification processing unit 103B may calculate and generate the information.

  The endpoint list 524 is a list of identifiers of endpoints of the observation range of each sensor. For example, the end point list 524 regarding the observation range 401 of the sensor 1_151 includes the identifiers of the end points 430A and 430B. Note that the identifier of the end point used here may not be unique within a plurality of observation ranges as long as it is unique within each observation range. In that case, in order to uniquely identify each end point within the observation range of each sensor within a plurality of observation ranges, it is necessary to use an identifier that combines the identifier of each sensor and the identifier of each end point. The endpoint identifiers used in the endpoint information table 530 and the endpoint network table 540, which will be described later, are information for identifying each endpoint within a plurality of observation ranges as described above. Alternatively, the end point list 524 may not be held in the sensor information table 520 but may be managed on the end point information table 530 side, and the end point information table 530 may be searched and used as necessary.

  The acquisition attribute 525 indicates attribute information acquired by each sensor. For example, when the sensor is a radar, information about the position of the target can be acquired, but information about the color cannot be acquired. In this case, “position” is retained as the acquisition attribute 525 related to the radar (if another attribute information can be acquired, that attribute is also retained).

  Although omitted in FIG. 5C, the sensor information table 520 further includes information such as observation data acquired by each sensor and the time when the sensor was acquired. Alternatively, it may be stored in another table and managed in association.

  FIG. 5D is an explanatory diagram of the endpoint information table 530 included in the endpoint information DB 106E of the sensor integration device 100 according to the embodiment of this invention. The end point information table 530 includes information on each end point (for example, end points 430A to 430D). In the example of FIG. 5D, the end point information table 530 includes an end point ID 531, a position 532, a passing target list 533, an intrusion target list 534, an intrusion prediction list 535, and the like.

  The end point ID 531 and the position 532 are an identifier of each end point and a coordinate value indicating the position of each end point. The passing target list 533 is a list of identifiers of the targets that have passed through the end points and have come out of the observation range. The intrusion target list 534 is a list of identifiers of targets that have entered the observation range from each end point.

  The intrusion prediction list 535 is a list of identifiers of target objects that are predicted to possibly enter the observation range from each end point. Prediction of the possibility of intrusion is performed based on association between end points by an end point network table 540 described later.

  As shown in FIG. 5D, the end point information table 530 may calculate and generate the intrusion prediction list 535 in advance and include it in the table. However, the end point information table 530 may be included in the table by searching the passing target list 533 as necessary. You may identify a target that has sex. A method of generating and using such information will be described later with reference to FIGS. 6C and 6D.

  As shown in FIG. 5D, each end point is managed in association with a target that has passed through each end point, a target that has entered from each end point, a target that is predicted to enter, and the like. Simplification and speeding up of processing in processing and the like can be realized.

  FIG. 5E is an explanatory diagram of the end-to-end network table 540 included in the end-to-end network DB 106F of the sensor integration device 100 according to the embodiment of this invention. The end-to-end point network table 540 includes information that associates two end points where the target can move between the regions outside the observation range and information about the route that passes through the region outside the observation range. In the example of FIG. 5E, the inter-endpoint network table 540 includes an end point start point 541, an end point end point 542, a distance 543, a related map graphic ID 544, a virtual flag 545, and the like.

  The end point start point 541 and the end point end point 542 are identifiers of the end point and the end point that are the start point and the end point of the route in which the target in the region outside the observation range can move, and the distance 543 is the route distance of the route, and the related map The graphic ID 544 is a graphic identifier indicating the shape of the route. For example, as shown in FIG. 4, when the target 411 or the like can move on the road 420 outside the observation range from the end points 430B to 430C, the identifiers of the end points 430B and 430C are the end point start point 541 and the end point end point 542. The route distance of the section from the end points 430B to 430C may be held as the distance 543, and the graphic identifier indicating the shape of the section may be held as the related map graphic ID 544.

  In FIG. 4, only a simple road 420 is shown to simplify the description, but actually, for example, a plurality of routes may exist between the end points 430B and 430C. In this case, for example, the map information processing unit 104 refers to the road network data included in the map information DB 106C to search for a route that can be passed between the end points 430B and 430C, and the route distance of the route having the shortest route distance and The identifier of the map figure used for the calculation of the route distance may be registered in the end-to-end network table 540 as the distance 543 and the related map figure ID 544, respectively, or some routes (for example, the route distance is relatively short). A predetermined number of routes) and a map figure identifier may be registered in ascending order. These may be realized using a known network analysis technique.

  If the map information DB 106C does not include road network data, information such as a route distance estimated based on information other than the road network data may be registered in the end-to-end network table 540. For example, when the map information DB 106C includes road shape vector data, the map information processing unit 104 may estimate road network data based on the vector data and perform a route search based on the road network data.

  Alternatively, when the map information DB 106C includes contour line vector data or terrain raster data, the map information processing unit 104 performs terrain analysis based on the data, and assumes a region where the target can move (for example, a vehicle is assumed as the target). In this case, the network data may be estimated based on the extracted ridge line, valley line, flat area, and the like that the vehicle is supposed to be able to pass through, or the database 106 may be a past moving object in the attention area. If the movement history data is included, the map information processing unit 104 or the like may extract a movement pattern from the movement history data and estimate the route in consideration of the movement pattern in addition to the shortest route analysis.

  As described above, the road network data is not stored in the map information DB 106C in advance, and the distance 543 and the related map graphic ID 544 estimated based on information other than the road network data (or network data estimated based on the information). Is registered in the end-to-end network table, a virtual flag 545 indicating that is registered. In the example of FIG. 5E, the value “Yes” of the virtual flag 545 indicates that the distance 543 and the related map graphic ID 544 estimated based on information other than the road network data are registered.

  In the above description, the road network data is exemplified. However, for example, when the target is a ship or an aircraft, network data related to the route may be used. With regard to the above information and aircraft, the route network may be estimated based on the arrangement of mountains and high-rise buildings.

  Next, with reference to FIG. 6A to FIG. 6E, a procedure of sensor integration processing executed by the sensor integration device 100 will be described.

  FIG. 6A is a flowchart illustrating sensor pre-processing executed by the sensor integration device 100 according to the embodiment of this invention. The sensor preprocessing shown in FIG. 6A is executed before each sensor starts observation.

  When the sensor integration device 100 receives information about each sensor via the input / output device 190 (step 601), the sensor integration device 100 identifies the sensor observation range and the like based on the information and registers it in the sensor information table 520 (step 602). Further, the type of attribute information that can be acquired by the sensor is specified and registered in the sensor information table 520 (step 603). For example, when the type of the sensor 1_151 is radar and the position where the sensor 1_151 is installed, the transmission power, and the like are input, the sensor integration device 100 based on the observation range of the sensor 1_151 and the information acquired thereby The attribute is specified, and the sensor type 522, the observation range 523, the acquired attribute 525, and the like are registered. When the sensor is mounted on the mobile platform, the sensor parameter may be held so that the observation range can be dynamically calculated and generated. This completes the sensor preprocessing.

  FIG. 6B is a flowchart illustrating sensor integration pre-processing executed by the sensor integration device 100 according to the embodiment of this invention. The sensor integration pre-process shown in FIG. 6B is executed at the latest before the process shown in FIGS. 6C to 6E described later.

  First, the sensor integration device 100 executes the sensor preprocessing shown in FIG. 6A for each of a plurality of sensors (for example, the sensor 1_151 and the sensor 2_152) (step 611).

  Next, the sensor integration device 100 collates the observation range of each sensor specified by the sensor preprocessing with the map included in the map information DB 106C (step 612), and specifies the end point of each observation range (step 613). ). The identifier of each identified end point is registered as the end point list 524 of the sensor information table 520 and the end point ID 531 of the end point information table 530, and the position of each end point is registered as the position 532.

  Next, the sensor integration device 100 generates a combination of each end point and an end point associated with the end point based on the specified end point information and the map included in the map information DB 106C, and calculates the path distance between these end points. Calculate (step 614). The association between the end points is as described with reference to FIG. 5E. Information on the generated combination of end points is registered in the end point network table 540. For combinations between end points, not all combinations may be generated, but only combinations having a short path distance may be selectively generated using a known network analysis technique. This completes the sensor integration preprocessing.

  FIG. 6C is a flowchart illustrating sensor observation range target passing processing executed by the sensor integration device 100 according to the embodiment of this invention.

  First, the target information processing unit 102 of the sensor integration device 100 identifies a target that has passed through each observation range, its speed, and the like (step 621). For example, when the collective target 411 moves on the road 420 in the direction from the end point 430A to the end point 430B in the observation range 401 and passes through the end point 430B and goes out of the observation range 401, the sensor 1_151 is within the observation range 401. The collective target 411 can be observed for a while, but cannot be observed after the observation is made outside the observation range 401. In this way, when the target that has been observed until then can no longer be observed, the target information processing unit 102 determines that the collective target 411 has passed through the observation range 401 (that is, has gone outside), A moving speed when the collective target object 411 passes through the observation range 401 (hereinafter, also simply referred to as “passing speed”) is calculated from the previous track (that is, the coordinate value at each time). For example, the target information processing unit 102 moves the collective target 411 immediately before passing through the observation range 401 from information on a wake including the last observed point before the collective target 411 passes through the observation range 401. The speed may be calculated and the moving speed may be specified as the passing speed.

  Next, the target object information processing unit 102 searches for an end point in the vicinity of the point where the target object that passed through was last observed (step 622). For example, the target information processing unit 102 may search for an end point whose distance from the point where the target was last observed is a predetermined value or less. At this time, the predetermined value of the distance may be changed according to the passing speed of the target. Alternatively, the search may be performed by predicting from the last track observed.

  As a result of the search in step 622, when an end point that satisfies the condition is acquired, the target information processing unit 102 determines that the target has passed (or at least may have passed) the acquired end point, The identifier of the passed target is registered in the acquired passing target list 533 of the end point (step 624). For example, when it is determined that the collective target object 411 has passed through the end point 430B and has gone out of the observation range 401, the target object information processing unit 102 passes the target object list of the record corresponding to the end point 430B of the end point information table 530. The identifier of the collective target 411 is added to 533. Further, the target object information processing unit 102 may specify the time at which the collective target object 411 passes through the end point 430B in step 622. For example, the time of passing through the end point 430B may be calculated based on the distance between the acquired end point 430 and the last observed point of the collective target 411, the last observed time, and the passing speed. Alternatively, the last observed time may be used as the time of passing through the (approximate) end point 430B. When a plurality of endpoints satisfying the condition are acquired in step 622, even if it is determined that the target has passed through the endpoint having the smallest distance from the point where the target was last observed. Alternatively, it may be determined that there is a possibility that the target has passed through all of these end points, and the identifier of the target may be registered in the passing target list 533 of the record corresponding to each of those end points.

  In step 624, the target object information processing unit 102 further refers to the end-to-end point network table 540, identifies the end point associated with the end point acquired in step 622, and records the end point information table 530 regarding the end point. The identifier of the target specified in step 621 may be registered in the intrusion prediction list 535. For example, as a result of executing Step 622, when it is determined that the collective target object 411 has passed the end point 430 </ b> B and has gone out of the observation range 401, the target information processing unit 102 refers to the end-point network table 540. The end point 430C associated with the end point 430B may be specified, and the identifier may be registered in the intrusion prediction list 535 of the record of the end point information table 530 regarding the end point 430C.

  On the other hand, for example, when there is no road within the observation range, or when the target travels in an area where no road map exists, an end point that satisfies the condition may not be acquired in step 622. In such a case, the target information processing unit 102 generates a temporary endpoint based on the track of the target and registers it in the endpoint information table 530 (step 623). For example, the target information processing unit 102 extrapolates the track of the target to estimate the track after the target is no longer observed, and a point where the estimated track and the boundary of the observation range intersect May be set and registered with the virtual flag 545 as a temporary endpoint. Subsequently, the target object information processing unit 102 executes Step 624 for the registered temporary endpoint.

  Thus, the sensor observation range target passing process is completed.

  FIG. 6D is a flowchart illustrating sensor observation range target entry processing executed by the sensor integration device 100 according to the embodiment of this invention.

  First, the target information processing unit 102 specifies a target that has entered each observation range, its speed, and the like (step 631). For example, when a collective target 412 that is not determined to be the same as any target that has been observed in the observation range 402 is newly observed, the target information processing unit 102 sets the collective target 412 in the observation range. The target object newly entered in 402 is identified, and the moving speed when the collective target object 412 enters the observation range 402 (hereinafter, also simply referred to as “intrusion speed”) is calculated from the track. This calculation can be performed in the same manner as in step 621.

  Next, the target object information processing unit 102 searches for an end point near the point where the intruding target object is first observed (step 632). This search can be performed in the same manner as step 622. The endpoints acquired in step 632 are endpoints determined that the target has entered (or at least that is likely). Furthermore, the time at which the target object has entered may be specified by the same method as in step 622.

  When the endpoint satisfying the condition in step 632 is acquired, the target object information processing unit 102 refers to the endpoint network table 540 and identifies the endpoint associated with the acquired endpoint (step 633). For example, it is determined that the collective target 412 has entered the observation range 402 from the end point 430C, and the end point 430C is associated with the end point 430B (more specifically, the end point of any record in the inter-endpoint network table 540) If the end points 430B and 430C are registered as the start point 541 and the end point end point 542), the end point 430B is specified in step 633.

  Next, the target object information processing unit 102 refers to the end point information table 530 and sets the target registered in the end point passing target list 533 specified in step 633 to the same target as the target that has entered. Are identified as candidates (step 634). For example, when the end point 430B is specified in step 633 and the collective target 411 is registered in the passing target list 533 of the end point 430B, the collective target 411 is specified in step 634.

  When the endpoint information table 530 includes the intrusion prediction list 535 and the identifier of any target is registered in the intrusion prediction list 535 regarding the endpoint acquired in step 632, the target object information processing unit 102 633 may not be executed. In this case, in step 634, the target object information processing unit 102 specifies the target registered in the intrusion prediction list 535 related to the end point acquired in step 632 as the same target candidate as the intruding target. This eliminates the need to refer to the end-to-end network table 540 after a target that has entered the observation range has been observed, thereby shortening the processing time.

  Next, the target object information processing unit 102 executes target target matching processing for the target candidate specified in step 634 (step 635). Details of this processing will be described later with reference to FIG. 6E. In practice, a plurality of end points may be specified in step 633, and a plurality of targets may be specified for each end point in step 634. In that case, target collation processing is executed for all the identified targets.

  Next, the target information processing unit 102 determines whether or not the collation is successful as a result of the target collation processing in step 635 (step 636). If any of the target candidates identified in step 634 has failed to be collated, it is determined that none of the target candidates is the same as the target that has entered, and therefore any other target is selected. It may have entered the observation range via the route. Therefore, the target object information processing unit 102 next uses the target registered in the passing target list 533 of the end points other than the end point specified in step 633 as the same target candidate as the intruding target object. Specify (step 637).

  Note that step 637 may be executed for all end points other than the end points specified in step 633. However, since the number of target end points is actually enormous, for example, the end points specified in step 631 Step 637 may be executed only for the end points whose distance from the distance is equal to or less than a predetermined value.

  In addition, if the endpoint that satisfies the condition is not acquired in step 632, it is considered that the target has entered the observation range via a route other than the road, and therefore the endpoint associated with the entered endpoint is specified. I can't. In this case, Steps 633 to 636 are not executed, and Step 637 is executed for all end points (or an end point whose distance from the temporary end point is equal to or less than a predetermined value) except for the temporary end point where the target has entered. The

  Next, the target object information processing unit 102 executes target object collation processing for all target objects specified in step 637 (step 638). This process is similar to step 635 and will be described later with reference to FIG. 6E.

  Next, the target information processing unit 102 determines whether or not the collation is successful as a result of the target collation processing in Step 638 (Step 639). If the verification fails for any target candidate specified in step 637, the target information processing unit 102 is different from any target that has been observed in any observation range in the past. A new target is determined, and a new target is generated and registered in the target information DB 106B (step 640).

  As a result of the collation in step 636 or step 639, when the collation succeeds for one or more target candidates specified in step 637 and when a new target is generated in step 640, the target information processing unit 102 Outputs the collation result (step 641). An example of the output information will be described later with reference to FIGS.

  FIG. 6E is a flowchart illustrating target collation processing executed by the sensor integration device 100 according to the embodiment of this invention. The target collation process executed in step 635 includes the target identified in step 631 (hereinafter referred to as an intrusion target) and the target candidate identified in step 634 (hereinafter referred to as a passing target candidate). It is performed for the pair. When a plurality of passing target candidates are specified in step 634, the process is performed for each set of the intrusion target and each passing target candidate. Similarly, the target collation process executed in step 638 is performed on the set of the intrusion target specified in step 631 and the passing target candidate specified in step 637.

  First, the target object information processing unit 102 determines whether or not there is a route map corresponding to the end point through which the passing target candidate has passed and the end point into which the intruding target has entered (step 651). If the endpoint satisfying the condition of step 622 could not be acquired for the passing target candidate or the endpoint satisfying the condition of step 632 could not be acquired for the intrusion target, it is determined that there is no corresponding route map. Otherwise, it is determined that there is a corresponding route map.

  If it is determined in step 651 that there is a corresponding route map, the target information processing unit 102 determines, based on the route map, between the end point where the passing target candidate has passed and the end point where the intruding target has entered. A route distance is specified (step 652). At this time, the map information processing unit 104 may perform route search between these end points using the road network data to calculate the route distance. However, a set of these end points is registered in the end-to-end point network table 540. If there is, the distance 543 of the record corresponding to the set can be acquired as the route distance. By calculating the path distance between the end points in advance based on the map information in this way, it is not necessary to search for the path between the end points after the target that has entered the observation range is observed. It can be shortened.

  If it is determined in step 651 that there is no corresponding route map, the route cannot be estimated based on the map information. Therefore, the target information processing unit 102 determines the end point and the intrusion target that the passing target candidate has passed. A straight line distance from the end point where the intruder enters is specified as a route distance (step 653). However, for example, when a vehicle or the like travels and moves between two points, it is considered that the shortest route between the two points is hardly passed, and actually, the route that is longer than the shortest route is moved. For this reason, the target object information processing unit 102 may correct the route distance by a method such as multiplying the linear distance by a predetermined coefficient that is empirically obtained (step 654).

  Next, the target information processing unit 102 assumes that the passing target candidate and the intrusion target are the same target, and that the target has moved along the path estimated in step 652 or the like. Are estimated based on the estimated path distance and / or the speed of at least one of the targets (step 655). Specifically, since the speed of the passing target candidate is calculated in step 621 and the speed of the intrusion target is calculated in step 631, the target information processing unit 102 determines either one of these speeds or these Based on the speed calculated based on the value of (for example, the average value of both) and the route distance specified in step 652 or 653 (or the route distance corrected in step 654), the passing target candidate passes. The time required for the target to move along the path between the end point that has entered and the end point into which the intruding target has entered is estimated. Hereinafter, this time is referred to as an estimated travel time.

  Next, the target information processing unit 102 determines the estimated travel time in step 655 and the time from the time when the passing target candidate passes through the end point to the time when the intruding target enters from the end point (hereinafter, this time is passed − (Described as intrusion time) (step 656). For example, the target information processing unit 102 may determine whether or not the passage-intrusion time is included in a predetermined time range including the estimated travel time based on the following equation (1).

d / v−αd / v <t <d / v + αd / v (1)
here,
d: Route distance specified in step 652 or the like v: Speed of target (that is, d / v: estimated travel time in step 655)
α: predetermined parameter t: passage-penetration time.

  The fact that the above equation (1) is not satisfied means that the passage-intrusion time is not included in the predetermined time range including the estimated movement time determined by the parameter α, in other words, the passage-intrusion time and the estimated movement time. This means that the magnitude of the deviation exceeds the range determined by the parameter α. A large divergence between the passage-intrusion time and the estimated travel time means that there is a large divergence between the movement speed of the target observed within the observation range and the speed required to move the route outside the observation range. To do. It is considered that the greater the difference, the lower the certainty (accuracy) that the passing target candidate and the intrusion target are the same target. For this reason, the target object information processing unit 102 determines in step 656 whether, for example, the condition of the expression (1) is satisfied, and if not satisfied, the passing target candidate and the intrusion target are the same target object. It is determined that it is not (that is, collation has failed) (step 662). Thereby, a passing target candidate with low accuracy can be excluded.

  On the other hand, when the condition of step 656 is satisfied, the target object information processing unit 102 may determine that the passing target candidate and the intrusion target are the same target (that is, the matching has succeeded), In the example of FIG. 6E, by collating one or more types of attributes other than the speed of the target, the passing target candidate and the intrusion target are the same target based on the respective attributes. In addition, the overall accuracy is determined based on the accuracy of the matching result of each attribute and travel time. For example, when the passing target candidate and the intrusion target are individual targets, the target information processing unit 102 may collate their types, sizes, colors, shapes, etc. In the case of objects, the number of individual targets included in them, the positional relationship, the type, size, etc. of each individual target may be collated. By collating these attributes, it is expected that the accuracy of collation is further improved.

  However, as already described, a plurality of types of sensors may be used, for example, the sensor 1_151 is a radar and the sensor 2_152 is a camera. May be different. For this reason, for example, the attribute information of the intrusion target includes information about the color, but the attribute information of the passing target candidate does not include information about the color. It is possible that you will not. Therefore, the target information processing unit 102 determines, for each attribute information type, whether the attribute information value of that type is included in any attribute information of the passing target candidate and the intrusion target. (Steps 657, 659, etc.) The attribute values are collated only when they are included (Steps 658, 660, etc.). As a result, even when a plurality of types of sensors coexist, it is possible to realize accurate collation.

  In the example of FIG. 6E, the target object information processing unit 102 determines whether or not any attribute information of the passing target candidate and the intrusion target includes the number of individual targets included in each target (step 657), if included, it is determined whether the difference between these values is within a predetermined parameter β (step 658). As a result, when the difference in the number of individual targets exceeds the parameter β, it is determined that the passing target candidate and the intrusion target are not the same target (step 662).

  When the difference in the number of individual targets does not exceed the parameter β, the target information processing unit 102 further selects other types of attribute values (positional relationships among a plurality of individual targets included in the collective target, each individual target The same determination can be made with respect to the type or the like. For example, the determination similar to the above may be repeatedly executed for n types (n is an arbitrary integer equal to or greater than 1) (steps 659 and 660). Specifically, for example, when both the attribute information of the passing target candidate and the intrusion target include information indicating the positional relationship of the individual targets included in each target, the target information processing unit 102 May determine the similarity of each positional relationship.

  Note that FIG. 6E is an example, and in actuality, any kind of attribute value may be collated, and any kind of attribute value may be collated. For example, the positional relationship of the individual targets included in the collective target may be collated without collating the number of individual targets included in the collective target.

  Eventually, the target object information processing unit 102 determines that all of the attribute values to be determined satisfy the predetermined condition, and the passing target candidate and the intrusion target are the same target ( That is, it is determined that the collation is successful), and the target collation process is terminated. The sensor integration device 100 may output the result of this collation from the input / output device 190 (see FIGS. 8 and 9).

  As described above, the matching between each passing target candidate and the intrusion target is performed based on the probability that they are the same. For this reason, for example, when a plurality of passing target candidates include targets that are not the same as the intrusion target but have similar attributes in addition to the actually same target as the intrusion target. As a result of the collation, there is a possibility that a plurality of passing target candidates are determined to be the same as one intrusion target. In that case, the sensor integration device may output only the passing target candidate with the highest calculated accuracy as the same as the intrusion target, or all the determined to be the same as the intrusion target. Passing target candidates may be output, and it may be left to the user to determine which of them is actually the same as the intrusion target.

  Furthermore, the contents of the database 106 may be updated according to the result of collation. For example, when it is determined that the collective target identified by the identifier “G001” and the collective target identified by the identifier “G002” shown in FIG. 5B are the same, information indicating that is the collective target information. One of the identifiers “G001” and “G002” registered as the collective target ID 511 of each record may be rewritten to the same value as the other.

  Further, in step 634, the passing target list 533 is referred to, and if it is determined that any passing target candidate included in the list is the same as the intrusion target, the identifier of the passing target candidate is the passing target. The item list 533 may be deleted. The same applies to the case where the intrusion prediction list 535 is referred to in step 634 and it is determined that any of the passing target candidates included in the list is the same as the intrusion target.

  Here, if even one of the estimated travel time and one or more other attribute values does not satisfy the condition, it is determined that the verification has failed. Such determination by the decision tree is merely an example of a method for determining the accuracy of the passing target candidate and the intrusion target being the same target, and other determination methods such as Bayesian inference and Demposter- A probabilistic determination method based on the Shafer theory, a determination method based on machine learning, or the like can also be employed. At that time, the type of attribute information used for the determination is specified in advance, and may be shared among various sensors and processing methods, or an arbitrary conversion rule is defined in a dictionary or ontology, and converted into the same type of concept. May be used.

  In addition, collation of each attribute value can be performed by arbitrary methods. For example, regarding the size of the individual target, for example, whether or not the difference in total length between the passing target candidate and the intrusion target is equal to or smaller than a predetermined parameter value (in other words, the similarity of the total length is equal to or larger than a predetermined value). Or the like, and regarding the color and shape of the individual target object, the positional relationship of the individual target object included in the collective target object, etc., the similarity of the feature quantity representing them is a predetermined parameter value. You may determine whether it is below.

  As for the type of the individual target, the type of the target may be pattern-recognized from image data or the like included in the sensor observation data, and then the type category may be matched, or the image data may be directly compared with each other. They may be collated and the degree of similarity calculated to determine their match. Since such attribute matching can be performed by a known method, detailed description thereof is omitted.

  The user of the sensor integration device 100 can arbitrarily set parameters such as α and β used for the above-described verification. For example, by selecting an appropriate parameter value according to the purpose of using the sensor integration device 100, it is easy to determine that the passing target candidate and the intrusion target are the same, or conversely, both are the same. Otherwise, it can be easily determined.

  The accuracy of the obtained wake varies depending on the type of sensor and the wake extraction process. For example, in a camera with high resolution, detailed target information can be obtained, and highly accurate track and attribute information is often obtained. Such high-accuracy information may be integrated with priority over other low-accuracy information. For example, the wakes may be integrated giving priority to the collation results such as the exact number of individual targets included in the group target and the presence / absence of characteristic individual targets included in the group target.

  Further, in practice, one individual target may move between observation ranges independently (that is, independently from other individual targets). In such a case, the sensor integration device 100 can execute the sensor integration process as described above for the individual target. In that case, the attributes of the individual target are collated in the process of FIG. 6E.

  FIG. 7 is a diagram illustrating a modified example of the sensor integration process executed by the sensor integration device 100 according to the embodiment of this invention. In FIG. 4, as a typical example, an example is shown in which a target 411 or the like that has passed through the observation range 401 enters the other observation range 402 through the road 420, but actually passes through one observation range. In some cases, the target has entered the original observation range again through a region outside the observation range. FIG. 7 shows a typical example, and shows a case where an occlusion due to a building or topography (such as a mountain) occurs in the observation range 702 of a single sensor 701 and there is a systematic missing range. In such an observation range 702, a plurality of targets traveling on the road 703 are observed as collective targets 711 including individual targets 711A, 711B, and 711C between the end points 721A to 721B in the observation range 702. , After passing through the end point 721B and out of the observation range 702, again enters the observation range 702 from the end point 721C, and the collective target including the individual targets 712A, 712B and 712C between the end points 721C to 721D. It may be observed as 712. Even in such a case, the collective targets 711 and 712 can be collated by the method described so far.

  In order to perform such collation, the inter-endpoint network table 540 includes not only information that associates end points of different observation ranges but also the same observation range connected by a path outside the observation range, such as the end points 721B and 721C. It is also possible to describe information that associates the end points of the.

  FIG. 8 is an explanatory diagram of information and a user interface output by the sensor integration device 100 according to the embodiment of this invention. As described with reference to FIG. 6E, the sensor integration device 100 determines that a plurality of passing target candidates having the highest probability of being the same as the intrusion target are the same as the intrusion target. Although the result may be output, it is also possible to output a plurality of passing target candidates and allow the user to select one of them as the same target as the intrusion target. FIG. 8 shows an example of a screen displayed by the input / output device 190 of the sensor integration device 100 for such an output and selection user interface. On the screen, the observation ranges 801 and 802 are superimposed on a map including the road 803, and further, the group targets 811 and 812 observed within the observation range 801 and the group targets observed within the observation range 802 are displayed. A symbol indicating 813 is displayed. The collective target 811 includes individual targets 811A, 811B and 811C, the collective target 812 includes individual targets 812A, 812B and 812C, and the collective target 813 includes individual targets 813A, 813B and 813C. Symbols indicating these individual targets may also be displayed. In this example, the collective target 813 is an intrusion target, and the collective targets 811 and 812 are passing target candidates that may be the same as the collective target 813.

  Further, a pointer 821 such as a mouse cursor operated by the user using a device such as a mouse is displayed on the screen. When the user places the pointer 821 on the collective target 813 that has entered the observation range 802, the sensor integrated device 100 may display a pop-up menu 831. In the pop-up menu 831, identifiers and the like of passing target candidates sorted in descending order of accuracy that are the same as the group target 813 that has entered are displayed together with the accuracy. The user can determine the collation result by selecting one of the passing target candidates.

  Note that the sensor integration device 100 displays one or more passing target candidates determined to be the same as the intrusion target as a result of the target integration process shown in FIG. 6E in the pop-up menu 831 in the order of accuracy. Alternatively, all the passing target candidates may be displayed in the pop-up menu 831 in the order of accuracy regardless of whether or not the target integration process determines that they are the same as the intrusion target.

  Further, for example, when the pointer 821 is placed on any of the plurality of passing target candidates displayed in the pop-up menu 831, the sensor integration device 100 displays the symbol of the target corresponding to the passing target candidate. May be visually assisted to highlight the user's situation. Thus, the user can easily execute association of appropriate targets, and can estimate the track of the target over a plurality of observation ranges. For example, the identification information of the group targets 811 and 812 that are passing target candidates is “A” and “B”, respectively, and the probability that they are the same as the group target 813 that is an intrusion target is “xx” and “B”, respectively. If “yy” is greater than “yy”, the sensor integration apparatus 100 displays “candidate group A” at the top of the pop-up menu 831 and then displays “candidate group B”. Good. Further, when the user places the pointer 821 on “candidate group A”, the sensor integration device 100 may highlight the symbol of the collective target 811.

  The pop-up menu 831 may further include options of “new group” and “cancel”. When the user selects “new group”, the sensor integration device 100 determines that the collective target 813 is a new target that is not identical to any passing target candidate, and the identifier and attribute of the collective target 813 Information and the like are newly registered in the database 106. Such new targets are observed when, for example, a plurality of collective targets are merged, a collective target that has not been observed before is newly observed, or a single collective target is observed. This is the case when things split. When a plurality of collective targets are merged, the user may select a plurality of passing target candidates that seem to have merged, and the sensor integration apparatus 100 may merge these passing target candidates. On the other hand, when the user selects “cancel”, the sensor integration device 100 holds (postpones) the verification result without confirming it. At this time, the sensor integration device 100 may not perform special processing, but a symbol indicating that the collation result is uncertain so that appropriate processing can be performed as necessary in the subsequent processing. May be displayed.

  Further, when the user places the pointer 821 on the symbol of the individual target, the sensor integration device 100 uses the observation data of the individual target (observation data by the radar or image data taken by the camera) and attribute information. A pop-up display 832 including the following display may be performed. Similarly, when the user places the pointer 821 on the symbol of the collective target, the sensor integration device 100 displays a pop-up display including observation data and attribute information display of all the individual targets included in the collective target. 833 may be performed. Thereby, it is possible to efficiently present information that supports the user's judgment.

  Further, when a path 841 that cannot be passed due to a disaster or the like occurs, the user operates the pointer 821 to instruct the path 841, and the sensor integration device 100 adds the attribute information that cannot be passed to the path 841 according to this instruction. Alternatively, the endpoint network table 540, the endpoint information table 530, or the map information DB 106C itself may be edited in accordance with the instruction. As a result, the route 841 is not used for route search, and the accuracy of wake integration is improved.

  FIG. 9 is an explanatory diagram of information output by the sensor integration device 100 according to the embodiment of this invention. The example of a display of a wake when a collation result is finalized (that is, when it is determined that one of a plurality of passing target candidates is the same as an intruding target) is shown. Specifically, when the collective target 411 that has passed through the observation range 401 and the collective target 412 that has entered the observation range 402 are determined to be the same, and the road 420 is estimated as a movement route between them. An example of the display of the wake is shown.

  A display example 901 is an example in which only tracks within each observation range are displayed. In addition to the wakes in each observation range, as shown in the display example 902, the wakes extending over a plurality of observation ranges are also displayed, so that the movement and presence status of a wide range of targets can be grasped. Alternatively, as shown in a display example 903, when the wakes are integrated by connecting the observation ranges, the line types displayed inside and outside the observation ranges may be switched and displayed. Alternatively, as shown in a display example 904, the track of the individual target may be switched and displayed within each observation range, and the track of the collective target may be switched outside the sensor observation range. With the display as described above, it becomes possible for the user to easily grasp the correspondence between the intrusion target and the passing target determined to be the same, and the route estimated to have passed. .

  Further, when there is no map information such as a road network outside the observation range, as shown in a display example 905, the wakes between them are displayed not as shapes along the roads but as straight wakes or other estimated routes. May be.

  Furthermore, the above-described arbitrary display contents may be switched according to an operation instruction from the user such as a display range or display scale of a map or a screen, or according to the size or distribution of a target. For example, when displaying a wide area or a plurality of targets, a track that covers a plurality of observation ranges is displayed as shown in a display example 902, and when displaying a partial area or a specific target in an enlarged manner, display examples 904 are displayed. As shown, the track of the individual target and the track of the group target may be switched and displayed. As a result, the visibility of the screen and information is improved, and it is possible to provide information that assists in grasping an appropriate situation.

  As described above, according to the present embodiment, a plurality of sensors of the same type, or a combination of a plurality of sensors having different characteristics such as a radar and a camera, a sensor mounted on a mobile platform, and a ground sensor are used to observe each sensor. Can complement the range. By integrating observation data of a plurality of sensors with different observation ranges, which has not been integrated in the past, it is possible to track a target in a wide area across the sensor observation range. Further, by combining observation data having different characteristics for each sensor type and combining information with different accuracy and items (data contents) related to the same target, it is possible to realize a highly reliable movement trajectory estimation. As a result, efficient (inexpensive) wide-area situation grasping by a small number of sensors can be realized.

  In addition, this invention is not limited to above-described embodiment, Various modifications are included. Further, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to the one having all the configurations described. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of the embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Further, each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function is stored in a non-volatile semiconductor memory, a hard disk drive, a storage device such as an SSD (Solid State Drive), or a computer-readable non-readable information such as an IC card, an SD card, or a DVD. It can be stored on a temporary data storage medium.

  In the drawings, control lines and information lines considered necessary for describing the embodiment are shown, and all control lines and information lines included in an actual product to which the present invention is applied are not necessarily shown. Not necessarily. Actually, it may be considered that almost all the components are connected to each other.

DESCRIPTION OF SYMBOLS 100 Sensor integration apparatus 101 Input / output part 102 Target object information processing part 102A Target object management part 102B Intrusion / passage determination part 102C Collective target collation part 102D Track integration processing part 103 Sensor information processing part 103A Track extraction processing part 103B Observation range specification Processing unit 104 Map information processing unit 105 Geographic information system 106 Database 106A Sensor information database (DB)
106B Target information DB
106C Map information DB
106D Processing parameter DB
106E Endpoint information DB
106F Network DB between endpoints
151, 152 Sensor 160 Network 171, 172 Sensor information 180 Other sensor integration device 181 Target information 190 Input / output device

Claims (4)

A sensor integration device comprising a processor, a storage device connected to the processor, and an output device connected to the processor,
The storage device holds sensor information, target information, and map information,
The sensor information includes information indicating observation ranges of a plurality of sensors,
The target information includes position information for each time of a plurality of individual targets in the plurality of observation ranges observed by the plurality of sensors, and for each time of a group target composed of the plurality of individual targets. Position information and information indicating attributes other than the position information of each collective target,
The information indicating the attribute of each collective target includes information indicating one or more types of the attribute, and information indicating the attribute value of each type,
The type of one or more attributes of each group target includes at least one of the number and positional relationship of the individual targets included in each group target , and at least one of the individual targets included in each group target. Including at least one of the type, size, color and shape of the target ,
Each individual target is a moving body that moves on the ground,
The map information includes road network information of areas included in the observation ranges and areas not included in any of the observation ranges adjacent to the observation ranges,
The processor is
Based on the target information, a passing point that is a point where each collective target that has passed through each observation range and moved to an area not included in any of the observation ranges has exited from each observation range, the passing A first procedure for specifying the passage time and passage speed of the point;
A second procedure for identifying an intrusion point, an intrusion time, and an intrusion speed of a group target invading any of the observation ranges from an area not included in any of the observation ranges based on the target information;
Based on the road network information, a third procedure for specifying a route distance through the road from the passing point of each group target that has passed through to the point of entry of the group target that has entered,
At least one of the passing speed of each passing group target and the entering speed of the entering group target, and the path distance from the passing point of each passing group target to the entering point of the entering group target; And a fourth procedure for estimating a travel time of a route distance from the passing point of each passing group target to the entering point of the invading group target, based on
The length of the movement time estimated based on the speed and the path distance, and the time of entry of the group target that has entered from the time of passage of each of the group targets that have passed, specified based on the target information. Performing a fifth step of evaluating the identity of each passed collective target with the invaded collective target by comparing the length of time until
The output device executes a sixth procedure for outputting the result of the identity evaluation,
The processor is
In the fifth procedure, if the similarity between the attribute of each passed group target and the attribute of the invaded group target does not satisfy a predetermined condition, each passed group target and the intruded group target Judge that the thing is not the same ,
In the fifth procedure, the length of time from the passage time specified based on the target object information to the intrusion time includes a length of travel time estimated based on the speed and path distance If it is not included in the range of the time length of, determine that each group target passed through and the group target that has entered are not the same,
The output device outputs identification information of one or more passed collective targets determined in the sixth procedure to be the same as the intruded collective target,
The storage device further holds end point information related to an end point where a boundary between the plurality of observation ranges and a road intersects, and end-point network information related to a road network between the plurality of end points,
The end point information includes information indicating the position of each end point,
The inter-endpoint network information is generated based on the end point information and the map information, and the other end points connected to the end points via roads in areas not included in any of the observation ranges. And information indicating a route distance via a road in a region not included in any of the observation ranges between the associated end points,
The processor is
Referring to the inter-endpoint network information, one or more end points associated with the end points corresponding to the passing points of each of the group objects that have passed are specified, and the one or more specified end points have passed A tenth procedure for adding information relating to each collective target to the end point information;
An eleventh step of referring to the end point information to identify each passed collective target associated with an end point corresponding to an intrusion point of the intruded collective target; and
Performing the third to fifth procedures for each of the passed collective targets identified in the eleventh procedure and the intruded collective target;
In the third procedure, the processor calculates a path distance between an end point corresponding to the passing point of each of the group objects that have passed and an end point corresponding to the intrusion point of the intruding group target. Sensor integration device characterized in that it is obtained from inter-network information . The sensor integration device according to claim 1,
  In the fifth procedure, the processor calculates the probability that each group target that has passed and the group target that has entered are the same, based on the result of matching the time and the attribute in the fifth procedure,
  In the sixth procedure, the output device displays the identification information of each of the group targets that have passed through, in order from the highest probability of being the same as the group target that has entered. The sensor integration device according to claim 1,
  When it is determined that any of the group objects that have passed is the same as the group object that has entered, the output device, in the sixth procedure, the observation range in which the group objects that have passed are observed The movement trajectory of the passed group target and the movement trajectory of the invaded group target in the observation range where the invaded group target is observed are displayed based on the target information, and The sensor integration apparatus, wherein the estimated route from the passing point of the passed collective target to the intrusion point of the invading collective target is displayed. A sensor integration method executed by a sensor integration device comprising a processor, a storage device connected to the processor, and an output device connected to the processor,
  The storage device holds sensor information, target information, and map information,
  The sensor information includes information indicating observation ranges of a plurality of sensors,
  The target information includes position information for each time of a plurality of individual targets in the plurality of observation ranges observed by the plurality of sensors, and for each time of a group target composed of the plurality of individual targets. Position information and information indicating attributes other than the position information of each collective target,
  The information indicating the attribute of each collective target includes information indicating one or more types of the attribute, and information indicating the attribute value of each type,
  The type of one or more attributes of each group target includes at least one of the number and positional relationship of the individual targets included in each group target, and at least one of the individual targets included in each group target. Including at least one of the type, size, color and shape of the target,
  The map information includes road network information of areas included in the observation ranges and areas not included in any of the observation ranges adjacent to the observation ranges,
  The sensor integration method includes:
  Based on the target information, the processor passes through each observation range, and each collective target that has moved to an area not included in any of the observation ranges is a point where the collective target has exited from each observation range. A first procedure for specifying a point, a passing time and a passing speed of the passing point;
  The processor specifies a point of entry, a time of entry, and a speed of entry of a group target that has entered one of the observation ranges from an area not included in any of the observation ranges based on the target information. Procedure and
  A third procedure in which the processor specifies a route distance via a road from a passing point of each passing group target to a point of entry of the invading group target based on the road network information;
  The processor includes at least one of a passing speed of each of the passed group targets and an intrusion speed of the invading group target, and from a passing point of each of the passing group targets to an entering point of the intruding group target. And a fourth procedure for estimating a travel time of a route distance from a passing point of each of the passed group targets to an intrusion point of the invading group target based on the route distance of
  The processor enters the collective target from the length of the travel time estimated based on the speed and the path distance and the passage time of each collective target passed based on the target information. A fifth procedure for evaluating the identity of each group object that has passed and the group object that has intruded by comparing the length of time until the intrusion time of
  A sixth procedure in which the output device outputs a result of the identity evaluation;
  The fifth procedure includes
  If the similarity between the attribute of each passed group target and the attribute of the invaded group target does not satisfy a predetermined condition, the processor includes the passed group target and the intruded group target. That are not the same,
  The length of time from the passage time specified based on the target object information to the intrusion time is a predetermined length of time including the length of travel time estimated based on the speed and path distance. If not included in range, the processor determines that each passed collective target and the intruded collective target are not the same, and
  The sixth procedure includes a procedure in which the output device outputs identification information of one or more of the passed collective targets determined to be the same as the intruded collective target,
  The storage device further holds end point information related to an end point where a boundary between the plurality of observation ranges and a road intersects, and end-point network information related to a road network between the plurality of end points,
  The end point information includes information indicating the position of each end point,
  The inter-endpoint network information is generated based on the end point information and the map information, and the other end points connected to the end points via roads in areas not included in any of the observation ranges. And information indicating a route distance via a road in a region not included in any of the observation ranges between the associated end points,
  The sensor integration method further includes:
  The processor refers to the network information between the end points, specifies one or more end points associated with the end points corresponding to the passing points of the group objects that have passed, and the specified one or more end points A tenth procedure for adding to the end point information information associating each collective target that has passed with
  An eleventh procedure wherein the processor refers to the end point information to identify each passed collective target associated with an end point corresponding to an intrusion point of the intruded collective target; and
  The third procedure to the fifth procedure are executed for each of the passed collective targets identified in the eleventh procedure and the intruded collective target,
  In the third procedure, the processor calculates a path distance between an end point corresponding to the passing point of each of the group objects that have passed and an end point corresponding to the intrusion point of the intruding group target. Sensor integration method characterized in that it is obtained from inter-network information.