JP6898883B2 - Connection device, connection method and connection program - Google Patents

Description

The present invention relates to a connecting device, a connecting method and a connecting program for connecting a connected graph showing a tracking result of an object.

Conventionally, in generating a flow line indicating a movement trajectory of an object in a predetermined area, the object is tracked based on a plurality of images acquired from a plurality of imaging devices (see, for example, Patent Document 1). ..

Japanese Unexamined Patent Publication No. 2016-139949

By the way, in order to track an object at high speed, the time interval of the tracking target is divided, and the tracking of the object in each divided interval is executed in parallel. However, when the time interval of the tracking target is divided, there arises a problem that the tracking result of the object is interrupted between each divided interval. Therefore, it is required to accurately connect the tracking results of objects that are interrupted by the time interval division.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to provide a connection device, a connection method, and a connection program capable of accurately connecting tracking results of objects interrupted by time interval division. And.

The connecting device according to the first aspect of the present invention shows the tracking results of one or more objects based on a plurality of images captured in the first period by the first imaging device that images a predetermined area, and shows the same object. As one or more first movement lines as movement lines indicating the positions at each time of the above, and as the movement lines corresponding to a plurality of images captured in the first period by the second imaging device that images the predetermined area. One or more connected graphs, which are connected graphs in which one or more second movement lines are connected based on the result of identification determination between the object indicated by the first movement line and the object indicated by the second movement line. An acquisition unit that acquires the first tracking result information including the first tracking result information and the second tracking result information including one or more second linked graphs that are the connected graphs corresponding to the second period adjacent to the first period, and the acquisition unit. The second connected graph to be connected to the first connected graph included in the first tracking result information acquired by the acquisition unit is one or more second connected graphs included in the second tracking result information acquired by the acquisition unit. When a connected graph in which the selection unit selected from the above, the first connected graph, and the second connected graph selected by the selection unit is connected to the first connected graph is generated, a plurality of flow lines are used. Therefore, a determination unit for determining whether or not there is a contradiction in the connected graph by determining whether or not a plurality of flow lines with overlapping corresponding imaging devices and corresponding times are included, and the above-mentioned When the determination unit determines that the connected graph does not have the inconsistency, the connecting unit that connects the first connected graph and the second connected graph selected for the first connected graph To be equipped.

When the selection unit includes the first movement line corresponding to the boundary position between the first period and the second period in the first connected graph, the first movement line and the second movement line are included. Based on the first flow line corresponding to the boundary position in the connected graph, the second connected graph to be connected to the first connected graph is selected, and the first connected graph corresponds to the boundary position. When the second moving line is included, the first connected graph is based on the second moving line and the second moving line corresponding to the boundary position in the second connected graph. You may select the second connected graph to be connected.

The selection unit calculates the degree of conformity between the flow line corresponding to the boundary position in the first connected graph and the flow line corresponding to the boundary position in the second connected graph, and the degree of conformity is relatively high. The second connected graph may be selected as the second connected graph to be connected to the first connected graph.

The selection unit includes a position in the predetermined area indicated by a flow line corresponding to the boundary position in the first connected graph and a position in the predetermined area indicated by a flow line corresponding to the boundary position in the second connected graph. The degree of conformity may be calculated based on the above.

The selection unit corresponds to an image showing an object indicated by a flow line corresponding to the first connected graph included in the image corresponding to the boundary position in the first period and the boundary position in the second period. The degree of conformity may be calculated based on the degree of similarity with the image showing the object indicated by the flow line corresponding to the second connected graph included in the image.

The selection unit calculates a degree of conformity indicating the degree of conformity between the first connected graph and one or more of the second connected graphs included in the second tracking result information, and one or more of the second connected graphs. Of the graphs, a second connected graph whose ratio to the degree of conformity corresponding to another second connected graph exceeds a predetermined threshold may be selected as the second connected graph to be connected to the first connected graph. Good.

The selection unit selects one first connected graph from the first tracking result information, and calculates the degree of conformity with respect to the flow line corresponding to the boundary position included in the selected first connected graph. Among one or more second connected graphs including the flow lines corresponding to, the second connected graph including the flow lines having a relatively high degree of conformity is selected as the connection target candidate for the selected first connected graph, and the connection target is selected. A flow line corresponding to the boundary position included in the second connected graph selected as a candidate and a flow line corresponding to the boundary position included in each of one or more first connected graphs included in the first tracking result information. When the first connected graph including the flow line with a relatively high degree of conformity and the selected first connected graph match among the one or more first connected graphs, the connection is made. The second connected graph selected as the target candidate may be selected as the second connected graph to be connected to the selected first connected graph.

The selection unit corresponds to the boundary position between the first period and the second period in the first connected graph included in the first tracking result information or the second connected graph included in the second tracking result information. When the flow line connected to the arrival flow line which is the flow line is a non-reachable flow line which is a flow line which has not reached the boundary position, based on the position information corresponding to the arrival flow line, The non-reachable flow line may be extended to the boundary position.

The connection unit has a relatively high degree of conformity with respect to the first connection graph among the plurality of second connection graphs for which the determination unit has determined that the connection graph does not have the contradiction. The two connected graph and the first connected graph may be connected.

In the connection method according to the second aspect of the present invention, each tracking result of one or more objects based on a plurality of images captured in the first period by a first imaging device that images a predetermined area, which is executed by a computer, is obtained. It corresponds to one or more first movement lines as movement lines indicating the positions of the same object at each time, and a plurality of images captured in the first period by the second imaging device that images the predetermined area. A first connected graph in which one or more second movement lines as the movement lines are connected based on the result of identification determination between the object indicated by the first movement line and the object indicated by the second movement line. A step of acquiring first trace result information including one or more connected graphs and second tracking result information including one or more second connected graphs which are the connected graphs corresponding to the second period adjacent to the first period. , The second connected graph to be connected to the first connected graph included in the acquired first tracking result information is selected from one or more second connected graphs included in the acquired second tracking result information. When a connected graph is generated in which the steps to be performed, the first connected graph, and the second connected graph selected for the first connected graph are connected, there are a plurality of flow lines and corresponding imaging. The step of determining whether or not there is a contradiction in the connected graph by determining whether or not a plurality of flow lines with overlapping devices and corresponding times are included, and the contradiction occurs in the connected graph. If it is determined that the case is not present, the first connected graph is provided with a step of connecting the second connected graph selected for the first connected graph.

The connection program according to a third aspect of the present invention shows a computer, each tracking result of one or more objects based on a plurality of images captured in a first period by a first imaging device that images a predetermined area. One or more first movement lines as movement lines indicating the positions of the same object at each time, and the movement corresponding to a plurality of images captured in the first period by a second imaging device that images the predetermined area. A first connected graph which is a connected graph in which one or more second flow lines as lines are connected based on the result of identification determination between the object indicated by the first movement line and the object indicated by the second movement line. The acquisition unit that acquires the first tracking result information including one or more and the second tracking result information including one or more second linked graphs corresponding to the second period adjacent to the first period. One or more second connected graphs to be connected to the first connected graph included in the first tracking result information acquired by the acquisition unit are included in the second tracking result information acquired by the acquisition unit. When a connected graph is generated in which a selection unit selected from a connected graph, the first connected graph, and the second connected graph selected by the selection unit are connected to the first connected graph, a plurality of flow lines are generated. Therefore, a determination unit for determining whether or not there is a contradiction in the connected graph by determining whether or not a corresponding image pickup device and a plurality of moving lines having overlapping corresponding times are included, and a determination unit. When the determination unit determines that the connected graph does not have the inconsistency, the connection unit that connects the first connected graph and the second connected graph selected for the first connected graph. To function as ,.

According to the present invention, there is an effect that the tracking results of objects interrupted by the time interval division can be connected with high accuracy.

It is a figure which shows the structure of the object tracking system which concerns on this embodiment. It is a figure which shows the installation position of the 1st image pickup apparatus and the 2nd image pickup apparatus which concerns on this Embodiment, and the image pickup range. It is a figure which shows the example of the tracking result of the object based on the moving image which the 1st image pickup apparatus and the 2nd image pickup apparatus imaged. It is a figure which shows the example of the tracking result of the object when the moving image corresponding to the tracking result shown in FIG. 3 is divided into the moving image corresponding to the 1st period, and the moving image corresponding to the 2nd period. It is a figure which shows the structure of the connection device which concerns on this embodiment. It is a flowchart which shows the flow of the process which concerns on the connection apparatus which concerns on this embodiment. It is a figure which shows the selection pattern when the 2nd connected graph is selected with respect to the 1st connected graph. It is a figure which shows the example which extended the non-reachable flow line to a boundary position. It is a figure which shows the example which repeated the connection of the connected graph for each of a plurality of periods.

[Overview of Object Tracking System S]
FIG. 1 is a diagram showing a configuration of an object tracking system S according to the present embodiment. The object tracking system S includes a first imaging device 1A, a second imaging device 1B, an object tracking device 2, and a connecting device 3.

The first imaging device 1A and the second imaging device 1B are imaging devices that generate moving images by imaging a predetermined area. Here, the predetermined area is, for example, the floor of a store. In the predetermined area, a person such as a clerk exists as an object to be tracked by the object tracking system S. Each of the first imaging device 1A and the second imaging device 1B generates a moving image including a plurality of images by capturing an omnidirectional image including at least a part of a predetermined area. It is assumed that the imaging time at which the image was captured is associated with each of the plurality of images.

Here, it is assumed that the range in which the first imaging device 1A can take an image and the range in which the second imaging device 1B can take an image overlap at least in part. FIG. 2 is a diagram showing the installation positions of the first imaging device 1A and the second imaging device 1B and the imaging range according to the present embodiment. As shown in FIG. 2, a first imaging device 1A and a second imaging device 1B are installed in the predetermined area A. Further, FIG. 2 shows an imaging range R1 of the first imaging device 1A and an imaging range R2 of the second imaging device 1B. As shown in FIG. 2, it can be confirmed that the imaging range R1 and the imaging range R2 partially overlap. In the present embodiment, each of the first imaging device 1A and the second imaging device 1B is intended to capture an omnidirectional image, but the present invention is not limited to this, and an image other than the omnidirectional image may be captured. ..

The object tracking device 2 acquires a moving image including a plurality of images from the first imaging device 1A, and acquires a moving image including a plurality of images from the second imaging device 1B. The object tracking device 2 tracks an object moving in a predetermined area by analyzing the acquired moving image, and generates a flow line as a movement locus of the object. FIG. 3 is a diagram showing an example of tracking results of objects based on moving images captured by the first imaging device 1A and the second imaging device 1B.

In the example shown in FIG. 3, the tracking results of objects based on a plurality of images captured by the first imaging device 1A are shown, and the flow lines LA1, LA2, and LA3 are used as the flow lines indicating the positions of the same object at each time. It can be confirmed that it has been identified. Further, in the example shown in FIG. 3, it can be confirmed that the flow lines LB1, LB2, and LB3 are specified as the flow lines showing the tracking results of the objects based on the plurality of images captured by the second imaging device 1B. In the following description, the flow line specified based on the image captured by the first imaging device 1A is the first flow line, and the flow line specified based on the image captured by the second imaging device 1B is the second flow line. That is.

The object tracking device 2 connects the first flow line and the second flow line based on the result of the identity determination between the object indicated by the first flow line and the object indicated by the second flow line. The object tracking device 2 connects the first flow line and the second flow line by generating tracking connection information indicating that the first flow line and the second flow line are connected. In the example shown in FIG. 3, it can be confirmed that the first flow line LA3 and the second flow line LB3 are connected by the tracking connection information C.

In the present embodiment, the object tracking device 2 divides the time interval of the tracking target and executes the tracking of the object in each divided section in parallel in order to track the object at high speed. That is, the object tracking device 2 divides the video reproduction time of the moving images generated by the first imaging device 1A and the second imaging device 1B into a plurality of periods, and tracks the objects in parallel based on the respective moving images. Execute.

FIG. 4 is a diagram showing an example of the tracking result of the object when the moving image corresponding to the tracking result shown in FIG. 3 is divided into the moving image corresponding to the first period and the moving image corresponding to the second period. Here, the second period is assumed to be a period adjacent to the first period. In FIG. 4, the first flow lines LA1-1, LA1-2 and LA1-3 and the second flow lines LB1-1, LB1-2 and LB1-3 are identified as the tracking results corresponding to the first period. It can be confirmed that it is. Further, in FIG. 4, it can be confirmed that the first flow lines LA2-1 and LA2-2 and the second flow lines LB2-1 and LB2-2 are specified as the tracking results corresponding to the second period. .. Further, in FIG. 4, it can be confirmed that the first flow line LA1-3 and the second flow line LB1-3 are connected by the tracking connection information C.

When the object is tracked by dividing it into a plurality of periods, the tracking result is interrupted as shown in FIG. Therefore, the connecting device 3 connects two tracking results that are likely to correspond. In this case, the connecting device 3 determines whether or not there is a contradiction in the tracking results after the connection, and if it is determined that there is no contradiction, connects these tracking results. By doing so, the connecting device 3 can accurately connect the tracking result of the object interrupted by the time interval division.

[Configuration of connection device 3]
Subsequently, the configuration of the connecting device 3 will be described. FIG. 5 is a diagram showing a configuration of a connecting device 3 according to the present embodiment. As shown in FIG. 5, the connecting device 3 includes a storage unit 31 and a control unit 32.

The storage unit 31 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The storage unit 31 stores a program executed by the control unit 32. For example, the storage unit 31 stores a connection program that causes the connection device 3 to function as the acquisition unit 321, the selection unit 322, the determination unit 323, and the connection unit 324.

The control unit 32 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The control unit 32 functions as an acquisition unit 321, a selection unit 322, a determination unit 323, and a connection unit 324 by executing the connection program stored in the storage unit 31.

Hereinafter, the details of the acquisition unit 321, the selection unit 322, the determination unit 323, and the connection unit 324 will be described with reference to the processing flow in the connection device 3. FIG. 6 is a flowchart showing a flow of processing according to the connecting device 3 according to the present embodiment.

First, the acquisition unit 321 acquires tracking result information indicating the tracking result of the object from the object tracking device 2 (S1). Specifically, the acquisition unit 321 obtains the first tracking result information indicating the tracking result of one or more objects in the first period and the second tracking result information indicating the tracking result of one or more objects in the second period. get.

The first tracking result information includes one or more first connected graphs. The first concatenated graph is composed of one or more first flow lines showing the tracking results of one or more objects based on a plurality of images captured in the first period by the first imaging device 1A, and the second imaging device 1B. The object indicated by the first flow line and the object indicated by the second flow line have one or more second flow lines indicating the tracking results of one or more objects based on a plurality of images captured in the first period. It is a concatenated graph connected based on the result of the identity determination.

Here, the flow line is indicated by the flow line information in which each time is associated with the position information indicating the position of the object at each time. For example, the position information of an object is a position coordinate indicating the position of the object in a predetermined area, or a position coordinate indicating the position of the object in an image captured by an imaging device. Further, the first flow line and the second flow line correspond to the result of the identity determination, and are connected by the tracking connection information indicating that the first flow line and the second flow line are connected. Therefore, the connected graph includes one or more first flow lines, one or more second flow lines, and one or more tracking connection information. In this embodiment, the first flow line alone or the second flow line alone is also one form of the connected graph.

The second tracking result information includes one or more second connected graphs. The second connected graph is composed of one or more first flow lines based on a plurality of images captured in the second period by the first imaging device 1A and a plurality of images captured in the second period by the second imaging device 1B. It is a connected graph which connected one or more 2nd flow lines based on the result of the identity determination of the object indicated by the 1st flow line and the object indicated by the 2nd flow line.

Subsequently, the selection unit 322 includes the second connected graph connected to the first linked graph included in the first tracking result information acquired by the acquisition unit 321 in the second tracking result information acquired by the acquisition unit 321. Select from the above second connected graph (S2 to S4).

Specifically, the selection unit 322 selects an unselected first connected graph from the plurality of first connected graphs (S2).
When the selected first connected graph includes the first flow line corresponding to the boundary position between the first period and the second period, the selection unit 322 sets the first flow line and the second connected graph. In, a second connected graph to be connected to the first connected graph is selected based on the first flow line corresponding to the boundary position. The selection unit 322 uses a Kalman filter or the like to supplement the flow line until it reaches the boundary position based on the flow line information of the past frame even for the flow line that has not reached the boundary position, and the boundary position. May be treated as having reached.

Further, when the selected first connected graph includes the second flow line corresponding to the boundary position between the first period and the second period, the selection unit 322 includes the second flow line and the second flow line. In the connected graph, the second connected graph to be connected to the first connected graph is selected based on the second flow line corresponding to the boundary position. Further, when the selected first connected graph includes the first movement line and the second movement line corresponding to the boundary position between the first period and the second period, the selection unit 322 performs the first movement. Based on the line and the first connected graph corresponding to the boundary position in the second connected graph, the second connected graph to be connected to the first connected graph is selected, and the second connected graph and the second connected graph are selected. In the two connected graph, the second connected graph to be connected to the first connected graph is selected based on the second flow line corresponding to the boundary position. In the following description, the boundary position between the first period and the second period is simply referred to as a boundary position.

Hereinafter, when the selected first connected graph includes the first flow line corresponding to the boundary position, the details of the processing when selecting the second connected graph to be connected to the first connected graph will be described. To do.

Specifically, the selection unit 322 calculates the goodness of fit between the flow line corresponding to the boundary position in the selected first connected graph and the flow line corresponding to the boundary position in each of one or more second connected graphs. (S3).

Here, the selection unit 322 is a position in a predetermined area indicated by a flow line corresponding to the boundary position in the first connected graph and a predetermined area indicated by a flow line corresponding to the boundary position in each of one or more second connected graphs. The goodness of fit is calculated based on the position. For example, the selection unit 322 includes a position indicated by the position information corresponding to the boundary position included in the flow line information indicating the flow line corresponding to the boundary position in the selected first connected graph, and one or more second connected graphs. In each case, the distance from the position indicated by the position information corresponding to the boundary position included in the flow line information indicating the flow line corresponding to the boundary position is calculated, and the degree of conformity is calculated based on the distance. The goodness of fit shall be higher as the distance between the flow lines corresponding to the boundary position is shorter. Since the distance between the corresponding flow lines at the division position becomes short, the connecting device 3 selects the corresponding connected graph with high accuracy by calculating the goodness of fit from the distance between the flow lines corresponding to the boundary position. Can be done. The calculation of the distance may be a world coordinate system which is a coordinate system in a predetermined area, or an image coordinate system which is a coordinate system in an omnidirectional image.

Further, the selection unit 322 calculates the goodness of fit based on the position indicated by the flow line corresponding to the boundary position, but the present invention is not limited to this. The selection unit 322 is included in the image showing the object indicated by the flow line corresponding to the first connected graph and the image corresponding to the boundary position in the second period, which is included in the image corresponding to the boundary position in the first period. The degree of conformity may be calculated based on the degree of similarity with the image showing the object indicated by the flow line corresponding to each of the one or more second connected graphs.

In this case, the selection unit 322 selects the image having the latest imaging time from the plurality of images corresponding to the first period captured by the first imaging device 1A and the second imaging device 1B. Then, the selection unit 322 specifies the position information associated with the imaging time included in the flow line corresponding to the selected first connected graph, and corresponds to the position indicated by the specified position information from the selected image. The image to be used is extracted as an image showing an object.

Further, the selection unit 322 selects the image having the oldest imaging time among the plurality of images corresponding to the second period captured by the first imaging device 1A and the second imaging device 1B. The selection unit 322 identifies one or more second connected graphs corresponding to the flow lines including the imaging time. Then, the selection unit 322 specifies the position information associated with the imaging time included in the flow line corresponding to one or more specified second connected graphs, and the specified position information is obtained from the selected image. The image corresponding to the indicated position is extracted as an image indicating the object.

The selection unit 322 includes an image showing an object extracted from the flow line corresponding to the selected first connected graph and an image showing a plurality of objects extracted from the flow line corresponding to one or more second connected graphs. The degree of similarity is calculated, and the degree of conformity is calculated based on the degree of similarity. The goodness of fit shall be higher the higher the similarity. Since the images of the objects corresponding to the corresponding flow lines at the division positions are similar, the connecting device 3 calculates the goodness of fit based on the images of the objects corresponding to the boundary positions, thereby accurately producing the corresponding connected graph. You can choose. The selection unit 322 compares the newest image in the first period with the oldest image in the second period, but the image used for the comparison is not limited to this. The selection unit 322 may compare the image corresponding to the first period in which the time interval between the images to be compared is within the threshold value and the image corresponding to the second period.

When the selection unit 322 calculates the degree of conformity of the flow line corresponding to the boundary position corresponding to each of the one or more second connected graphs with respect to the flow line corresponding to the boundary position included in the selected first connected graph, the said unit 322. Among the one or more second connected graphs, the second connected graph including the flow line having a relatively high degree of conformity is selected as the second connected graph to be connected to the first connected graph (S4). For example, the selection unit 322 selects a second connected graph including the flow line having the highest degree of conformity and a second connected graph including the flow line having the second highest degree of conformity among the one or more second connected graphs. Identify. Then, the selection unit 322 corresponds to another second connected graph, and when the ratio of the second highest goodness of fit exceeds a predetermined threshold, the selection unit 322 includes the flow line having the highest goodness of fit. The biconnected graph may be selected as the second connected graph to be connected to the first connected graph.

The selection unit 322 may select a second connected graph including a flow line having a relatively high degree of conformity from one or more second connected graphs as a connection target candidate. Then, the selection unit 322 corresponds to the flow line corresponding to the boundary position included in the second connected graph selected as the connection target candidate and the boundary position among the plurality of first connected graphs included in the first tracking result information. You may calculate the degree of conformity with the flow line included in one or more first connected graphs including the flow line. Then, the selection unit 322 is a first connected graph including a flow line having a relatively high degree of conformity with the flow line included in the second connected graph selected as a connection target candidate among the one or more first connected graphs. When the selected first connected graph matches, the second connected graph selected as the connection target candidate may be selected as the second connected graph to be connected to the selected first connected graph. By doing so, the connecting device 3 can select a second connected graph that is likely to correspond to the selected first graph.

When the selected first connected graph includes the second flow line corresponding to the boundary position, the process for selecting the second connected graph to be connected to the first connected graph is the first process. Since the process is the same as the process for selecting the second connected graph when the flow line is included, detailed description thereof will be omitted.

Subsequently, the selection unit 322 determines whether or not all the first connected graphs including the flow lines corresponding to the boundary positions have been selected (S5). When the selection unit 322 determines that all the first connected graphs have been selected, the process is transferred to S6, and when it is determined that all the first connected graphs have not been selected, the selection unit 322 shifts the processing to S2.

The result of selecting the second connected graph with respect to the selected first connected graph can be divided into the following five types of selection patterns. FIG. 7 is a diagram showing a selection pattern when the second connected graph is selected with respect to the first connected graph.

The first pattern is a flow line corresponding to the same image pickup device with respect to the flow line included in the first connected graph, and the flow line corresponding to another image pickup device is not connected by the tracking connection information. The second connected graph connected by the line is the selected pattern. As an example of the first pattern, as shown in FIG. 7, the first connected graph G1-1 containing only the first flow line LA1-1 is compared with the second connected graph G2-containing only the second flow line LA2-1. An example in which 1 is selected is given. Here, it is assumed that the bidirectional arrows AS shown in FIG. 7 indicate the flow lines to be connected to each other.

The second pattern corresponds to the same imaging device with respect to the flow line included in the first connected graph, and is connected by the flow line to which the flow line corresponding to the other imaging device is connected by the tracking connection information. The second connected graph is the selected pattern. As an example of the second pattern, as shown in FIG. 7, with respect to the first connected graph G1-1 including only the first flow line LA1-1, the second flow line LA2-1 and the second flow line LB2- An example in which the second connected graph G2-2 to which 1 is connected by the tracking connection information is selected can be mentioned.

The third pattern is a pattern in which the second connected graph connected by the two flow lines connected by the tracking connection information is selected for each of the two flow lines included in the first connected graph. As an example of the third pattern, as shown in FIG. 7, a first connected graph G1-1 containing only the first flow line LA1-1 and a first connected graph G1-2 containing only the second flow line LB1-1. On the other hand, there is an example in which the second connected graph G2-2 including the first flow line LA2-1 and the second flow line LB2-1 connected by the tracking connection information is selected.

The fourth pattern is a pattern in which two flow lines are connected to one first connected graph in the third pattern by tracking connection information. As an example of the fourth pattern, as shown in FIG. 7, a first connected graph G1-3 in which the first flow line LA1-1 and the second flow line LB1-2 are connected by tracking connection information, and a first connected graph G1-3. A second connected graph including the first flow line LA2-1 and the second flow line LB2-1 connected by tracking connection information with respect to the first connected graph G1-2 containing only the two flow lines LB1-1. An example in which G2-2 is selected can be mentioned. In this case, if the second flow line LB1-1 and the second flow line LB1-2 indicating different objects include positions corresponding to the same time, the connected graph connecting these connected graphs is inconsistent. Will have occurred.

That is, contradiction means that the connected graph contains a plurality of flow lines corresponding to different objects. When the connected graph contains a plurality of flow lines in which the corresponding imaging device and the corresponding time overlap, the connected graph is inconsistent.

In the example shown in the fourth pattern, the second flow line LB1-2 and the first flow line LA1-1 in the first connected graph G1-3 are flow lines determined to be the same object by the tracking connection information. The first flow line LA1-1 and the first flow line LA2-1 are the flow lines of the same object by connecting the connected graphs.

Further, the first flow line LA2-1 and the second flow line LB2-1 in the second connected graph G2-2 are flow lines determined to be the same object by the tracking connection information, and the second flow line LB2-1. And the second flow line LB1-1 are flow lines that are made into the same object by the connection of the connected graph. Therefore, with respect to the second flow line LB1-2 and the second flow line LB1-1, the corresponding imaging device and the corresponding time overlap, and it is derived that they are the same object flow lines. Therefore, as a result of the connection of the connected graphs shown in the fourth pattern, there is a contradiction that two flow lines of the same object exist corresponding to the images of the same imaging device at the same time.

The fifth pattern is a pattern in which two flow lines are connected by tracking connection information to the other first connected graph in the fourth pattern. As an example of the fifth pattern, as shown in FIG. 7, a first connected graph G1-3 in which the first flow line LA1-1 and the second flow line LB1-2 are connected by tracking connection information, and a first connected graph G1-3. The first flow line LA2-, in which the first flow line LA1-2 and the second flow line LB1-1 are connected by the tracking connection information to the first connected graph G1-4 connected by the tracking connection information. There is an example in which the second connected graph G2-2 including 1 and the second flow line LB2-1 is selected. In this case, the first flow line LA1-1 and the first flow line LA1-2 indicating different objects include positions corresponding to the same time, or the second flow line LB1-1 indicating different objects. , If the second flow line LB1-2 includes a position corresponding to the same time, the connected graph connecting these connected graphs will contain a contradiction.

In each pattern of FIG. 7, the same can be considered for a pattern in which the structure of the connected graph in the first period and the structure of the connected graph in the second period are interchanged. The description of the pattern in which the structure of the connected graph of the first period and the structure of the connected graph of the second period are exchanged will be omitted.

The determination unit 323 selects an unselected combination of the connected first connected graph and second connected graph based on the result selected by the selection unit 322 (S6).
Subsequently, the determination unit 323 generated a connected graph in which the first connected graph and the second connected graph selected by the selection unit 322 with respect to the first connected graph were connected based on the combination selected in S6. In this case, whether or not there is a contradiction in the connected graph by determining whether or not a plurality of flow lines include a plurality of flow lines in which the corresponding imaging device and the corresponding time overlap. Is determined (S7).

For example, when a plurality of first flow lines specified based on an image captured by the first imaging device 1A exist at the same time, these first flow lines indicate different objects and are the same. It does not indicate an object. Similarly, when a plurality of second flow lines specified based on the image captured by the second image pickup device 1B exist at the same time, these second flow lines indicate different objects and are the same. Does not indicate an object of.

Therefore, when the determination unit 323 generates a connected graph based on the combination selected in S6, the connected graph includes a plurality of first flow lines corresponding to the same time, or at the same time. If the corresponding second flow line is included, it is determined that the connected graph is inconsistent.

When the determination unit 323 determines that there is a contradiction in the connected graph, the process is transferred to S9, and when it is determined that there is no contradiction in the connected graph, the process is transferred to S8.

When the determination unit 323 determines that there is no contradiction in the connected graph to be determined, the connecting unit 324 selects the first connected graph corresponding to the combination selected in S6 and the first connected graph. A connected graph is generated by connecting to a second connected graph (S8).

Further, when the determination unit 323 determines that the connected graph to be determined is inconsistent, the connection unit 324 connects the object indicated by the first connected graph selected in S2 and the first connected graph in S4. It is determined that the objects indicated by the selected second connected graph are different objects, and the first connected graph and the second connected graph are controlled so as not to be connected.

Subsequently, the determination unit 323 determines whether or not all combinations of the connected first connected graph and the second connected graph have been selected (S9). When the selection unit 322 determines that all the combinations have been selected, it ends the process according to this flowchart, and when it determines that all the combinations have not been selected, it shifts the process to S6.

In the processing in this flowchart, the selection unit 322 selects one second connected graph as the second connected graph including the flow lines having a relatively high degree of conformity, but the present invention is not limited to this, and the degree of conformity is relative. You may select a plurality of second connected graphs that include a particularly high flow line. In this case, when the determination unit 323 connects each of the plurality of second connected graphs selected by the selection unit 322 to the selected first connected graph to generate a connected graph, the determination unit 323 is inconsistent with the connected graph. May be determined whether or not is occurring.

Then, the connecting unit 324 conforms to the flow line included in the selected first connected graph among the plurality of second connected graphs determined that there is no contradiction in the connected graph generated by the determination unit 323. A second connected graph including a flow line having a relatively high degree may be connected to the first connected graph. By doing so, the connecting device 3 accurately identifies the second connected graph to be connected to the selected first connected graph, and connects the first connected graph and the second connected graph. Can be done.

Further, in the process in this flowchart, the selection unit 322 does not perform the process of editing the flow line in the connected graph including the flow line corresponding to the boundary position, but the process is not limited to this, and the flow line is edited. You may do so.

The selection unit 322 is a flow line corresponding to the boundary position between the first period and the second period in the first connected graph included in the tracking result information or the second connected graph included in the second tracking result information. When the flow line and the flow line connected by the tracking connection information are non-reachable flow lines that have not reached the boundary position, the relevant flow line is based on the position information corresponding to the reached flow line. The non-reachable flow line may be extended to the boundary position.

FIG. 8 is a diagram showing an example in which the non-reachable flow line is extended to the boundary position. As shown in FIG. 8, the first flow line LA1-3 in the first period is a non-reachable flow line, and the second flow line LB1-3 connected by the tracking connection information C is a reachable flow line. In this case, as shown in FIG. 8, the selection unit 322 specifies the latest time T1 among the times corresponding to the non-reachable flow lines. The selection unit 322 calculates the amount of change in the position on the arrival flow line based on the position corresponding to the specified time T1 and the position corresponding to the time T2 next to the time T1. Then, the selection unit 322 specifies the position corresponding to the time T2 by adding the calculated change amount to the position corresponding to the time T1 on the non-reachable flow line. The selection unit 322 repeats the above-mentioned process until the position corresponding to the time corresponding to the boundary position is specified in the non-reachable flow line. By doing so, the non-reachable flow line can be treated as a reachable flow line and the connected graph can be connected.

[Effect in this embodiment]
As described above, the connecting device 3 according to the present embodiment has one or more connected second connected graphs to be connected to the first connected graph included in the first tracking result information, which is included in the second tracking result information. When a connected graph is generated by selecting from the second connected graph of the above and connecting the first connected graph and the second connected graph selected for the first connected graph, there are a plurality of flow lines. By determining whether or not a plurality of flow lines with overlapping corresponding imaging devices and corresponding times are included, it is determined whether or not there is a contradiction in the connected graph. Then, when the connecting device 3 determines that the inconsistency does not occur in the connected graph, the connecting device 3 connects the first connected graph and the second connected graph selected for the first connected graph. By doing so, the connecting device 3 can realize high speed by performing the tracking process in parallel for each section by dividing the time interval, and the first one corresponding to the tracking result of the interrupted object. The connected graph and the second connected graph can be connected with high accuracy.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments.

In the above embodiment, the second connected graph connected to the first connected graph included in the first tracking result information corresponding to the first period is selected from the second linked graph, but the present invention is not limited to this, and three or more are not limited to this. You may want to connect the connected graphs included in the tracking results corresponding to each of the periods.

FIG. 9 is a diagram showing an example in which the connection of the connected graph is repeated for each of the plurality of periods. As shown in FIG. 9, the control unit 32 of the connecting device 3 connects the connected graph included in the tracking result information corresponding to the first period and the connected graph included in the tracking result information corresponding to the second period. By doing so, a consistently connected graph is generated. Then, the control unit 32 further generates a consistently connected graph by connecting the generated consistently connected graph and the connected graph included in the tracking result information corresponding to the third period. By doing so, the connecting device 3 can generate a consistently connected graph by connecting the connected graphs corresponding to a plurality of periods.

Further, in particular, the specific embodiments of the distribution / integration of the devices are not limited to those shown above, and all or a part thereof may be arbitrarily added according to various additions or functional loads. It can be functionally or physically distributed / integrated in units.

For example, in the above embodiment, the connecting device 3 and the object tracking device 2 are different devices, but the device is not limited to this. The connecting device 3 and the object tracking device 2 may be realized by one device.

1A ... 1st imaging device, 1B ... 2nd imaging device, 2 ... object tracking device, 3 ... connection device, 31 ... storage unit, 32 ... control unit, 321 ... Acquisition unit, 322 ... Selection unit, 323 ... Judgment unit, 324 ... Connection unit, S ... Object tracking system

Claims (11)

Shows the tracking result of each of one or more objects based on a plurality of images captured in the first period by the first imaging device that images a predetermined area, and one or more as a moving line indicating the position of the same object at each time. The first movement line and one or more second movement lines as the movement lines corresponding to the plurality of images captured in the first period by the second imaging device that images the predetermined area are the first. In the first tracking result information including one or more first connected graphs which are connected graphs connected based on the result of the identity determination between the object indicated by the movement line and the object indicated by the second movement line, and the first period. An acquisition unit that acquires second tracking result information including one or more second connected graphs, which are the connected graphs corresponding to adjacent second periods, and an acquisition unit.
A second connected graph to be connected to a first connected graph included in the first tracking result information acquired by the acquisition unit is one or more of the second linked graphs included in the second tracking result information acquired by the acquisition unit. A selection section to select from a 2 connected graph and
When a connected graph in which the first connected graph and the second connected graph selected by the selection unit are connected to the first connected graph is generated, there are a plurality of flow lines and corresponding imaging devices. And a determination unit that determines whether or not there is a contradiction in the connected graph by determining whether or not a plurality of flow lines with overlapping corresponding times are included.
When the determination unit determines that the connected graph does not have the contradiction, the connection unit that connects the first connected graph and the second connected graph selected for the first connected graph ,
Connecting device. When the selection unit includes the first movement line corresponding to the boundary position between the first period and the second period in the first connected graph, the first movement line and the second movement line are included. Based on the first flow line corresponding to the boundary position in the connected graph, the second connected graph to be connected to the first connected graph is selected, and the first connected graph corresponds to the boundary position. When the second moving line is included, the first connected graph is based on the second moving line and the second moving line corresponding to the boundary position in the second connected graph. Select the second connected graph to connect to,
The connecting device according to claim 1. The selection unit calculates the degree of conformity between the flow line corresponding to the boundary position in the first connected graph and the flow line corresponding to the boundary position in the second connected graph, and the degree of conformity is relatively high. The second connected graph is selected as the second connected graph to be connected to the first connected graph.
The connecting device according to claim 2. The selection unit includes a position in the predetermined area indicated by a flow line corresponding to the boundary position in the first connected graph and a position in the predetermined area indicated by a flow line corresponding to the boundary position in the second connected graph. The degree of conformity is calculated based on
The connecting device according to claim 3. The selection unit corresponds to an image showing an object indicated by a flow line corresponding to the first connected graph included in the image corresponding to the boundary position in the first period and the boundary position in the second period. The degree of conformity is calculated based on the degree of similarity with the image showing the object indicated by the flow line corresponding to the second connected graph included in the image.
The connecting device according to claim 3 or 4. The selection unit calculates a degree of conformity indicating the degree of conformity between the first connected graph and one or more of the second connected graphs included in the second tracking result information, and one or more of the second connected graphs. Among the graphs, the second connected graph whose ratio with the degree of conformity corresponding to the other second connected graph exceeds a predetermined threshold is selected as the second connected graph to be connected to the first connected graph.
The connecting device according to any one of claims 3 to 5. The selection unit selects one first connected graph from the first tracking result information, and calculates the degree of conformity with respect to the flow line corresponding to the boundary position included in the selected first connected graph. Among one or more second connected graphs including the flow lines corresponding to, the second connected graph including the flow lines having a relatively high degree of conformity is selected as the connection target candidate for the selected first connected graph, and the connection target is selected. A flow line corresponding to the boundary position included in the second connected graph selected as a candidate and a flow line corresponding to the boundary position included in each of one or more first connected graphs included in the first tracking result information. When the first connected graph including the flow line with a relatively high degree of conformity and the selected first connected graph match among the one or more first connected graphs, the connection is made. The second connected graph selected as the target candidate is selected as the second connected graph to be connected to the selected first connected graph.
The connecting device according to any one of claims 3 to 6. The selection unit corresponds to the boundary position between the first period and the second period in the first connected graph included in the first tracking result information or the second connected graph included in the second tracking result information. When the flow line connected to the arrival flow line which is the flow line is a non-reachable flow line which is a flow line which has not reached the boundary position, based on the position information corresponding to the arrival flow line, Extending the non-reachable flow line to the boundary position,
The connecting device according to any one of claims 1 to 7. The connection unit has a relatively high degree of conformity with respect to the first connection graph among the plurality of second connection graphs for which the determination unit has determined that the connection graph does not have the contradiction. Connecting the two connected graph and the first connected graph,
The connecting device according to any one of claims 1 to 8. Computer runs,
Shows the tracking result of each of one or more objects based on a plurality of images captured in the first period by the first imaging device that images a predetermined area, and one or more as a flow line indicating the position of the same object at each time. The first flow line and one or more second flow lines as the flow lines corresponding to the plurality of images captured in the first period by the second imaging device that images the predetermined area. In the first tracking result information including one or more first connection graphs which are connected graphs connected based on the result of the identity determination between the object indicated by the flow line and the object indicated by the second flow line, and the first period. A step of acquiring a second tracking result information including one or more second linked graphs, which is the linked graph corresponding to the adjacent second period, and a step of acquiring the second tracking result information.
A second connected graph to be connected to the first connected graph included in the acquired first tracking result information is selected from one or more second connected graphs included in the acquired second tracking result information. Steps and
When a connected graph in which the first connected graph and the second connected graph selected for the first connected graph are connected is generated, there are a plurality of flow lines, and the corresponding imaging device and the corresponding image pickup device and the corresponding A step of determining whether or not there is a contradiction in the connected graph by determining whether or not a plurality of flow lines with overlapping times are included, and
When it is determined that the inconsistency does not occur in the connected graph, a step of connecting the first connected graph and the second connected graph selected for the first connected graph,
Connection method including. Computer,
Shows the tracking result of each of one or more objects based on a plurality of images captured in the first period by the first imaging device that images a predetermined area, and one or more as a flow line indicating the position of the same object at each time. The first flow line and one or more second flow lines as the flow lines corresponding to the plurality of images captured in the first period by the second imaging device that images the predetermined area. In the first tracking result information including one or more first connection graphs which are connected graphs connected based on the result of the identity determination between the object indicated by the flow line and the object indicated by the second flow line, and the first period. An acquisition unit that acquires second tracking result information including one or more second connected graphs, which are the connected graphs corresponding to adjacent second periods.
A second connected graph to be connected to the first connected graph included in the first tracking result information acquired by the acquisition unit is one or more of the second linked graphs included in the second tracking result information acquired by the acquisition unit. Selection part to select from 2 connected graphs,
When a connected graph in which the first connected graph and the second connected graph selected by the selection unit are connected to the first connected graph is generated, there are a plurality of flow lines and corresponding imaging devices. And a determination unit that determines whether or not there is a contradiction in the connected graph by determining whether or not a plurality of flow lines with overlapping corresponding times are included.
When the determination unit determines that the connected graph does not have the contradiction, the connecting unit connecting the first connected graph and the second connected graph selected for the first connected graph.
A connection program that functions as.

Images