JP7238536B2 - Specific object tracking device and specific object tracking system - Google Patents

Description

The present invention relates to a specific object tracking device that tracks a specific object to be monitored.

In order to realize a society where people can live safely and securely, there is a demand for monitoring systems for early resolution of crimes and monitoring systems for children and the elderly. In particular, it is necessary to quickly find specific objects to be investigated and monitored, such as stolen vehicles and missing persons, and to continuously monitor and track the situation thereafter.

For example, by tracking a vehicle in which a criminal boarded or a stolen vehicle, it is possible to arrest the criminal and secure an opportunity to bring the criminal to justice. In addition, by continuing the pursuit, it is possible to make known that escape is impossible, reduce the motivation to commit crimes, and reduce the number of people who actually commit crimes.

Furthermore, by monitoring the situation of a loitering old man or a person who is about to run out into the roadway and taking immediate action, it is possible to reduce the risk of such a person being involved in a traffic accident, kidnapping, or other crime.

In order to make the tracking of a specific object more effective, it is necessary to realize both wide-area and wide-range tracking and detailed tracking from the viewpoint of space. Also, it is necessary to realize continuous tracking in terms of time.

Here, wide-area and wide-range tracking refers to tracking when the target to be tracked moves greatly. For example, when chasing a fleeing criminal, it is assumed that the criminal will run farther to avoid being chased, so wide-area and wide-range chasing is required.

On the other hand, detailed tracking refers to thorough tracking from a spatial point of view. When chasing a fleeing criminal in the same way as above, in addition to the above-mentioned wide-area and wide-ranging pursuit, criminals do not always escape along roads and passages, so there is no omission in the tracking range. Is required.

In addition, continuous tracking refers to tracking without omission in terms of time. Even if the target to be tracked can be tracked over a wide area, in a wide range, and in detail, if there is a period of time when the target cannot be tracked, the tracking up to that point will be lost. For example, when chasing an escaped criminal using a vehicle, there will be a period of time during which the criminal cannot be tracked, and if the criminal gets out of the vehicle and escapes during that time, the chase will not be possible, so continuous tracking is required.

As a method of identifying an object to be searched, tracked, or monitored, a traveling locus identification device has been proposed that identifies the traveling locus of a target vehicle based on feature information relating to the appearance of the target vehicle. In this running locus identification device, a plurality of sensors are arranged on the road, and the running locus identification device, which is a central device, determines whether or not there is the same characteristic information from a plurality of pieces of characteristic information acquired by the sensors. When there is characteristic information, the passage time of the sensor from which the characteristic information was acquired is acquired to identify the travel locus (see Patent Document 1).

In addition, a vehicle search system that recognizes the vehicle number information of surrounding vehicles to increase the detection rate of the vehicle to be searched and searches for the vehicle to be detected in real time, and a vehicle search device and a vehicle search center device that constitute the system have been proposed. ing. In this vehicle search system, a vehicle search device is mounted on a vehicle to acquire vehicle number information of surrounding vehicles. It is sent to the device (see Patent Document 2).

In particular, realizing both detailed tracking in terms of space and continuous tracking in terms of time contributes to improvement of tracking accuracy. Since the information obtained from the sensors used for tracking varies greatly depending on space and time, it is difficult to identify the same target captured by the sensor as the same tracked target. For example, if a camera is used as a sensor, it becomes difficult to identify the same object because the lighting conditions change if the position of the object changes significantly even within the same time period. Also, even if the position is the same, if the tracking time changes significantly, the lighting conditions will change. Furthermore, especially when the tracking target is a person, if the tracking time changes significantly, it can be identified as the same object due to changes in clothes such as changing clothes or putting on and taking off jackets, etc., and changes in appearance caused by removing hats, glasses, etc. becomes difficult to do.

JP 2008-217432 JP 2007-305155

The travel locus identification device described in Patent Literature 1 is capable of tracking over a wide area by arranging sensors at a plurality of points over a wide area. However, placing more sensors to enable detailed tracking would dramatically increase the installation cost of the sensors and the processing load on the running locus identification device, which is the central device. is difficult.

In the vehicle search system described in Patent Literature 2, by mounting a vehicle search device on a moving vehicle, restrictions on sensor installation positions are reduced, and both wide-area tracking and detailed tracking are achieved from the viewpoint of space. . However, the information about the tracked object is processed in the vehicle search center device and then transmitted from the vehicle search center device to the vehicle search device mounted on each vehicle. Although it is possible for each vehicle search device that receives information about a tracked object to search for the target, it is difficult to continuously track the tracked object in terms of time.

In the road traffic environment, the present invention realizes both detailed tracking in a wide area from the viewpoint of space and continuous tracking from the viewpoint of time toward the realization of a safe and secure road environment and society. To provide a specific object tracking device capable of

A specific object tracking device according to the present invention is connected to M number of sensor devices (M is an integer satisfying 2≦M) each having a sensor via an arbitrary connection means, and a communication unit that communicates with the sensor devices; An access control unit that switches the sensor device with which the communication unit communicates, and a sensing information acquisition that acquires sensing information acquired by the m-th (m is an integer satisfying 1≤m<M) sensor device connected via the communication unit. a tracked object search unit that detects a tracked object based on the sensing information acquired by the sensing information acquisition unit; a tracking data generation unit that generates tracking data regarding the tracked object detected by the tracked object search unit; A tracking continuation determination unit that determines whether or not tracking can be continued by the device, a tracking request target selection unit that selects the (m+1)th sensor device that requests continuation of tracking, and another specific object tracking device that requests tracking assistance is selected. Consists of the selection department for relief requests. In the specific object tracking device according to another aspect of the present invention, the tracking request target selection unit determines that tracking continuation by the m-th sensor device is not permitted by the tracking continuation permission/inhibition determination unit, and the tracking request target selection unit When the (m+1)th sensor device requesting continuation of tracking of the tracked object is selected, a tracking request signal for the tracked object to the (m+1)th sensor device selected by the tracking request target selection unit is sent to the access control unit. . Alternatively, in the specific object tracking device according to another aspect of the present invention, the help request target selection unit determines that the m-th sensor device cannot continue tracking by the tracking continuation possibility determination unit, and the tracking request target is selected. If the m+1-th sensor device requesting continued tracking of the tracked object is not selected by the access control unit, the access control unit sends a help request signal for tracking the tracked object to the other specific object tracking device selected by the help request target selection unit. send to The access control unit in the specific object tracking device according to another aspect of the present invention switches the connection destination to the (m+1)-th sensor device according to the acquired tracking request signal, and transmits the tracking request signal via the communication unit, or , switching the connection destination to another specific object tracking device according to the acquired help request signal, and transmitting the help request signal through the communication unit to request the tracking and rescue of the object to be tracked.

According to the specific object tracking device of the present invention, both detailed tracking in a wide area from the spatial point of view and continuous tracking from the time point of view can be realized.

1 is a configuration block diagram of a specific object tracking system 1000 according to the first embodiment of the present invention; FIG. 3 is a flow chart showing the processing procedure of the specific object tracking device 100 in the specific object tracking system 1000 of the first embodiment of the present invention; 3 is a flowchart showing detailed procedures of processing step S111 of the specific object tracking device 100 according to the first embodiment of the present invention; Image when the estimated course of the tracked object 135 is not included in the blind spot of the sensor 152A Image when the estimated course of the tracked object 135 is included in the blind spot of the sensor 152A Image when the detection position of the tracked object 135 does not change within the sensing range of the sensor 152A Image when the detection position of the tracked object 135 changes in the sensing range of the sensor 152A Image when the detection size of the tracked object 135 in the sensing range of the sensor 152A changes 3 is a flowchart showing detailed procedures of processing step S112 of the specific object tracking device 100 according to the first embodiment of the present invention; FIG. 4 is a configuration block diagram of a specific object tracking system 2000 according to the second embodiment of the present invention; 10 is a flow chart showing the processing procedure of the specific object tracking device 200 in the specific object tracking system 2000 of the second embodiment of the present invention; Flowchart showing a detailed procedure of processing step S211 of the specific object tracking device 200 according to the second embodiment of the present invention An image of when the estimated course of the tracked object 235 and the planned course of the vehicle, which is the sensor device 251A, are the same. An image of a case in which the estimated route of the tracked object 235 and the planned route of the vehicle, which is the sensor device 251A, are different. Image when the detection position of the tracked object 235 does not change in the sensing range of the sensor 252A Image when the detection position of the tracked object 235 changes in the sensing range of the sensor 252A Image when the detection size of the tracked object 235 in the sensing range of the sensor 252A changes Flowchart showing a detailed procedure of processing step S212 of the specific object tracking device 200 according to the second embodiment of the present invention FIG. 3 is a configuration block diagram of a specific object tracking device 300 according to the third embodiment of the present invention; FIG. 11 is a configuration block diagram of a specific object tracking system 3000 according to the third embodiment of the present invention; 13 is a flow chart showing the processing procedure of the specific object tracking device 300 in the specific object tracking system 3000 according to the third embodiment of the present invention; Flowchart showing a detailed procedure of processing step S311 of the specific object tracking device 300 according to the third embodiment of the present invention Flowchart showing detailed procedure of processing step S312 of specific object tracking device 300 according to the third embodiment of the present invention Flowchart showing a detailed procedure of processing step S315 of the specific object tracking device 300 according to the third embodiment of the present invention Configuration block diagram of specific object tracking system 4000 according to the fourth embodiment of the present invention 14 is a flow chart showing the processing procedure of the specific object tracking device 400 in the specific object tracking system 4000 of the fourth embodiment of the present invention; Flowchart showing a detailed procedure of processing step S411 of the specific object tracking device 400 according to the fourth embodiment of the present invention Flowchart showing a detailed procedure of processing step S415 of the specific object tracking device 400 according to the fourth embodiment of the present invention Configuration block diagram of a specific object tracking system 5000 according to the fifth embodiment of the present invention 12 is a flow chart showing the processing procedure of the specific object tracking device 500 in the specific object tracking system 5000 of the fifth embodiment of the present invention; Flowchart showing a detailed procedure of processing step S511 of the specific object tracking device 500 according to the fifth embodiment of the present invention Flowchart showing a detailed procedure of processing step S515 of the specific object tracking device 500 according to the fifth embodiment of the present invention

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, the shape, size, and arrangement of each component are only schematically shown to the extent that the present invention can be fully understood. Also, the numerical conditions of each component are merely preferred examples. Accordingly, the present invention is not limited to the illustrated examples only, and many modifications and variations can be made that achieve the effects of the present invention without departing from the scope of the construction of the present invention. Moreover, for each component, any suitable known element or the like that realizes the functions described below can be used. Furthermore, the same reference numerals are given to common or similar components in each drawing, and duplicate descriptions thereof will be omitted. Further, when distinguishing the same constituent elements from each other, they are denoted by adding A, B, C, . . .

(First embodiment)
A specific object tracking system 1000 according to the first embodiment of the present invention will be described with reference to FIG. A specific object tracking system 1000 is composed of a specific object tracking device 100 and a plurality of sensor devices 151 . FIG. 1 is a configuration block diagram of a specific object tracking system 1000. As shown in FIG. A specific object tracking system 1000 is configured by connecting a specific object tracking device 100 and a plurality of sensor devices 151A, 151B, 151C, . In the following description, when the specific object tracking device 100 connects to the sensor device 151A for the purpose of communication, the specific object tracking device 100 accesses the sensor device 151A. A connection state in which no access is made is described as a link between the specific object tracking device 100 and the sensor device 151A.

In this embodiment, an example in which the sensor device 151 is of a fixed-point installation type will be described. Fixed-point installation examples of the sensor device 151 include a case where the sensor device 151 is installed on a utility pole, a street lamp, or an outer wall of a building. An example in which the specific object tracking device 100 according to the present embodiment is of a fixed point installation type like the sensor device 151 will be described. An example of fixed-point installation of the specific object tracking device 100 corresponds to a case where it is installed at a central office such as an information management center.

The sensor device 151 is a device that generates sensing information 156 from image data acquired by a sensor 152, which will be described later. The sensor device 151 is composed of a sensor 152 , a control section 153 and a transmission/reception section 154 .

The sensor 152 is a moving image camera called a PTZ (pan-tilt-zoom) camera. However, the sensor 152 is not limited to a moving image camera, and may be a still image camera that captures still images, a ranging sensor such as LIDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) or a millimeter wave radar. good. Further, the sensor 152 does not need to be single, and a plurality of types of the sensors described above may be used together, or a GPS (Global Positioning System) sensor for acquiring position information as world coordinates may be provided together.

The sensor 152 sends the acquired image data, the distance to the subject, and the relative positional relationship to the control unit 153 .

The control unit 153 generates sensing information 156 from the image data acquired from the sensor 152, the distance to the subject, and the relative positional relationship. The control unit 153 generates sensor device information 157 from the information (type, number and sensing range) of the sensors 152 and the installation position information of the sensor device 151 . The control unit 153 controls the rotation angle (pan), the elevation angle or depression angle (tilt), and the imaging magnification of the sensor 152 based on the sensor control signal 108 acquired from the sensor control unit 123 of the specific object tracking device 100 (to be described later) via the transmission/reception unit 154. (zoom) and controls. The control unit 153 sends the generated sensing information 156 and sensor device information 157 to the transmission/reception unit 154 .

The transmission/reception unit 154 is connected to the communication unit 121 of the specific object tracking device 100, which will be described later, via connection means such as a communication network. The transmitting/receiving unit 154 transmits the sensing information 156 and the sensor device information 157 generated by the control unit 153 to the communication unit 121 of the specific object tracking device 100 . The transmission/reception unit 154 acquires the sensor control signal 108 from the sensor control unit 123 of the specific object tracking device 100 via the communication unit 121 and sends it to the control unit 153 .

The sensing information 156 includes image data acquired by the sensor 152, the distance between the sensor 152 and the subject, and the relative positional relationship.

The sensor device information 157 includes information on the sensors 152 (type, number and sensing range) and installation position information on the sensor device 151 .

The sensor device 151 may be connected to the specific object tracking device 100 via a connection means such as a communication network, and an existing sensor device can be used.

The specific object tracking device 100 accesses the sensor devices 151 in a predetermined order, acquires sensing information 156 from the sensor devices 151 being accessed, and searches for the tracked object 135 . For example, when the tracking target object 135 cannot be detected with the sensing information 156A acquired from the sensor device 151A, the access destination is changed to the next sensor device 151B to search for the tracking target object 135 with the sensing information 156B. The sensor devices 151 to be accessed are sequentially changed, and the search for the tracked object 135 is repeated by each sensor device 151 until the tracked object 135 is detected.

When the specific object tracking device 100 detects the tracked object 135, the specific object tracking device 100 tracks the tracked object 135 using the sensing information 156A acquired from the sensor device 151 (for example, the sensor device 151A). When the specific object tracking device 100 determines that the tracking by the sensor device 151A cannot be continued, the specific object tracking device 100 selects the sensor device 151G that can continue tracking the tracked object 135 from the sensor devices 151B, 151C, . Tracking of the tracked object 135 is requested and tracking of the tracked object 135 is continued.

The specific object tracking device 100 acquires sensor device information 157A, 157B, 157C, . . . from all the sensor devices 151A, 151B, 151C, . The acquired sensor device information 157 is stored in the sensor device information storage unit 142, which will be described later.

The specific object tracking device 100 includes a sensing information acquisition unit 111, a tracking target object search unit 112, an operation mode control unit 113, a communication unit 121, an access control unit 122, a sensor control unit 123, a tracking data generation unit 131, and a tracking continuation propriety determination. It is composed of a unit 132 , a tracking request target selection unit 133 , a tracking object information storage unit 141 and a sensor device information storage unit 142 .

The sensing information acquisition unit 111 acquires sensing information 156 via the communication unit 121 from the accessed sensor device 151 . The sensing information acquisition unit 111 sends the acquired sensing information 156 to the tracked object search unit 112 .

The tracked object search unit 112 acquires the tracked object information 105 stored in the tracked object information storage unit 141, and compares the sensing information 156 acquired from the sensing information acquisition unit 111 with the tracked object information 105. Search for tracked object 135 . When searching for the tracked object 135, the tracked object search unit 112 generates a sensor control signal 108 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 152. and sent to the sensor control unit 123 .

If the tracked object 135 is a vehicle, the tracked object information 105 includes information such as the license plate (vehicle number), vehicle type, and body color. It is information such as an image or feature amount extraction data based on the image.

The tracked object search unit 112 searches for a subject from the moving image included in the sensing information 156, and collects information such as the license plate (vehicle number), vehicle type, and body color for each detected vehicle, and calculates feature values for each detected person. Information such as extraction data is extracted as identification information for identifying a subject.

Tracked object search unit 112 collates identification information for identifying a subject extracted from sensing information 156 with tracked object information 105 related to tracked object 135, and extracts a subject that matches tracked object information 105. to detect the tracked object 135 .

Here, from the relative positional relationship between the sensor 152 included in the sensing information 156 and the tracked object 135, which is the subject, the position of the tracked object 135 can be converted into world coordinates. If sensor 152 includes a GPS sensor, the position of sensor 152 or sensor device 151 can be obtained as world coordinates. Since the mounting states (positions, angles, etc.) of the sensor device 151 and the sensor 152 are known, it is possible to convert the position of the tracked object 135 from the world coordinates of the sensor device 151 into world coordinates. In the following description, it is not particularly specified whether the positions of the sensor device 151 and the tracked object 135 are relative positions between the sensor device 151 and the tracked object 135, or world coordinates converted as described above. It is simply referred to as location information.

The tracked object search unit 112 newly adds detection coordinates in the imaging area in the sensing information 156 of the detected tracked object 135 to the tracked object information 105, and searches for the tracked object 135 as search result information 117. is sent to operation mode control unit 113 . The search result information 117 includes detection result of the tracked object 135 , position information of the sensor device 151 and the tracked object 135 , sensing information 156 and tracked object information 105 .

The operation mode control unit 113 acquires the search result information 117 from the tracked object search unit 112, sets the operation mode according to the detection status of the tracked object 135 in the tracked object search unit 112, and stores it. The set operation modes are a search mode in which the tracked object search unit 112 searches for the tracked object 135, and a tracking mode in which the tracked object search unit 112 repeatedly detects the tracked object 135 and tracks the tracked object 135. any of the modes.

When the operation mode control unit 113 acquires the search result information 117 from the tracked object search unit 112, the operation mode control unit 113 confirms whether the set operation mode is the search mode or the tracking mode.

When the set operation mode is the search mode, the operation mode control unit 113 confirms the detection status of the tracked object 135 by the tracked object search unit 112 based on the search result information 117 . When the tracked object search unit 112 detects the tracked object 135, the operation mode control unit 113 switches the operation mode from the search mode to the tracking mode and resets the operation mode. 156 acquisition continues. When the tracked object 135 is not detected by the tracked object search unit 112 , the operation mode control unit 113 generates an access control signal 124 for changing the sensor device 151 to be accessed and sends it to the access control unit 122 . .

When the set operation mode is the tracking mode, the operation mode control unit 113 sends the search result information 117 of the tracked object 135 acquired from the tracked object search unit 112 to the tracking data generation unit 131 .

The communication unit 121 accesses and communicates with the transmission/reception unit 154 of the sensor device 151 set as an access target by the access control unit 122 . The communication unit 121 acquires the sensing information 156 and the sensor device information 157 from the sensor device 151 via the transmission/reception unit 154 of the sensor device 151, transmits the sensing information 156 to the sensing information acquisition unit 111, and transmits the sensor device information 157 to the sensor device information. Each is sent to the storage unit 142 . The communication unit 121 sends the sensor control signal 108 acquired from the sensor control unit 123 to the transmission/reception unit 154 of the sensor device 151 .

The communication means and communication method between the communication unit 121 and the communication unit 154 of the sensor device 151 are not particularly limited. For example, wireless communication lines such as wireless LAN (Local Area Network) communication and mobile phone lines, or wired communication lines such as fixed telephone lines and optical fiber lines can be used.

The access control unit 122 acquires the access control signal 124 generated by the operation mode control unit 113 or the tracking request target selection unit 133, and controls which sensor device 151 the specific object tracking device 100 accesses. The switching means of the sensor device 151 to be accessed and the method of connection with the sensor device 151 are not particularly limited. can do.

The sensor control unit 123 acquires the sensor control signal 108 generated by the tracked object search unit 112 or the tracking data generation unit 131, and transmits the signal to the sensor device via the communication unit 121 and the transmission/reception unit 154 of the sensor device 151 being accessed. 151 to the control unit 153 .

The tracking data generation unit 131 acquires the history of the position information associated with the tracking of the tracked object 135 included in the search result information 117 acquired from the tracked object search unit 112 via the operation mode control unit 113, and the image is captured at a new angle. Tracking data 107 including new information such as the captured image or feature amount data extracted from the image is generated and sent to the tracked object information storage unit 141 .

The tracking data generator 131 extracts feature amount data from an arbitrary image frame in the search result information 117 acquired from the tracked object searcher 112 via the operation mode controller 113 . The tracking data generation unit 131 not only tracks the tracked object 135 by searching for feature amounts that match or are similar to the feature amount data in subsequent image frames, but also uses a kernelized correlation filter to perform faster tracking. Target object 135 may also be tracked to generate tracking data 107 .

When generating the tracking data 107, the tracking data generation unit 131 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 152 so that the tracking target object 135 does not deviate from the imaging area. It generates a sensor control signal 108 for controlling and sends it to the sensor control unit 123 .

The tracking continuation enable/disable determination unit 132 determines whether tracking can be continued by the sensor device 151 being accessed (for example, the sensor device 151A). Based on the tracking data 107 acquired from the tracking data generation unit 131 and the sensor device information 157 acquired from the sensor device information storage unit 142, the tracking continuation possibility determination unit 132 determines the state of the tracking target object 135 and the tracking in the sensing range of the sensor 152A. The course of the tracked object 135 is estimated from the change history of the detected position of the target object 135 or the like. For example, if the tracked object 135 is a vehicle, the route of the tracked object 135 is estimated based on whether the tracked object 135 is in the straight lane or in the right turn (or left turn) lane. The tracking continuation possibility determination unit 132 sends the estimated course of the tracked object 135 to the tracking request target selection unit 133 .

The tracking continuation possibility determination unit 132 determines the relationship between the estimated course and the sensing range of the sensor 152A provided in the sensor device 151A, and the change history of the detection position and detection size of the tracked object 135 in the sensing range of the sensor 152A. 151A determines whether or not tracking can be continued.

If it is determined that the sensor device 151A can continue tracking, the search (detection) of the tracked object 135 is repeated using the sensing information 156A. When it is determined that the sensor device 151A cannot continue tracking, the tracking continuation possibility determination section 132 sends the determination result to the tracking request target selection section 133 .

For example, when the estimated course of the tracked object 135 is included in the blind spot of the sensor 152A, or when the detection size of the tracked object 135 in the sensing range of the sensor 152A changes small (or large), the magnification of the sensor 152A is changed. If it is assumed that the tracking target object 135 cannot be detected even if the tracking target object 135 is detected, the tracking continuation possibility determination unit 132 determines that the tracking continuation by the sensor device 151A is not permitted.

Tracking request target selection unit 133 selects sensor device information 157B, 157C, . . . related to other sensor devices 151B, 151C, . A sensor device 151G that is a candidate for a tracking request for the tracked object 135 is selected from the estimated track of the tracked object 135 and the positional relationship between the sensor devices. The tracking request target selection unit 133 sends the tracking data 107 acquired from the tracking object information storage unit 141 to the selected sensor device 151G, and acquires the result of whether or not the tracking target object 135 can be detected from the sensor device 151G. If the tracking target object 135 can be detected by the sensor device 151G, the sensor device 151G is selected as the request target.

Here, if the determination result of the tracking continuation permission determination unit 132 is negative, the tracking request target selection unit 133 generates an access control signal 124 for changing the access destination sensor device 151 to the sensor device 151G, and controls access control. 122. If the determination result of the tracking continuation permission determination unit 132 is yes, the tracking request target selection unit 133 generates the tracking request candidate information 137 of the sensor device 151G selected as the request target, and stores it in the sensor device information storage unit 142. do.

If the tracking request target object 135 is not detected by the sensor device 151G, the tracking request target selection unit 133 redoes selection of the sensor device 151 as a tracking request candidate, and selects another sensor device as a new tracking request candidate. 151H is selected.

The tracked object information storage unit 141 stores the tracked object information 105 as well as the tracked data 107 generated by the tracked data generating unit 131 .

The sensor device information storage unit 142 stores information (type and number) of the sensors 152 included in all the sensor devices 151 linked to the specific object tracking device 100, as well as information such as installation position information and sensing range of the sensor devices 151. and store the sensor device information 157 .

With reference to FIG. 2, the tracking processing procedure of the tracking target object 135 of the specific object tracking device 100 in the specific object tracking system 1000 of this embodiment will be described. FIG. 2 is a flowchart showing the processing procedure of the specific object tracking device 100 in the specific object tracking system 1000. As shown in FIG.

When the specific object tracking device 100 starts searching for the tracked object 135 (S100), the specific object tracking device 100 acquires sensor device information 157 from all linked sensor devices 151 through the communication unit 121. (S100-1). The specific object tracking device 100 stores the acquired sensor device information 157 in the sensor device information storage unit 142 (S100-2).

The specific object tracking device 100 accesses the access target sensor device 151 set by the access control unit 122 through the communication unit 121 . The sensor device 151 accessed by the specific object tracking device 100 is selected by the access control section 122 based on the access control signal 124 . The first setting accessed at the start of a search is sensor device 151A. When the sensor device 151A is selected based on the access control signal 124, the access control unit 122 sets the operation mode to the search mode by the operation mode control unit 113 (S101).

The specific object tracking device 100 acquires the sensing information 156A from the accessing sensor device 151A by the sensing information acquisition unit 111 (S102). The sensing information acquiring unit 111 sends the acquired sensing information 156A to the tracking target object searching unit 112 (S102).

The tracked object search unit 112 acquires the tracked object information 105 from the tracked object information storage unit 141, compares the sensing information 156A acquired from the sensing information acquisition unit 111 with the tracked object information 105, and finds the tracked object 135. (S103).

When searching for the tracked object 135, the tracked object search unit 112 generates the sensor control signal 108 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 152A. , to the sensor control unit 123 (S103).

The search for the tracked object 135 by the tracked object search unit 112 will be specifically described for the case where the tracked object 135 is a vehicle and the case where the tracked object 135 is a person.

When the tracked object 135 is a vehicle, the tracked object searching unit 112 searches for the vehicle included in the acquired sensing information 156A, and checks the license plate (vehicle number), vehicle type, body color, etc. of each detected vehicle. is extracted as identification information for identifying the vehicle. The tracked object search unit 112 collates the extracted identification information with the acquired tracked object information 105 , extracts vehicles that match the tracked object information 105 , and detects the tracked object 135 .

When the tracked object 135 is a person, the tracked object searching unit 112 searches for the person included in the acquired sensing information 156A, and for each detected person, an image such as age, sex, face photograph, etc. Information such as feature amount extraction data based on this is extracted as identification information for identifying a person. The tracked object search unit 112 collates the extracted identification information with the acquired tracked object information 105 to extract a person who matches the tracked object information 105 and detects the tracked object 135 .

Next, the specific object tracking device 100 confirms whether the set operation mode is the search mode or the tracking mode by the operation mode control unit 113 (S104).

If the operation mode confirmed by the operation mode control unit 113 in step S104 is the search mode set in step S101 (search mode in S104), the operation mode control unit 113 controls the tracking object search unit in step S103. It is checked whether the tracked object 135 has been detected by the search in 112 (S105).

When the tracked object 135 has been detected (detected in S105), the operation mode control unit 113 switches the operation mode from the search mode to the tracking mode (S106). After that, the process returns to step S102.

If the tracked object 135 has not been detected (not detected in S105), the operation mode control unit 113 generates an access control signal 124 for changing the sensor device 151 to be accessed and sends it to the access control unit 122 (S107). ). After that, the process proceeds to step S120.

If the operation mode confirmed by the operation mode control unit 113 in step S104 is the tracking mode (tracking mode in S104), the operation mode control unit 113 retrieves the search result information of the tracked object 135 acquired from the tracked object search unit 112. 117 to the tracking data generator 131 . The tracking data generator 131 generates the tracking data 107 from the acquired search result information 117 and stores it in the tracked object information storage 141 (S110).

Next, the specific object tracking device 100 determines whether or not the tracking of the tracking target object 135 by the sensor device 151A can be continued by the tracking continuation possibility determination unit 132 (S111), and the tracking request target selection unit 133 determines whether the sensor device In parallel with selection of a target for which tracking continuation is requested (S112) when tracking continuation is not possible in 151A.

3 to 8, a detailed procedure for determining whether or not tracking can be continued in the sensor device 151A in the tracking continuation determination unit 132 (S111) will be described. FIG. 3 is a flow chart showing the detailed processing procedure of the tracking continuation permission determination (S111) in the sensor device 151A in the tracking continuation permission determination unit 132. As shown in FIG. FIG. 4 is an image when the estimated course of the tracked object 135 is not included in the blind spot of the sensor 152A, and FIG. 5 is an image when the estimated course of the tracked object 135 is included in the blind spot of the sensor 152A. 6 is an image when the detection position of the tracked object 135 does not change in the sensing range of the sensor 152A, FIG. 7 is an image when the detection position of the tracked object 135 changes in the sensing range of the sensor 152A, and FIG. It is an image when the detection size of the tracked object 135 changes in the sensing range of the sensor 152A.

The tracking continuation possibility determination unit 132 acquires the tracking data 107 from the tracking data generation unit 131 and the tracking object information storage unit 141 (S111-1).

The tracking continuation possibility determination unit 132 estimates the course of the tracked object 135 from the acquired tracking data 107 (S111-2). By comparing the estimated course of the tracked object 135 and the blind spot of the sensor 152A, it is determined whether or not the tracking of the tracked object 135 by the sensor device 151A can be continued (S111-3).

Alternatively, the tracking continuation possibility determination unit 132 extracts the change history of the detected position and size of the tracked object 135 from the tracking data 107 (S111-4). The extracted change history and the sensing range of the sensor 152A are compared to determine whether or not the tracking of the tracked object 135 by the sensor device 151A can be continued (S111-5).

In the determination in step S111-3, for example, when the estimated course of the tracked object 135 is not included in the blind spot of the sensor 152A as shown in FIG. 4, it is determined that the sensor device 151A can continue tracking. Here, the blind spot of the sensor 152A refers to a non-sensable area of the sensor 152A that is generated when the outer wall of the building in which the sensor device 151A is installed is included in the sensing range of the sensor 152A. On the other hand, when the estimated course of the tracked object 135 is included in the blind spot of the sensor 152A as shown in FIG. 5, it is determined that the sensor device 151A cannot continue tracking.

In the determination in step S111-5, for example, as shown in FIG. 6, if the detected position and size of the tracked object 135 in the sensing range of the sensor 152A do not change, the sensor device 151A can continue tracking. I judge. On the other hand, as shown in FIG. 7, the detection position of the tracked object 135 in the sensing range of the sensor 152A is moving in the horizontal direction (or in the vertical direction), and it is assumed that it is out of the sensing range of the sensor 152A (into a blind spot). 8, the detection size of the tracked object 135 in the sensing range of the sensor 152A is small (or large), and even if the magnification of the sensor 152A is changed, the tracked object 135 can be detected ( identification), it is determined that the sensor device 151A cannot continue tracking.

If both the determination result in step S111-3 and the determination result in step S111-5 are acceptable, the tracking continuation possibility determination unit 132 determines that the tracking of the tracking target object 135 by the sensor device 151A can be continued. (S111-6), and if both the determination result in step S111-3 and the determination result in step S111-5 or either one is negative, the tracking target object 135 is tracked by the sensor device 151A. It is determined that continuation is not possible (S111-6).

When the tracking continuation possibility determination unit 132 determines that the sensor device 151A can continue tracking the tracked object 135 (Yes in S111-6), the tracking continuation possibility determination unit 132 transmits the sensing information via the communication unit 121. The acquisition unit 111 continues acquiring new sensing information 156A, returns to step S102, and repeats the search and detection of the tracked object 135 to track the tracked object 135. FIG.

When the tracking continuation possibility determination unit 132 determines that the sensor device 151A cannot continue tracking the tracked object 135 (No in S111-6), the tracking continuation possibility determination unit 132 determines whether the sensor device 151A can continue tracking. A determination result indicating that the continuation is not permitted is sent to the tracking request target selection unit 133 .

Referring to FIG. 9, a detailed procedure for selecting another sensor device 151 candidate (S112) for requesting continuation of tracking by the sensor device 151A in the tracking request target selection unit 133 will be described. FIG. 9 is a flow chart showing the detailed processing procedure of candidate selection (S112) of the other sensor device 151 in the tracking request target candidate selection unit 133. As shown in FIG.

The tracking request target selection unit 133 checks whether the tracking request candidate information 137 is stored in the tracking object information storage unit 141 (S112-1).

If the tracking request candidate information 137 is not stored in the tracking object information storage unit 141 (NO in S112-1), the sensor device information 157B, 157C, . (S112-2).

Further, the tracking request target selection unit 133 acquires the estimated course of the tracking target object 135 estimated in step S111-2 from the tracking continuation possibility determination unit 132 (S112-3).

Then, the relationship between the acquired estimated course of the tracked object 135 and the sensing ranges of the sensor devices 151B, 151C, . . . obtained from the sensor device information 157B, 157C, . , the sensor device 151G whose sensing range includes the estimated course of the tracked object 135 is selected as a request candidate sensor device (S112-4).

The tracking request target selection unit 133 sends the tracking data 107 via the communication unit 121 to the sensor device 151G selected as a request candidate (S112-5). The tracking request target selection unit 133 acquires and confirms whether or not the tracking target object 135 can be detected by the sensor device 151G (S112-6).

If the object 135 to be tracked is not detected by the sensor device 151G (NO in S112-6), the selection of sensor devices as request candidates is redone in step S112-4. If the object 135 to be tracked is detected by the sensor device 151G (possible in S112-6), the sensor device 151G is again selected as a request candidate sensor device (S112-7).

The tracking request target selection unit 133 confirms whether or not there is a determination result in step S111 by the tracking continuation possibility determination unit 132 (S112-8). If there is no determination result in step S111 (not in S112-8=determination of S111 “Yes”), the tracking request target selection unit 133 stores the information of the sensor device 151G as the tracking request candidate information 137 in the sensor device information storage unit. 142 (S112-9).

If there is a determination result in step S111 (yes in S112-8=determination "no" in S111), the tracking request target selection unit 133 generates an access control signal 124 for changing the access target to the sensor device 151G. It is sent to the access control unit 122 (S112-10). After that, the process proceeds to step S120.

When the specific object tracking device 100 fails to detect the tracked object 135 (undetected in S105), the access control unit 122 accesses the specific object tracking device 100 based on the access control signal 124 generated by the operation mode control unit 113. 151 is changed (S120). For example, if the sensor device 151A fails to detect the tracked object 135, the sensor device 151 to be accessed next by the specific object tracking device 100 is selected in a predetermined order by the access control unit 122. FIG.

In the specific object tracking device 100, when the tracking continuation possibility determination unit 132 determines that the sensor device 151A cannot continue tracking (S111) and the tracking request target selection unit 133 selects the request candidate sensor device 151G. At (S112), based on the access control signal 124 generated by the tracking request target selection unit 133, the access control unit 122 changes the sensor device 151 to be accessed to the sensor device 151G (S120).

The specific object tracking device 100 is not limited to the fixed point installation type described in the present embodiment, and may be a mobile installation type mounted on a mobile such as a vehicle.

(Effect of the first embodiment)
The specific object tracking system 1000 of this embodiment is composed of the specific object tracking device 100 and a plurality of sensor devices 151 that are installed at fixed points. Detailed tracking is possible in large areas. Furthermore, the specific object tracking device 100 of this embodiment changes the access target to the surrounding sensor devices 151B, 151C, . , . . . , continuous tracking in terms of time is also possible.

The specific object tracking device 100 of this embodiment autonomously executes a series of processing operations for detecting and tracking the tracked object 135 and requesting the other sensor device 151 to track it. Therefore, it is not necessary for the operator of the specific object tracking device or the like to perform the processing operations related to detecting and tracking the tracked object 135 or requesting the other sensor device 151 to track it.

(Second embodiment)
A specific object tracking system 2000 according to the second embodiment of the present invention will be described with reference to FIG. A specific object tracking system 2000 is composed of a specific object tracking device 200 and a plurality of sensor devices 251 . FIG. 10 is a configuration block diagram of the specific object tracking system 2000. As shown in FIG. This embodiment differs from the specific object tracking system 1000 in the first embodiment in that the sensor device 251 is a mobile object such as a vehicle. An example in which the specific object tracking device 200 according to the present embodiment is of a fixed point installation type like the specific object tracking device 100 according to the first embodiment will be described. An example of fixed-point installation of the specific object tracking device 200 corresponds to a case where it is installed at a central office such as an information management center.

The sensor device 251 is a vehicle as described above, and the sensor device 251 may be referred to as the vehicle 251 in the following description.

The sensor device 251 is a device that generates sensing information 156 from image data acquired by a sensor 252, which will be described later. The sensor device 251 includes a sensor 252, a control section 253, and a transmission/reception section 254 (not shown in FIG. 10), like the sensor device 151 in the first embodiment.

Sensor 252 is a video camera such as a drive recorder. However, the sensor 252 is not limited to a moving image camera, and may be a still image camera that captures still images, a distance measurement sensor such as LIDAR or millimeter wave radar, or the like. Moreover, the sensor 252 does not need to be single, and a plurality of types of the sensors described above may be used together, or a GPS sensor or the like for acquiring position information as world coordinates may be provided together.

The sensor 252 sends the acquired image data, the distance to the subject, and the relative positional relationship to the control unit 253 . The control unit 253 generates sensing information 256 from the image data acquired from the sensor 252, the distance to the subject, and the relative positional relationship. The control unit 253 generates sensor device information 257 from the information of the sensors 252 (type, number and sensing range) and the installation position information of the sensor device 251 . The control unit 253 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification of the sensor 252 based on the sensor control signal 208 acquired from the sensor control unit 223 of the specific object tracking device 200 (to be described later) via the transmission/reception unit 254. (zoom) and controls. The control unit 253 sends the generated sensing information 256 and sensor device information 257 to the transmission/reception unit 254 .

The transmission/reception unit 254 is connected to the communication unit 221 of the specific object tracking device 200, which will be described later, via connection means such as a communication network. The transmission/reception unit 254 sends the sensing information 256 and the sensor device information 257 generated by the control unit 253 to the communication unit 221 of the specific object tracking device 200 . The transmission/reception unit 254 acquires the sensor control signal 208 from the sensor control unit 223 of the specific object tracking device 200 via the communication unit 221 and sends it to the control unit 253 .

The sensing information 256 includes image data acquired by the sensor 252, the distance between the sensor 252 and the subject, and the relative positional relationship.

The sensor device information 257 includes information on the sensors 252 (type, number, and sensing range), position information on the sensor devices 251, and a planned route.

The specific object tracking device 200 accesses the sensor device 251 in a predetermined order, acquires sensing information 256 from the sensor device 251 , and searches for the tracked object 235 . For example, when the tracking target object 235 cannot be detected by the sensing information 256A acquired from the sensor device 251A, the access destination is changed to the next sensor device 251B and the tracking target object 235 is searched by the sensing information 256B. The sensor devices 251 to be accessed are sequentially changed, and the search for the tracked object 235 by each sensor device 251 is repeated until the tracked object 235 is detected.

When the specific object tracking device 200 detects the tracked object 235, the specific object tracking device 200 tracks the tracked object 235 using the sensing information 256A acquired from the sensor device 251 (for example, the sensor device 251A). When the specific object tracking device 200 determines that the tracking by the sensor device 251A cannot be continued, the specific object tracking device 200 selects the sensor device 251G that can continue tracking the tracked object 235 from the sensor devices 251B, 251C, . Tracking of the tracked object 235 is requested and tracking of the tracked object 235 is continued.

The specific object tracking device 200 acquires sensor device information 257A, 257B, 257C, . . . from all the sensor devices 251A, 251B, 251C, . The acquired sensor device information 257 is stored in the sensor device information storage unit 242, which will be described later. Unlike the sensor device 151 in the first embodiment, the sensor device 251 in the present embodiment is a moving object (vehicle), so the position information of the sensor device 251 in particular changes from moment to moment.

The specific object tracking device 200 periodically acquires the sensor device information 257 from the sensor device 251 being accessed and updates its position information. In addition, sensor device information 257 is periodically obtained for other linked sensor devices 251, and the position information thereof is updated.

The specific object tracking device 200 includes a sensing information acquisition unit 211, a tracked object search unit 212, an operation mode control unit 213, a communication unit 221, an access control unit 222, a sensor control unit 223, a tracking data generation unit 231, and a determination as to whether or not tracking can be continued. It is composed of a unit 232 , a tracking request target selection unit 233 , a tracking object information storage unit 241 and a sensor device information storage unit 242 .

The sensing information acquisition unit 211 acquires sensing information 256 via the communication unit 221 from the accessed sensor device 251 . The sensing information acquisition unit 211 sends the acquired sensing information 256 to the tracked object search unit 212 .

The tracked object search unit 212 acquires the tracked object information 205 stored in the tracked object information storage unit 241, and compares the sensing information 256 acquired from the sensing information acquisition unit 211 with the tracked object information 205. Search for tracked object 235 . The tracked object search unit 212 generates a sensor control signal 208 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 252 when searching for the tracked object 235. and sent to the sensor control unit 223 .

If the tracked object 235 is a vehicle, the tracked object information 205 includes information such as the license plate (vehicle number), vehicle type, and body color. It is information such as an image or feature amount extraction data based on the image.

The tracked object search unit 212 searches for a subject from the moving image included in the sensing information 256, and obtains information such as the license plate (vehicle number), vehicle type, and body color for each detected vehicle, and calculates feature values for each detected person. Information such as extraction data is extracted as identification information for identifying a subject.

Tracked object search unit 212 collates identification information for identifying a subject extracted from sensing information 256 with tracked object information 205 related to tracked object 235, and extracts a subject that matches tracked object information 205. to detect the tracked object 235 .

Here, from the relative positional relationship between the sensor 252 included in the sensing information 256 and the tracked object 235, which is the subject, the position of the tracked object 235 can be converted into world coordinates. If sensor 252 includes a GPS sensor, the position of sensor 252 or sensor device 251 can be obtained as world coordinates. Since the mounting state (position, angle, etc.) of the sensor 252 on the sensor device (vehicle) 251 is known, it is also possible to convert the position of the tracked object 235 from the world coordinates of the sensor device 251 to the world coordinates. In the following description, it is not particularly specified whether the positions of the sensor device 251 and the tracked object 235 are relative positions between the sensor device 251 and the tracked object 235 or world coordinates converted as described above. It is simply referred to as location information.

The tracked object search unit 212 newly adds the detection coordinates in the imaging area in the sensing information 256 of the detected tracked object 235 to the tracked object information 205, and searches for the tracked object 235 as search result information 217. is sent to the operation mode control unit 213 . The search result information 217 includes detection result of the tracked object 235 , position information of the sensor device 251 and the tracked object 235 , sensing information 256 and tracked object information 205 .

The operation mode control unit 213 acquires the search result information 217 from the tracked object search unit 212, sets the operation mode according to the detection status of the tracked object 235 in the tracked object search unit 212, and stores it. The set operation modes are a search mode in which the tracked object search unit 212 searches for the tracked object 235, and a tracking mode in which the tracked object search unit 212 repeatedly detects the tracked object 235 and tracks the tracked object 235. any of the modes.

When the operation mode control unit 213 acquires the search result information 217 from the tracked object search unit 212, it confirms whether the set operation mode is the search mode or the tracking mode.

When the set operation mode is the search mode, the operation mode control unit 213 confirms the detection status of the tracked object 235 by the tracked object search unit 212 based on the search result information 217 . When the tracked object search unit 212 detects the tracked object 235, the operation mode control unit 213 switches the operation mode from the search mode to the tracking mode and resets the operation mode. 256 acquisition continues. If the tracked object search unit 212 does not detect the tracked object 235 , the operation mode control unit 213 generates an access control signal 224 for changing the access target sensor device 251 and sends it to the access control unit 222 .

When the set operation mode is the tracking mode, the operation mode control unit 213 sends the search result information 217 of the tracked object 235 acquired from the tracked object search unit 212 to the tracking data generation unit 231 .

The communication unit 221 accesses and communicates with the transmission/reception unit 254 of the sensor device 251 set as an access target by the access control unit 222 . The communication unit 221 acquires the sensing information 256 and the sensor device information 257 from the sensor device 251 via the transmission/reception unit 254 of the sensor device 251, transmits the sensing information 256 to the sensing information acquisition unit 211, and transmits the sensor device information 257 to the sensor device information. Each is sent to the storage unit 242 . The communication unit 221 sends the sensor control signal 208 acquired from the sensor control unit 223 to the transmission/reception unit 254 of the sensor device 251 .

The communication means and communication method between the communication unit 221 and the communication unit 254 of the sensor device 251 are not particularly limited. For example, wireless communication lines such as wireless LAN communication and mobile phone lines, or wired communication lines such as fixed telephone lines and optical fiber lines can be used.

The access control unit 222 acquires an access control signal 224 generated by the operation mode control unit 213 or the tracking request target selection unit 233, and controls which sensor device 251 the specific object tracking device 200 accesses. The switching means of the sensor device 251 to be accessed and the method of connection with the sensor device 251 are not particularly limited. can do.

The sensor control unit 223 acquires the sensor control signal 208 generated by the tracked object search unit 212 or the tracking data generation unit 231, and transmits the sensor control signal 208 to the sensor device via the communication unit 221 and the transmission/reception unit 254 of the sensor device 251 being accessed. 251 to the control unit 253 . The sensor 252 of the sensor device 251 being accessed cannot control the rotation angle (pan), elevation angle or depression angle (tilt), imaging magnification (zoom), etc. like a general drive recorder exemplified in this embodiment. In this case, the sensor control signal 208 is invalidated in the sensor control section 223 or the control section 253 of the sensor device 251 .

The tracking data generation unit 231 acquires the history of the position information associated with the tracking of the tracked object 235 included in the search result information 217 acquired from the tracked object search unit 212 via the operation mode control unit 213 and the image at a new angle. Tracking data 207 including new information such as the captured image or feature amount data extracted from the image is generated and sent to the tracked object information storage unit 241 .

The tracking data generation unit 231 extracts feature amount data from an arbitrary image frame in the search result information 217 acquired from the tracked object search unit 212 via the operation mode control unit 213 . The tracking data generation unit 231 not only tracks the tracked object 235 by searching for feature amounts that match or are similar to the feature amount data in subsequent image frames, but also uses a kernelized correlation filter to perform faster tracking. Target object 235 may also be tracked to generate tracking data 207 .

When generating the tracking data 207, the tracking data generation unit 231 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 252 so that the tracking target object 235 does not deviate from the imaging area. It generates a sensor control signal 208 for controlling and sends it to the sensor control unit 223 .

The tracking continuation enable/disable determination unit 232 determines whether tracking can be continued by the sensor device 251 being accessed (for example, the sensor device 251A). The tracking continuation possibility determination unit 232 uses the tracking data 207 acquired from the tracking object information storage unit 241 to determine the tracking target object based on the state of the tracking target object 235 and the change history of the detection position of the tracking target object 235 in the sensing range of the sensor 252A. Estimate the path of 235. For example, if the tracked object 235 is a vehicle, the route of the tracked object 235 is estimated based on whether the tracked object 235 is in the straight lane or in the right turn (or left turn) lane. The tracking continuation possibility determination unit 232 sends the estimated course of the tracked object 235 to the tracking request target selection unit 233 .

Furthermore, the tracking continuation permission determination unit 232 acquires the planned route of the sensor device 251A that is being accessed from the sensor device information storage unit 242 . The tracking continuation possibility determination unit 232 determines the relationship between the estimated course of the tracked object 235 and the planned route of the sensor device 251A, or the change history of the detected position and detection size of the tracked object 235 in the sensing range of the sensor 252A. 251A determines whether or not tracking can be continued.

If it is determined that the sensor device 251A can continue tracking, the search (detection) of the tracked object 235 is repeated using the sensing information 256A. When it is determined that the sensor device 251A cannot continue tracking, the tracking continuation possibility determination section 232 sends the determination result to the tracking request target selection section 233 .

For example, when the estimated route of the tracked object 235 and the planned route of the sensor device 251A are different, or when the detection size of the tracked object 235 in the sensing range of the sensor 252A changes small (or large), the magnification of the sensor 252A If it is assumed that the tracking target object 235 cannot be detected even if the .

Tracking request target selection unit 233 selects sensor device information 257B, 257C, . . . related to other sensor devices 251B, 251C, . A sensor device 251</b>G is selected as a tracking request candidate for the tracked object 235 based on the relationship between the estimated track of the tracked object 235 and the current position and planned route of each sensor device 251 . The tracking request target selection unit 233 sends the tracking data 207 acquired from the tracking object information storage unit 241 to the selected sensor device 251G, and acquires the detection result of the tracking target object 235 from the sensor device 251G. If the sensor device 251G can detect the tracked object 235, the sensor device 251G is selected as the request target.

Here, if the determination result of the tracking continuation permission determination unit 232 is negative, the tracking request target selection unit 233 generates an access control signal 224 for changing the access destination sensor device 251 to the sensor device 251G, and controls the access. 222. If the determination result of the tracking continuation permission determination unit 232 is yes, the tracking request target selection unit 233 generates tracking request candidate information 237 regarding the sensor device 251G selected as the request target, and stores it in the sensor device information storage unit 242. do. If the tracking target object 235 is not detected by the sensor device 251G, the selection of the sensor device 251 as a tracking request candidate is redone, and another sensor device 251H is selected as a new tracking request candidate.

The tracked object information storage unit 241 stores the tracked object information 205 as well as the tracked data 207 generated by the tracked data generation unit 231 .

The sensor device information storage unit 242 stores the information (type and number) of the sensors 252 included in all the sensor devices 251 linked to the specific object tracking device 200, as well as the position information, planned route, sensing range, etc. of the sensor devices 251. The sensor device information 257 including the information of is stored.

With reference to FIG. 11, the tracking processing procedure of the tracking target object 235 of the specific object tracking device 200 in the specific object tracking system 2000 in this embodiment will be described. FIG. 11 is a flow chart showing the processing procedure of the specific object tracking device 200 in the specific object tracking system 2000. As shown in FIG.

When the specific object tracking device 200 starts searching for the tracking target object 235 (S200), the specific object tracking device 200 receives the sensor device information 257 from all the linked sensor devices 251 via the communication unit 221. (S200-1). The specific object tracking device 200 stores the acquired sensor device information 257 in the sensor device information storage unit 242 (S200-2).

The specific object tracking device 200 accesses the access target sensor device 251 set by the access control unit 222 through the communication unit 221 . The sensor device 251 accessed by the specific object tracking device 200 is selected by the access control section 222 based on the access control signal 224 . The first setting accessed at the start of a search is sensor device 251A. When the sensor device 251A is selected based on the access control signal 124, the access control unit 222 sets the operation mode to the search mode by the operation mode control unit 213 (S201).

The specific object tracking device 200 acquires the sensing information 256A and the sensor device information 257A from the accessing sensor device 251A by the sensing information acquisition unit 211 (S202). The sensing information acquisition unit 211 stores (overwrites) the acquired sensor device information 257A in the sensor device information storage unit 242 (S202-1). The sensing information acquisition unit 211 sends the acquired sensing information 256A to the tracked object search unit 212 (S202-2).

The tracked object search unit 212 acquires the tracked object information 205 from the tracked object information storage unit 241, compares the sensing information 256A acquired from the sensing information acquisition unit 211 with the tracked object information 205, and finds the tracked object 235. (S203).

When searching for the tracked object 235, the tracked object search unit 212 generates a sensor control signal 208 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 252A. , to the sensor control unit 223 (S203).

The search for the tracked object 235 by the tracked object search unit 212 will be specifically described for the case where the tracked object 235 is a vehicle and the case where the tracked object 235 is a person.

When the tracked object 235 is a vehicle, the tracked object search unit 212 searches for the vehicle included in the acquired sensing information 256A, and checks the license plate (vehicle number), vehicle type, body color, etc. of each detected vehicle. is extracted as identification information for identifying the vehicle. The tracked object search unit 212 compares the extracted identification information with the acquired tracked object information 205 , extracts vehicles matching the tracked object information 205 , and detects the tracked object 235 .

When the tracked object 235 is a person, the tracked object searching unit 212 searches for the person included in the acquired sensing information 256A, and for each detected person, an image such as age, sex, face photograph, etc. Information such as feature quantity extraction data based on this is extracted as identification information for identifying a person. The tracked object search unit 212 compares the extracted identification information with the acquired tracked object information 205 , extracts a person who matches the tracked object information 205 , and detects the tracked object 235 .

Next, the specific object tracking device 200 uses the operation mode control unit 213 to confirm whether the recorded operation mode is the search mode or the tracking mode (S204).

If the operation mode confirmed by the operation mode control unit 213 in step S204 is the search mode set in step S201 (search mode in S204), the operation mode control unit 213 controls the tracked object search unit in step S203. It is checked whether the tracked object 235 has been detected by the search in 212 (S205).

If the tracked object 235 has been detected (detected in S205), the operation mode control unit 213 switches the operation mode from the search mode to the tracking mode (S206). After that, the process returns to step S202.

If the tracked object 235 has not been detected (not detected in S205), the operation mode control unit 213 generates an access control signal 224 for changing the access target sensor device 251 and sends it to the access control unit 222 (S207). . After that, the process proceeds to step S220.

If the operation mode confirmed by the operation mode control unit 213 in step S204 is the tracking mode (tracking mode in S204), the operation mode control unit 213 retrieves the search result information of the tracked object 135 acquired from the tracked object search unit 212. 217 to the tracking data generator 231 . The tracking data generator 231 generates the tracking data 207 from the acquired search result information 217 and stores it in the tracked object information storage 241 (S210).

Next, the specific object tracking device 200 determines whether or not the tracking of the tracking target object 235 by the sensor device 251A can be continued by the tracking continuation possibility determination unit 232 (S211), and the tracking request target selection unit 233 determines whether the sensor device If 251A is unable to continue tracking, selection of a target for which continued tracking is requested (S212) is executed in parallel.

12 to 14, a detailed procedure for determining whether or not tracking can be continued in the sensor device 251A in the tracking continuation determination unit 232 (S211) will be described. FIG. 12 is a flow chart showing the detailed processing procedure of the tracking continuation permission/inhibition determination (S211) in the sensor device 251A in the tracking continuation permission/inhibition determination unit 232. As shown in FIG. FIG. 13 shows an image when the estimated course of the tracked object 235 and the planned route of the sensor device 251A (=vehicle 251A) are the same, and FIG. 15 is an image of when the detection position of the tracked object 235 does not change in the sensing range of the sensor 252A, and FIG. 16 is an image of when the detection position of the tracked object 235 changes in the sensing range of the sensor 252A. An image, FIG. 17 is an image when the detection size of the tracked object 235 changes in the sensing range of the sensor 252A.

The tracking continuation possibility determination unit 232 acquires the tracking data 207 from the tracking data generation unit 231 and the tracking object information storage unit 241 (S211-1).

The tracking continuation possibility determination unit 232 estimates the course of the tracked object 235 from the acquired tracking data 207 (S211-2). The estimated route of the tracked object 235 and the planned route of the sensor device 251A are compared to determine whether or not the sensor device 251A can continue to track the tracked object 235 (S211-3).

Alternatively, the tracking continuation possibility determination unit 232 extracts the change history of the detected position and detected size of the tracked object 235 from the tracking data 207 (S211-4). The extracted change history and the sensing range of the sensor 252A are compared to determine whether or not the tracking of the tracked object 235 by the sensor device 251A can be continued (S211-5).

The determination in step S211-3 is that, for example, as shown in FIG. 13, if the estimated course of the tracked object 235 and the planned route of the sensor device 251A are the same, it is possible to continue tracking with the sensor device 251A. judge. On the other hand, as shown in FIG. 14, when the estimated course of the tracked object 235 and the planned course of the sensor device 251A are different, it is determined that the sensor device 251A cannot continue tracking.

In the determination in step S211-5, for example, as shown in FIG. 15, if the detected position and size of the tracked object 235 in the sensing range of the sensor 252A do not change, the sensor device 251A can continue tracking. I judge. On the other hand, as shown in FIG. 16, the detection position of the tracked object 235 in the sensing range of the sensor 252A is moving in the horizontal direction (or in the vertical direction), and it is assumed that it is out of the sensing range of the sensor 252A (into a blind spot). 17, the detection size of the tracked object 235 in the sensing range of the sensor 252A is small (or large), and even if the magnification of the sensor 252A is changed, the tracked object 235 can be detected ( identification), it is determined that the sensor device 251A cannot continue tracking.

If both the determination result in step S211-3 and the determination result in step S211-5 are acceptable, the tracking continuation possibility determination unit 232 determines that the tracking of the tracking target object 235 by the sensor device 251A can be continued. (S211-6), and if both the determination result in step S211-3 and the determination result in step S211-5, or either one is negative, the tracking target object 235 is tracked by the sensor device 251A. It is determined that continuation is not possible (S211-6).

If the tracking continuation possibility determination unit 232 determines that the sensor device 251A can continue tracking the tracked object 235 (Yes in S211-6), the tracking continuation possibility determination unit 232 transmits the sensing information via the communication unit 221. The acquisition unit 211 continues acquiring new sensing information 256A, returns to step S202, repeats the search and detection of the tracking target object 235, and tracks the tracking target object 235. FIG.

When the tracking continuation possibility determination unit 232 determines that the sensor device 251A cannot continue tracking the tracked object 235 (No in S211-6), the tracking continuation possibility determination unit 232 determines whether the sensor device 251A can continue tracking. A determination result that the continuation is not permitted is sent to the tracking request target selection unit 233 .

Referring to FIG. 18, a detailed procedure for selecting another sensor device 251 candidate for requesting continuation of tracking by sensor device 251A in tracking request target selection unit 233 (S212) will be described. FIG. 18 is a flow chart showing a detailed processing procedure of candidate selection (S212) of the other sensor device 251 in the tracking request target candidate selection unit 233. As shown in FIG.

The tracking request target selection unit 233 checks whether the tracking request candidate information 237 is stored in the tracking object information storage unit 241 (S212-1).

If the tracking request candidate information 237 is not stored in the tracking object information storage unit 241 (NO in S212-1), the sensor device information 257B, 257C, . (S212-2).

Further, the tracking request target selection unit 233 acquires the estimated course of the tracking target object 235 estimated in step S211-2 from the tracking continuation possibility determination unit 232 (S212-3).

Then, the estimated path of the tracked object 235 obtained is compared with the positional relationship between the sensor devices 251B, 251C obtained from the sensor device information 257B, 257C, . The sensor device 251G located on the estimated route of 235 is selected as a request candidate sensor device (S212-4).

The tracking request target selection unit 233 sends the tracking data 207 via the communication unit 221 to the sensor device 251G selected as a request candidate (S212-5). The tracking request target selection unit 233 acquires and confirms whether or not the tracking target object 235 can be detected by the sensor device 251G (S212-6).

If the object 235 to be tracked is not detected by the sensor device 251G (No in S212-6), the selection of sensor devices as request candidates is redone in step S212-4. If the tracking target object 235 is detected by the sensor device 251G (possible in S212-6), the sensor device 251G is again selected as a request candidate sensor device (S212-7).

The tracking request target selection unit 233 confirms whether or not there is a determination result in step S211 by the tracking continuation possibility determination unit 232 (S212-8). If there is no determination result in step S211 (not S212-8=determination “Yes” in S211), the tracking request target selection unit 233 stores the information of the sensor device 251G as the tracking request candidate information 237 in the sensor device information storage unit. 242 (S212-9).

If there is a determination result in step S211 (yes in S212-8=determination "no" in S211), the tracking request target selection unit 233 generates an access control signal 224 for switching the access target to the sensor device 251G, and accesses the sensor device 251G. It is sent to the control unit 222 (S212-10). After that, the process proceeds to step S220.

When the specific object tracking device 200 fails to detect the tracked object 235 (undetected in S205), the access control unit 222 accesses the sensor based on the access control signal 224 generated by the tracked object search unit 212. The device 251 is changed (S220). For example, if the sensor device 251A fails to detect the tracked object 235, the sensor device 251 to be accessed next by the specific object tracking device 200 is selected by the access control unit 222 in a predetermined order.

In the specific object tracking device 200, the tracking continuation possibility determination unit 232 determines that the sensor device 251A cannot continue tracking (S211), and the tracking request target selection unit 233 selects the sensor device 251G as a request candidate. In the case (S212), the sensor device 251 to be accessed is changed to the sensor device 251G by the access control unit 222 based on the access control signal 224 generated by the tracking request target selection unit 233 (S220).

The specific object tracking device 200 is not limited to the fixed-point installation type described in this embodiment, and may be a mobile installation type mounted on a mobile such as a vehicle.

(Effect of Second Embodiment)
Since the specific object tracking system 2000 of this embodiment is composed of the specific object tracking device 200 and the sensor devices 251, which are a plurality of moving bodies, detailed tracking is possible in a wide area. Furthermore, the specific object tracking device 200 of this embodiment changes the target of access to the peripheral sensor devices 251B, 251C, . , . . . , continuous tracking in terms of time is also possible.

The specific object tracking device 200 of this embodiment autonomously executes a series of processing operations for detecting and tracking the tracked object 235 and requesting another sensor device 251 to track it. Therefore, the operator of the specific object tracking device 200, the driver of the vehicle that is the sensor device 251, and his fellow passengers can perform the processing operations related to the detection and tracking of the tracked object 235 or requesting the other sensor device 251 to track it. need not be borne by In addition, the driver of the vehicle, which is the sensor device 251, and his fellow passengers do not change the planned route or take a detour from the planned route in order to continue the tracking by the sensor device 251.例文帳に追加

(Third Embodiment)
A specific object tracking system 3000 according to a third embodiment of the present invention will be described with reference to FIGS. 19 and 20. FIG. A specific object tracking system 3000 is composed of a plurality of specific object tracking subsystems 3001 . A specific object tracking subsystem 3001 is composed of a specific object tracking device 300 and a plurality of sensor devices 351 . In the following description of this embodiment, when distinguishing between the specific object tracking subsystem 3001 and the specific object tracking device 300, reference numerals A, B, C, . 3001B, . . . , specific object tracking devices 300A, 300B, . When distinguishing between the sensor device 351 and the sensor 352 provided in the sensor device 351, the code of the specific object tracking subsystem 3001 (or the specific object tracking device 300) to which the sensor device 351 and the sensor 352 belong is A combination of reference numerals is attached to indicate sensor devices 351AA, 351AB, . . .

FIG. 19 is a configuration block diagram of the specific object tracking device 300. As shown in FIG. FIG. 20 is a configuration block diagram of a specific object tracking system 3000 composed of a plurality of specific object tracking devices 300 and a plurality of sensor devices 351. As shown in FIG.

In this embodiment, an example in which the sensor device 351 is of a fixed-point installation type will be described. Fixed-point installation examples of the sensor device 351 include a case where the sensor device 351 is installed on a utility pole, a street lamp, or an exterior wall of a building. An example in which the specific object tracking device 300 in this embodiment is of a fixed point installation type like the sensor device 351 will be described. An example of fixed-point installation of the specific object tracking device 300 corresponds to a case where it is installed at a central office such as an information management center.

The sensor device 351 is a device that generates sensing information 356 from image data acquired by a sensor 352, which will be described later. The sensor device 351 includes a sensor 352, a controller 353, and a transmitter/receiver 354 (not shown in FIG. 20), like the sensor device 151 in the first embodiment.

The sensor 352 is a moving image camera called a PTZ camera, like the sensor 152 in the first embodiment. However, the sensor 352 is not limited to a moving image camera, and may be a still image camera that captures still images, a distance measurement sensor such as LIDAR or millimeter wave radar, or the like. Moreover, the sensor 352 does not need to be single, and a plurality of types of the sensors described above may be used together, or a GPS sensor or the like for acquiring position information as world coordinates may be provided together.

The sensor 352 sends the acquired image data, the distance to the subject, and the relative positional relationship to the control unit 353 .

The control unit 353 generates sensing information 356 from the image data acquired from the sensor 352, the distance to the subject, and the relative positional relationship. The control unit 353 generates sensor device information 357 from the information (type, number, and sensing range) of the sensors 352 and the installation position information of the sensor device 351 . The control unit 353 controls the rotation angle (pan), the elevation angle or depression angle (tilt), and the imaging magnification of the sensor 352 according to the sensor control signal 308 acquired from the sensor control unit 323 of the specific object tracking device 300 (to be described later) via the transmission/reception unit 354. (zoom) and controls. The control unit 353 sends the generated sensing information 356 and sensor device information 357 to the transmission/reception unit 354 .

The transmission/reception unit 354 is connected to the communication unit 321 of the specific object tracking device 300, which will be described later, via connection means such as a communication network. The transmission/reception unit 354 sends the sensing information 356 and the sensor device information 357 generated by the control unit 353 to the communication unit 321 of the specific object tracking device 300 . The transmission/reception unit 354 acquires the sensor control signal 308 from the sensor control unit 323 of the specific object tracking device 300 via the communication unit 321 and sends it to the control unit 353 .

The sensing information 356 includes image data acquired by the sensor 352, the distance between the sensor 352 and the subject, and the relative positional relationship.

The sensor device information 357 includes information on the sensors 352 (type, number and sensing range) and installation position information on the sensor device 351 .

The sensor device 351 can be connected to the specific object tracking device 300 via a connection means such as a communication network, and an existing sensor device can be used.

Similar to the specific object tracking device 100 in the first embodiment, the specific object tracking device 300 accesses the sensor device 351 in a predetermined order, acquires sensing information 356 from the sensor device 351, and detects the tracked object 335. Search. For example, when the tracking target object 335 cannot be detected by the sensing information 356AA acquired from the sensor device 351AA, the access destination is changed to the next sensor device 351AB and the tracking target object 335 is searched by the sensing information 356AB. The sensor devices 351 to be accessed are sequentially changed, and the search for the tracked object 335 by each sensor device 351 is repeated until the tracked object 335 is detected.

When the specific object tracking device 300 detects the tracked object 335, the specific object tracking device 300 tracks the tracked object 335 using the sensing information 356AA acquired from the sensor device 351 (for example, the sensor device 351AA). When the specific object tracking device 300A determines that the tracking by the sensor device 351AA cannot be continued, the specific object tracking device 300A selects the sensor device 351AG that can continue tracking the tracked object 335 from the sensor devices 351AB, 351AC, . Tracking of the tracked object 335 is requested and tracking of the tracked object 335 is continued.

The specific object tracking device 300A acquires sensor device information 357AA, 357AB, 357AC, . . . from all sensor devices 351AA, 351AB, 351AC, . The acquired sensor device information 357 is stored in the sensor device information storage unit 342, which will be described later.

The specific object tracking subsystem 3001A is configured by connecting a specific object tracking device 300A and a plurality of sensor devices 351AA, 351AB, 351AC, . A specific object tracking system 3000 is configured by connecting a plurality of specific object tracking subsystems 3001A, 3001B, 3001C, . . . via connection means such as a communication network.

At this time, for example, in the specific object tracking device 300A constituting the specific object tracking subsystem 3001A, it is determined that the tracking by the sensor device 351AA cannot be continued, and the tracking target object 335 is tracked by the peripheral sensor devices 351AB, 351AC, . A case may occur in which the sensor device 351 capable of continuing the operation cannot be selected. In such a case, the specific object tracking device 300A accesses the specific object tracking devices 300B, 300C, . . . of the other specific object tracking subsystems 3001B, 3001C, . The specific object tracking device 300A asks whether the tracking target object 335 can be tracked by other specific object tracking subsystems 3001B, 3001C, . A rescue request signal is sent to 300G to request continued tracking of the tracked object 335 .

The specific object tracking device 300 includes a sensing information acquisition unit 311, a tracked object search unit 312, an operation mode control unit 313, a communication unit 321, an access control unit 322, a sensor control unit 323, a tracking data generation unit 331, and a determination as to whether or not tracking can be continued. It consists of a unit 332 , a tracking request target selection unit 333 , a help request target selection unit 334 , a tracking object information storage unit 341 , a sensor device information storage unit 342 , a self device information storage unit 343 and a peripheral device information storage unit 344 .

The sensing information acquisition unit 311 acquires sensing information 356 from the accessed sensor device 351 via the communication unit 321 . The sensing information acquisition unit 311 sends the acquired sensing information 356 to the tracked object search unit 312 .

The tracked object search unit 312 acquires the tracked object information 305 stored in the tracked object information storage unit 341, and compares the sensing information 356 acquired from the sensing information acquisition unit 311 with the tracked object information 305. Search for tracked object 335 . When searching for the tracked object 335, the tracked object search unit 312 generates a sensor control signal 308 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 352. and sent to the sensor control unit 323 .

If the tracked object 335 is a vehicle, the tracked object information 305 includes information such as the license plate (vehicle number), vehicle type, and body color. It is information such as an image or feature amount extraction data based on the image.

The tracked object search unit 312 searches for a subject from the moving image included in the sensing information 356, and obtains information such as the license plate (vehicle number), vehicle type, and body color for each detected vehicle, and calculates feature values for each detected person. Information such as extraction data is extracted as identification information for identifying a subject.

Tracked object search unit 312 collates identification information for identifying a subject extracted from sensing information 356 with tracked object information 305 related to tracked object 335, and extracts a subject that matches tracked object information 305. to detect the tracked object 335 .

Here, from the relative positional relationship between the sensor 352 included in the sensing information 356 and the tracked object 335, which is the subject, the position of the tracked object 335 can be converted into world coordinates. If sensor 352 includes a GPS sensor, the position of sensor 352 or sensor device 351 can be obtained as world coordinates. Since the mounting states (positions, angles, etc.) of the sensor device 351 and the sensor 352 are known, it is possible to convert the position of the tracked object 335 from the world coordinates of the sensor device 351 into world coordinates. In the following description, it is not particularly specified whether the positions of the sensor device 351 and the tracked object 335 are relative positions between the sensor device 351 and the tracked object 335 or world coordinates converted as described above. It is simply notated as location information.

The tracked object search unit 312 newly adds the detection coordinates in the imaging area in the sensing information 356 of the detected tracked object 335 to the tracked object information 305, and searches for the tracked object 335 as search result information 317. is sent to the operation mode control unit 313 . The follow-up search result information 317 includes detection result of the tracked object 335 , position information of the sensor device 351 and the tracked object 335 , sensing information 356 and tracked object information 305 .

The operation mode control unit 313 acquires the search result information 317 from the tracked object search unit 312, sets the operation mode according to the detection status of the tracked object 335 in the tracked object search unit 312, and stores it. The set operation modes are a search mode in which the tracked object search unit 312 searches for the tracked object 335, and a tracking mode in which the tracked object search unit 312 repeatedly detects the tracked object 335 and tracks the tracked object 335. any of the modes.

When the operation mode control unit 313 acquires the search result information 317 from the tracked object search unit 312, the operation mode control unit 313 confirms whether the set operation mode is the search mode or the tracking mode.

When the set operation mode is the search mode, the operation mode control unit 313 confirms the detection status of the tracked object 335 by the tracked object search unit 312 based on the search result information 317 . When the tracked object search unit 112 detects the tracked object 335, the operation mode control unit 313 switches the operation mode from the search mode to the tracking mode and resets the operation mode. 356 acquisition continues. If the tracked object search unit 312 does not detect the tracked object 335 , the operation mode control unit 313 generates an access control signal 324 for changing the access target sensor device 351 and sends it to the access control unit 322 .

When the set operation mode is the tracking mode, operation mode control section 313 sends search result information 317 of tracked object 335 acquired from tracked object search section 312 to tracking data generation section 331 .

The communication unit 321 accesses and communicates with the transmission/reception unit 354 of the sensor device 351 set as an access target by the access control unit 322 . The communication unit 321 acquires the sensing information 356 and the sensor device information 357 from the sensor device 351 via the transmission/reception unit 354 of the sensor device 351, transmits the sensing information 356 to the sensing information acquisition unit 311, and transmits the sensor device information 357 to the sensor device information. Each is sent to the storage unit 342 . The communication unit 321 sends the sensor control signal 308 acquired from the sensor control unit 323 to the transmission/reception unit 354 of the sensor device 351 .

The communication means and communication method between the communication unit 321 and the communication unit 354 of the sensor device 351 are not particularly limited. For example, wireless communication lines such as wireless LAN communication and mobile phone lines, wired communication lines such as fixed telephone lines and optical fiber lines, and wired communication lines such as fixed telephone lines and optical fiber lines can be used.

The access control unit 322 acquires an access control signal 324 generated by the operation mode control unit 313 or the tracking request target selection unit 333, and controls which sensor device 351 the specific object tracking device 300 accesses. The switching means of the sensor device 351 to be accessed and the method of connection with the sensor device 351 are not particularly limited. can do.

The sensor control unit 323 acquires the sensor control signal 308 generated by the tracked object search unit 312 or the tracking data generation unit 331, and transmits the signal to the sensor device via the communication unit 321 and the transmission/reception unit 354 of the sensor device 351 being accessed. 351 to the control unit 353 .

The tracking data generation unit 331 acquires the history of the position information associated with the tracking of the tracked object 335 included in the search result information 317 acquired from the tracked object search unit 312 via the operation mode control unit 313, and the image is captured at a new angle. Tracking data 307 including new information such as image information or feature amount data extracted from the image is generated and sent to the tracking object information storage unit 341 .

The tracking data generation unit 331 extracts feature amount data from an arbitrary image frame in the search result information 317 acquired from the tracked object search unit 312 via the operation mode control unit 313 . The tracking data generation unit 331 not only tracks the tracked object 335 by searching for feature quantities that match or are similar to the feature quantity data in subsequent image frames, but also uses a kernelized correlation filter to speed up tracking. Target object 335 may also be tracked to generate tracking data 307 .

When generating the tracking data 307, the tracking data generation unit 331 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 352 so that the tracking target object 335 does not deviate from the imaging area. It generates a sensor control signal 308 for controlling and sends it to the sensor control unit 323 .

The tracking continuation enable/disable determination unit 332 determines whether tracking can be continued by the sensor device 351 being accessed (for example, the sensor device 351AA). The tracking continuation possibility determination unit 332 uses the tracking data 307 acquired from the tracking object information storage unit 341 to determine the tracking target object based on the state of the tracking target object 335 and the change history of the detection position of the tracking target object 335 in the sensing range of the sensor 352AA. Estimate the course of 335. For example, if the tracked object 335 is a vehicle, the route of the tracked object 335 is estimated based on whether the tracked object 335 is in the straight lane or in the right turn (or left turn) lane. The tracking continuation possibility determination unit 332 sends the estimated course of the tracking target object 335 to the tracking request target selection unit 333 .

The tracking continuation possibility determination unit 332 determines the relationship between the estimated course and the sensing range of the sensor 352AA provided in the sensor device 351AA, or the change history of the detection position and detection size of the tracked object 335 in the sensing range of the sensor 352AA. 351AA determines whether or not tracking can be continued.

If it is determined that the sensor device 351AA can continue tracking, the search (detection) of the tracked object 335 from the sensing information 356AA is repeated. If it is determined that the sensor device 351AA cannot continue tracking, the determination result is sent to the tracking request target selection section 333 .

For example, when the estimated course of the tracked object 335 is included in the blind spot of the sensor 352AA, or when the detection size of the tracked object 335 in the sensing range of the sensor 352AA changes small (or large), the magnification of the sensor 352AA is changed. If it is assumed that the tracking target object 335 cannot be detected even if the sensor device 351AA detects the tracking target object 335, it is determined that the sensor device 351AA cannot continue tracking.

Tracking request target selection unit 333 selects sensor device information 357AB, 357AC, . . . related to other sensor devices 351AB, 351AC, . A sensor device 351AG that is a candidate for a request to track the tracked object 335 is selected from the estimated track of the tracked object 335 and the positional relationship between the sensor devices. The tracking request target selection unit 333 sends the tracking data 307 acquired from the tracking object information storage unit 341 to the selected sensor device 351AG, and acquires the detection result of the tracking target object 335 from the sensor device 351AG. If the sensor device 351AG can detect the tracked object 335, the sensor device 351AG is selected as the request target.

Here, if the determination result of the tracking continuation permission determination unit 332 is negative, the tracking request target selection unit 333 generates an access control signal 324 for changing the access destination sensor device 351 to the sensor device 351AG, and performs access control. 322. If the determination result of the tracking continuation possibility determination unit 332 is yes, the tracking request target selection unit 333 generates tracking request candidate information 337 regarding the sensor device 351AG selected as the request target, and stores it in the sensor device information storage unit 342. do.

If the tracking request target object 335 is not detected by the sensor device 351AG, the tracking request target selection unit 333 redoes selection of the sensor device 351 as a tracking request candidate, and selects another sensor device as a new tracking request candidate. Select 351AH.

The help request target selection unit 334 accesses the specific object tracking devices 300B, 300C, . . . of the other specific object tracking subsystems 3001B, 3001C, . Acquisition of peripheral device information 309B, 309C, . Then, the specific object tracking device 300G to be requested for tracking assistance of the tracked object 335 is selected from the relationship between the estimated course of the tracked object 335 and the sensing range of each specific object tracking subsystem.

The rescue request target selection unit 334 sends the tracking data 307 acquired from the tracked object information storage unit 341 to the selected specific object tracking device 300G, and acquires the result of whether or not the tracking target object 335 can be detected from the specific object tracking device 300G. do. If the tracking target object 335 can be detected by the specific object tracking device 300G, the specific object tracking device 300G is selected as the target for the rescue request. If the tracking target object 335 is not detected by the specific object tracking device 300G, the specific object tracking device 300 to be the target of the rescue request is selected again, and the specific object tracking device 300H to be the new target of the rescue request is selected. do.

Here, if the determination result of the tracking continuation possibility determination unit 332 is negative and none of the sensor devices 351 is selected as a request target in the tracking request target selection unit 333, the rescue request target selection unit 334 selects a rescue request target. A help request signal is generated and sent to the specific object tracking device 300G selected as the request target. If the determination result of the tracking continuation possibility determination unit 332 is yes, or if the sensor device 351 is selected as the request target in the tracking request target selection unit 333, the information about the specific object tracking device 300G is added to the help request candidate information 338. , and stored in the peripheral device information storage unit 344 . The tracked object information storage unit 341 stores the tracked object information 305 as well as the tracked data 307 generated by the tracked data generating unit 331 .

The sensor device information storage unit 342 stores information (type and number) of the sensors 352 included in all the sensor devices 351 linked to the specific object tracking device 300, as well as information such as installation position information and sensing range of the sensor devices 351. and sensor device information 357 is stored.

The device information storage unit 343 stores sensor device information 357 stored in the sensor device information storage unit 342 as well as information on the specific object tracking subsystem 3001 to which the device belongs as device information 306 .

Peripheral device information storage unit 344 stores information about other specific object tracking devices 300B, 300C, . , peripheral device information 309B, 309C, .

With reference to FIG. 21, the tracking processing procedure of the tracking target object 335 of the specific object tracking device 300 by the specific object tracking system 3000 in this embodiment will be described. FIG. 21 is a flow chart showing the processing procedure of the specific object tracking device 300 in the specific object tracking system 3000. As shown in FIG.

When the specific object tracking device 300 starts searching for the tracked object 335 (S300), the specific object tracking device 300 acquires sensor device information 357 from all linked sensor devices 351 through the communication unit 321. (S300-1). The specific object tracking device 300 stores the obtained sensor device information 357 in the sensor device information storage section 342 (S300-2).

The specific object tracking device 300 accesses the access target sensor device 351 set by the access control unit 322 through the communication unit 321 . The sensor device 351 accessed by the specific object tracking device 300 is selected by the access control section 322 based on the access control signal 324 . The first setting accessed at the start of a search is sensor device 351AA. The access control unit 322 sets the operation mode to the search mode by the operation mode control unit 313 (S301).

The specific object tracking device 300 acquires the sensing information 356AA from the accessing sensor device 351AA by the sensing information acquisition unit 311 (S302). The sensing information acquiring unit 311 sends the acquired sensing information 356AA to the tracking target object searching unit 312 (S302).

The tracked object search unit 312 acquires the tracked object information 305 from the tracked object information storage unit 341, compares the sensing information 356AA acquired from the sensing information acquisition unit 311 with the tracked object information 305, and finds the tracked object 335. (S303).

When searching for the tracked object 335, the tracked object search unit 312 generates a sensor control signal 308 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 352AA. , to the sensor control unit 323 (S303).

The search for the tracked object 335 by the tracked object search unit 312 will be specifically described for the case where the tracked object 335 is a vehicle and the case where the tracked object 335 is a person.

When the tracked object 335 is a vehicle, the tracked object search unit 312 searches for the vehicle included in the acquired sensing information 356AA, and checks the license plate (vehicle number), vehicle type, body color, etc. for each detected vehicle. is extracted as identification information for identifying the vehicle. The tracked object search unit 312 collates the detected identification information with the acquired tracked object information 305 , extracts vehicles that match the tracked object information 305 , and detects the tracked object 335 .

When the tracked object 335 is a person, the tracked object searching unit 312 searches for the person included in the acquired sensing information 356AA, and for each detected person, an image such as age, gender, face photograph, etc. Information such as feature quantity extraction data based on this is extracted as identification information for identifying a person. The tracked object search unit 312 collates the extracted identification information with the acquired tracked object information 305 to extract a person who matches the tracked object information 305 and detects the tracked object 335 .

Next, the specific object tracking device 300 confirms whether the set operation mode is the search mode or the tracking mode by the operation mode control unit 313 (S304).

If the operation mode confirmed by the operation mode control unit 313 in step S304 is the search mode set in step S301 (search mode in S304), the operation mode control unit 313 controls the tracked object search unit in step S303. It is checked whether the tracked object 335 has been detected by the search in 312 (S305).

When the tracked object 335 has been detected (detected in S305), the operation mode control unit 313 switches the operation mode from the search mode to the tracking mode (S306). After that, the process returns to step S302.

If the tracked object 335 has not been detected (not detected in S305), the operation mode control unit 313 generates an access control signal 324 for changing the access target sensor device 351 and sends it to the access control unit 322 (S307). . After that, the process proceeds to step S320.

If the operation mode confirmed by the operation mode control unit 313 in step S304 is the tracking mode (tracking mode in S304), the operation mode control unit 313 retrieves the search result information of the tracked object 335 acquired from the tracked object search unit 312. 317 is sent to the tracking data generator 331 . The tracking data generator 331 generates the tracking data 307 from the acquired search result information 317 and stores it in the tracked object information storage 341 (S310).

Next, in the specific object tracking device 300A, the tracking continuation possibility determination unit 332 determines whether or not the sensor device 351AA can continue tracking the tracking target object 335 (S311), and the tracking request target selection unit 333 determines whether the sensor device 335 If continuation of tracking by 351AA is not possible, selection of a target for which continuation of tracking is requested (S312) is executed in parallel.

22 and FIG. 4 to FIG. 8 again, the detailed procedure of the tracking continuation possibility determination (S311) by the sensor device 351AA in the tracking continuation possibility determination unit 332 will be described. FIG. 22 is a flow chart showing a detailed processing procedure of the tracking continuation permission/inhibition decision (S311) in the sensor device 351AA in the tracking continuation permission/inhibition decision section 332. As shown in FIG.

The tracking continuation possibility determination unit 332 acquires the tracking data 307 from the tracking data generation unit 331 and the tracking object information storage unit 341 (S311-1).

The tracking continuation possibility determination unit 332 estimates the course of the tracked object 335 from the acquired tracking data 307 (S311-2). The estimated course of the tracked object 335 and the blind spot of the sensor 352AA are compared to determine whether or not the tracking of the tracked object 335 by the sensor device 351AA can be continued (S311-3).

Alternatively, the tracking continuation possibility determination unit 332 extracts the change history of the detected position and size of the tracked object 335 from the tracking data 307 (S311-4). The extracted change history and the sensing range of the sensor 352AA are compared to determine whether or not the tracking of the tracked object 335 by the sensor device 351AA can be continued (S311-5).

In the determination in step S311-3, for example, as shown in FIG. 4, if the estimated course of the tracked object 335 is not included in the blind spot of the sensor 352AA, it is determined that the sensor device 351AA can continue tracking. . Here, the blind spot of the sensor 352AA refers to a non-sensable area of the sensor 352AA that is generated when the outer wall of the building in which the sensor device 351AA is installed is included in the sensing range of the sensor 352AA. On the other hand, when the estimated course of the tracked object 335 is included in the blind spot of the sensor 352AA as shown in FIG. 5, it is determined that the sensor device 351AA cannot continue tracking.

In the determination in step S311-5, for example, as shown in FIG. 6, if the detected position and size of the tracked object 335 in the sensing range of the sensor 352AA do not change, the sensor device 351AA can continue tracking. Determine that there is. On the other hand, as shown in FIG. 7, the detection position of the tracked object 335 in the sensing range of the sensor 352AA moves in the horizontal direction (or in the vertical direction), and it may deviate from the sensing range of the sensor 352AA (into a blind spot). In the assumed case, the detection size of the tracked object 335 in the sensing range of the sensor 352AA is small (or large) as shown in FIG. If it is assumed that detection (identification) will not be possible, it is determined that the sensor device 351AA cannot continue tracking.

If both the determination result in step S311-3 and the determination result in step S311-5 are yes, the tracking continuation possibility determination unit 332 determines that the sensor device 351AA can continue to track the tracked object 335. determination (S311-6), and if both the determination result in step S311-3 and the determination result in step S311-5 or either one is negative, the tracking target object 335 is tracked by the sensor device 351AA. It is determined that continuation is not possible (S311-6).

If the tracking continuation possibility determination unit 332 determines that the sensor device 351AA can continue tracking the tracked object 335 (Yes in S311-6), the tracking continuation possibility determination unit 332 transmits the sensing information via the communication unit 321. The acquisition unit 311 continues acquiring new sensing information 356AA, returns to step S302, and the search unit 310 repeats the search and detection of the tracked object 335 to track the tracked object 335. FIG.

When the tracking continuation possibility determination unit 332 determines that the sensor device 351AA cannot continue tracking the tracked object 335 (No in S311-6), the tracking continuation possibility determination unit 332 determines whether the sensor device 351AA can continue tracking. A determination result indicating that the continuation is not permitted is sent to the tracking request target selection unit 333 .

Referring to FIG. 23, a detailed procedure of selection (S312) of another sensor device 351 to request continuation of tracking by the sensor device 351AA in the tracking request target selection unit 333 will be described. FIG. 23 is a flow chart showing a detailed processing procedure of target selection (S312) of the other sensor device 351 in the tracking request target selection unit 333. As shown in FIG.

The tracking request target selection unit 333 checks whether the tracking request candidate information 337 is stored in the tracking object information storage unit 341 (S312-1).

If the tracking request candidate information 337 is not stored in the tracking object information storage unit 341 (NO in S312-1), the sensor device information 357AB, 357AC, . (S312-2).

Further, the tracking request target selection unit 333 acquires the estimated course of the tracking target object 335 estimated in step S311-2 from the tracking continuation possibility determination unit 332 (S312-3).

Then, the relationship between the acquired estimated course of the tracked object 335 and the sensing ranges of the sensor devices 351AB, 351AC, . . . obtained from the sensor device information 357AB, 357AC, . , the sensor device 351AG whose sensing range includes the estimated course of the tracked object 335 is selected as a request candidate sensor device (S312-4).

The tracking request target selection unit 333 sends the tracking data 307 via the communication unit 321 to the sensor device 351AG selected as a request candidate (S312-5). The tracking request target selection unit 333 acquires and confirms whether or not the tracking target object 335 can be detected by the sensor device 351AG (S312-6).

If the tracking target object 335 is not detected by the sensor device 351AG (NO in S312-6), the tracking request target selection unit 333 redoes the selection of the sensor device as a request candidate in step S312-4, and selects the specific object tracking device 300A. It sends to the rescue request target selection unit 334 the judgment result that the tracking continuation is not possible.

If the object 335 to be tracked is detected by the sensor device 351AG (possible in S312-6), the sensor device 351AG is again selected as a request candidate sensor device (S312-7).

The tracking request target selection unit 333 confirms whether or not there is a determination result of step S311 by the tracking continuation possibility determination unit 332 (S312-8), and there is no determination result of step S311 (not S312-8 = determination “possible” in S311). ”), the information of the sensor device 351AG is stored in the sensor device information storage section 342 as the tracking request candidate information 337 (S312-9).

If there is a determination result in step S311 (yes in S312-8=determination "no" in S311), the tracking request target selection unit 333 generates an access control signal 324 for changing the access target to the sensor device 351AG. It is sent to the access control unit 322 (S312-10). After that, the process proceeds to step S320.

When the specific object tracking device 300 fails to detect the tracked object 335 (undetected in S305), the access control unit 322 accesses the sensor based on the access control signal 324 generated by the tracked object search unit 312. The device 351 is changed (S320). For example, if the sensor device 351AA cannot detect the tracked object 335, the sensor device 351 to be accessed next by the specific object tracking device 300 is selected by the access control unit 322 in a predetermined order.

In the specific object tracking device 300, the tracking continuation possibility determination unit 332 determines that the sensor device 351AA cannot continue tracking (S311), and the tracking request target selection unit 333 selects the sensor device 351AG as a request candidate. In the case (S312), the sensor device 351 to be accessed is changed to the sensor device 351AG by the access control unit 322 based on the access control signal 324 generated by the tracking request target selection unit 333 (S320).

Referring to FIG. 24, a detailed procedure of target selection (S315) of the other specific object tracking device 300 for which the specific object tracking device 300A is requested to track the specific object tracking device 300A in the help request target selection unit 334 will be described. FIG. 24 is a flow chart showing a detailed processing procedure of target selection (S315) of another specific object tracking device 300 in the help request target selection unit 334. As shown in FIG.

The help request target selection unit 334 checks whether the help request candidate information 338 is stored in the peripheral device information storage unit 344 (S315-1).

If the help request candidate information 338 is not stored in the peripheral device information storage unit 344 (NO in S315-1), the peripheral device information 309B, 309C, . (S315-2).

Furthermore, the relief request target selection unit 334 acquires the estimated course of the tracked object 335 acquired in step S312-3 (S315-3).

Then, the relationship between the obtained estimated course of the tracked object 335 and the sensing range of each of the specific object tracking devices 300B, 300C, . . . obtained from the peripheral device information 309B, 309C, . Then, the specific object tracking device 300G, which includes the estimated path of the tracked object 335 in its sensing range, is selected as the specific object tracking device for which assistance is requested (S315-4).

The rescue request target selection unit 334 sends the tracking data 307 via the communication unit 321 to the specific object tracking device 300G selected as the rescue request target (S315-5). The help request target selection unit 334 acquires and confirms whether or not the tracking target object 335 can be detected by the specific object tracking device 300G (S315-6).

If the tracking target object 335 is not detected by the specific object tracking device 300G (NO in S315-6), the selection of the specific object tracking device 300 to be the rescue request target is redone in step S315-4. When the tracking target object 335 is detected by the specific object tracking device 300G (possible in S315-6), the specific object tracking device 300G is again selected as the specific object tracking device to be requested for help (S315-7).

The rescue request target selection unit 334 confirms the determination result of step S312 by the tracking request target selection unit 333 (S315-8). If there is no determination result in step S312 (no S315-8 = tracking request target selected in S312), the relief request target selection unit 334 selects the information of the specific object tracking device 300G as the help request candidate information 338 from the peripheral device It is stored in the information storage unit 344 (S315-9).

If there is a determination result in step S312 (yes in S315-8 = tracking request target cannot be selected in S312), the help request target selection unit 334 generates a help request signal for the selected specific object tracking device 300G ( S315-10). After that, the process proceeds to step S325.

In the rescue request target selection unit 334, the tracking continuation possibility determination unit 332 determines that the sensor device 351AA cannot continue tracking, and the tracking request target selection unit 333 selects another sensor device 351 that can continue tracking with the sensor device 351AA. If not selected, the help request signal for the other specific object tracking device 300G generated by the help request target selection unit 334 is sent to the other specific object tracking device 300G via the communication unit 321 (S325).

The specific object tracking device 300 is not limited to the fixed point installation type described in the present embodiment, and may be a mobile installation type mounted on a mobile such as a vehicle.

(Effect of the third embodiment)
A specific object tracking system 3000 of this embodiment is composed of a plurality of specific object tracking subsystems 3001 . Each of the specific object tracking subsystems 3001 is composed of the specific object tracking device 300 and a plurality of fixed-point installation type sensor devices 351. Therefore, if a plurality of sensor devices 351 are installed, for example, on a building or on the side of a road, they can be detected in a wide area. can be tracked in detail.

Furthermore, the specific object tracking device 300A of the present embodiment changes the target of access to the peripheral sensor devices 351AB, 351AC, . , . . . , continuous tracking in terms of time is also possible.

Further, in the specific object tracking system 3000 of the present embodiment, for example, when tracking by the specific object tracking subsystem 3001A cannot be continued in all the sensor devices 351AA, 351AB, . Since the specific object tracking subsystem 3001 (for example, the specific object tracking subsystem 3001B) is requested for help and tracking is continued, detailed and continuous tracking in a wide area is possible in terms of time. .

The specific object tracking device 300 of this embodiment autonomously executes a series of processing operations for detecting and tracking a tracked object 335 and requesting another sensor device 351 to track it. In addition, the specific object tracking device 300 of this embodiment also autonomously executes a series of processing operations related to a rescue request to another specific object tracking device 300 . Therefore, the operator of the specific object tracking device or the like can perform the processing operations related to the detection and tracking of the tracked object 335, the request for tracking to the other sensor device 351, or the request for assistance to the other specific object tracking device 300. need not be borne by

(Fourth embodiment)
A specific object tracking system 4000 according to the fourth embodiment of the present invention will be described with reference to FIG. A specific object tracking system 4000 is composed of a plurality of specific object tracking devices 400 . FIG. 25 is a configuration block diagram of the specific object tracking system 4000. As shown in FIG. A specific object tracking device 400 in this embodiment differs from the specific object tracking devices in the first to third embodiments in that a sensor device 451 having a sensor 452 is integrated. When the specific object tracking device 400 and the sensor device 451 are integrated, in which case the specific object tracking device 400 is added to the sensor device 451 or when the specific object tracking device 400 incorporates the sensor device 451 Also includes Alternatively, the specific object tracking device 400 includes the sensor 452 itself.

Each of the specific object tracking devices 400 in this embodiment is of a fixed point installation type. Fixed-point installation examples of the specific object tracking device 400 correspond to cases where it is installed on utility poles, street lamps, exterior walls of buildings, and the like arranged on the roadside.

The sensor device 451 is a device that generates sensing information 456 from image data acquired by a sensor 452, which will be described later. The sensor device 451 is composed of a sensor 452 and a controller 453 .

The sensor 452 is a moving image camera called a PTZ camera, like the sensor 152 in the first embodiment. However, the sensor 452 is not limited to a moving image camera, and may be a still image camera that captures still images, a distance measurement sensor such as LIDAR or millimeter wave radar, or the like. Moreover, the sensor 452 does not need to be single, and a plurality of types of the sensors described above may be used together, or a GPS sensor or the like for acquiring position information as world coordinates may be provided together.

The sensor 452 sends the acquired image data, the distance to the subject, and the relative positional relationship to the control unit 453 .

The control unit 453 generates sensing information 456 from the image data acquired from the sensor 452, the distance to the subject, and the relative positional relationship. The control unit 453 generates sensor device information 457 from the information (type, number, and sensing range) of the sensors 452 and the installation position information of the sensor device 451 . The control unit 453 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 452 using a sensor control signal 408 obtained from the sensor control unit 423 of the specific object tracking device 400, which will be described later. do. The control unit 453 sends the generated sensing information 456 to the sensing information acquisition unit 411 of the specific object tracking device 400, which will be described later, and the sensor device information 457 to the own device information storage unit 443 of the specific object tracking device 400, which will be described later. .

The sensing information 456 includes image data acquired by the sensor 452, the distance between the sensor 452 and the subject, and the relative positional relationship.

The sensor device information 457 includes information on the sensors 452 (type, number and sensing range) and installation position information of the sensor device 451 , that is, the specific object tracking device 400 .

When the tracking target object 435 is detected, the specific object tracking device 400 tracks the tracking target object 435 using the sensing information 456 acquired from the sensor device 451 .

When the specific object tracking device 400A determines that tracking by the sensor device 451A cannot be continued, it selects the specific object tracking device 400G that can continue tracking the tracked object 435 from the surrounding specific object tracking devices 400B, 400C, . Tracking of the tracked object 435 is requested and tracking of the tracked object 435 is continued.

The specific object tracking device 400A acquires peripheral device information 409B, 409C, . . . from all the specific object tracking devices 400B, 400C, . The acquired peripheral device information 409 is stored in a peripheral device information storage unit 444, which will be described later. The peripheral device information 409 is sensor device information 457 generated by each of the specific object tracking devices 400B, 400C, .

The specific object tracking device 400 includes a sensing information acquisition unit 411, a tracking target object search unit 412, a communication unit 421, an access control unit 422, a sensor control unit 423, a tracking data generation unit 431, a tracking continuation determination unit 432, and a help request target. It is composed of a selection unit 434 , a tracked object information storage unit 441 , a device information storage unit 443 and a peripheral device information storage unit 444 .

The sensing information acquisition unit 411 acquires sensing information 456 from the sensor device 451 . The sensing information acquisition unit 411 sends the acquired sensing information 456 to the tracked object search unit 412 .

The tracked object search unit 412 acquires the tracked object information 405 stored in the tracked object information storage unit 441, and compares the sensing information 456 acquired from the sensing information acquisition unit 411 with the tracked object information 405. Search for tracked object 435 . The tracked object search unit 412 generates a sensor control signal 408 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 452 when searching for the tracked object 435. and sent to the sensor control unit 423 .

If the tracked object 435 is a vehicle, the tracked object information 405 includes information such as the license plate (vehicle number), vehicle type, and body color. It is information such as an image or feature amount extraction data based on the image.

The tracked object search unit 412 searches for a subject from the moving image included in the sensing information 456, and obtains information such as the license plate (vehicle number), vehicle type, and body color for each detected vehicle, and calculates feature values for each detected person. Information such as extraction data is extracted as identification information for identifying a subject.

Here, from the relative positional relationship between the sensor 452 included in the sensing information 456 and the tracked object 435, which is the subject, the position of the tracked object 435 can be converted into world coordinates. If sensor 452 includes a GPS sensor, the position of sensor 452 or sensor device 451 can be obtained as world coordinates. Since the mounting state (position, angle, etc.) of the sensor 452 in the sensor device 451 is known, it is also possible to transform the position of the tracked object 435 from the world coordinates of the sensor device 451 into world coordinates. In the following description, it is not particularly specified whether the positions of the sensor device 451 and the tracked object 435 are relative positions between the sensor device 451 and the tracked object 435 or world coordinates converted as described above. It is simply referred to as location information.

The tracked object search unit 412 collates the identification information for identifying the subject extracted from the sensing information 456 with the tracked object information 405 regarding the tracked object 435, and searches for a subject that matches the tracked object information 405. A tracked object 435 is detected by extraction.

The tracked object search unit 412 newly adds detection coordinates in the imaging area in the sensing information 456 of the detected tracked object 435 to the tracked object information 405, and searches for the tracked object 435 as search result information 417. is sent to the trace data generator 431 . The search result information 417 includes detection result of the tracked object 435 , position information of the sensor device 451 and the tracked object 435 , sensing information 456 and tracked object information 405 .

If the tracked object 435 cannot be detected, the tracked object search unit 412 acquires new sensing information 456 and repeats the search for the tracked object 435 .

The communication unit 421 accesses and communicates with the communication unit 421B of another specific object tracking device 400B set as an access target by the access control unit 422 . The communication unit 421 acquires the peripheral device information 409B from the communication unit 421B of the other specific object tracking device 400B and sends it to the peripheral device information storage unit 444 .

The communication means and communication method between the other specific object tracking devices 400B, 400C, . . . by the communication unit 421 are not particularly limited. For example, wireless communication lines such as wireless LAN communication and mobile phone lines can be used.

The specific object tracking device 400A acquires peripheral device information 409B, 409C, . . . from other specific object tracking devices 400B, 400C, . The specific object tracking device 400A sends the acquired peripheral device information 409B, 409C, . Further, the specific object tracking device 400A transmits its own device information 406A related to the specific object tracking device 400A stored in its own device information storage unit 443 to the other specific object tracking devices 400B, 400C, . do. Own device information 406A is sensor device information 457A generated by sensor device 451A.

The access control unit 422 acquires the access control signal 424 generated by the help request target selection unit 434 and controls which specific object tracking device 400 to access. Means for switching access to other specific object tracking devices 400 and connection methods with other specific object tracking devices 400 are not particularly limited, and each specific object tracking device 400 is given an identification number. or a unique code or the like can be given.

The sensor control unit 423 acquires the sensor control signal 408 generated by the tracked object search unit 412 or the tracking data generation unit 431 and sends it to the control unit 453 of the sensor device 451 .

The tracking data generation unit 431 extracts from the history of the position information accompanying the tracking of the tracked object 435 included in the search result information 417 acquired from the tracked object search unit 412 and the image information captured at a new angle or the image. The tracking data 407 including the new information such as the feature amount data obtained from the tracking data is generated and sent to the tracking object information storage unit 441 .

The tracking data generation unit 431 extracts feature amount data from an arbitrary image frame in the search result information 417 acquired from the tracking target object search unit 412 . The tracking data generator 431 not only tracks the tracked object 435 by searching for feature amount data that matches or is similar to the feature amount data in subsequent image frames, but also uses a kernelized correlation filter to perform faster tracking. Tracked object 435 may also be tracked to generate tracking data 407 .

When generating the tracking data 407, the tracking data generation unit 431 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 452 so that the tracking target object 435 does not deviate from the imaging area. A sensor control signal 408 is generated and sent to the sensor control unit 423 .

The tracking continuation availability determination unit 432 determines whether or not tracking by the specific object tracking device 400A can be continued. Based on the tracking data 407 acquired from the tracking object information storage unit 441, the tracking continuation determination unit 432 determines the tracking target object based on the state of the tracking target object 435 and the change history of the detection position of the tracking target object 435 in the sensing range of the sensor 452A. Estimate the course of 435. For example, if the tracked object 435 is a vehicle, the route of the tracked object 435 is estimated based on whether the tracked object 435 is in the straight lane or the right turn (or left turn) lane.

The tracking continuation possibility determination unit 432 determines the relationship between the estimated course and the sensing range of the sensor 452A, or the change history of the detection position and detection size of the tracking target object 435 in the sensing range of the sensor 452A. Determine whether to continue tracking. For example, when the estimated course of the tracked object 435 is included in the blind spot of the sensor 452A, or when the detection size of the tracked object 435 in the sensing range of the sensor 452A changes small (or large), the magnification of the sensor 452A is changed. If it is assumed that the tracking target object 435 cannot be detected even if the tracking target object 435 is detected, it is determined that the specific object tracking device 400A cannot continue tracking.

If it is determined that the specific object tracking device 400A can continue tracking, the search (detection) of the tracking target object 435 from the sensing information 456A is repeated. When it is determined that the specific object tracking device 400A cannot continue tracking, the determination result is sent to the help request target selection unit 434 .

The rescue request target selection unit 434 stores the peripheral device information 406B, 406C, . , and from the estimated path of the tracked object 435 and the positional relationship between the specific object tracking devices 400B, 400C, . . . The rescue request target selection unit 434 sends the tracking data 407 acquired from the tracked object information storage unit 441 to the selected specific object tracking device 400G, and acquires the result of whether or not the tracking target object 435 can be detected from the specific object tracking device 400G. do. If the tracking target object 435 can be detected by the specific object tracking device 400G, the specific object tracking device 400G is selected as the target for the rescue request.

Here, if the determination result of the tracking continuation possibility determination unit 432 is negative, the help request target selection unit 434 generates a help request signal to the specific object tracking device 400G, and sends the specific object tracking device through the communication unit 421 Send to 400G. If the determination result of the tracking continuation possibility determination unit 432 is yes, the help request candidate information 438 of the specific object tracking device 400G selected as the target of the help request is generated and stored in the peripheral device information storage unit 444 . If the tracking target object 435 is not detected by the specific object tracking device 400G, the selection of the specific object tracking device 400 to be the target of the rescue request is repeated, and another specific object tracking device 400H to be the new target of the rescue request. to select.

The tracked object information storage unit 441 stores the tracked object information 405 as well as the tracked data 407 generated by the tracked data generation unit 431 .

In the own device information storage unit 443, sensor device information 457 including information (type, number and sensing range) of the sensors 452 provided in the sensor device 451 and own device information 406 including installation position information of the specific object tracking device 400 are stored. Stored.

Generated by the other specific object tracking devices 400B, 400C, . . . are stored in the peripheral device information storage unit 444 as peripheral device information 409B, 409C, . Help request candidate information 438 generated by the help request target selection unit 434 is also stored in the peripheral device information storage unit 444 .

With reference to FIG. 26, the specific object tracking processing procedure by the specific object tracking system 4000 in this embodiment will be described. FIG. 26 is a flow chart showing the processing procedure of the specific object tracking device 400 in the specific object tracking system 4000. FIG.

When the specific object tracking device 400 starts searching for the tracked object 435 (S400), the specific object tracking device 400 acquires sensor device information 457 from the sensor device 451 (S400-1). The specific object tracking device 400 stores the acquired sensor device information 457 in the sensor device information storage section 442 (S400-2).

The specific object tracking device 400 acquires the sensing information 456 from the sensor device 451 by the sensing information acquisition unit 411 (S401). The sensing information acquisition unit 411 sends the acquired sensing information 456 to the tracked object search unit 412 .

The tracked object search unit 412 acquires the tracked object information 405 from the tracked object information storage unit 441, compares the sensing information 456 acquired from the sensing information acquisition unit 411 with the tracked object information 405, and finds the tracked object 435. (S402). The tracked object search unit 412 generates a sensor control signal 408 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 452 when searching for the tracked object 435. , to the sensor control unit 423 (S402).

The search for the tracked object 435 by the tracked object search unit 412 will be specifically described for the case where the tracked object 435 is a vehicle and the case where the tracked object 435 is a person.

When the tracked object 435 is a vehicle, the tracked object search unit 412 searches for the vehicle included in the acquired sensing information 456, and checks the license plate (vehicle number), vehicle type, body color, etc. of each detected vehicle. is extracted as identification information for identifying the vehicle. The tracked object search unit 412 collates the extracted identification information with the acquired tracked object information 405 , extracts vehicles that match the tracked object information 405 , and detects the tracked object 435 .

When the tracked object 435 is a person, the tracked object searching unit 412 searches for the person included in the acquired sensing information 456, and for each detected person, an image such as age, gender, face photograph, etc. Information such as feature amount extraction data based on this is extracted as identification information for identifying a person. The tracked object search unit 412 collates the extracted identification information with the acquired tracked object information 405 , extracts a person who matches the tracked object information 405 , and detects the tracked object 435 .

The tracked object search unit 412 checks whether the tracked object 435 has been detected (S403). If the tracked object 435 can be detected (detected in S403), the tracked object search unit 412 generates search result information 417 as the search result of the tracked object 435 and sends it to the tracking data generation unit 431. FIG. If the tracked object 435 has not been detected (not detected in S403), the process is repeated from step S401.

The tracking data generator 431 generates the tracking data 407 from the search result information 417 of the tracked object 435 acquired from the tracked object searcher 412, and stores it in the tracked object information storage 441 (S410).

Next, the specific object tracking device 400A determines whether or not the specific object tracking device 400A can continue tracking the tracking target object 435 by the tracking continuation possibility determination unit 432 (S411), and the relief request target selection unit 434 In parallel, selection of a target for which tracking assistance is requested when the specific object tracking device 400A cannot continue tracking (S415) is executed.

27 and FIG. 4 to FIG. 8 again, a detailed procedure for determining whether or not tracking can be continued in the specific object tracking device 400A by the tracking continuation determination unit 432 (S411) will be described. FIG. 27 is a flow chart showing the detailed processing procedure of the tracking continuation possibility determination unit 432 for tracking continuation possibility determination (S411) in the specific object tracking device 400A.

The tracking continuation possibility determination unit 432 acquires the tracking data 407 from the tracking data generation unit 431 and the tracking object information storage unit 441 (S411-1).

The tracking continuation possibility determination unit 432 estimates the course of the tracked object 435 from the acquired tracking data 407 (S411-2). The estimated course of the tracked object 435 and the blind spot of the sensor 452A are compared to determine whether or not the tracking of the tracked object 435 by the specific object tracking device 400A can be continued (S411-3).

Alternatively, the tracking continuation possibility determination unit 432 extracts the change history of the detected position and size of the tracked object 435 from the tracking data 407 (S411-4). The extracted change history and the sensing range of the sensor 452A are compared to determine whether or not the specific object tracking device 400A can continue tracking the tracked object 435 (S411-5).

In the determination in step S411-3, for example, as shown in FIG. 4, if the estimated course of the tracked object 435 is not included in the blind spot of the sensor 452A, it is determined that the specific object tracking device 400A can continue tracking. judge. Here, the blind spot of the sensor 452A refers to a non-sensable area of the sensor 452A that is generated when the outer wall of the building in which the specific object tracking device 400A is installed is included in the sensing range of the sensor 452A. On the other hand, when the estimated course of the tracked object 435 is included in the blind spot of the sensor 452A as shown in FIG. 5, it is determined that the specific object tracking device 400A cannot continue tracking.

In the determination in step S411-5, if the detected position and size of the tracked object 435 in the sensing range of the sensor 452A do not change as shown in FIG. It is judged to be acceptable. On the other hand, as shown in FIG. 7, the detection position of the tracked object 435 in the sensing range of the sensor 452A is moving in the horizontal direction (or in the vertical direction), and may deviate from the sensing range of the sensor 452A (into a blind spot). In the assumed case, the detection size of the tracked object 435 in the sensing range of the sensor 452A is small (or large) as shown in FIG. If it is assumed that detection (identification) will not be possible, it is determined that the specific object tracking device 400A cannot continue tracking.

If both the determination result in step S411-3 and the determination result in step S411-5 are acceptable, the tracking continuation possibility determination unit 432 determines that tracking of the tracking target object 435 by the specific object tracking device 400A can be continued. It is determined that there is (S411-6), and if both the determination result in step S411-3 and the determination result in step S411-5, or either one is negative, the tracking target in the specific object tracking device 400A It is determined that tracking of the object 435 should not be continued (S411-6).

If the tracking continuation possibility determination unit 432 determines that the tracking of the tracking target object 435 by the specific object tracking device 400A is allowed (Yes in S411-6), the specific object tracking device 400A returns to step S401 and returns to step S401. The search and detection of the object 435 are repeated, and the tracked object 435 is tracked.

If the tracking continuation possibility determination unit 432 determines that the specific object tracking device 400A cannot continue tracking the tracked object 435 (No in S411-6), the tracking continuation possibility determination unit 432 determines that the specific object tracking device 400A cannot continue tracking. 400A sends to the rescue request target selection unit 434 the judgment result that the tracking continuation is not possible.

Referring to FIG. 28, a detailed procedure of target selection (S415) of other specific object tracking devices 400B, 400C, . . . . FIG. 28 is a flow chart showing a detailed processing procedure for target selection (S415) of another specific object tracking device 400 in the help request target selection unit 434. FIG.

The help request target selection unit 434 checks whether the help request candidate information 438 is stored in the peripheral device information storage unit 444 (S415-1).

If the help request candidate information 438 is not stored in the peripheral device information storage unit 444 (No in S415-1), the peripheral device information 409B, 409C, . -2).

Further, the rescue request target selection unit 434 acquires the estimated course of the tracked object 435 estimated in step S411-2 from the tracking continuation possibility determination unit 432 (S415-3).

Then, the relationship between the estimated course of the tracked object 435 obtained and the sensing ranges of the specific object tracking devices 400B, 400C, . By comparison, the specific object tracking device 400G, which includes the estimated path of the tracked object 435 in its sensing range, is selected as a help request target device (S415-4).

The rescue request target selection unit 434 sends the tracking data 407 via the communication unit 421 to the specific object tracking device 400G selected as the rescue request target (S415-5). The help request target selection unit 434 acquires and confirms whether or not the tracking target object 435 can be detected by the specific object tracking device 400G (S415-6).

If the tracking target object 435 is not detected by the specific object tracking device 400G (No in S415-6), the selection of the rescue request target device in step S415-4 is redone. If the tracking target object 435 is detected by the specific object tracking device 400G (possible in S415-6), the specific object tracking device 400G is again selected as a help request target device (S415-7).

The relief request target selection unit 434 confirms whether or not there is a determination result in step S411 by the tracking continuation possibility determination unit 432 (S415-8), and there is no determination result in step S411 (not S415-8 = determination “possible” in S411). ”), the information of the specific object tracking device 400G is stored in the peripheral device information storage unit 444 as the help request candidate information 438 (S415-9).

If there is a determination result in step S411 (yes in S415-8=determination "no" in S411), the help request target selection unit 434 generates a help request signal for the selected specific object tracking device 400G (S415-10). ). After that, the process proceeds to step S425.

In the specific object tracking device 400, the tracking continuation possibility determination unit 432 determines that the specific object tracking device 400A cannot continue tracking (S411), and the help request target selection unit 434 determines that the help request target specific object tracking device 400G If selected (S415), a help request signal is sent to the specific object tracking device 400G to request tracking help for the tracked object 435 (S425).

(Effect of the fourth embodiment)
Since the specific object tracking system 4000 of the present embodiment is composed of a plurality of fixed-point specific object tracking devices 400, if the specific object tracking devices 400 are installed, for example, on a building or roadside, detailed tracking can be performed over a wide area. It is possible. Furthermore, the specific object tracking device 400 of the present embodiment continues tracking with the surrounding specific object tracking devices 400B, 400C, . Tracking is also possible.

The specific object tracking device 400 of this embodiment autonomously executes a series of processing operations for detecting and tracking the tracked object 435 and requesting help from another specific object tracking device 400 . Therefore, it is not necessary for the operator of the specific object tracking device or the like to perform the processing operations related to the detection and tracking of the tracked object 435 or the request for tracking to another specific object tracking device 400 .

(Fifth embodiment)
A specific object tracking system 5000 according to the fifth embodiment of the present invention will be described with reference to FIG. A specific object tracking system 5000 is composed of a plurality of specific object tracking devices 500 . FIG. 29 is a configuration block diagram of a specific object tracking system 5000. As shown in FIG. A specific object tracking device 500 in this embodiment is integrated with a sensor device 551, like the specific object tracking device 400 in the fourth embodiment.

The specific object tracking device 500 according to the present embodiment is different from the specific object tracking device 400 according to the fourth embodiment in that each device is of a mobile installation type installed in a vehicle or the like.

The sensor device 551 is a device that generates sensing information 556 from image data acquired by a sensor 552, which will be described later. The sensor device 556 is composed of a sensor 552 and a controller 553, like the sensor device 451 in the fourth embodiment.

Sensor 552 is a video camera such as a drive recorder. However, the sensor 552 is not limited to a moving image camera, and may be a still image camera that captures still images, a distance measurement sensor such as LIDAR or millimeter wave radar, or the like. Moreover, the sensor 552 does not need to be single, and a plurality of types of the sensors described above may be used together, or a GPS sensor or the like for acquiring position information as world coordinates may also be provided.

The sensor 552 sends the acquired image data, the distance to the subject, and the relative positional relationship to the control unit 553 .

The control unit 553 generates sensing information 556 from the image data acquired from the sensor 552, the distance to the subject, and the relative positional relationship. The control unit 553 generates sensor device information 557 from the information of the sensor 552 (type, number and sensing range) and the installation position information of the sensor device 551 . The control unit 553 controls the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 552 using a sensor control signal 508 obtained from the sensor control unit 523 of the specific object tracking device 500, which will be described later. do. The control unit 553 sends the generated sensing information 556 to the sensing information acquisition unit 511 of the specific object tracking device 500, which will be described later, and the sensor device information 557 to the own device information storage unit 543 of the specific object tracking device 500, which will be described later. .

The sensing information 556 includes image data acquired by the sensor 552, the distance between the sensor 552 and the subject, and the relative positional relationship.

The sensor device information 557 includes information (type, number and sensing range) of the sensors 552 and position information of the sensor device 551 , that is, the specific object tracking device 500 .

When the tracking target object 535 is detected, the specific object tracking device 500 tracks the tracking target object 535 using the sensing information 556 acquired from the sensor device 551 .

When the specific object tracking device 500A determines that tracking by the sensor device 551A cannot be continued, it selects the specific object tracking device 500G that can continue tracking the tracked object 535 from the surrounding specific object tracking devices 500B, 500C, . Tracking of the tracked object 535 is requested and tracking of the tracked object 535 is continued.

The specific object tracking device 500A acquires peripheral device information 509B, 509C, . . . from all the specific object tracking devices 500B, 500C, . The acquired peripheral device information 509 is stored in a peripheral device information storage unit 544, which will be described later. Peripheral device information 509 includes information (type and number) of sensors 552B, 552C, . information is included.

The specific object tracking device 500 acquires sensor device information 557 from the sensor device 551 . The acquired sensor device information 557 is stored in the sensor device information storage unit 542, which will be described later. The sensor device information 557 includes information (type and number) of the sensors 552 included in the sensor device 551 as well as information such as the position information and planned route of the sensor devices 552 and the sensing range of the sensors 552 . Unlike the specific object tracking device 400 in the fourth embodiment, the specific object tracking device 500 in this embodiment is installed in a moving body (vehicle), so the position information of the specific object tracking device 500 changes every moment. do.

The specific object tracking device 500 periodically acquires peripheral device information 509 from other linked specific object tracking devices 500, and especially updates its position information.

The specific object tracking device 500 includes a sensing information acquisition unit 511, a tracking target object search unit 512, a communication unit 521, an access control unit 522, a sensor control unit 523, a tracking data generation unit 531, a tracking continuation possibility determination unit 532, and a help request target. It is composed of a selection unit 534 , a tracked object information storage unit 541 , a device information storage unit 543 and a peripheral device information storage unit 544 .

The sensing information acquisition unit 511 acquires sensing information 556 from the sensor device 551 . The sensing information acquisition unit 511 sends the acquired sensing information 556 to the tracked object search unit 512 .

The tracked object search unit 512 acquires the tracked object information 505 stored in the tracked object information storage unit 541, and compares the sensing information 556 acquired from the sensing information acquisition unit 511 with the tracked object information 505. Search for tracked object 535 . The tracked object search unit 512 generates a sensor control signal 508 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 552 when searching for the tracked object 535. and sent to the sensor control unit 523 .

If the tracked object 535 is a vehicle, the tracked object information 505 includes information such as the license plate (vehicle number), vehicle type, and body color. It is information such as an image or feature amount extraction data based on the image.

The tracked object search unit 512 searches for a subject from the moving image included in the sensing information 556, and obtains information such as the license plate (vehicle number), vehicle type, and body color for each detected vehicle, and calculates feature values for each detected person. Information such as extraction data is extracted as identification information for identifying a subject.

The tracked object search unit 512 compares the identification information extracted from the sensing information 556 with the tracked object information 505 regarding the tracked object 535, extracts a subject that matches the tracked object information 505, and searches for the tracked object. 535 is detected.

Here, from the relative positional relationship between the sensor device 551 and the tracked object 535, the position of the tracked object 535 can be transformed into world coordinates. If sensor 552 includes a GPS sensor, the position of sensor device 551 can be obtained as world coordinates. Since the mounting state (position, angle, etc.) of the sensor 552 in the sensor device 551 is known, it is also possible to transform the position of the tracked object 535 from the world coordinates of the sensor device 551 into world coordinates. In the following description, it is not particularly specified whether the positions of the sensor device 551 and the tracked object 535 are relative positions between the sensor device 551 and the tracked object 535 or world coordinates converted as described above. It is simply referred to as location information.

The tracked object search unit 512 newly adds the detected coordinates of the detected tracked object 535 in the imaging area in the sensing information 556 to the tracked object information 505, and searches for the tracked object 535 as search result information 517. is sent to the trace data generator 531 . The search result information 517 includes detection result of the tracked object 535 , position information of the sensor device 551 and the tracked object 535 , sensing information 556 and tracked object information 505 .

If the tracked object 535 cannot be detected, the tracked object search unit 512 acquires new sensing information 556 and repeats the search for the tracked object 535 .

The communication unit 521 acquires peripheral device information 509B, 509C, . . . from other specific object tracking devices 500B, 500C, . The communication means 521 transmits the acquired peripheral device information 509B, 509C, . . . Further, the communication unit 521 sends own device information 506A regarding the specific object tracking device 500A stored in the own device information storage unit 543 to the other specific object tracking devices 500B, 500C, . Own device information 506A is sensor device information 557A generated by sensor device 551A.

The communication means and communication method between the other specific object tracking devices 500B, 500C, . . . by the communication unit 521 are not particularly limited. For example, wireless communication lines such as wireless LAN communication and mobile phone lines, or wired communication lines such as fixed telephone lines and optical fiber lines can be used.

The access control unit 522 acquires the access control signal 524 generated by the help request target selection unit 534 and controls which specific object tracking device 500 to access. Means for switching access to other specific object tracking devices 500 and connection methods with other specific object tracking devices 500 are not particularly limited, and each specific object tracking device 500 is given an identification number. or a unique code or the like can be given.

The sensor control unit 523 acquires the sensor control signal 508 generated by the tracked object search unit 512 or the tracking data generation unit 531 and sends it to the control unit 553 of the sensor device 551 . In this embodiment, if the sensor 552 of the sensor device 551 cannot control the rotation angle (pan), elevation or depression angle (tilt), and imaging magnification (zoom) like a general drive recorder, the sensor The sensor control signal 508 is invalidated in the controller 523 or the controller 553 of the sensor device 551 .

The tracking data generation unit 531 extracts from the history of the position information accompanying the tracking of the tracked object 535 included in the search result information 517 acquired from the tracked object search unit 512 and the image information captured at a new angle or the image. The tracking data 507 including the new information such as the feature amount data obtained from the tracking data is generated and sent to the tracking object information storage unit 541 .

The tracking data generation unit 531 extracts feature amount data from an arbitrary image frame in the search result information 517 acquired from the tracking target object search unit 512 . The tracking data generator 531 not only tracks the tracked object 535 by searching for feature amount data that matches or is similar to the feature amount data in subsequent image frames, but also uses a kernelized correlation filter to perform faster tracking. Tracked object 535 may also be tracked to generate tracking data 507 .

When generating the tracking data 507 , the tracking data generation unit 531 generates a sensor control signal 508 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 552 . It is sent to the control unit 523 .

The tracking continuation possibility determination unit 532 determines whether or not the tracking by the specific object tracking device 500A can be continued. Based on the tracking data 507 acquired from the tracking object information storage unit 541, the tracking continuation determination unit 532 determines the tracking target object based on the state of the tracking target object 535 and the change history of the detection position of the tracking target object 535 in the sensing range of the sensor 552A. Estimate the course of 535. For example, if the tracked object 535 is a vehicle, the route of the tracked object 535 is estimated based on whether the tracked object 535 is in the straight lane or in the right turn (or left turn) lane.

The tracking continuation possibility determination unit 532 acquires the planned route of the specific object tracking device 500A from the own device information storage unit 543 . From the relationship between the estimated route of the tracked object 535 and the planned route of the specific object tracking device 500A, or the change history of the detected position and detection size of the tracked object 535 in the sensing range of the sensor 552A, the specific object tracking device 500A Determine whether to continue tracking.

When it is determined that tracking can be continued by the specific object tracking device 500A, the search (detection) of the tracking target object 535 is repeated using the sensing information 556A. When it is determined that the specific object tracking device 500A cannot continue tracking, the determination result is sent to the relief request target selection unit 534 .

For example, when the estimated route of the tracked object 535 and the planned route of the specific object tracking device 500A are different, or when the detection size of the tracked object 535 in the sensing range of the sensor 552A changes small (or large), the sensor 552A If it is assumed that the tracking target object 535 cannot be detected even if the magnification of . .

The rescue request target selection unit 534 stores the peripheral device information 509B, 509C, . , and selects a specific object tracking device 500G that is a candidate for a rescue request for the tracked object 535 from the estimated route of the tracked object 535 and the positional relationship between the specific object tracking devices 500B, 500C, .

The rescue request target selection unit 534 sends the tracking data 507 acquired from the tracked object information storage unit 541 to the selected specific object tracking device 500G, and acquires the result of whether or not the tracking target object 535 can be detected from the specific object tracking device 500G. do. If the tracking target object 535 can be detected by the specific object tracking device 500G, the specific object tracking device 500G is selected as the target for the rescue request.

Here, if the determination result of the tracking continuation possibility determination unit 532 is negative, the help request target selection unit 534 generates a help request signal to the specific object tracking device 500G, Send to 500G. If the determination result of the tracking continuation possibility determination unit 532 is yes, the help request candidate information 538 of the specific object tracking device 500G selected as the target of the help request is generated and stored in the peripheral device information storage unit 544 . If the tracking target object 535 is not detected by the specific object tracking device 500G, the help request target selection unit 534 starts again from extraction of the specific object tracking device 500 to be the target of the help request, and selects a new help request target. Other specific object tracking devices 500H are extracted.

The tracked object information storage unit 541 stores the tracked object information 505 as well as the tracked data 507 generated by the tracked data generating unit 531 .

The sensor device information storage unit 542 stores sensor device information 557 including information (type, number and sensing range) of the sensors 552 included in the sensor device 551 .

The own device information storage unit 543 stores sensor device information 557 including information (type, number and sensing range) of the sensors 551 provided in the sensor device 551 and own device information 506 including position information of the specific object tracking device 500. . Since the specific object tracking device 500 in this embodiment is installed on a moving object as described above, the position information of the specific object tracking device 500 included in the own device information 506 is regularly updated.

The peripheral device information storage unit 544 stores the own device information 506B, 506C, . . . generated by the other specific object tracking devices 500B, 500C, . Help request candidate information 538 generated by the help request target selection unit 534 is also stored in the peripheral device information storage unit 544 .

With reference to FIG. 30, the specific object tracking processing procedure by the specific object tracking system 5000 in this embodiment will be described. FIG. 30 is a flow chart showing the processing procedure of the specific object tracking device 500 in the specific object tracking system 5000. FIG.

When the specific object tracking device 500 starts searching for the tracked object 535 (S500), the specific object tracking device 500 acquires sensor device information 557 from the sensor device 551 (S500-1). The specific object tracking device 500 stores the acquired sensor device information 557 in the sensor device information storage section 542 (S500-2).

The specific object tracking device 500 acquires the sensing information 556 from the sensor device 551 by the sensing information acquisition unit 511 (S501). The sensing information acquiring unit 511 sends the acquired sensing information 556 to the tracking target object searching unit 512 .

The tracked object search unit 512 acquires the tracked object information 505 from the tracked object information storage unit 541, compares the sensing information 556 acquired from the sensing information acquisition unit 511 with the tracked object information 505, and finds the tracked object 535. (S502). The tracked object search unit 512 generates a sensor control signal 508 for controlling the rotation angle (pan), elevation angle or depression angle (tilt), and imaging magnification (zoom) of the sensor 552 when searching for the tracked object 535. , to the detailed examination control unit 523 (S502).

The search for the tracked object 535 by the tracked object search unit 512 will be specifically described for the case where the tracked object 535 is a vehicle and the case where the tracked object 535 is a person.

When the tracked object 535 is a vehicle, the tracked object search unit 512 searches for the vehicle included in the acquired sensing information 556, and checks the license plate (vehicle number), vehicle type, body color, etc. of each detected vehicle. is extracted as identification information for identifying the vehicle. The tracked object search unit 512 collates the extracted identification information with the acquired tracked object information 505 , extracts a vehicle that matches the tracked object information 505 , and detects the tracked object 535 .

When the tracked object 535 is a person, the tracked object searching unit 512 searches for the person included in the acquired sensing information 503, and for each detected person, an image such as age, sex, face photograph, etc. Information such as feature amount extraction data based on this is extracted as identification information for identifying a person. The tracked object search unit 512 collates the extracted identification information with the acquired tracked object information 505 , extracts a person who matches the tracked object information 505 , and detects the tracked object 535 .

The tracked object search unit 512 checks whether the tracked object 535 has been detected (S503). If the tracked object 535 can be detected (detected in S503), the tracked object search unit 512 generates search result information 517 as the search result of the tracked object 535 and sends it to the tracking data generation unit 531. FIG. If the tracked object 535 has not been detected (not detected in S503), the process is repeated from step S501.

The tracking data generator 531 generates the tracking data 507 from the search result information 517 of the tracked object 535 acquired from the tracked object searcher 512, and stores it in the tracked object information storage 541 (S510).

Next, the specific object tracking device 500A determines whether or not the specific object tracking device 500A can continue tracking the tracking target object 535 by the tracking continuation possibility determination unit 532 (S511), and the relief request target selection unit 534 Selection of a target for which tracking continuation is requested when tracking continuation by the specific object tracking device 500A is impossible (S515) is executed in parallel.

31 and FIG. 13 to FIG. 17 again, the detailed procedure for determining whether or not tracking can be continued in the specific object tracking device 500A by the tracking continuation determination unit 532 (S511) will be described. FIG. 31 is a flow chart showing the detailed processing procedure of the tracking continuation possibility determination unit 532 for tracking continuation possibility determination (S511) in the specific object tracking device 500A.

The tracking continuation possibility determination unit 532 acquires the tracking data 507 from the tracking data generation unit 531 and the tracking object information storage unit 541 (S511-1).

The tracking continuation possibility determination unit 532 estimates the course of the tracked object 535 from the acquired tracking data 507 (S511-2). The estimated route of the tracked object 535 and the planned route of the specific object tracking device 500A are compared to determine whether or not the tracking of the tracked object 535 by the specific object tracking device 500A can be continued (S511-3). .

Alternatively, the tracking continuation possibility determination unit 532 extracts the change history of the detected position and size of the tracked object 535 from the tracking data 507 (S511-4). The extracted change history and the sensing range of the sensor 552A are compared to determine whether or not the specific object tracking device 500A can continue tracking the tracked object 535 (S511-5).

In the determination in step S511-3, for example, as shown in FIG. 13, if the estimated route of the tracking target object 535 and the planned route of the specific object tracking device 500A are the same, the specific object tracking device 500A continues tracking. is determined to be acceptable. On the other hand, when the estimated route of the tracked object 535 and the planned route of the specific object tracking device 500A are different as shown in FIG. 14, it is determined that the specific object tracking device 500A cannot continue tracking.

In the determination in step S511-5, if the detected position and size of the tracked object 535 in the sensing range of the sensor 552A do not change as shown in FIG. It is judged to be acceptable. On the other hand, as shown in FIG. 16, the detection position of the tracked object 535 in the sensing range of the sensor 552A moves in the horizontal direction (or in the vertical direction), and it may deviate from the sensing range of the sensor 552A (into a blind spot). In the assumed case, the detection size of the tracked object 535 in the sensing range of the sensor 552A is small (or large) as shown in FIG. If it is assumed that (identification) will not be possible, it is determined that the specific object tracking device 500A cannot continue tracking.

If both the determination result in step S511-3 and the determination result in step S511-5 are acceptable, the tracking continuation possibility determination unit 532 determines that tracking of the tracking target object 535 by the specific object tracking device 500A can be continued. It is determined that there is (S511-6), and if both the determination result in step S511-3 and the determination result in step S511-5, or either one is negative, the tracking target in the specific object tracking device 500A It is determined that the tracking of the object 535 should not be continued (S511-6).

If the tracking continuation possibility determination unit 532 determines that the tracking of the tracking target object 535 by the specific object tracking device 500A is allowed (Yes in S511-6), the specific object tracking device 500A returns to step S501 and returns to the tracking target. The search and detection of the object 535 are repeated, and the tracked object 535 is tracked.

If the tracking continuation possibility determination unit 532 determines that the specific object tracking device 500A cannot continue tracking the tracking target object 535 (No in S511-6), the tracking continuation possibility determination unit 532 determines that the specific object tracking device 500A cannot continue tracking. 500A sends to the rescue request target selection unit 534 the judgment result that the tracking continuation is not possible.

Referring to FIG. 32, a detailed procedure of target selection (S515) of other specific object tracking devices 500B, 500C, . . FIG. 32 is a flow chart showing a detailed processing procedure of target selection (S515) of another specific object tracking device 500 in the help request target selection unit 534. FIG.

The help request target selection unit 534 checks whether the help request candidate information 538 is stored in the peripheral device information storage unit 544 (S515-1).

If the help request candidate information 538 is not stored in the peripheral device information storage unit 544 (NO in S515-1), the other specific object tracking devices 500B, 500C, . are acquired as peripheral device information 509B, 509C, . . . (S515-2). In this embodiment, since any of the specific object tracking devices 500 is installed in a mobile object such as a vehicle, its position information in particular changes from moment to moment. Therefore, instead of acquiring the peripheral device information 509B, 509C, . . . to get

Furthermore, the relief request target selection unit 534 acquires the estimated course of the tracked object 535 estimated in step S511-2 from the tracking continuation possibility determination unit 532 (S515-3).

Then, the relationship between the estimated course of the tracked object 535 obtained and the sensing ranges of the specific object tracking devices 500B, 500C, obtained from the peripheral device information 509B, 509C, obtained in step 515-2. By comparison, the specific object tracking device 500G, which includes the estimated path of the tracked object 535 in its sensing range, is selected as a help request target device (S515-4).

The rescue request target selection unit 534 sends the tracking data 507 via the communication unit 521 to the specific object tracking device 500G selected as the rescue request target (S515-5). The rescue request target selection unit 534 acquires and confirms the result of whether or not the tracking target object 535 can be detected by the specific object tracking device 500G (S515-6).

If the tracking target object 535 is not detected by the specific object tracking device 500G (No in S515-6), the selection of the rescue request target device is redone in step S515-4. If the tracking target object 435 is detected by the specific object tracking device 500G (possible in S515-6), the specific object tracking device 500G is again selected as a rescue request target device (S515-7).

The relief request target selection unit 534 confirms whether or not there is a determination result of step S511 by the tracking continuation possibility determination unit 532 (S515-8), and there is no determination result of step S511 (not S515-8 = determination “possible” in S511). ”), the information of the specific object tracking device 400G is stored in the peripheral device information storage unit 544 as the help request candidate information 538 (S515-9).

If there is a determination result in step S511 (yes in S515-8=determination "no" in S511), the relief request target selection unit 534 generates a relief request signal for the selected specific object tracking device 500G (S515- 10). After that, the process proceeds to step S525.

In the specific object tracking device 500, the tracking continuation possibility determination unit 532 determines that the specific object tracking device 500A cannot continue tracking (S511), and the help request target selection unit 534 selects the specific object tracking device 500G for the help request. If selected (S515), a help request signal is sent to the specific object tracking device 500G to request the tracking help of the tracked object 535 (S525).

(Effect of the fifth embodiment)
Since the specific object tracking system 5000 of this embodiment is composed of a plurality of specific object tracking devices 500, detailed tracking in a wide area is possible. Furthermore, the specific object tracking device 500A of the present embodiment continues tracking with the surrounding specific object tracking devices 500B, 500C, . Tracking is also possible.

The specific object tracking device 500 of this embodiment autonomously executes a series of processing operations for detecting and tracking the tracked object 535 and requesting other specific object tracking devices 500 for tracking assistance. Therefore, the operator of the specific object tracking device 500, the driver of the vehicle, and the passenger of the vehicle bear the processing operations related to the detection and tracking of the tracked object 535 or requesting assistance from another specific object tracking device 500. No need. In addition, the driver of the vehicle and his fellow passengers do not change the planned route or take a detour from the planned route in order to continue tracking with the specific object tracking device 500 .

Description of usage

In the specific object tracking systems 1000 to 5000 described in each embodiment of the present invention described above,
Although detection and tracking of a single tracked object 135 to 535 has been described as an example, it is also possible to detect and track two or more multiple tracked objects 135 to 535 in parallel.

In the specific object tracking systems 1000 to 5000 described in each embodiment of the present invention, when the tracked objects 135 to 535 being tracked by the specific object tracking devices 100 to 500 are criminal vehicles, wanted criminals, etc. can be reported to the local police, etc.

In addition, the specific object tracking devices 100 to 500 described in each embodiment of the present invention respond to an emergency search request from the local police, etc., or a missing person report from a facility such as a nursing home. It is also possible to terminate the tracking of the tracked objects 135-535 and start searching and tracking the tracked objects 135'-535' based on the new tracked object information 105'-505' according to the emergency request.

In such a case, the specific object tracking devices 100 to 500 are used when the tracked objects 135' to 535', which are criminals such as criminal vehicles and wanted criminals, are arrested or detained by the police, etc. under jurisdiction. When all information related to the tracked objects 135' to 535' is transferred (provided) to, or when a missing person's securing report is received from a facility such as a nursing home, search/ It is also possible to terminate the search/tracking of the tracked objects 135' to 535' that are being tracked. When the specific object tracking devices 100 to 500 terminate the search and tracking of the tracked objects 135' to 535', the access control units 122 to 522 change the operation mode of the sensor devices 151 to 551 under access to the standby mode. This is done by changing to and canceling access.

100, 200, 300, 400, 500 Specific object tracking device 105, 205, 305, 405, 505 Tracked object information 106, 206, 306, 406, 506 Own device information 107, 207, 307, 407, 507 Tracking data 108 , 208, 308, 408, 508 sensor control signal 309, 409, 509 peripheral device information 111, 211, 311, 411, 511 sensing information acquisition unit 112, 212, 312, 412, 512 tracking object search unit 113, 213, 313 Operation mode control unit 117, 217, 317, 417, 517 Search result information 121, 221, 321, 421, 521 Communication unit 122, 222, 322, 422, 522 Access control unit 123, 223, 323, 423, 523 Sensor Control units 124, 224, 324, 424, 524 Access control signals 131, 231, 331, 431, 531 Tracking data generators 132, 232, 332, 432, 532 Tracking continuation determination units 133, 233, 333 Tracking request target selection Units 334, 434, 534 Help request target selection units 135, 235, 335, 435, 535 Tracked objects 137, 237, 337 Tracking request candidate information 338, 438, 538 Help request candidate information 141, 241, 341, 441, 541 tracked object information storage units 142, 242, 342 sensor device information storage units 343, 443, 543 own device information storage units 344, 444, 544 peripheral device information storage units 151, 251, 351, 451, 551 sensor devices 152, 252, 352, 452, 552 sensors 153, 253, 353, 453, 553 control units 154, 254, 354 transmission/reception units 156, 256, 356, 456, 556 sensing information 157, 257, 357, 457, 557 sensor device information 1000, 2000 , 3000, 4000, 5000 Specific Object Tracking System

Claims (10)

A communication unit that is interconnected with M (M is an integer satisfying 2≦M) sensor devices each provided with a sensor via arbitrary connection means and communicates with the sensor devices, and the sensors that the communication unit communicates with an access control unit for switching devices, a sensing information acquisition unit for acquiring sensing information acquired by an m-th (m is an integer satisfying 1≦m<M) sensor device connected via the communication unit, and the sensing information A tracked object search unit for detecting a tracked object based on the sensing information acquired by the acquisition unit, a tracking data generation unit for generating tracking data regarding the tracked object detected by the tracked object search unit, and the m-th sensor. A tracking continuation determination unit that determines whether or not tracking can be continued by the device, a tracking request target selection unit that selects the m+1th sensor device that requests continuation of tracking, and another specific object tracking device that requests tracking assistance. It consists of a selection department for relief requests,
The tracking request target selection unit
The (m+1)-th sensor device for which the tracking continuation enable/disable determination unit determines that the m-th sensor device cannot continue tracking, and the tracking request target selection unit requests continuation of tracking of the tracked object. is selected, sending a tracking request signal for the tracked object to the m+1-th sensor device selected in the tracking request target selection to the access control unit;
The rescue request target selection unit
The (m+1)-th sensor device for which the tracking continuation possibility determination unit determines that the m-th sensor device cannot continue tracking, and the tracking request target selection unit requests continuation of tracking of the tracked object. is not selected, sending a help request signal for tracking the tracked object to the other specific object tracking device selected by the help request target selection unit to the access control unit;
The access control unit
Switching a connection destination to the m+1 sensor device according to the acquired tracking request signal, and transmitting the tracking request signal via the communication unit;
or
A specific object tracking device that switches a connection destination to the other specific object tracking device according to the acquired help request signal, and transmits the help request signal via the communication unit. The specific object tracking device according to claim 1 ,
Further, a tracked object information storage unit for storing information about the tracked object, a sensor device information storage unit for storing information about the connected sensor device, and a peripheral device information storage unit for storing information about the other specific object tracking device. prepared,
The tracking request target selection unit
When the tracking continuation possibility determination unit determines that the m-th sensor device can continue tracking, tracking request candidate information about the m+1-th sensor device selected by the tracking request target selection unit stored in the sensor device information storage unit;
The rescue request target selection unit
Help request candidate information about the other specific object tracking device selected by the help request target selection unit when the tracking continuation possibility determination unit determines that the tracking continuation by the m-th sensor device is possible. in the peripheral device information storage unit. The specific object tracking device according to claim 2 ,
further comprising an operation mode control unit that sets and stores an operation mode of the specific object tracking device according to a search result of the tracked object by the tracked object search unit;
The operation mode control unit
switching the operation mode from a search mode to a tracking mode when the tracked object is detected by the tracked object searching unit;
when the operation mode is the tracking mode, sending a search result from the tracked object search unit to the tracking data generation unit;
When the operation mode is the search mode and the tracked object is not detected by the tracked object search unit, a switching signal for the connected sensor device is generated and sent to the access control unit. A specific object tracking device characterized by: The specific object tracking device according to any one of claims 1 to 3 ,
The tracking continuation possibility determination unit
Either the case where the estimated course of the tracked object estimated from the tracking data is different from the planned route of the sensor device or the specific object tracking device, or the case where the detection of the tracked object in the sensing information becomes difficult. A specific object tracking device that determines that tracking by the specific object tracking device is not to be continued by matching one or both. The specific object tracking device according to claim 3 ,
The tracking request target selection unit
It is possible to detect the tracked object by the tracking data sent from the other specific object tracking device that is estimated to be able to track the tracked object from the relative position history with the tracked object and the tracking data sent from the own device. Any one or both of the other specific object tracking devices confirmed to be a specific object tracking device is selected as a target for which tracking of the tracked object is requested. The specific object tracking device according to any one of claims 2 or 3 ,
The rescue request target selection unit
It is possible to detect the tracked object by the tracking data sent from the other specific object tracking device that is estimated to be able to track the tracked object from the relative position history with the tracked object and the tracking data sent from the own device. A specific object tracking device, wherein any one of the other specific object tracking devices confirmed to exist is selected as a target for which tracking of the tracked object is requested. a specific object tracking device according to any one of claims 1, 4, and 5 ;
A specific object tracking system configured by connecting N (N is an integer satisfying 2≦N) sensor devices each provided with a sensor to the specific object tracking device via arbitrary connection means,
The tracking continuation possibility determination unit of the specific object tracking device determines that tracking continuation by the n-th sensor device (n is an integer satisfying 1≦n<N) responsible for detecting and tracking the tracking target object is not possible. A series of processes for requesting tracking to the n+1-th sensor device selected by the tracking request target selection unit and switching connection destinations are autonomously processed by the specific object tracking device when it is determined that A specific object tracking system characterized by: A specific object tracking system configured by connecting a plurality of specific object tracking subsystems to each other via a first connection means,
The specific object tracking subsystem includes:
7. The specific object tracking device according to any one of claims 2, 3 or 4 to 6, and M pieces (M is an integer satisfying 2≤M) mutually connected to the specific object tracking device through a second connection means ) sensor device and
The specific object tracking device provided in each of the plurality of specific object tracking subsystems,
The tracking continuation possibility determination unit determines that the m-th sensor device (m is an integer satisfying 1≤m<M) cannot continue tracking, and the tracking request target selection unit determines that the tracking target object When the m+1-th sensor device requesting continuation of tracking is selected, a request for tracking to the m+1-th sensor device selected by the tracking request target selection unit and switching of the connected sensor device a series of processes,
or
The tracking continuation possibility determination unit determines that the m-th sensor device (m is an integer satisfying 1≤m<M) cannot continue tracking, and the tracking request target selection unit determines that the tracking target object a series of processes for requesting help from the other specific object tracking device selected by the help request target selection unit when the m+1-th sensor device requesting continuation of tracking is not selected;
A specific object tracking system characterized by performing any processing autonomously. A specific object tracking system configured by connecting J (J is an integer satisfying 2≦J) specific object tracking devices according to claim 6 to each other via arbitrary connection means,
The tracking continuation determination unit of the j-th specific object tracking device (j is an integer satisfying 1≦j<J) responsible for detecting and tracking the tracking target object determines whether or not the specific object tracking device can continue tracking. If it is determined that there is, the j-th specific object tracking device autonomously processes a series of processing related to a request for tracking help to the j+1th specific object tracking device selected by the rescue request target selection unit. A specific object tracking system characterized by: The specific object tracking device according to any one of claims 1 to 6 ,
The specific object tracking device, wherein the sensor device includes a moving image camera as the sensor.

Images