JP6934386B2 - Moving object tracking device and its program - Google Patents

Description

The present invention relates to a moving object tracking device and a program thereof.

Conventionally, inventions have been proposed in which an aircraft takes off and landing at an airport while being automatically tracked by a robot camera (for example, Patent Documents 1 and 2).
The invention described in Patent Document 1 uses an ADS-B (Automatic Dependent Surveillance-Broadcast) signal emitted by an aircraft to identify the approach path and gliding direction of the aircraft, points the robot camera in the specified direction, and performs template matching. It tracks an aircraft.
Further, the invention described in Patent Document 2 predicts the approach path and the gliding direction of the aircraft from the measurement data of the wind direction and the wind speed, points the robot camera in the specified direction, and tracks the aircraft by template matching.

Japanese Patent No. 5858741 Japanese Unexamined Patent Publication No. 2013-119328

However, in the inventions described in Patent Documents 1 and 2, since template matching is used, there is a problem that the aircraft is difficult to be reflected at night and the detection accuracy of the aircraft is lowered. Further, in the invention described in Patent Document 1, it is difficult to track an aircraft that does not emit an ADS-B signal.

Therefore, an object of the present invention is to provide a moving object tracking device capable of tracking various moving objects day and night and a program thereof.

In view of the above-mentioned problems, the moving object tracking device according to the present invention is a moving object tracking device that tracks a moving object to be tracked by using a sensor image obtained by photographing the moving object with a sensor camera, and is a differential image generation means. The configuration includes a difference area detecting means, a storage means, a moving object detecting means, and a tracking means.

According to such a moving object tracking device, the difference image generation means generates a difference image from the sensor image. Further, the difference region detecting means detects the difference region from the difference image in each of the low sensitivity mode and the high sensitivity mode in which the detection sensitivity is higher than the low sensitivity mode.

Further, the storage means detects a tracking target identification area for identifying a moving object as a tracking target in the sensor image in the low-sensitivity mode, a mask area through which the tracking target does not pass, and a detection area error of the tracking target in the high-sensitivity mode. Area information indicating an erroneous detection prevention area for prevention is stored in advance.

Further, the moving body detecting means extracts a difference region whose position changes over a preset number of frames or more, refers to the area information, detects the moving body based on the position and detection sensitivity of the extracted difference region, and detects the moving body. Generates motion detection information that represents the position and affiliation area. Then, the tracking means calculates a score for each moving object based on the moving object detection information, and determines the tracking target based on the calculated score.

In this way, since the moving object tracking device uses the sensor image, it is possible to track various tracking targets such as an aircraft that does not emit an ADS-B signal. Further, since the moving object tracking device does not use template matching, it detects the moving object by the difference between frames and determines the tracking target by the area information, so that it can be accurately tracked even at night.

According to the present invention, the following effects are obtained.
Since the moving object tracking device according to the present invention uses the sensor image, detects the moving object based on the difference between frames, and determines the tracking target based on the area information, it can track various moving objects day and night.

It is an overall block diagram of the aircraft tracking photography system which concerns on embodiment of this invention. It is explanatory drawing explaining the installation of the sensor camera of FIG. It is an enlarged view of the sensor image of FIG. It is a block diagram which shows the structure of the aircraft tracking photography system of FIG. In the sensor image on the right side of FIG. 3, it is explanatory drawing of the area information set in the low sensitivity mode. In the sensor image on the right side of FIG. 3, it is explanatory drawing of the area information set in a high-sensitivity mode. In the sensor image on the right side of FIG. 3, it is explanatory drawing of the area information set at night. FIG. 5 is an explanatory diagram of area information set in the low sensitivity mode in the sensor image in the center of FIG. FIG. 5 is an explanatory diagram of area information set in the high sensitivity mode in the sensor image in the center of FIG. It is explanatory drawing explaining the detection of a moving body by the moving body detecting means of FIG. It is explanatory drawing explaining the detection of a moving body by the moving body detecting means of FIG. It is explanatory drawing explaining the erroneous detection of a moving body in the moving body detecting means of FIG. It is explanatory drawing explaining the erroneous detection of a moving body in the moving body detecting means of FIG. It is explanatory drawing explaining the calculation of the score by the tracking target determination means of FIG. It is explanatory drawing explaining the tracking of an aircraft by the aircraft tracking photography system of FIG. It is explanatory drawing explaining the tracking of an aircraft by the aircraft tracking photography system of FIG. It is a sequence diagram which shows the operation of the aircraft tracking photography system of FIG.

[Outline of aircraft tracking shooting system]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
An outline of the aircraft tracking imaging system 1 according to the embodiment of the present invention will be described with reference to FIG.

The aircraft tracking imaging system 1 tracks an aircraft (tracking target) departing from and arriving at an airport with a robot camera 40 and captures the image. As shown in FIG. 1, the sensor cameras 10 (10 _R , 10 _C , 10 _L ) A motion detection device 20 (20 _R , 20 _C , 20 _L ), a position identification / pan head control device 30, and a robot camera 40 are provided. Here, the moving object detection device 20 and the position identification / pan head control device 30 correspond to the moving object tracking device described in the claims.

The sensor camera 10 captures an aircraft and generates a sensor image. For example, the sensor camera 10 is a fixed camera having a constant shooting direction and shooting angle of view. Then, the sensor camera 10 outputs the generated sensor image to the motion detecting device 20.

Here, one or more sensor cameras 10 can be installed as required. In the present embodiment, as shown in FIG. 2, three sensor cameras 10 are installed on the roof of the building in the center of the airport so that the entire runway can be photographed. At this time, the imaging ranges of the sensor cameras 10 overlap each other. In FIG. 2, the sensor camera 10 _R faces the right side of the runway, the sensor camera 10 _C faces the center of the runway, and the sensor camera 10 _L faces the left side of the runway. Then, as shown in FIG. 3, the sensor cameras 10 _R , 10 _C , and 10 _L generate _{sensor images 90 R} , 90 _C , and 90 _L in which the runway is photographed in each photographing direction. The ends of the sensor images 90 _R , 90 _C , and 90 _{L overlap each other.}
In FIG. 2, the shooting angles of view of the sensor cameras 10 _R , 10 _C , and 10 _L are shown by broken lines so as not to overlap, but they actually overlap.

Returning to FIG. 1, the description of the aircraft tracking imaging system 1 will be continued.
The moving object detection device 20 analyzes an image of the sensor image from the sensor camera 10, detects a moving object, and outputs the detection result (moving object detection information) to the position identification / pan head control device 30. In the present embodiment, the moving object detection device 20 detects not only the aircraft to be tracked but also moving objects such as automobiles, people, and birds, and outputs moving object detection information about the moving objects.

In the present embodiment, three motion detection devices 20 are installed at a broadcasting station away from the airport so as to have a one-to-one correspondence with the sensor camera 10. Then, the moving object detection device 20 _R image-analyzes the sensor image 90 _R of the sensor camera 10 _R , the moving object detecting device 20 _C image-analyzes the sensor image 90 _C of the sensor camera 10 _C , and the moving object detecting device 20 _L is the sensor camera. the 10 _L of the sensor image 90 _L for image analysis.

The position identification / pan head control device 30 identifies an aircraft to be tracked from among the moving objects detected by the moving object detecting device 20 based on the trajectory of the moving object calculated from the moving object detection information. Then, the position identification / pan head control device 30 controls the robot camera 40 so as to take a picture of the specified aircraft. In the present embodiment, one position identification / pan head control device 30 is installed in the same broadcasting station as the motion detection device 20.
The tracking target is one moving object to be tracked by the aircraft tracking imaging system 1. In the present embodiment, among the aircraft at the airport, the aircraft taking off and landing is the target of tracking.

The robot camera 40 is driven according to a control signal from the position identification / pan head control device 30 to photograph an aircraft. The robot camera 40 is a general robot camera capable of panning, tilting, and zooming, and includes a camera body 41 that generates a captured image 91 of an aircraft, and a pan head 43 on which the camera body 41 is mounted.
In the present embodiment, one robot camera 40 is installed on the roof of the building in the center of the airport, like the sensor camera 10.

At night, it becomes difficult for the aircraft to appear in the sensor image, so the following measures are taken compared to daytime. For nighttime aircraft tracking imaging system 1 sets a nighttime approach path area A _{N (FIG.} 7) to the sensor image 90 to detect the aircraft lights from the nighttime approach path area A _N.

[Configuration of motion detector]
The configuration of the moving object detection device 20 will be described with reference to FIG.
As shown in FIG. 4, the moving object detecting device 20 includes a day / night determining means 21, a storage means 22, a difference image generating means 23, a difference area detecting means 24, a light detecting means 25, and a moving body detecting means 26. Be prepared.
Since the three motion detection devices 20 have the same configuration, only one of them will be described.

The day / night determination means 21 determines whether or not it is nighttime based on the time. In the present embodiment, the day / night determination means 21 calculates the sunrise time and the sunset time based on the latitude information and the longitude information, and determines the daytime time zone and the nighttime time zone. Then, the day / night determination means 21 determines daytime or nighttime depending on whether the current time belongs to the daytime time zone or the nighttime time zone. After that, the day / night determination means 21 outputs the daytime or nighttime determination result to the difference area detecting means 24 and the moving object detecting means 26.

The storage means 22 is a storage device such as a memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive) that stores various information necessary for processing of the motion detection device 20 in advance. For example, the administrator of the aircraft tracking imaging system 1 manually sets the area information in the storage means 22 via operating means such as a mouse and a keyboard (not shown). Further, the administrator of the aircraft tracking imaging system 1 sets the conversion information calculated in advance in the storage means 22.

The area information includes each area such as an aircraft specific area for identifying an aircraft (tracking target specific area), a mask area where an aircraft does not pass, and an erroneous detection prevention area for preventing an aircraft detection area error. This is the information set in advance.

<Area information setting>
The setting of the area information will be specifically described with reference to FIG.
Here, the area information is set in the sensor image 90 _R of the sensor camera 10 _R.

In the low-sensitivity mode, as shown in FIG. 5, the approach road area _AL , the runway area R, the mask area M, and the sky area S are set in the _{sensor image 90 R as area information.} In the present embodiment, the approach road area _AL and the runway area R correspond to the aircraft specific area. In the tracking target determination means 35 described later, among the moving bodies detected by the moving body detecting means 26, the moving body that has passed through the aircraft specific area even once is specified as an aircraft.

The approach road area A _L represents the area in which the aircraft taking off and landing enters in the sensor image 90 _R.
The runway area R represents an area of the runway included in the _{sensor image 90 R.}
Sky area S represents the airspace other than the approach path area A _L.
The mask area M represents an area where there are many false positives or it is not necessary to track an aircraft, such as an observation deck, an airport facility, a place with many birds, vegetation, and a puddle.

In the high-sensitivity mode, as shown in FIG. 6, the approach road area A _H and the guide road area T are set in the _{sensor image 90 R as area information.}
The approach road area A _H represents an area where aircraft taking off and landing enter, similar to the approach road area A _{L in FIG.} Since noise and the like are more likely to be erroneously detected in the high-sensitivity mode than in the low-sensitivity mode, the approach road areas A _H and A _L can be set in the sensor image 90 _{R at different positions and sizes.}
The guide path area T represents an area of the guide path included in the _{sensor image 90 R.} In the present embodiment, the guide path area T corresponds to the false detection prevention area.

Furthermore, as shown in FIG. 7, as the area information, to set the nighttime approach path area A _N sensor image 90 in _R.
Nighttime approach path area A _N, the approach path area A _H, similar to A _L, represents an area where aircraft taking off and landing enters. For night, nighttime approach path area A _N, since the area to detect the aircraft lights, it is preferable to set so as not to erroneously detect a non-aircraft lights.

Area information is set in the _{sensor image 90 C} as well as the sensor image 90 _R.
As shown in FIG. 8, in the low-sensitivity mode, as the area information, runway and area R, and the mask area M, sets the sky area S in the sensor image 90 in _C. Further, as shown in FIG. 9, in the high-sensitivity mode, the guide path area T is set in the _{sensor image 90 R as the area information.}
Since approach path is not displayed on the sensor image 90 _C, it does not set the access road area.
Further, since the area information can be set for the sensor image 90 _L in the same _{manner as the sensor image 90 R, the description thereof will be omitted.}

In this way, the area information can be set separately for daytime and nighttime. Therefore, in the area information, it is possible to exclude places where there is a high possibility of false detection, such as places where many birds fly in the daytime and places where lights other than aircraft are reflected at night.

Returning to FIG. 4, the motion detection device 20 of FIG. 1 will be described.
The conversion information is information on parameters for converting the angle of view between the sensor camera 10 and the robot camera 40. This conversion information is information necessary for matching the speed and size of the aircraft detected by the sensor camera 10 with the speed and size of the aircraft reflected on the robot camera 40. In the present embodiment, the conversion information is calculated in advance for each sensor camera 10 according to the shooting angle of view of each sensor camera 10.

The difference image generation means 23 receives a sensor image from the sensor camera 10 and generates a difference image from the input sensor image. That is, the difference image generation means 23 obtains the brightness difference for each pixel between the frames before and after the sensor image, and generates a difference image representing the obtained brightness difference for each pixel. Then, the difference image generating means 23 outputs the generated difference image to the difference area detecting means 24.

When the determination result from the day / night determination means 21 is daytime, the difference region detecting means 24 detects the difference region from the difference image in each of the low sensitivity mode and the high sensitivity mode. That is, the difference region detecting means 24 generates a difference region by grouping close pixels for each pixel whose luminance difference is equal to or greater than the threshold value.

Specifically, the difference area detecting means 24 determines the pixel value (luminance value) of each pixel of the difference image as a threshold value, and extracts pixels having a pixel value equal to or higher than a preset threshold value from the difference image. At this time, in the high-sensitivity mode, the threshold value is set to a small value so that the detection sensitivity of the difference region is higher than that in the low-sensitivity mode. Then, the difference area detecting means 24 detects the pixel area where the distance between the extracted pixels is a certain value or less as the difference area.

Also. When the determination result from the day / night determination means 21 is nighttime, the difference region detecting means 24 detects the difference region from the difference image only in the low sensitivity mode. Even at night, the procedure itself in the low-sensitivity mode is the same as in the daytime, so further description will be omitted.

After that, the difference area detecting means 24 outputs the information of the position of the difference area and the detection time zone of the difference area to the moving object detecting means 26 as the information of the detected difference area. Here, the position of the difference region represents the center position or the center of gravity position of the difference region. Further, the detection time zone of the difference region represents the daytime or nighttime division and the time information (time code) of the sensor image 90 that is the detection source of the difference region.

The light detecting means 25 detects the light of the aircraft from the sensor image 90 when the determination result from the day / night determining means 21 is at night. In the present embodiment, the light detecting means 25 refers to the area information storage means 22, in the nighttime approach path area A _N set in the sensor image 90, the pixel region becomes equal to or higher than the threshold luminance value is set in advance Is detected, and the position of the center of gravity or the center of the pixel area is detected as the light position of the aircraft.

After that, the light detecting means 25 outputs the light position and the detection time zone to the moving object detecting means 26 as the information of the detected aircraft lights.
The light detecting means 25 does not perform processing when the determination result from the day / night determining means 21 is daytime.

The moving body detecting means 26 extracts a difference region whose position changes over a preset number of frames or more from the difference region input from the difference region detecting means 24. That is, the moving body detecting means 26 extracts a moving body region that is continuously moving over several frames. As a result, the moving body detecting means 26 can distinguish between a moving body region that is continuously moving and a noise region other than that.

Next, the moving body detecting means 26 refers to the area information of the storage means 22, detects the moving body based on the position of the extracted difference region and the detection sensitivity, and generates the moving body detection information described later. Hereinafter, the moving object detecting means 26 will be described separately when the determination result from the day / night determining means 21 is daytime and nighttime.
In the following example, it is described that the aircraft is a moving body, but in reality, a moving body other than the aircraft may be detected.

<Detection of moving objects: in the daytime>
In the daytime, the moving object detecting means 26 refers to the area information of the storage means 22 and detects the moving object according to the area to which the extracted difference area belongs. In the present embodiment, the moving object detecting means 26 detects a moving object from each area (approach road area A _L , sky area S, runway area R, guide road area T) other than the mask area M.

Specifically point, moving object detection means 26, as shown in FIG. 10, for detecting a moving object Obj ₁ included in the approach path area A _L in the low-sensitivity mode. Similar to the approach path area A _L, moving object detection means 26 detects a moving object included in the sky area S in the low-sensitivity mode. On the other hand, as shown in FIG. 11, the moving object detecting means 26 _{does not detect the object Obj 2} (for example, a towing car) included in the mask area M in the low sensitivity mode.

Here, as shown in FIG. 12, when viewed from the sensor camera 10, _{a part of the aircraft Obj 3} (for example, a tail wing having a color or pattern different from that of the aircraft) may overlap the runway. In this case, since only the tail wing portion is likely to be detected in the inter-frame difference, it may be erroneously determined that the aircraft is located on the runway even though the aircraft is actually located on the guideway. On the other hand, as shown in FIG. 13, since the high-sensitivity mode has a higher detection sensitivity than the low-sensitivity mode, when the movement of only the tail portion is detected in the low-sensitivity mode, the aircraft Obj ₁ is in the high-sensitivity mode. It is highly possible that the entire movement can be detected and it can be correctly determined that the aircraft is located on the guideway.
Incidentally, 12 and 13, among the aircraft Obj _3, illustrating the position detected as the moving object by a chain line.

Therefore, the moving object detecting means 26 reduces erroneous detection of area information of the moving object by combining two detection modes, a low sensitivity mode and a high sensitivity mode. Specifically, the moving body detecting means 26 detects a moving body included in the runway area R in the low-sensitivity mode and not included in the guideway area T in the high-sensitivity mode as a moving body in the runway area R (). Not shown). _{On the other hand, the moving body detecting means 26 detects the moving body Obj 3} included in the runway area R in the low-sensitivity mode (FIG. 12) and included in the guideway area T in the high-sensitivity mode as a moving body in the guideway area T. (Fig. 13).

<Detection of moving objects: at night>
At night, the moving object detecting means 26 detects a moving object only in the low-sensitivity mode as in the daytime. Specifically, the moving object detecting means 26 detects a moving object included _{in each area (approach path area A L} , sky area S) other than the mask area M in the low sensitivity mode. On the other hand, the moving object detecting means 26 does not detect the moving object included in the mask area M in the low sensitivity mode.
At night, the moving object detecting means 26 does not use the low-sensitivity mode and the high-sensitivity mode in combination, so that the moving objects in the runway area R and the guideway area T are not detected.

After that, the moving object detecting means 26 generates the determined moving object detecting information, and outputs the generated moving object detecting information to the position identification / pan head control device 30 (information collecting means 31). This moving object detection information represents the position, belonging area (approach road area, runway area, etc.), size, and time information of the sensor image 90 of each moving object reflected in the sensor image 90.

At this time, the moving object detecting means 26 obtains the detection size of the moving object on the sensor camera 10 from the height and width of the rectangular region circumscribing the difference region. Then, the moving object detecting means 26 refers to the conversion information of the storage means 22 and converts the moving object detection size on the sensor camera 10 into the moving object size on the robot camera 40.

<Detection of distant aircraft: in the daytime>
If the aircraft is located far away and the speed is slow, such as when the aircraft starts to run at takeoff or when it enters the runway at landing, the detection accuracy of the difference region will be low. At this time, if the robot camera 40 takes a picture, there is a high possibility that the aircraft deviates from the shooting angle of view due to erroneous detection of the difference region. Therefore, instead of tracking control, it is necessary to perform fixed position control so that a distant aircraft does not deviate from the shooting angle of view.

In the daytime, the moving object detecting means 26 determines whether or not a distant aircraft is reflected in the sensor image 90. That is, the moving object detecting means 26 determines a moving object that _{is not included in the approach road area A L} in the low sensitivity mode and is included in the approach road area A _H in the high sensitivity mode as a distant aircraft.

<Detection of distant aircraft: at night>
At night, the aircraft body is not reflected in the sensor image 90, and it is unlikely that the aircraft can be detected. Therefore, moving object detection means 26, if the low-sensitivity mode motion is not detected, the light at night for access road in the area A _N detected by the light detection means 25, detected as a remote aircraft. As a result, the aircraft tracking imaging system 1 can control the fixed position even at night.

After that, the moving object detecting means 26 outputs the distant moving object detecting information indicating that the distant aircraft has been detected to the position identification / pan head control device 30 (tracking target determining means 35).

[Positioning / pan head control device configuration]
Returning to FIG. 4, the configuration of the position identification / pan head control device 30 will be described.
As shown in FIG. 4, the position identification / pan head control device 30 includes an information aggregation means 31, an identification information addition means 33, a tracking target determination means (tracking means) 35, and a pan head control means (robot camera control means). ) 37.

The information aggregating means 31 acquires moving object detection information from the moving object detecting device 20 (moving object detecting means 26) and aggregates the acquired moving object detection information. In the present embodiment, the information collecting means 31 determines whether or not the moving object detection information has been acquired from all three moving object detecting devices 20. Then, when the information collecting means 31 acquires the moving object detection information for three vehicles, for example, the moving object detection information at the same time is associated with the time code and output to the identification information adding means 33.

The identification information adding means 33 adds the identification information to the motion detection information from the information aggregating means 31. In the present embodiment, the identification information adding means 33 refers to the moving object detection information from the past to the present, determines a moving object that seems to be the same in the preceding and following frames based on the position, and adds the same identification information. Specifically, the identification information adding means 33 adds the same identification information to the moving object detection information when the moving object is moving at a constant speed and direction over several frames in order to suppress erroneous detection of a bird or the like.

Further, the identification information adding means 33 refers to the area to which the moving object detection information belongs, and if the moving object has passed through the aircraft specific area (approach road area and runway area) even once in the past, it has been identified as an aircraft. The specific information indicating that is added to the motion detection information. Further, the identification information adding means 33 calculates the speed of the moving body from the amount of change in the position of the moving body with respect to the moving body detection information to which the same identification information is given, and adds the calculated speed to the moving body detection information.
After that, the identification information adding means 33 outputs the moving object detection information to the tracking target determining means 35.

As a result, even if the aircraft deviates from the runway at the time of landing or the aircraft re-climbs due to a go-around, the same identification information is added and tracking can be continued. Furthermore, even if the aircraft moves to the shooting angle of view of another sensor camera 10, the tracking can be continued because the same identification information is added.

The tracking target determining means 35 calculates a score for each aircraft based on the motion detection information from the identification information adding means 33, and determines the tracking target aircraft based on the calculated score. Then, the tracking target determining means 35 outputs the position, speed, acceleration, and size of the aircraft to the pan head control means 37 as information on the tracking target.

<Determination of aircraft based on score>
Hereinafter, the determination of the aircraft based on the score will be specifically described.
In the present embodiment, the tracking target determining means 35 calculates a score for the moving object detection information to which the specific information is added. That is, the tracking target determining means 35 does not calculate the score from the moving object detection information to which the specific information is not added.

First, the tracking target determining means 35 adds a speed score according to the speed of the aircraft. That is, the tracking target determining means 35 refers to the speed of the moving object detection information, and adds a speed score according to the speed. This speed score is preset according to the speed of the aircraft (for example, 0 points to 500 points).

Further, the tracking target determining means 35 adds the sky area score (first area score) when the aircraft is detected in the sky. That is, the tracking target determining means 35 adds the sky area score when the area to which the motion detection information belongs is the sky area S (FIG. 5). This sky area score is preset with a value higher than the speed score (for example, 1000 points).

Further, the tracking target determining means 35 adds a specific area score (second area score) when the aircraft is detected on the approach road or the runway. That is, the tracking target determining means 35 adds the specific area score when the area to which the motion detection information belongs is the runway area R or the approach road area _{AL (FIG. 5).} This specific area score is preset with a value higher than the sky area score (for example, 2000 points).

Then, the tracking target determining means 35 determines the aircraft having the highest total value of the speed score and the area score as the tracking target. As shown in FIG. 14, consider a case _{where the aircraft Obj 4} slides on the runway and the aircraft Obj _{5 is located on the guideway.} In this case, the aircraft Obj ₄ in the runway because faster than aircraft Obj _5, speed score of aircraft Obj ₄ is higher than the aircraft Obj _5. Further, since the aircraft Obj ₄ is located on the runway and the aircraft Obj ₅ is located on the guideway, the area score of the _{aircraft Obj 4} is also higher than _{that of the aircraft Obj 5.} As a result, tracking target decision means 35, the sensor image 90 _C in FIG. 14, to determine the aircraft Obj ₄ as a tracking target.

In addition, when the tracking target determination means 35 has only one moving object detection information to which specific information is added (that is, when only one aircraft is detected), the tracking target determining means 35 does not calculate the score and uses the aircraft. It may be determined immediately as a tracking target.

<Other processing in the tracking target determination means 35>
Here, the tracking target determination means 35 may perform the following processing in addition to determining the aircraft based on the score.
For example, the tracking target determining means 35 determines that the fixed position control is performed when the aircraft to be tracked does not exist and the distant moving object detection information is input from the moving object detecting device 20 (moving object detecting means 26). .. In this case, the tracking target determining means 35 outputs a fixed position control command to the pan head control means 37. The A position control, for example, at a pre-set photographing direction and photographing field angle as fall within the shot image of the approach path area A _H of FIG. 6, is to taken the robot camera 40.

Further, when the detection of the aircraft is interrupted, such as when the aircraft is hidden behind a building or another aircraft, the tracking target determination means 35 uses the tracking speed of the aircraft based on the Bayesian estimation for a preset re-detection time. Find the estimated position and wait for the aircraft to be rediscovered at that estimated position. Then, the tracking target determining means 35 continues tracking when the aircraft is rediscovered at the estimated position.

Further, the tracking target determining means 35 outputs an estimated position control command to the pan head control means 37 when the aircraft is not rediscovered even after the rediscovery time has elapsed. This estimated position control is to have the robot camera 40 take a picture with a wide shooting angle of view centering on the estimated position of the aircraft at the present time in order to increase the possibility of taking an accident image.

Further, the tracking target determining means 35 outputs the standby position control command to the pan head control means 37 when the aircraft is not re-detected even after the preset standby time elapses after the estimated position control command is output. This standby position control means, for example, causing the robot camera 40 to take a picture in a shooting direction and a shooting angle of view set in advance so as to capture a place having a high accident rate such as an approach road area in the shot image.

Returning to FIG. 4, the description of the position identification / pan head control device 30 will be continued.
The pan head control means 37 controls the robot camera 40 (head 43) based on the position of the tracking target from the tracking target determining means 35. In the present embodiment, the pan head control means 37 performs general feedback control so that the robot camera 40 can track and photograph the aircraft to be tracked.

Here, the pan head control means 37 controls the zoom of the robot camera 40 according to the size of the aircraft input from the tracking target determination means 35. Specifically, the pan head control means 37 controls to zoom out as the size of the aircraft increases and zoom in as the size of the aircraft decreases. Further, when the speed of the aircraft exceeds the maximum operating speed of the robot camera 40, the pan head control means 37 may perform control to zoom out.

Further, the pan head control means 37 controls the fixed position of the robot camera 40 when a fixed position control command is input from the tracking target determining means 35. Further, the pan head control means 37 controls the estimated position of the robot camera 40 when the estimated position control command is input from the tracking target determining means 35. Further, the pan head control means 37 controls the standby position of the robot camera 40 when a standby position control command is input from the tracking target determination means 35.

From the above, as shown in FIG. 15, the aircraft tracking and photographing system 1 can track the aircraft during takeoff and landing shown in the sensor image 90 and photograph it with the robot camera 40. Further, as shown in FIG. 16, the aircraft tracking imaging system 1 can track the aircraft by the tracking target determining means 35 even when _{the same aircraft is shown in the sensor images 90 C} and 90 _R.
In addition, in FIGS. 15 and 16, the subscript t represents the time, and the region detected as an aircraft is shown by a dashed line.

[Operation of aircraft tracking shooting system]
The operation of the aircraft tracking imaging system 1 will be described with reference to FIG.
In FIG. 17, only one sensor camera 10 and one motion detecting device 20 are shown in order to make the drawings easier to see. Further, since the processing of each sensor camera 10 and each moving object detection device 20 is the same, duplicate description will be omitted.
Further, in FIG. 17, it is assumed that the storage means 22 stores the area information and the conversion information, and the day / night determination means 21 determines that it is daytime.

The sensor camera 10 generates a sensor image of an aircraft, and outputs the generated sensor image 90 to the motion detection device 20 (step S1).
The difference image generation means 23 generates a difference image from the sensor image. That is, the difference image generation means 23 obtains the brightness difference for each pixel between the frames before and after the sensor image, and generates a difference image representing the obtained brightness difference (step S2).

The difference region detecting means 24 detects the difference region from the difference image generated in step S2 in each of the low sensitivity mode and the high sensitivity mode. That is, the difference region detecting means 24 generates a difference region by grouping close pixels for each pixel whose luminance difference is equal to or greater than the threshold value (step S3).

The moving object detecting means 26 extracts a difference region whose position changes over a predetermined number of frames or more from the difference region generated in step S3. Then, the moving body detecting means 26 refers to the area information of the storage means 22, detects the moving body based on the position of the extracted difference region and the detection sensitivity, and generates the moving body detection information. For example, the moving object detecting means 26 detects a moving object from each area other than the mask area M, and does not detect a moving object from the mask area M (step S4).
The moving object detecting means 26 outputs the moving object detecting information generated in step S4 to the information collecting means 31 (step S5).

The information aggregating means 31 determines whether or not the motion detection information has been acquired from all three motion detection devices 20 (step S6), and waits for processing until all the motion detection information is acquired (No in step S6). ..

When all the moving object detection information is acquired (Yes in step S6), the identification information adding means 33 adds the identification information to the moving object detection information acquired in step S6. Here, the identification information adding means 33 refers to the moving body detection information from the past to the present, determines a moving body that seems to be the same in the preceding and following frames based on the detection position of the moving body, and the moving body detection information corresponding to the moving body. The same identification information is added to. Further, since the identification information adding means 33 can know the trajectory of the moving object by referring to the moving object detection information, if the moving object has passed through the aircraft specific area even once in the past, the identification information indicating that the moving object has been identified as an aircraft can be obtained. It is added to the motion detection information. Further, the identification information adding means 33 calculates the velocity of the moving object from the amount of change in the position of the moving object with respect to the moving object detection information to which the same identification information is given, and adds the calculated velocity to the moving object detection information (step S7). ..

The tracking target determining means 35 calculates a score for each moving object specified as an aircraft based on the area and speed to which the moving object detection information belongs. Specifically, the tracking target determining means 35 adds a velocity score according to the velocity of the moving object. Further, the tracking target determining means 35 adds or subtracts an area score according to the area to which the moving object belongs. Then, the tracking target determining means 35 determines the moving object having the highest total value of the speed score and the area score as the tracking target aircraft (step S8).

The pan head control means 37 generates a control signal for the robot camera 40 based on the position of the aircraft determined in step S8. Here, the pan head control means 37 performs general feedback control so that the robot camera 40 can track and photograph the aircraft to be tracked (step S9).

The pan head control means 37 outputs the control signal generated in step S9 to the robot camera 40 (step S10).
The robot camera 40 tracks and photographs the aircraft based on the control signal (step S11).

[Action / Effect]
As described above, the aircraft tracking imaging system 1 according to the embodiment of the present invention detects the moving object by the difference between frames and determines the tracking target by the area information, so that the detection accuracy is rapidly improved at night like template matching. It does not drop and can accurately track the aircraft day and night.

Further, since the aircraft tracking imaging system 1 determines the tracking target based on the past trajectory of the moving object, it is possible to accurately track the moving object deviating from an unexpected course.
Further, since the aircraft tracking imaging system 1 uses the sensor image 90, it is possible to track an aircraft that does not emit an ADS-B signal.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and includes design changes and the like within a range that does not deviate from the gist of the present invention.
In the above-described embodiment, it has been described that the number of sensor cameras is three, but the present invention is not limited thereto. For example, one ultra-wide-angle camera may be used as a sensor camera, or two or more sensor cameras may be used.

In the above-described embodiment, it has been described that each sensor camera is provided with a motion detection device, but the present invention is not limited thereto. For example, the sensor image of each sensor camera may be image-analyzed by one motion detection device.
Further, in the above-described embodiment, the motion detection device and the position identification / pan head control device have been described as having separate configurations, but the present invention integrates the motion detection device and the position identification / pan head control device. May be good.

In the above-described embodiment, it has been described that the moving body is an aircraft, but the present invention is not limited thereto. That is, in the present invention, there is no particular limitation as long as the moving body moves. In particular, the moving body is preferably one that moves in a certain direction, and includes, for example, a ship, a railroad, an automobile, and the like. Further, the present invention can also be applied to a sports image in which an athletics athlete or the like is photographed as a moving object.

In the above-described embodiment, the robot camera is used to track the aircraft, but the present invention is not limited thereto. For example, the present invention may be applied to a method of tracking an aircraft on an image.

In the above-described embodiment, a specific score (number of points) has been illustrated, but the present invention is not limited thereto. For example, the score for each area may be set manually according to the environment of each airport.

In the above-described embodiment, the moving object tracking device has been described as independent hardware, but the present invention is not limited thereto. For example, the present invention can also be realized by a moving object tracking program in which hardware resources such as a CPU, a memory, and a hard disk included in a computer are cooperatively operated as the above-mentioned moving object tracking device. These programs may be distributed via a communication line, or may be written and distributed on a recording medium such as a CD-ROM or a flash memory.

1 Aircraft tracking imaging system 10 _{, 10 R} , 10 _C , 10 _L Sensor camera 20 _{, 20 R} , 20 _C , 20 _L Motion detection device 21 Day / night determination means 22 Storage means 23 Difference image generation means 24 Difference area detection means 25 Motion detection Means 26 Light detection means 30 Position identification / pan head control device 31 Information aggregation means 33 Identification information addition means 35 Tracking target determination means (tracking means)
37 Head control means (robot camera control means)
40 robot camera

Claims (8)

It is a moving object tracking device that tracks the moving object to be tracked by using a sensor image of the moving object taken by a sensor camera.
A difference image generation means for generating a difference image from the sensor image, and
In each of the low-sensitivity mode and the high-sensitivity mode in which the detection sensitivity is higher than that of the low-sensitivity mode, the difference region detecting means for detecting the difference region from the difference image and the difference region detecting means.
In the low-sensitivity mode, a tracking target specific area for identifying the moving object as the tracking target in the sensor image, a mask area through which the tracking target does not pass, and an error in the detection area of the tracking target in the high-sensitivity mode. A storage means for storing area information in advance, which represents an erroneous detection prevention area for preventing
The difference region whose position changes over a preset number of frames is extracted, the area information is referred to, the moving body is detected by the extracted position of the difference region and the detection sensitivity, and the position of the detected moving body is detected. And the moving object detection means that generates the moving object detection information indicating the area to which it belongs,
A tracking means that calculates a score for each moving object based on the moving object detection information and determines the tracking target based on the calculated score.
A moving object tracking device characterized by being equipped with. The tracking target is an aircraft,
The storage means stores the area information representing the approach road area and the runway area in the sensor image as the tracking target specific area, and the guide road area in the sensor image as the false detection prevention area.
It is determined whether or not the moving body is the same based on the moving body detection information, the same identification information is added to the same moving body, and the moving body has passed the approach road area or the runway area in the past. In the case, it is further provided with identification information addition means for identifying the moving object as an aircraft.
The moving object tracking device according to claim 1, wherein the tracking means calculates a score of a moving object identified as an aircraft by the identification information adding means. The storage means stores the area information in which the sky area in the sensor image is further set in the low sensitivity mode.
The moving object detecting means
In the low sensitivity mode, a moving object included in the approach road area or the sky area is detected.
A moving body included in the runway area in the low-sensitivity mode and not included in the guideway area in the high-sensitivity mode is detected as a moving body in the runway area.
A moving body included in the runway area in the low-sensitivity mode and included in the guidance path area in the high-sensitivity mode is detected as a moving body in the guidance path area.
The moving object tracking device according to claim 2, wherein the moving object included in the mask area is not detected in the low sensitivity mode. The identification information adding means calculates the velocity of the moving body based on the amount of change in the position of the moving body with respect to the moving body detection information to which the same identification information is given.
The tracking means
A preset speed score is added according to the moving object, and the speed score is added.
When the area to which the moving object belongs is the sky area, a first area score higher than the speed score is added.
When the area to which the moving object belongs is the approach road area or the runway area, a second area score higher than the first area score is added.
The moving object tracking device according to claim 3, wherein the moving object having the highest total value of the speed score and the area score is determined as the tracking target. Further equipped with a day / night determination means for determining whether or not it is nighttime based on the time of day,
The difference region detecting means detects the difference region only in the low sensitivity mode when it is determined to be at night.
The moving object detecting means is characterized in that, when it is determined at night, the moving object included in the approach road area is detected in the low sensitivity mode, and the moving object included in the mask area is not detected in the low sensitivity mode. The moving object tracking device according to any one of claims 2 to 4. The invention according to any one of claims 2 to 5, further comprising a robot camera control means for controlling the robot camera that captures the moving object based on the position of the tracking target output by the tracking means. The described moving object tracking device. The storage means stores the area information in which the approach road area and the nighttime approach road area are further set in the high sensitivity mode.
Further provided with a light detecting means for detecting a pixel area having a brightness value equal to or higher than a preset brightness value as a light of the aircraft in the night approach road area in the sensor image when it is determined to be at night.
The moving object detecting means
When it is determined to be daytime, a moving object that is not included in the approach road area in the low sensitivity mode and is included in the approach road area in the high sensitivity mode is further detected as a distant moving body.
If the moving object is not detected in the low sensitivity mode when it is determined to be at night, the light of the aircraft detected by the light detecting means is further detected as the distant moving object.
The tracking means determines that the fixed position control is performed when the tracking target does not exist and the distant moving object is detected.
The robot camera control means is characterized in that, when the tracking means determines to perform the fixed position control, the robot camera controls the robot camera so as to shoot in a preset shooting direction and shooting angle of view. 6. The moving object tracking device according to 6. A moving object tracking program for causing a computer to function as the moving object tracking device according to any one of claims 1 to 7.

Images