JP7053057B2 - Mobile tracking device and mobile tracking method - Google Patents

Description

The present invention relates to a moving body tracking device and a moving body tracking method for tracking a moving body based on an object image in a captured image captured by an imaging unit.

In recent years, for example, many methods for detecting objects such as people and automobiles using a deep neural network (DNN) have been studied. By evaluating the captured image of the camera with an image processing device equipped with artificial intelligence that performed deep learning using images identified for each type such as private cars, buses, trucks, people, and bicycles in advance, it is included in the image. It is possible to estimate the type of each object and its position in the image. In the existing identification by image processing, it was necessary to create an algorithm for each type, but by using DNN, it is possible to more easily detect a plurality of types of objects. Further, a technique of using DNN to track not only the position of an object in a captured image but also the subsequent movement of the object as described above is also being researched. For example, a tracking device using a particle filter is being studied. Etc. (Patent Document 1 etc.).

Further, a vector focus method is known as a method of recognizing a predetermined object from an image captured by a camera. This vector focal method creates standard body data based on a standard model of an object to be recognized, and recognizes an object that matches the standard body data from captured images (Patent Documents 2, 3, and 4). ..

Japanese Unexamined Patent Publication No. 2019-185210 Japanese Patent No. 3406587 Japanese Patent No. 3390426 Japanese Patent No. 340657577

When object detection and tracking are performed using DNN, since the amount of calculation is large, a calculation processing device capable of high-speed calculation processing is required in practice. However, when it is necessary to operate on a small and inexpensive system such as traffic volume measurement, it is possible to use arithmetic processing due to the need for low power consumption for long-term stable operation and various economic circumstances. The equipment is limited. Therefore, as a practical matter, it is desired to provide a moving object tracking device that can recognize various objects from captured images with high accuracy, has low power consumption, has a power supply method, and has almost no heat dissipation problem.

In addition, when there are various types of tracking targets such as automobiles and people, the regularity and speed of movement are different, and in order to accurately track moving objects over various moving objects with the same algorithm, It is necessary to increase the number of times the captured image is processed per second (FPS: Frames Per Second). For example, when a moving object is moving irregularly instead of a constant velocity linear motion, the feature points obtained from a captured image are changing, or objects having similar feature points intersect with each other. The lower the frequency of image acquisition, the lower the degree of matching of feature points and the degree of matching of positions obtained from captured images, and the lower the tracking accuracy. Therefore, it is necessary to increase the FPS of the captured image in order to accurately track the moving object identified for each type using DNN, but there is a limit in the arithmetic processing device with limited performance as described above. be.

Further, in order to trace a moving body with high accuracy by using the vector focal method, it is necessary to set standard body data corresponding to an object image that changes variously and individually over time. Moreover, it was difficult to anticipate and set standard data corresponding to individual changes in advance.

The present invention has been made in view of the above circumstances, and is a moving body tracking device and a moving body capable of accurately tracking a predetermined moving body from a captured image while being able to operate with low power consumption. The purpose is to provide a tracking method.

The present inventors operate an object identification unit that can be executed at high speed intermittently, for example, by about 1/10 to operate it with low power consumption to recognize an object, or simple hardware with low power consumption but low speed. The object is recognized using the object identification unit of the execution, and the object tracking is performed by the vector focus method that can be executed with low power consumption. Completed the device and method.

That is, the mobile tracking device according to the present invention is
It is a mobile object tracking device that tracks a moving object based on the object image in the captured image captured by the imaging unit.
An image distribution unit that distributes the captured image into two systems, an output that delays the captured image in a FIFO type according to the image processing speed and an output that thins the captured image without delay, and outputs the captured image.
An object identification unit that performs processing to detect an object to be tracked from the captured images input from the image distribution unit by object detection by artificial intelligence or the vector focus method.
The position and range of the recognition result of the object in the object identification unit are treated as a standard model of image processing by the vector focus method, and the moving object to be tracked that matches the standard model is selected from the captured images input from the image distribution unit. A moving object tracking unit that performs tracking processing, and
It is equipped with a match determination unit that integrates the results of the object identification unit and the moving object tracking unit.
The image distribution unit periodically outputs captured images to the object identification unit at intervals of a fixed number of frames, and outputs the object identification unit to the moving object tracking unit and the moving object tracking unit. The configuration is such that the captured image is output each time so that the output is at the same timing.
The object identification unit is configured to perform intermittent operations by high-speed arithmetic processing or to perform low-speed arithmetic processing.
When the same moving object exists from the outputs from the object identification unit and the moving object tracking unit, the matching determination unit detects the moving object tracked by the moving object tracking unit by the object identification unit. It is configured to perform processing to match with.

One form of the mobile tracking method according to the present invention is
It is a moving body tracking method that tracks a moving body based on an object image in the captured image captured by the imaging unit.
An image distribution step in which the image distribution unit distributes the captured image into two systems, an output that delays the captured image in a FIFO type according to the image processing speed and an output that thins out without delay, and outputs the captured image to each processing unit.
In the object identification unit, an object identification step that performs a process of detecting an object to be tracked from the captured images of the frame input in the image distribution step by artificial intelligence, and an object identification step.
A mobile tracking step that performs processing to track a moving object to be tracked from the captured images of the frame input in the image distribution step by image processing by the vector focusing method in the moving object tracking unit.
The match determination unit includes a match determination step for performing a process for matching a moving object to be tracked by the moving object tracking step with an object detected by the object identification step.
The image distribution step periodically outputs captured images to the object identification unit at intervals of a fixed number of frames, and outputs the object identification unit to the moving object tracking unit and the moving object tracking unit. A process is performed to output the captured image each time with a delay so that the output is at the same timing.
In the object identification step, a trained model is constructed by machine-learning image data of various shapes of an actual object to be tracked in the artificial intelligence, and the object is periodically input by the image distribution step. A process is performed to accurately detect the area of a predetermined object, the position of the object, and the type of the object in the captured image of the frame to be performed.
In the moving body tracking step, standard body data based on a standard model of the moving body to be tracked is set by image processing by the vector focusing method, and a representative point is a moving body that matches the standard body data in the captured image. A process of tracking the representative points of the same moving body from the captured images of the frames that are arranged and input each time by the image distribution step is performed.
The match determination step is
Regarding the captured images of the same frame input to the object identification step and the moving object tracking step, among the objects detected in the object identification step, the same moving object as the moving object being tracked in the moving object tracking step exists. Make a match judgment and decide whether or not to do it.
When it is determined in the matching determination that the same moving object exists, in the processing of the captured image of the next frame one after the current frame used in the matching determination in the moving object tracking step, the object identification step is performed. The object detected in is used as a standard model, and it is processed.
When it is determined in the matching determination that the same moving object does not exist, in the moving object tracking step, in the processing of the captured image of the next frame, the object detected only in the object identification step is a new moving object. Is registered as, and processing is performed using the object as a standard model.

Another embodiment of the mobile tracking method according to the present invention is
It is a moving body tracking method that tracks a moving body based on an object image in the captured image captured by the imaging unit.
An image distribution step of acquiring and outputting the captured image in the image distribution unit,
In the object identification unit, an object identification step that performs a process of detecting an object to be tracked from the captured images input in the image distribution step by object detection by the vector focus method, and an object identification step.
The moving object tracking unit includes a moving object tracking step that performs a process of tracking a moving object to be tracked from the captured images input in the image distribution step by image processing by the vector focus method.
In the moving object tracking step, standard body data based on a standard model of the moving body to be tracked is set by image processing by the vector focusing method, and a representative point is set to a moving body that matches the standard body data in the captured image. It is a step of tracking the representative points of the same moving body from the captured images that are arranged and then input each time in the order of passage of time in the image distribution step, and is one step before the captured image as the standard model. A moving object being tracked in the captured image or an object detected by the object identification step is used.

From the above, according to the present invention, when tracking a moving body using the vector focal method, it is possible to prepare standard body data corresponding to an object image that changes variously and individually over time, and therefore, It is possible to track the target moving object at high speed and with high accuracy. In addition, by reducing the processing frequency of the object identification unit, which has a high computational load, and tracking the moving object with high frequency using the vector focus method, multiple types of movement can be performed even in a low power consumption and low performance operating environment. Body-classified tracking can be performed.

It is a block diagram which shows the structure of the moving body tracking apparatus of Embodiment 1. FIG. It is a schematic diagram for demonstrating the distribution output method of the captured image by an image distribution part. It is a schematic diagram for demonstrating image processing by artificial intelligence (AI) in an object identification part. It is a schematic diagram for demonstrating the image processing by the vector focus method in the moving body tracking part. It is a schematic diagram for demonstrating the case where the range of a circular Fourier transform is set to a predetermined voting range at the time of image processing by the vector focal method. It is a schematic diagram for demonstrating the case where the range of a circular Fourier transform is set to a predetermined voting range when there is a tracking history of a moving object at the time of image processing by the vector focal method. It is a schematic diagram for demonstrating how to use a standard model in a vector focal method. It is a schematic diagram which shows the flow of the production instruction of the standard model to the moving body tracking part by a match determination part. It is a schematic diagram which shows each step of the moving body tracking method by the moving body tracking apparatus of Embodiment 1. FIG. It is a flowchart which shows the procedure of the moving body tracking step in Embodiment 1. It is a block diagram which shows the structure of the moving body tracking apparatus of Embodiment 2.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
(Embodiment 1)
The moving body tracking device 1 of the first embodiment is a device that tracks a moving body from an object image in the captured image 20 captured by the camera 10. In the first embodiment, the objects of image processing and determination from the captured image 20 are images such as an object image and a moving body image, but in the following description, they are simply referred to as "object" and "moving body". In some cases. Further, in the first embodiment, the object or the moving body is assumed to be a person, but the present invention is not limited to the person, but the present invention is not limited to the person, but also animals other than the person, vehicles such as private cars, buses, trucks, bicycles, etc. An object that can be identified by image processing from the captured image of the above can be an object to be tracked or a moving object.

As shown in FIG. 1, the mobile object tracking device 1 of the first embodiment is connected to the camera 10, and its main configuration is an image distribution unit 2, an object identification unit 3, a mobile object tracking unit 5, and a matching determination unit. Has 6. The camera 10 is an imaging unit for imaging a predetermined area in which a moving object to be tracked exists, and is composed of, for example, a CMOS camera or the like. The camera 10 is installed so that a predetermined area in which the moving object to be tracked can move can be imaged. For example, when tracking the movement of a passerby in the aisle in front of the store, the movement of a person in the store, etc., the camera 10 should be installed on the ceiling or the wall surface so that the aisle and a predetermined area in the store can be imaged from above. Can be done. The camera 10 may be communicated and connected to the mobile tracking device 1 by wire or wirelessly, or may be communicated and connected by a network such as a LAN. Further, the camera 10 is not connected to the moving body tracking device 1, and the image pickup data captured by the camera 10 is stored in a storage device, and the moving body tracking device 1 is configured to read the image pickup data from this storage device. You may be.

The moving object tracking device 1 detects a person as an object image by the object identification unit 3 in the image 20 captured by the camera 10, and the object detected by the object identification unit 3 is used as a moving object by the moving object tracking unit 5. It is configured to track the movement of the object. The moving object tracking unit 5 periodically detects an object image of a person from the captured image 20 by the object identification unit 3, and the moving object tracking unit 5 actually tracks the object based on the object information in the object identification unit 3. It is configured to correct the tracking information of the moving object.

The image captured by the camera 10 is input to the object identification unit 3 and the moving object tracking unit 5 from the image distribution unit 2 in frame units. In the object identification unit 3 and the moving object tracking unit 5, the processing frequency (FPS: Frames Per Second) per second of the captured image 20 is lower in the object identification unit 3, for example, moving with respect to the FPS of the object identification unit 3. The FPS of the body tracking unit 5 is constructed to be about 5 times larger. In the moving object tracking device 1, for example, the moving object tracking unit 5 is processed mainly by using a CPU (Central Processing Unit), and the object identification unit 3 is mainly processed by using a GPU (Graphics Processing Unit), which are separate. The moving object tracking system can be configured more efficiently by dividing it into arithmetic units.

The image distribution unit 2 delays the captured image captured by the camera 10 in a FIFO (First-In, First-Out) type according to the image processing speed of the processing unit (3 or 5), and outputs thinning without delay. It is distributed to the two systems of and outputs the captured image. The image distribution unit 2 temporarily records the captured images 20 over a plurality of frames in the order of passage of time from the data captured by the camera 10, and moves the captured images 20 to the object identification unit 3 and the object identification unit 3 at predetermined time intervals in the order of passage of time. It is provided with a configuration for outputting to the body tracking unit 5. For example, in the object identification unit 3 and the moving object tracking unit 5, the moving object tracking unit 5 is faster than the object identification unit 3 in terms of image processing capacity (FPS: Frames Per Second). The image processing time for the captured image 20 of one frame is long in the object identification unit 3 and short in the moving object tracking unit 5. The image distribution unit 2 acquires and records the captured image 20 from the image captured data of the camera 10 at a time interval corresponding to the short image processing time in the moving object tracking unit 5, and at least the long image processing time in the object identification unit 3. The captured images 20 for the number of frames that can be acquired in the image 20 are temporarily recorded.

Then, as the output of the captured image 20, the image distribution unit 2 outputs the captured image 20 to the object identification unit 3 at intervals of a fixed number of frames in the order of passage of time, and outputs the captured image 20 to the moving object tracking unit 5 in the order of passage of time. The captured image 20 is output for each frame. At this time, the image distribution unit 2 has the object identification unit 3 and the moving object tracking unit 5 for the same captured image 20 as the captured image 20 output to the object identification unit 3 as the output of the captured image 20 to the moving object tracking unit 5. The output to the moving object tracking unit 5 is delayed so that the end timing of the image processing becomes the same timing. As a result, the image processing results of the object identification unit 3 and the moving object tracking unit 5 can be obtained at the same timing for the same captured image 20. By reducing only the FPS of the object identification unit 3 having a high calculation load, it is possible to reduce the amount of calculation per hour and reduce the power consumption while maintaining the FPS as the moving object tracking device 1.

To explain an example based on FIG. 2, when the object identification unit 3 performs image processing operation at 1 FPS (1 frame per second) and the moving object tracking unit 5 performs image processing operation at 5 FPS (5 frames per second), the image distribution unit In 2, the captured image 20 is acquired from the captured data of the camera 10 at a time interval of 5 FPS, and temporarily recorded over 5 frames. That is, the image distribution unit 2 acquires one captured image 20 from the captured data of the camera 10 at 1/5 second intervals and temporarily records it over time. In the image distribution unit 2, as the number of recorded images 20 per unit time increases, the tracking process that goes back to the past becomes possible, but at least the captured image portion during the image processing operation of the object identification unit 3, that is, the above-mentioned In this example, it is preferable to record the captured images 20 for 5 frames per second in the order of passage of time. Further, it is preferable that the operating speed of the object identification unit 3 and the operating speed of the moving object tracking unit 5 are adjusted in advance according to the performance of the operating environment and the properties of the moving object to be tracked.

Then, the image distribution unit 2 obtains and records the frame No. 1 from the image pickup data of the camera 10. The captured image 20 of 1 is output to the object identification unit 3 at the time of frame recording. The moving object tracking unit 5 has an image processing capacity of 5 FPS, but the object identification unit 3 has an image processing capacity of 1 FPS, which requires five times as long as the image processing time of the moving object tracking unit 5. Therefore, the object identification unit 3 has the frame NO. Although the image processing of the captured image 20 of 1 is completed, the image distribution unit 2 is set to the frame NO. It takes until the time when the captured image 20 of 5 is acquired and recorded. Therefore, the image distribution unit 2 has the frame NO. At the time when the captured image 20 of No. 5 was acquired and recorded, this frame No. The same frame number as the captured image 20 of the frame 4 frames before 5, that is, the captured image 20 output to the object identification unit 3. The captured image 20 of 1 is output to the mobile tracking unit 5. As a result, the same frame No. With respect to the captured image 20 of 1, the output of the image processing results of the object identification unit 3 and the moving object tracking unit 5 is obtained at the same timing.

After that, similarly, the image distribution unit 2 sends the moving body tracking unit 5 to the frame NO. Frame No. following 1 2. Frame No. The captured image 20 is output for each frame as in 3, ..., And the frame NO. To the object identification unit 3 is adjusted to the processing capacity (1 FPS) of the object identification unit 3. Next to 1, the frame number. 6. Next is the frame number. As in 11, ..., the captured image 20 of the frame is output periodically at intervals of 5 frames. As a result, in the same manner as above, NO. 6, NO. For the captured image 20 of the same frame such as 11, the image processing results of the object identification unit 3 and the moving object tracking unit 5 are obtained at the same timing.

In this way, by obtaining the image processing results of the object identification unit 3 and the moving object tracking unit 5 for the captured image 20 of the same frame at the same timing, the image processing result of the moving object tracking unit 5 is obtained at this timing. By verifying with the image processing result of the identification unit 3, the tracking information of the moving object tracking unit 5 is periodically (time interval of the long image processing time of the object identification unit 3) by the detection information of the object identification unit 3. ), And as a result, the tracking accuracy by the moving object tracking unit 5 can be improved.

The object identification unit 3 includes a feature extraction unit 31 and an object detection unit 32, and the feature extraction unit 31 and / or the object detection unit 32 are configured by artificial intelligence. With reference to FIG. 3, the object identification unit 3 detects an object 21 that is a moving object to be tracked by artificial intelligence constructed by the trained model from the captured images 20 of the frame input from the image distribution unit 2. The image processing result is output to the match determination unit 6.

The feature extraction unit 31 is a functional unit that performs a process of extracting features from the captured image 20, and obtains, for example, image feature quantities such as brightness and color of each pixel in the captured image 20. The feature extraction unit 31 can be configured by a program according to an algorithm of image analysis processing for obtaining the feature amount, and has constructed a trained model in which the data for obtaining the feature amount is machine-learned. It may be composed of artificial intelligence.

The object detection unit 32 is a functional unit that performs processing for obtaining the type of object, the area of the object, the position of the object, the center point of the object, and the like from the data of the feature amount extracted by the feature extraction unit 31, and is in the captured image 20. Detects a moving object to be tracked with high accuracy. The object detection unit 32 can be configured by artificial intelligence trained to detect a predetermined object image 21 captured from the captured image 20, but the predetermined object image captured from the captured image 20. It may be composed of a program according to an algorithm for detecting 21.

The artificial intelligence that constitutes the object detection unit 32 is a large amount including various shapes of the object to be detected (for example, other animals such as a person, a dog, a cat, a private car, a bus, a truck, a bicycle, etc.). It is composed of a trained model in which image data is machine-learned. For this artificial intelligence, for example, a convolutional neural network (CNN) is used. Neural networks learn and input how to distinguish between correct object images and other images (background images, etc.) by machine learning using a large amount of training image data (teacher data) consisting of image data of correct objects. A trained neural network model constructed so as to accurately output the type of the object, the area of the object, the position of the object, the center point of the object, and the like from the captured image 20 is used. Here, the area of the object refers to a candidate area in which the object 21 exists in the captured image 20, for example, a rectangular frame that surrounds the object image 21 shown on the captured image 20 or is attached to the object image 21. It can be represented by a Bounding Box B, which is a line. The center point m of the object image 21 can be represented by the intersection of two diagonal lines of the bounding box B.

It is preferable to adopt a method that can be calculated at a higher speed, such as providing a neural network for the appearance of the area of the object, the position of the object, the center point of the object, and the like. Further, the object detection unit 32 includes an algorithm for excluding detected objects that have been detected in duplicate and an algorithm for estimating the position coordinates of the objects, and includes the same object from one captured image 20. Is not output in duplicate. For example, an algorithm called Non-Maximum-Suppression (NMS) is provided, and by using this NMS to obtain the IoU (Intersection over Union) value for multiple borders on the same object, the IoU value is large and the overlapping region. Duplicate output can be suppressed by excluding those with a large proportion of.

The above-mentioned neural network and algorithm are examples of the algorithm used for object identification, and the object identification unit 3 uses the input captured image 20 to determine the type of object and the captured image of the moving object to be tracked. It suffices if it is composed of a neural network or an algorithm constructed so as to obtain the area of the object in 20, the position coordinates of the object in the captured image 20, the center point of the object in the captured image 20, the evaluation value, and the like. It is possible to select a method that can be calculated quickly and accurately. When a neural network is used for both the feature extraction unit 31 and the object detection unit 32, a plurality of neural networks are combined to form a complex neural network in which the feature extraction unit 31 and the object detection unit 32 are integrated. May be good.

The moving body tracking unit 5 includes an image processing unit 51 and a tracking unit 52 by the vector focusing method, and performs a process of tracking a predetermined moving body 21 in the captured image 20 of the frame input from the image distribution unit 2. It is provided with a configuration in which the image processing result is output to the match determination unit 6. Since the object identification unit 3 is heavier in arithmetic processing than the moving object tracking unit 5 and requires many times the execution time of the moving object tracking unit 5 in the arithmetic processing, in the first embodiment, the moving object tracking unit 5 is the object identification unit. It is also a functional component for making it possible to practically handle the object detection algorithm in No. 3 as moving object tracking.

The moving body tracking unit 5 can track the moving body 21 from the captured image 20 at high speed by using the image processing technique by the vector focusing method. That is, in the moving object tracking unit 5, the image processing unit 51 by the vector focal method arranges a standard model of the moving object to be tracked on the image, and with respect to the center point in this standard model and the contour line of the standard model. Conforms with the standard body data in the captured image 20 based on the standard body data by the direction (angle information of the normal vector) and the length (coordinate information of the position of the normal vector) between the positions of the normal vector. A representative point is attached to the object image (moving body) to be used. The tracking unit 52 tracks the moving object to be tracked at high speed by tracking the representative points over time from the captured images 20 in the order of passage of time. For example, the tracking unit 52 tracks the combination in which the representative points exist within a certain range in the front and rear frames and the closest distance is regarded as the same moving body. Here, the fixed range to be regarded as the same moving body may be set through verification, experiment, etc. according to the type of the moving body, or may be set as the moving speed from the difference in the position coordinates in each frame of the moving body. It may be set by estimating the position of the current frame on the premise that the linear motion is constant velocity in the direction.

With reference to FIG. 4, as a procedure for tracking a moving object using the vector focus method, first, a standard model S of an object image to be a moving object 21 to be tracked is arranged on the captured image 20, and the captured image is obtained. The brightness gradient and the gradient direction of each pixel are calculated by performing a Fourier transform (circular Fourier transform) on each of the 20 pixel points into a circular shape centered on the processing point. The luminance gradient appears along the characteristics of the object image (standard model S), and pixels having a luminance gradient of a certain value or more are regarded as effective pixels, and an aggregate of these effective pixels is a region reflecting the standard model S of the object image. The gradient direction is a vector (normal vector) b in the normal direction with respect to the feature boundary of the object image. For this gradient direction, for example, each direction obtained by dividing 360 degrees into 16 directions may be selected. Then, for the position of the brightness gradient of each pixel (the position corresponding to the contour of the standard model S), the angle (angle information) and the length (coordinate information) in the gradient direction are obtained from the center point p of the standard model S. , This is stored as standard data according to the standard model S.

Next, the circular Fourier transform is performed on the actual captured image 20 captured by the camera 10 to calculate the brightness gradient and the gradient direction, and a vote is performed for the center point p based on the standard body data to obtain a predetermined range. If it can be evaluated that a certain number of points or more are gathered in the image, it can be determined that an object image (moving body) corresponding to the standard model S exists in the captured image 20, and the point of the aggregated result having the highest voting result is this. It is the position coordinate of the representative point d in which the moving body 21 exists in the captured image 20. As described above, the position coordinates of the representative point d of the moving body 21 are obtained in the same manner for each captured image 20 input in the order of passage of time by the image processing unit 51, and the representative points of the captured images 20 are obtained by the tracking unit 52 in the order of passage of time. By tracking d, the moving body 21 indicated by the representative point d is tracked. By tracking in this way, it is resistant to changes in illuminance, and it is possible to faithfully and at high speed track the external shape of the moving body, the internal pattern of the moving body, and the like as feature points. Although the vector focus method has been described above based on the brightness of each pixel, of course, the color gradient of the color of each pixel and the gradient direction thereof may be used.

When the number of objects detected by the object identification unit 3 in the moving object tracking unit 5 is small (for example, several or less in the captured image 20), the movable range is based on the type of the detected object. The circular Fourier transform may be performed only for a predetermined voting range. The movable range can be obtained, for example, from the maximum moving speed defined in advance for each type of object and the image processing ability (FPS) in the object identification unit 3. Further, whether or not to limit the range of the circular Fourier transform to the voting range can be determined according to the number of objects detected by the object identification unit 3. As a result, the processing speed of the mobile tracking unit 5 can be further improved.

In the image processing by the vector focus method, when the voting process is performed on the entire image when the voting is performed, the processing range is wide and the calculation load becomes high. Therefore, by defining a voting range in which the voting process for the central point p is performed and performing the voting process only in the voting range, the calculation load can be reduced and the operation can be performed at high speed. At the same time, power consumption is also reduced.

With reference to FIG. 5, in the voting range R1, for example, the size s of the moving body 21 is added to the maximum moving amount j between one frame of the moving body 21 set according to the type of the moving body 21. It can be expressed as a circle with the value as the radius. The search range dR of the representative point d can be a range of a circle whose radius is the maximum movement amount j. The voting range R1 is not limited to the expression in the circle, but may be expressed in a box or the like.
At this time, the maximum movement amount j between one frame is, for example, if the moving body 21 is a person, the maximum speed of the person is about 25 km / h, and the measurement target is converted into the movement distance per frame. It is obtained by defining each moving object and converting the moving distance of the representative point d per frame into the coordinate system in the captured image 20 from the installation angle and the installation position of the camera 10.
For the size s of the moving body 21, for example, if the moving body 21 is a person, the distance on the captured image 20 from the center position of the person to the head or foot is defined in advance as the size s of the moving body 21. back.

Further, referring to FIG. 6, when the moving body 21 to be tracked has a tracking history, the voting range R2 is a moving body on the premise that it is performing a constant velocity linear motion from the position coordinates in the plurality of past frames. The predicted point e in the next frame of the representative point d of 21 can be estimated. The search range eR of the prediction point e can be an elliptical range in which the value obtained by adding the correction distance c to the distance half of the maximum movement amount j is the major axis and the correction distance c is the minor axis. The voting range R2 is based on the position coordinates of the representative point d of the moving body 21 in the current frame and the midpoint ed of the predicted point e, the correction distance c, and the size s of the moving body 21.
Major axis a = e-ed + c + s
Minor diameter b = c + s
And the ellipse formula x ² / a ² + y ² / b ² = 1
Can be expressed by.
Here, the correction distance c is an error amount from the prediction point e, and if it is set to about half of the maximum movement amount j, for example, the calculation amount can be reduced and the accuracy can be achieved at the same time. The correction distance c is preferably defined for each moving body from experiments and the like.

With reference to FIG. 7, the standard model placed on the captured image 20fn of the current frame in the above-mentioned vector focusing method standardizes the moving body 21 being tracked by the captured image 20fn-1 of the previous frame immediately before the current frame. Used as model S-2. As a result, even if the moving body 21 changes variously and individually over time, the center point p and the standard body data of the standard model S-2 based on the standard model S-2 corresponding to the latest moving body 21 can be obtained. It can be manufactured and the tracking accuracy can be improved. The standard model S-2 is not limited to using the moving body 21 that is actually being tracked, and the shape of the object or the moving body that is captured in the captured image 20 captured by the fixed camera 10 is preliminarily used. You may also use the standard model created in. For example, when the moving body 21 to be tracked moves at a constant speed, or when the shape of the moving body 21 itself does not change much, an object or a moving body whose position, shape, etc. do not change in various ways over time. If so, even the standard model created in advance can be traced accurately.

However, when the object is detected in the captured image 20 by the object identification unit 3, the object image output as the image processing result by the object identification unit 3 is used as the standard model S-1 (details thereof are as follows. This will be described in the match determination unit 6 of.). For example, the center point m of the object 21 detected by the object identification unit 3 is used as the center point p of the standard model S-1 in the mobile object tracking unit 5. As described above, the object identification unit 3 can accurately detect a predetermined object from the captured image 20 by artificial intelligence. Therefore, the object that first appears in the captured image 20 can be accurately detected by the object identification unit 3, and this object can be reliably tracked by the mobile object tracking unit 5 as a new moving object. In addition, the standard model (center point p of the standard model S and standard body data) for the moving body being tracked by the moving body tracking unit 5 is periodically corrected so as to correspond to the object accurately detected by the object identification unit 3. This makes it possible to further improve the tracking accuracy of the moving object by the moving object tracking unit 5.

The match determination unit 6 has a configuration for performing a process for matching a moving object to be tracked by the moving object tracking unit 5 with an object detected by the object identification unit 3, and includes a determination unit 61, a correction unit 62, and a new one. It has a creating unit 63.

With reference to FIG. 8, the determination unit 61 has the object identification unit 3 and the moving object tracking unit 5 for the captured image 20 of the same frame input to the object identification unit 3 and the moving object tracking unit 5 by the image distribution unit 2. By comparing the image processing results with the above, it is determined whether or not the same moving object as the moving object being tracked by the moving object tracking unit 5 exists among the objects detected by the object identification unit 3. That is, when the image processing result for the captured image 20 of the same frame is input to the match determination unit 6 from the object identification unit 3 and the moving object tracking unit 5, the determination unit 61 moves with the processing result in the object identification unit 3. The moving body 21 that matches the processing result of the body tracking unit 5 is searched for. Specifically, the determination unit 61 has the position coordinates of the center point m of the object 21 detected by the object identification unit 3 and the position coordinates of the representative point d of the moving body 21 being tracked by the moving body tracking unit 5. , The combination that exists within a certain range and is the closest to the distance is regarded as the "same moving object" (see FIG. 8 (A)). The fixed range to be regarded as the same moving body may be determined and set by verification, experiment, or the like.

Further, regarding the object that first appears in the captured image 20, the moving object tracking unit 5 does not yet have the standard data of the object, so that the representative point d of the object is not attached, but the object identification unit 3 has the captured image. An object that first appears in 20 is also detected, and the center point m of the object is also detected. In this case, the representative point d in the moving object tracking unit 5 that is paired with the center point m detected by the object identification unit 3 does not exist, and the object is detected only in the image processing result in the object identification unit 3. Become. Therefore, in the determination unit 61, it is determined that the same moving object does not exist in the object identification unit 3 and the moving object tracking unit 5 (see FIG. 8B).

When the determination unit 61 determines that the same moving object 21 exists, the correction unit 62 tells the moving object tracking unit 5 the frame next to the frame input to the match determination unit 6, that is, the determination unit. In the next frame 20fn + 1 one after the current frame 20fn used in the determination of 61, the process of modifying the object detected by the object identification unit 3 as the standard body data into the standard body data as the standard model S-1 and using it. I do. On the other hand, when the determination unit 61 determines that the same moving object 21 does not exist, that is, the object detected by the object identification unit 3 can be regarded as the same as the moving object being tracked by the moving object tracking unit 5. If an object that exists only in the image processing result of the object identification unit 3 is detected, the new creation unit 63 detects the moving object tracking unit 5 only in the object identification unit 3 in the next frame 20fn + 1. The created object is registered as a new moving object, and the process of using the standard object data using the object as the standard model S-1 is performed.

Further, a moving body that can be regarded as "the same moving body" with respect to the position coordinates of the representative point d of the moving body 21 being tracked by the moving body tracking unit 5 can be found from the processing result of the object identification unit 3. If not, the correction unit 62 corrects the position coordinates of the representative point d to the standard body data as the standard model S-1 based on the brightness gradient of the moving body 21 tracked by the moving body tracking unit 5. Perform processing. As a result, even if there is a momentary omission of detection or erroneous detection due to a lack of learned data in the artificial intelligence of the object identification unit 3, it is possible to perform accurate tracking.

In the matching determination unit 61, the moving body can be regarded as the "same moving body" with respect to the position coordinates of the representative point d of the moving body 21 being tracked by the moving body tracking unit 5 in succession in a plurality of frames. If 21 cannot be found from the processing result of the object identification unit 3, it may be considered that the tracking target is an object that has been erroneously tracked and may be excluded from the tracking target. As a result, the result of erroneous detection of the moving body tracking unit 5 and the object identification unit 3 can be absorbed, and the moving body 21 can be tracked with high accuracy. Here, the number of times that the moving body 21 that can be continuously regarded as the "same moving body" cannot be found may be set through experiments or the like, and is preferably adjusted according to the detection performance of artificial intelligence. ..

From the above, the moving body 21 being tracked by the moving body tracking unit 5 can be periodically verified and corrected by the object identification unit 3, and the tracking accuracy of the moving body 21 by the moving body tracking unit 5 can be further improved. can. Further, an object that first appears in the captured image 20 can be accurately detected by the object identification unit 3, and this object can be accurately tracked by the mobile object tracking unit 5 as a new moving object 21. Further, the moving body tracking unit 5 frequently tracks the moving body 21 by using the vector focus method while reducing the processing frequency of the object identification unit 3 having a high calculation load, resulting in low power consumption and low performance. It is possible to classify and track multiple types of moving objects even in various operating environments.

As described above, the mobile tracking device 1 can track the moving body 21 and output the moving body information 4 to the outside. The moving body information 4 records and outputs not only the position coordinates and type of the moving body 21, but also information associated with the moving body 21, such as a tracking start time, an identification number, and a speed information, according to the purpose and use. You may.

Next, a moving body tracking method using the moving body tracking device 1 will be described.
As shown in FIG. 9, the moving object tracking method includes an image distribution step, an object identification step, a moving object tracking step, and a matching determination step. Each of these steps is executed by the functions of the image distribution unit 2, the object identification unit 3, and the match determination unit 6. In each step, the description of each part of the mobile tracking device 1 described above is appropriately incorporated.

In the image distribution step, the image distribution unit 2 distributes the captured image 20 captured by the camera 10 into two systems, an output that delays the image in the FIFO type and an output that does not delay, and outputs the captured image 20 to the processing units 3 and 5. It is a step to perform the processing to be performed. In this image distribution step, the captured image 20 is periodically output to the object identification unit 3 at intervals of a fixed number of frames, and the captured image 20 is output to the moving object tracking unit 5 for each frame. At this time, the output timing of the captured image 20 to the moving object tracking unit 5 is such that the image processing end timing of the moving object tracking unit 5 is the image processing end timing of the object identification unit 3 for the captured image 20 of the same frame output to the object identification unit 3. The output is delayed so as to match the image processing end timing.

The object identification step is a step in which the object identification unit 3 performs a process of detecting an object to be tracked from the captured images 20 of the frame input in the image distribution step by artificial intelligence. That is, a captured image of a frame periodically input from the image distribution step by artificial intelligence constructed by a trained model in which image data of various shapes of an actual object to be a moving object 21 to be tracked is machine-learned. A process of accurately detecting the region of a predetermined object, the position of the object, and the type of the object in 20 is performed. As a result, the same captured image 20 is periodically image-processed at intervals of a fixed number of frames among the captured images 20 image-processed in the moving object tracking step, and the moving object tracking information in the moving object tracking unit 5 is performed. The tracking accuracy of the moving body 21 can be improved by modifying the moving body 21 on a regular basis.

The moving object tracking step is a step in which the moving object tracking unit 5 performs a process of tracking the moving object 21 to be tracked from the captured images 20 of the frame input in the image distribution step by image processing by the vector focus method. be. In this moving body tracking step, standard body data based on the standard model of the moving body 21 to be tracked is set by image processing by the vector focus method, and is represented by the moving body 21 that matches the standard body data in the captured image 20. A point d is arranged, and a process of tracking the representative point d of the same moving body 21 from the captured image 20 of the frame input each time by the image distribution step is performed. The procedure in this mobile tracking step will be described later.

The match determination step is a step in which the match determination unit 6 performs a process for matching the moving object 21 to be tracked by the moving object tracking step with the object detected in the object identification step. In this match determination step, first, the determination unit 61 takes a moving object tracking step among the objects detected in the object identification step for the captured image 20 of the same frame input to the object identification step and the moving object tracking step. It is determined whether or not the same moving body 21 as the moving body 21 being tracked exists in the above. Then, when it is determined in the matching determination that the same moving object 21 exists, the correction unit 62 captures an image of the next frame one frame after the current frame used in the matching determination in the moving object tracking step. In the process of 20, the object detected in the object identification step is modified into standard body data as a standard model and used. When it is determined in the matching determination that the same moving body 21 does not exist, it is detected only in the object identification step in the processing of the captured image 20 of the next frame in the moving body tracking step in the new creation unit 63. The object is registered as a new moving body 21, and a process of using the standard body data using the object as a standard model is performed.

Next, the processing procedure in the moving body tracking step will be described.
As shown in FIG. 10, when the moving body tracking operation is started, the captured image 20 is input to the moving body tracking unit 5 by the image distribution step (S1), and it is determined whether or not the captured image 20 is input for the first time (S2). ). When the captured image 20 is input for the first time (that is, when the captured image 20 of the first frame is input), the moving body 21 is detected from the captured images 20 (S3), and the moving body detection result is determined by the match determination unit. Output to 6 (S4).

When the captured image 20 is not input for the first time in step S1 (that is, when the captured image 20 of the second and subsequent frames is input), the process shifts to procedure S5, and the matching determination unit 6 commands the determination result. Judge whether there is an input of. When the command of the determination result is input and the same moving body 21 as the object detected by the object identification unit 3 exists in the moving body tracking unit 5 (“yes” in S6), the captured image input in the procedure S7. In the image processing of No. 20, as the standard body data, the detected object in the object identification unit 3 used in the matching determination is used for the standard model S-1 to correct the standard body data (see FIG. 8).

On the other hand, in the procedure S6, when the same moving object 21 as the object detected by the object identification unit 3 does not exist in the moving object tracking unit 5 (“no” in S6), the object detected only by the object identification unit 3 is new. It is registered as the moving body 21 appearing in the above and the tracking is started (S8), and when the input image processing of the captured image 20 is performed, the detected object by the object identification unit 3 used in the matching determination is used as the standard body data. Is used for the standard model S-1 to newly create standard data (S9).

Then, the moving body is detected from the input captured image 20 (S10), and the detected moving body 21 is tracked (S11). Further, the moving object detection result of the moving object tracking unit 5 is output to the matching determination unit 6 (S11). It should be noted that the output to the match determination unit 6 is not performed every time for each captured image 20 input to the moving object tracking unit 5, but to process the same captured image 20 as the captured image 20 input to the object identification unit 3. The moving object detection result may be output only when.

If there is no input of the determination result command from the match determination unit 6 in the procedure S5 (“no” in S5), the input image 20 is input as standard data in the image processing of the input image 20 in the procedure S13. The moving body 21 being tracked in the captured image 20 of the previous frame immediately before the frame of the captured image 20 is used for the standard model S-2 to generate standard data (see FIG. 7).

Then, the moving body 21 is detected from the input captured image 20 (S14), and the same moving body 21 as the moving body 21 being tracked is tracked (S15, S16). Further, the moving object detection result of the moving object tracking unit 5 is output to the matching determination unit 6 (S18). Also in this case, the output to the match determination unit 6 is not performed every time for each captured image 20 input to the moving object tracking unit 5, but is the same as the captured image 20 input to the object identification unit 3. The moving object detection result may be output only at the time of the processing of. If there is no moving body 21 that is the same as the moving body 21 being tracked in the input captured image 20 (“no” in S15), the tracking of the moving body 21 is terminated (S17).
By repeating the above operation, the predetermined moving body 21 can be accurately tracked from the captured image 20 of the camera 10.

If there is no moving body 21 that is the same as the moving body 21 being tracked in the procedure S15, the tracking may not be terminated immediately in the procedure S17. That is, there are cases where the image 20 does not appear (cannot be seen) even though it actually exists due to the positional relationship with the occlusion and the light source. Therefore, the tracking is not terminated immediately, and the position of the moving body in the current frame is assumed to be moving linearly at a constant velocity depending on the moving speed and the direction due to the difference in the position coordinates of the moving body 21 in each frame over time in the past. If this estimated position is within the range of the captured image 20, the tracking information may be updated and left with this estimated position as the position coordinates of the representative point of the moving body 21. If the estimated position is outside the range of the captured image 20, the tracking may be terminated assuming that the moving body 21 is out of frame.

(Embodiment 2)
As shown in FIG. 11, the moving body tracking device 1X of the second embodiment has, as the main configuration, the image distribution unit 2X, the object identification unit 3X, the moving body tracking unit 5X, and the matching determination as in FIG. 1 of the first embodiment. It has a part 6X.

In the first embodiment, the object identification unit 3 may detect the object by using artificial intelligence, but in the second embodiment, the object identification unit 3X is an object by image processing by the vector focus method. Is to be detected. The object identification unit 3X using the vector focus method has a small arithmetic processing load for image processing and can achieve low power consumption. As for the method of detecting an object by the vector focus method, the corresponding part of the description of the moving body detection by the moving body tracking unit 5 of the first embodiment is referred to.

According to the second embodiment, both the object identification unit 3X and the moving object tracking unit 5X are configured to use the image processing of the vector focus method, whereby the calculation processing load in the object identification unit 3X and the moving object tracking unit 5X is reduced. , Low power consumption can be achieved. Therefore, the mobile tracking device 1X can be a device having low power consumption, small size, and easy heat countermeasures.

Since the object identification unit 3X of the second embodiment performs object detection by image processing of the vector focus method, the image processing can be performed at a relatively high speed as compared with the object identification unit 3 of the first embodiment using artificial intelligence. Therefore, the object identification unit 3X of the second embodiment may be operated intermittently. In this case, the match determination operation of the match determination unit 6X is also intermittent. As a result, the power consumption of the object identification unit 3X and the match determination unit 6X can be reduced, and the mobile object tracking device 1X with low power consumption can be realized. Further, since the match determination operation of the match determination unit 6X is performed periodically, the moving object tracking unit 5X periodically performs the same operation in accordance with the intermittent operation of the object identification unit 3X and the match determination unit 6X. The object detected by the object identification unit 3X as the same moving body 21 from the captured image 20 is set as the standard model S-1, and the moving body 21 is tracked. When the object identification unit 3X is stopped, the standard model S used in the image processing by the vector focus method in the moving object tracking unit 5X uses a preset standard model imitating a moving object image. May be good.
In the second embodiment, the description in the first embodiment is incorporated for other configurations and actions and effects.

The present invention is not limited to the above-described embodiment, and can be arbitrarily modified within the scope of the present invention.

1 Moving object tracking device 2 Image distribution unit 3 Object identification unit 4 Moving object information 5 Moving object tracking unit 6 Match determination unit 20 Captured image 21 Moving object image, object image 31 Feature extraction unit 32 Object detection unit 51 Image processing unit 52 Tracking unit Part B Bounding box d Representative point m Center point of object p Center point of standard model S, S-1, S-2 Standard model

Claims (7)

It is a mobile object tracking device that tracks a moving object based on the object image in the captured image captured by the imaging unit.
An image distribution unit that distributes the captured image into two systems, an output that delays the captured image in a FIFO type according to the image processing speed and an output that thins the captured image without delay, and outputs the captured image.
An object identification unit that performs processing to detect an object to be tracked from the captured images input from the image distribution unit by object detection by artificial intelligence or the vector focus method.
The position and range of the recognition result of the object in the object identification unit are treated as a standard model of image processing by the vector focus method, and the moving object to be tracked that matches the standard model is selected from the captured images input from the image distribution unit. A moving object tracking unit that performs tracking processing, and
It is equipped with a match determination unit that integrates the results of the object identification unit and the moving object tracking unit.
The image distribution unit periodically outputs captured images to the object identification unit at intervals of a fixed number of frames, and outputs the object identification unit to the moving object tracking unit and the moving object tracking unit. The configuration is such that the captured image is output each time so that the output is at the same timing.
The object identification unit is configured to perform intermittent operations by high-speed arithmetic processing or to perform low-speed arithmetic processing.
When the same moving object exists from the outputs from the object identification unit and the moving object tracking unit, the matching determination unit detects the moving object tracked by the moving object tracking unit by the object identification unit. A moving object tracking device configured to perform processing for matching with. It is a mobile object tracking device that tracks a moving object based on the object image in the captured image captured by the imaging unit.
An image distribution unit that distributes the captured image to two systems, an output that is delayed by a FIFO type according to the image processing speed and an output that is thinned out without delay, and outputs the captured image to each processing unit.
An object identification unit that performs processing to detect an object to be tracked from the captured image of the frame input from the image distribution unit using artificial intelligence, and an object identification unit.
A mobile tracking unit that performs processing to track a moving object to be tracked from the captured image of a frame input from the image distribution unit by image processing by the vector focus method.
A matching determination unit that performs processing for matching a moving object to be tracked by the moving object tracking unit with an object detected by the object identification unit is provided.
The image distribution unit periodically outputs captured images to the object identification unit at intervals of a fixed number of frames, and outputs the object identification unit to the moving object tracking unit and the moving object tracking unit. It has a configuration in which the captured image is output each time with a delay so that the output is at the same timing.
The object identification unit constructs a trained model in which image data of various shapes of an actual object to be tracked by the artificial intelligence is machine-learned, and is periodically input from the image distribution unit. It has a configuration that accurately detects the region of a predetermined object, the position of the object, and the type of the object in the captured image of the frame to be formed.
In the moving body tracking unit, standard body data based on a standard model of the moving body to be tracked is set by image processing by the vector focus method, and a representative point is a moving body that matches the standard body data in the captured image. It has a configuration in which the representative points of the same moving body are tracked from the captured images of the frames that are arranged and input from the image distribution unit each time.
The match determination unit
Regarding the captured image of the same frame input to the object identification unit and the moving object tracking unit, the same moving object as the moving object being tracked by the moving object tracking unit exists among the objects detected by the object identification unit. Make a match judgment and decide whether or not to do it.
When it is determined by the matching determination that the same moving object exists, the object is processed by the moving object tracking unit in the processing of the captured image of the next frame one frame after the current frame used in the matching determination. Performs processing to use the object detected by the identification unit as a standard model,
When it is determined in the matching determination that the same moving object does not exist, the object detected only in the object identification unit is newly added to the moving object tracking unit in the processing of the captured image of the next frame. A moving object tracking device having a configuration in which a moving object is registered and processing is performed using the object as a standard model. In the mobile tracking device according to claim 2,
The moving object tracking unit is one of the frames as the standard model used in the processing of the captured image of the frame, except when the object detected by the object identification unit by the matching determination unit is used as the standard model. A moving object tracking device having a configuration in which the moving object being tracked in the captured image of the previous frame is used as a standard model. It is a mobile object tracking device that tracks a moving object based on the object image in the captured image captured by the imaging unit.
An image distribution unit that acquires and outputs the captured image, and
An object identification unit that performs processing to detect an object to be tracked from the captured images input from the image distribution unit by object detection by the vector focus method.
It is provided with a moving object tracking unit that performs processing for tracking a moving object to be tracked from the captured image input from the image distribution unit by image processing by the vector focus method.
In the moving body tracking unit, standard body data based on a standard model of the moving body to be tracked is set by image processing by the vector focus method, and a representative point is a moving body that matches the standard body data in the captured image. It is provided with a configuration for tracking a representative point of the same moving object from the captured images that are arranged and input from the image distribution unit each time in the order of passage of time, and is one before the captured image as the standard model. A moving object tracking device having a configuration in which a moving object being tracked in an captured image or an object detected by the object identification unit is used. It is a moving body tracking method that tracks a moving body based on an object image in the captured image captured by the imaging unit.
An image distribution step in which the image distribution unit distributes the captured image into two systems, an output that delays the captured image in a FIFO type according to the image processing speed and an output that thins out without delay, and outputs the captured image to each processing unit.
In the object identification unit, an object identification step that performs a process of detecting an object to be tracked from the captured images of the frame input in the image distribution step by artificial intelligence, and an object identification step.
A mobile tracking step that performs processing to track a moving object to be tracked from the captured images of the frame input in the image distribution step by image processing by the vector focusing method in the moving object tracking unit.
The match determination unit includes a match determination step for performing a process for matching a moving object to be tracked by the moving object tracking step with an object detected by the object identification step.
The image distribution step periodically outputs captured images to the object identification unit at intervals of a fixed number of frames, and outputs the object identification unit to the moving object tracking unit and the moving object tracking unit. A process is performed to output the captured image each time with a delay so that the output is at the same timing.
In the object identification step, a trained model is constructed by machine-learning image data of various shapes of an actual object to be tracked in the artificial intelligence, and the object is periodically input by the image distribution step. A process is performed to accurately detect the area of a predetermined object, the position of the object, and the type of the object in the captured image of the frame to be performed.
In the moving body tracking step, standard body data based on a standard model of the moving body to be tracked is set by image processing by the vector focusing method, and a representative point is a moving body that matches the standard body data in the captured image. A process of tracking the representative points of the same moving body from the captured images of the frames that are arranged and input each time by the image distribution step is performed.
The match determination step is
Regarding the captured images of the same frame input to the object identification step and the moving object tracking step, among the objects detected in the object identification step, the same moving object as the moving object being tracked in the moving object tracking step exists. Make a match judgment and decide whether or not to do it.
When it is determined in the matching determination that the same moving object exists, in the processing of the captured image of the next frame one after the current frame used in the matching determination in the moving object tracking step, the object identification step is performed. The object detected in is used as a standard model, and it is processed.
When it is determined in the matching determination that the same moving object does not exist, in the moving object tracking step, in the processing of the captured image of the next frame, the object detected only in the object identification step is a new moving object. A moving object tracking method that registers as a standard model and performs processing using the object as a standard model. In the mobile tracking method according to claim 5,
The moving object tracking step is one of the frames, except when the object detected in the object identification step by the matching determination step is used as the standard model as the standard model used in the processing of the captured image of the frame. A moving object tracking method that uses the moving object being tracked in the captured image of the previous frame as a standard model. It is a moving body tracking method that tracks a moving body based on an object image in the captured image captured by the imaging unit.
An image distribution step of acquiring and outputting the captured image in the image distribution unit,
In the object identification unit, an object identification step that performs a process of detecting an object to be tracked from the captured images input in the image distribution step by object detection by the vector focus method, and an object identification step.
The moving object tracking unit includes a moving object tracking step that performs a process of tracking a moving object to be tracked from the captured images input in the image distribution step by image processing by the vector focus method.
In the moving object tracking step, standard body data based on a standard model of the moving body to be tracked is set by image processing by the vector focusing method, and a representative point is set to a moving body that matches the standard body data in the captured image. It is a step of tracking the representative points of the same moving body from the captured images that are arranged and then input each time in the order of passage of time in the image distribution step, and is one step before the captured image as the standard model. A moving object tracking method using a moving object being tracked in an captured image or an object detected by the object identification step.

Images