WO2023166648A1 - Movement trajectory information processing device - Google Patents

Abstract

This movement trajectory information processing device comprises a calculation means and a conversion means. The calculation means calculates movement trajectory information, which is a time series of information pertaining to the position of a moving object in an image, from a plurality of images in which the object is continuously captured by a camera for capturing the image within an imaging range. The conversion means calculates a function for mapping from the image to a reference image, converts the information pertaining to the position of the object in the image to information pertaining to the position of the object in the reference image by using the calculated mapping function, and converts the information pertaining to the position of the object in the reference image that was obtained through conversion to information pertaining to the position of the object in a global coordinate system using camera calibration information.

Description

Movement trajectory information processing device

The present invention relates to a movement trajectory information processing device, a movement trajectory information processing method, and a recording medium.

An example of a conventional device that calculates movement trajectory information is described in Patent Document 1. A conventional apparatus uses a camera with fixed pan, tilt and zoom values (PTZ values) to continuously photograph an object such as a moving person. In addition, the conventional apparatus extracts the position of the object from the image obtained by the camera at the time of each photographing, and uses the camera calibration information (intrinsic parameters and extrinsic parameters) to extract the position of the object on the image of the object. Converts coordinate values to coordinate values in real space. The conventional apparatus uses the time series of the position of the object on the real space obtained as described above as movement trajectory information.

WO2014/010174

A camera with a fixed PTZ value has a limited field of view. Therefore, for the purpose of monitoring a wider range, it is possible to calculate the movement trajectory information of an object from a plurality of images continuously captured by a camera while changing the PTZ value and capturing the moving object within the imaging range. is important. However, when the PTZ value of a camera changes, the camera calibration information for that camera changes. Further, in order to acquire the camera calibration information corresponding to the changed PTZ value, calibration work by the operator is required. For this reason, the conventional apparatus that directly converts the coordinate values on the image of the target object to the coordinate values on the real space using the camera calibration information from each of a plurality of images that are continuously captured cannot capture the moving target object. It has been difficult to calculate movement trajectory information from a plurality of images of an object continuously captured by a camera that captures it within its range.

An object of the present invention is to provide a movement trajectory information processing apparatus that solves the above-described problems.

A movement trajectory information processing device according to one aspect of the present invention includes:
Calculation means for calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; conversion means for converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
is configured to include

A movement trajectory information processing method according to another aspect of the present invention includes:
calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
is configured as

A computer-readable recording medium according to another aspect of the present invention comprises:
to the computer,
A process of calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; a process of converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
It is configured to record a program for causing the

According to the present invention, having the above-described configuration, a time series of the position of the object in the real space is obtained from a plurality of images of the object continuously photographed by a camera that captures the moving object within the photographing range. Movement trajectory information can be calculated.

It is a mimetic diagram showing an example of composition of a tracking system concerning one embodiment of the present invention. It is a block diagram which shows an example of the control apparatus in the tracking system which concerns on one Embodiment of this invention. FIG. 4 is a diagram showing an example of tracked object movement trajectory information used by the control device in the tracking system according to one embodiment of the present invention; FIG. 4 is a diagram showing an example of to-be-verified trajectory information used by the control device in the tracking system according to one embodiment of the present invention; FIG. 4 is a schematic diagram for explaining an example of a method for converting position information on an image from an image coordinate system to a world coordinate system in the control device in the tracking system according to one embodiment of the present invention; 4 is a flowchart showing an example of tracking processing performed by a control device using one camera in the tracking system according to one embodiment of the present invention; 4 is a flowchart showing an example of tracking processing performed by the control device using the other camera in the tracking system according to one embodiment of the present invention; 1 is a block diagram of an embodiment of a movement trajectory information processing device according to the present invention; FIG.

Next, embodiments of the present invention will be described in detail with reference to the drawings. In addition, in the description in the specification, the descriptions with the same meaning of "tracking" and "tracking" coexist, but "tracking" mainly refers to the structure of the part that is strongly related to people, and "tracking" It is only for the sake of convenience to explain the configuration of the part that is mainly related to the device.
[First embodiment]
First, in order to facilitate understanding of the first embodiment of the present invention, problems assumed by the first embodiment of the present invention will be described.

Detecting and tracking people from surveillance camera images is useful for crime prevention and marketing. A camera with fixed PTZ values has a limited field of view of the camera. By changing the PTZ value of the camera, the field of view of the camera can be expanded. However, the field of view of a single camera is still limited. Therefore, a system is considered in which a plurality of PTZ cameras are distributed over a monitoring area with a part of the camera field of view shared, and a person is tracked in a wider range spatially and temporally. In such tracking systems, the system automatically recognizes when a person being tracked by one camera comes into the field of view of another camera, and the other camera begins tracking that person. I hope you can. A person's movement trajectory can be used to determine which person should take over tracking from many persons present in the field of view of the camera. That is, by collating information on the movement trajectory of a person being tracked by one camera (tracking target movement trajectory information) with information on the movement trajectory of a person within the field of view of the other camera (matched movement trajectory information). , determine the tracking target. In order to perform such collation, it is necessary that the tracking target trajectory information and the collated trajectory information are expressed in the same coordinate system.

In general, the coordinate system that expresses the position of a person captured by a camera includes an image coordinate system and a camera coordinate system unique to each camera, and a world coordinate system that is common to multiple cameras. The image coordinate system is a two-dimensional coordinate system on the image pickup device, and the position of a point on the image is usually represented by this image coordinate system. The camera coordinate system is a coordinate system defined by the image coordinate system and internal parameters of the camera. Therefore, the image coordinate system of the camera and the camera coordinate system of the camera can be mutually transformed using the intrinsic parameters of the camera. The world coordinate system is a coordinate system defined by camera calibration information composed of camera intrinsic and extrinsic parameters and an image coordinate system. Therefore, the image coordinate system of a camera and the world coordinate system can be transformed to each other using the camera calibration information of the camera.

A conventional device extracts the position of a person from a photographed image in an image coordinate system, and uses camera calibration information to convert the coordinate values of the extracted person in the image coordinate system into coordinate values in the world coordinate system. do. However, changing the PTZ value of the camera to track a person changes the camera calibration information. Also, it is difficult to acquire camera calibration information corresponding to the changed PTZ value in real time. Therefore, in the conventional apparatus, it is difficult to calculate movement locus information while tracking a moving object with a camera having a tracking function that captures the moving object within an imaging range. As a result, it has been difficult to collate the movement trajectory of a person between a plurality of PTZ cameras in real time.

This embodiment solves the above problems. A tracking system 10 according to an embodiment to which the present invention is applied will now be described in detail with reference to the drawings.

FIG. 1 is a schematic diagram showing a configuration example of a tracking system 10 according to one embodiment of the present invention. Referring to FIG. 1, the tracking system 10 is a system for detecting and tracking a person in a surveillance area, and includes two PTZ cameras 11 and 12 and a control device 20.

The PTZ cameras (hereinafter simply referred to as cameras) 11 and 12 include solid-state imaging devices such as CMOS sensors and CCD sensors, and pan heads. The cameras 11 and 12 generate a plurality of images by continuously photographing the photographing range at regular intervals, for example, in accordance with instructions from the control device 20 . It is preferable that the imaging cycles and imaging timings of the cameras 11 and 12 are the same or similar, but they may be different. The cameras 11 and 12 can change the imaging range by changing the pan, tilt, and zoom in accordance with commands from the control device 20 . Here, "pan" means moving the camera left and right, "tilt" means moving the camera up and down, and "zoom" means changing the angle of view to telephoto or wide angle. The cameras 11 and 12 are installed dispersedly in the monitoring area so that a part of their photographing ranges are shared. In the example shown in FIG. 1, the camera 11 monitors monitoring areas 13A, 13B, and 13C set on a passage extending in the horizontal direction of the paper. In addition, the camera 12 monitors a monitoring area 13C and a monitoring area 13D set on a passage extending in the vertical direction of the paper. The monitoring area 13C corresponds to the corner portion of the passage. That is, the corner area of the passage is the surveillance area 13C that is common to the two cameras 11 and 12 . On the aisle, a person enters from the left side of the paper surface, passes through the corner of the passage, and exits from the lower side of the paper surface. Also, the monitoring areas 13A to 13D are assumed to be flat surfaces.

The control device 20 is connected to the cameras 11 and 12 by wire or wirelessly. The control device 20 calculates camera calibration information (intrinsic parameters and extrinsic parameters) of each camera when the cameras 11 and 12 are installed. This camera calibration information is calculated by setting the PTZ values of the cameras 11 and 12 as the reference PTZ of each camera. For example, the reference PTZ of the camera 11 is set so as to provide a camera field of view capable of photographing a person in the monitoring area 13A. Also, the reference PTZ of the camera 12 is set so as to provide a camera field of view capable of photographing a person in the monitoring area 13C. At the start of system operation, the control device 20 monitors and displays on the screen display unit images continuously captured by the camera 11 fixed to the reference PTZ. When the operator designates a person to be tracked on the image displayed on the monitor, the control device 20 tracks the designated person while changing the PTZ value of the camera 11 . Then, the control device 20 calculates in real time information on the movement trajectory of the person being tracked (tracking target movement trajectory information) in the world coordinate system.

On the other hand, at the start of system operation, the control device 20 obtains information on the movement trajectories of all persons in the monitoring area 13C (matched movement trajectory information) in real time in the world coordinate system. Then, the control device 20 collates the tracked movement trajectory information of the person being tracked by the camera 11 with the verified movement trajectory information of all the persons captured by the camera 12 in real time. Thereby, the control device 20 determines which person captured by the camera 12 is the person being tracked by the camera 11 . Next, the control device 20 continues tracking with the camera 12 while catching the determined person in the imaging range of the camera 12, and displays the image on the screen display unit as a monitor.

The above is the outline of the tracking system 10. Next, the control device 20 will be described in detail.

FIG. 2 is a block diagram showing an example of the control device 20. As shown in FIG. Referring to FIG. 2 , the control device 20 includes a communication I/F (interface) section 21 , an operation input section 22 , a screen display section 23 , a storage section 24 and an arithmetic processing section 25 .

The communication I/F unit 21 is composed of a data communication circuit, and is configured to perform data communication with the cameras 11 and 12 and other external devices (not shown) by wire or wirelessly. The operation input unit 22 is composed of an operation input device such as a keyboard and a mouse, and is configured to detect an operator's operation and output it to the arithmetic processing unit 25 . The screen display unit 23 is composed of a display device such as an LCD (Liquid Crystal Display), and is configured to display images captured by the cameras 11 and 12 and the like.

The storage unit 24 is composed of one or more types of storage devices such as hard disks and memories, and is configured to store processing information and programs 241 required for various processes in the arithmetic processing unit 25 . The program 241 is a program that realizes various processing units by being read and executed by the arithmetic processing unit 25, and can be read from an external device or a recording medium (not shown) via a data input/output function such as the communication I/F unit 21. It is read in advance and stored in the storage unit 24 . Main processing information stored in the storage unit 24 includes camera calibration information 242-1, 242-2, reference images 243-1, 243-2, image DB (database) 244-1, 244-2, tracking target There is moving track information 245 and collated moving track information 246 .

The camera calibration information 242-1 is the camera calibration information of the camera 11 (internal parameters and external parameters of the camera 11). The camera calibration information 242-2 is camera calibration information of the camera 12 (intrinsic parameters and extrinsic parameters of the camera 12).

A reference image 243-1 is an image of the surveillance area 13A captured by the camera 11 set to the pan value, tilt value, and zoom value when obtaining the camera calibration information 242-1. The reference image 243-1 includes, in addition to the image, the pan value, tilt value, and zoom value (reference PTZ value of the camera 11) of the camera 11 when the reference image was captured, and the camera position (the position of the camera 11 in the world coordinate system). may be included. The reference image 243-2 is an image of the monitoring area 13C captured by the camera 12 set to the pan value, tilt value, and zoom value when the camera calibration information 242-2 was obtained. The reference image 243-2 includes the pan value, tilt value, zoom value (reference PTZ value of the camera 12) and camera position (position of the camera 12 in the world coordinate system) at the time of photographing the reference image, in addition to the image. It's okay.

The image DB 244-1 accumulates the time series of images taken by the camera 11. Each image stored in the image DB 244-1 is added with the camera ID of the camera 11, the shooting time, and the PTZ value at the time of shooting. The image DB 244-2 accumulates the time series of images taken by the camera 12. FIG. Each image stored in the image DB 244-2 is added with the camera ID of the camera 12, the shooting time, and the PTZ value at the time of shooting.

The tracked object movement trajectory information 245 is information related to the movement trajectory of the tracked person calculated based on the image captured by the camera 11 . FIG. 3 is a diagram showing a configuration example of the tracked object movement trajectory information 245. As shown in FIG. The tracked object movement trajectory information 245 in this example includes an entry 2451 composed of a tracked object ID that uniquely identifies the tracked object, the camera ID of the camera 11, and associated information, and a plurality of It is composed of a plurality of entries 2452 corresponding to images one-to-one. A plurality of entries 2452 are linked in a line in order of photographing time by the association information in the entry 2451 and the association information in the entry 2452 .

Each entry 2452 consists of shooting time, person area, position information (image coordinate system), image feature, position information (world coordinate system), movement vector, movement speed, acceleration, and association information. The shooting time represents the shooting time of the corresponding image. A person area represents, for example, a bounding rectangle of the person area in the corresponding image. The position information (image coordinate system) represents the coordinate values of one point in the image coordinate system that represents the person area in the same entry. One point representing the person area may be, for example, the center of gravity of the person area, but is not limited to this, and may be the center of gravity of the face, feet, or the like. An image feature represents a feature amount of an image extracted from a human region in the corresponding image. Image features may include, but are not limited to, face features, clothing features, and person size. For the position information (world coordinate system), coordinate values obtained by converting the position information (image coordinate system) in the same entry from the image coordinate system to the world coordinate system are set. A motion vector represents the amount and direction of movement of the tracked person between the corresponding image and an adjacent image. The moving speed represents the moving speed of the tracked person. Acceleration represents the acceleration of the tracked person. The movement vector, movement velocity, and acceleration are calculated, for example, based on the position information (world coordinate system) in the entry 2452 of the image adjacent to the corresponding image. The type of data included in entry 2452 is an example, and is not limited to the above.

The to-be-verified movement trajectory information 246 is information related to the movement trajectory of the person extracted from the image captured by the camera 12 . The to-be-verified movement trajectory information 246 exists for each person extracted from the image captured by the camera 12 . FIG. 4 is a diagram showing a configuration example of the to-be-verified movement trajectory information 246. As shown in FIG. The to-be-verified movement trajectory information 246 in this example includes an entry 2461 composed of a person ID that uniquely identifies a person, a camera ID of the camera 12, and association information, and a plurality of images continuously shot by the camera 12. It is composed of a plurality of entries 2462 in one-to-one correspondence. A plurality of entries 2462 are linked in a line in order of photographing time by the association information in the entry 2461 and the association information in the entry 2462 . Each entry 2462 is composed of data similar to the entry 2452 of the tracked object movement trajectory information 245 described with reference to FIG. The type of data included in entry 2462 is an example and is not limited to the above.

The arithmetic processing unit 25 has one or more microprocessors such as an MPU and their peripheral circuits, and by reading and executing the program 241 from the storage unit 24, the hardware and the program 241 cooperate to perform various processes. It is configured to realize the part. Main processing units realized by the arithmetic processing unit 25 include a camera calibration unit 251 and two monitoring units 252 and 253 .

The camera calibration unit 251 is configured to acquire camera calibration information of the cameras 11 and 12 through interactive processing with the operator via the operation input unit 22 and screen display unit 23 . The camera calibration method used is not particularly limited.

The monitoring unit 252 is configured to use the camera 11 to detect and track a person moving within the monitoring areas 13A to 13C. The monitoring unit 252 includes a detection unit 2521 , a tracking unit 2522 and a coordinate conversion unit 2523 .

The detection unit 2521 detects a specific person as a tracking target from among all persons present in the monitoring area 13A based on images of the monitoring area 13A continuously captured by the camera 11 set to the reference PTZ value. is configured as

The tracking unit 2522 calculates information on the movement trajectory of the tracking target in the image coordinate system of the camera 11 while tracking the tracking target detected by the detection unit 2521 with the camera 11 so as to capture it within the imaging range, and calculates the movement of the tracking target. It is configured to be stored in the storage unit 24 as locus information 245 . Since the tracking unit 2522 changes the PTZ value of the camera 11 to track the tracking target, even if the tracking target moves from the monitoring area 13A to the monitoring area 13B or the monitoring area 13C, the tracking target is captured within the shooting range. be able to.

The coordinate transformation unit 2523 is configured to transform the tracked object movement trajectory information 245 calculated by the tracking unit 2522 from the image coordinate system of the camera 11 to the world coordinate system. That is, the coordinate transformation unit 2523 transforms the position information on the image from the image coordinate system to the world coordinate system. Here, the coordinate transformation unit 2523 may use a plane (common map) where z=0 (that is, height 0) as the world coordinate system. Transformation into the two-dimensional world coordinate system reduces the amount of calculation and errors compared to the configuration of transforming points on the image coordinate system into the three-dimensional world coordinate system. However, the coordinate transformation unit 2523 may be configured to transform points in the image coordinate system into the three-dimensional world coordinate system.

FIG. 5 is a schematic diagram for explaining an example of a method for converting position information on an image from the image coordinate system to the world coordinate system. In FIG. 5, an image I ₀ is an image captured by the camera 11 set to the same PTZ value as when the camera calibration information 242-1 was calculated. Images I ₁ , I ₂ , and I ₃ are a plurality of images taken by camera 11 continuously while changing the PTZ value after image I ₀ was taken.

The coordinate transformation unit 2523 transforms the position information on the image I ₀ from the image coordinate system to the world coordinate system using the camera calibration information 242-1.

In addition, the coordinate conversion unit 2523 converts the position information on the image I ₁ to the world coordinate system by first converting the position information on the image I ₁ to the corresponding position on the image I ₀ using the mapping function f ₁ . Convert to information. Next, the coordinate transformation unit 2523 transforms the position information on the image I ₀ obtained by the above transformation from the image coordinate system to the world coordinate system using the camera calibration information 242-1. The coordinate transformation unit 2523 also calculates the mapping function f ₁ as a planar projective transformation matrix H ₁ from the image I ₁ to the image I ₀ . That is, f ₁ =H ₁ . In the calculation of the planar projective transformation matrix H ₁ , the coordinate transformation unit 2523 obtains a group of corresponding points between the image I ₁ and the image I ₀ using a known feature point extraction method. Calculate the planar projective transformation matrix H ₁ from the group. The corresponding point cloud between image I ₁ and image I ₀ is, in other words, pairs of corresponding feature points between image I ₁ and image I ₀ .

In addition, the coordinate conversion unit 2523 converts the position information on the image I ₂ to the world coordinate system by converting the position information on the image I ₂ to the corresponding position on the image I ₀ using the mapping function f ₂ . information, and then position information on the image I ₀ obtained by this transformation is transformed from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Also, the coordinate transformation unit 2523 obtains the mapping function f ₂ from the mapping function f ₁ and the planar projective transformation matrix H ₂ from the image I ₂ to the image I ₁ . That is, f ₂ =f ₁ *H ₂ . Further, the coordinate transformation unit 2523 obtains the planar projective transformation matrix H ₂ by using a known feature point extraction method to obtain the corresponding point group between the image I ₂ and the image I ₁ , and from the obtained corresponding point group A planar projective transformation matrix H ₂ is calculated.

In addition, the coordinate conversion unit 2523 converts the position information on the image I ₃ to the world coordinate system by first converting the position information on the image I ₃ to the corresponding position on the image I ₀ using the mapping function f ₃ . information, and then position information on the image I ₀ obtained by this transformation is transformed from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Also, the coordinate transformation unit 2523 obtains the mapping function _f3 from the mapping function _f2 and the planar projective transformation matrix _H3 from the image _I3 to the image _I2 . That is, f ₃ =f ₂ *H ₃ . In addition, the coordinate transformation unit 2523 obtains the planar projective transformation matrix H ₃ by using a known feature point extraction method to obtain a group of corresponding points between the images I ₃ and I ₂ , and from the obtained corresponding point group A planar projective transformation matrix _H3 is calculated.

The method of obtaining the mapping function is not limited to the above. For example, the coordinate transformation unit 2523 calculates the mapping function _f3 as _f3 = _f1 * _H31 using the mapping function _f1 and the planar projective transformation matrix _H31 from the image _I3 to the image _I1 . good too. Alternatively, the coordinate transformation unit 2523 may calculate the mapping function f ₃ as f ₃ =H ₃₀ using the planar projective transformation matrix H ₃₀ from the image I ₃ to the image I ₀ . However, the common area between adjacent images shot continuously by the camera 11 while tracking a moving person in the shooting range is relatively large, but the common area between the images becomes smaller as the time increases. tend to become Between images with a small common area, it is difficult to obtain corresponding point groups between both images. Also, if the difference between the two images is large, it may not be possible to obtain the corresponding point group accurately even if there is a common area. Therefore, it is desirable that the photographing time interval between the two images for obtaining the planar projective transformation matrix is determined in consideration of the above circumstances.

Referring to FIG. 2 again, the monitoring unit 253 uses the camera 12 to detect the person being tracked by the camera 11 from among the persons in the monitoring area 13C, and monitors the detected person in the monitoring areas 13C and 13D. configured to track. The monitoring unit 253 includes a matching unit 2531 , a tracking unit 2532 and a coordinate conversion unit 2533 .

The matching unit 2531 extracts information on the movement trajectory of each person moving in the monitoring area 13C (matched movement trajectory information) from a plurality of images of the monitoring area 13C continuously captured by the camera 12 set to the reference PTZ value. 246) in the image coordinate system of the camera 12 .

The coordinate transformation unit 2533 is configured to transform the collated movement trajectory information 246 calculated by the collation unit 2531 from the image coordinate system of the camera 12 to the world coordinate system using the camera calibration information 242-2. . Here, the coordinate transformation unit 2533 uses a plane (common map) where z=0 (that is, height 0) as the world coordinate system, similarly to the coordinate transformation unit 2523 .

The collation unit 2531 is configured to collate the collated trajectory information 246 converted into the world coordinate system and the tracked trajectory information 245 of the person being tracked by the monitoring unit 252 . In addition, based on the result of the matching, the matching unit 2531 determines which of the people photographed by the camera 12 and moving in the monitoring area 13C is the person being tracked by the camera 11. It is configured.

An example of a method for collating the tracking target movement trajectory information 245 and the to-be-verified movement trajectory information 246 in the verification unit 2531 will be described.

First, the matching unit 2531 matches the latest N frames of each trajectory. Here, N is a predetermined value of 2 or more. Therefore, the matching unit 2531 extracts N entries 2452 in order from the last entry 2452 of the tracked object movement trajectory information 245, and uses the extracted N entries 2452 as the tracked object movement trajectory information. If there are no N entries 2452 in the tracking target movement trajectory information 245, the matching unit 2531 does not perform matching at that point. On the other hand, the matching unit 2531 extracts N entries 2462 in order from the last entry 2462 from each of the one or more to-be-matched movement trajectory information 246, and converts the extracted N entries 2462 to the to-be-matched movement trajectory. information. The collation unit 2531 does not collate the to-be-verified movement trajectory information 246 in which N entries 2462 do not exist at that time. The collation unit 2531 does not perform collation at that time if there is no collated movement trajectory information 246 in which N entries 2462 exist.

Next, the collation unit 2531 calculates collation between the movement trajectory information of the tracked object and the individual trajectory information to be collated based on the matching degree of the shape and the matching degree of the direction.

The matching unit 2531 first normalizes each movement trajectory by the center of gravity in order to eliminate the influence of the absolute position when calculating the matching degree of the shapes. Here, the movement trajectory of the tracked object is obtained by connecting the position information (in the world coordinate system) in the N entries 2452 constituting the movement trajectory information of the tracked object with line segments in order of photographing time. Also, the location information (in the world coordinate system) in the N entries 2462 constituting the movement trajectory information to be verified is connected by line segments in the order of photographing time to become the movement trajectory to be verified. Next, the collation unit 2531 obtains the error between the positions of the movement trajectories of the tracked object and the object to be collated at the same time by using the method of least squares or the like, and calculates a value that increases as the error decreases as the matching degree of the shape. .

In addition, in calculating the matching degree of orientation, the matching unit 2531 calculates a value that increases as the sum of errors between movement vectors at the same time of the movement trajectories of the tracked object and the object to be matched becomes smaller, as the degree of orientation matching. .

Next, the matching unit 2531 calculates a matching result from the shape matching degree and orientation matching degree. For example, the collation unit 2531 may obtain a value obtained by multiplying the matching degree of the shape by the matching degree of the orientation as the collation result. Alternatively, the collation unit 2531 may use, for example, a value obtained by adding together the matching degree of the shape and the matching degree of the orientation as the matching result.

The above is an example of a method of collating the tracked movement trajectory information 245 and the collated movement trajectory information 246 in the collation unit 2531 . However, the matching method is not limited to the above. For example, the collation unit 2531 also considers the image features, movement speed, and acceleration included in the entry 2452 of the tracked movement trajectory information 245 and the entry 2462 of the to-be-verified movement trajectory information 246, and collates both movement trajectory information. you can

As described above, the collation unit 2531 identifies a person moving in the monitoring area 13C captured by the camera 12 based on the result of collating the tracked movement trajectory information 245 and one or more to-be-verified movement trajectory information 246. Among them, it is configured to determine which person is the person being tracked by the camera 11 . For example, if the highest matching result is equal to or greater than a certain threshold, the matching unit 2531 determines that the person associated with the to-be-matched movement trajectory information 246 with the highest matching result is the person being tracked by the camera 11. do. On the other hand, for example, the matching unit 2531 determines that the person being tracked by the camera 11 does not exist among the people photographed by the camera 12 unless the highest matching result is equal to or greater than a certain threshold.

The tracking unit 2532 is configured so that the tracking target determined by the matching unit 2531 (the person captured by the camera 12 and being tracked by the camera 11) is tracked by the camera 12 so as to be captured within the shooting range. The tracking unit 2532 changes the PTZ value of the camera 12 to track the tracking target, so even if the tracking target moves from the monitoring area 14C to the monitoring area 13D, the tracking target can be captured within the imaging range.

Next, the operation of the control device 20 will be explained. First, camera calibration performed before system operation will be described.

The camera calibration unit 251 of the control device 20 calibrates the cameras 11 and 12 at any time before system operation, such as when the cameras 11 and 12 are installed.

In calibrating the camera 11, the camera calibration unit 251 first adjusts the PTZ value of the camera 11 so that the entire monitoring area 13A can be captured. Next, the camera calibration unit 251 calculates camera calibration information of the camera 11 using a predetermined camera calibration method through interactive processing with the operator through the operation input unit 22 and the screen display unit 23 . The camera calibration unit 251 stores the calculated camera calibration information in the storage unit 24 as the camera calibration information 242-1. Also, the camera calibration unit 251 acquires the PTZ value of the camera 11 at the time of calibration as the reference PTZ value. Further, the camera calibration unit 251 acquires an image of the area 13A captured by the camera 11 set to the reference PTZ value as a reference image. The camera calibration unit 251 adds a reference PTZ value to the obtained reference image and stores it in the storage unit 24 as a reference image 243-1.

In calibrating the camera 12, the camera calibration unit 251 first adjusts the PTZ value of the camera 12 so that the entire monitoring area 13C (common area with the camera 11) can be captured. Next, the camera calibration unit 251 calculates camera calibration information of the camera 12 using a predetermined camera calibration method through interactive processing with the operator through the operation input unit 22 and the screen display unit 23 . The camera calibration unit 251 stores the calculated camera calibration information in the storage unit 24 as camera calibration information 242-2. Also, the camera calibration unit 251 acquires the PTZ value of the camera 12 at the time of calibration as the reference PTZ value. Further, the camera calibration unit 251 acquires an image of the area 15 captured by the camera 12 set to the reference PTZ value as a reference image. The camera calibration unit 251 adds a reference PTZ value to the obtained reference image and stores it in the storage unit 24 as a reference image 243-2.

Next, the operation of the control device 20 during system operation will be described. FIG. 6 is a flowchart showing an example of tracking processing performed using the camera 11. As shown in FIG. The tracking process performed by the control device 20 using the camera 11 will be described below with reference to FIG.

First, the detection unit 2521 in the monitoring unit 252 of the control device 20 initializes (step S11). In this initialization, the detection unit 2521 sets the camera 11 to the reference PTZ value. Further, the detection unit 2521 clears all entries of the tracked object movement trajectory information 245 in the initialization. Next, the detection unit 2521 acquires an image of the surveillance area 13A captured by the camera 11 set to the reference PTZ value, saves it in the image DB 244-1, and displays it on the screen display unit 23 (step S12). Next, the detection unit 2521 detects all persons from the image saved this time using various techniques such as pattern recognition and machine learning (step S13). Next, the detection unit 2521 displays the detection result on the screen display unit 23 (step S14). For example, the detection unit 2521 generates an image in which the circumscribed rectangle for each person detected from the image is superimposed on the image, and displays the image on the screen display unit 23 . Next, the detection unit 2521 determines whether or not a person to be tracked has been designated by the operation input unit 22 (step S15). For example, the operator can specify a person to be tracked by performing an operation such as clicking a rectangle of the person on the image of the camera 11 displayed on the screen display unit 23 . If the person to be tracked is not specified, the detection unit 2521 returns to step S12 and repeats the same processing as described above. On the other hand, when a person to be tracked is designated, the detection unit 2521 updates the tracked object movement trajectory information 245 (step S16).

The detection unit 2521 sets the ID assigned to the tracked person specified by the operator and the camera ID of the camera 11 in the entry 2451 when updating the tracked object movement trajectory information 245 in step S16. Further, the detection unit 2521 secures one empty entry 2452 and sets association information to the secured entries 2452 and 2451 , thereby mutually relating the secured entries 2452 and 2451 . Next, the detection unit 2521 focuses on the secured entry 2452, sets the shooting time, person area, position information (image coordinate system), and image features to the focused entry 2452, and sets the position information (world coordinate system) and The movement vector, movement speed, and acceleration are NULL values.

Next, the coordinate transformation unit 2523 of the monitoring unit 252 performs coordinate transformation on the noted entry 2452 of the tracked object movement trajectory information 245 (step S17). In this step S17, the coordinate conversion unit 2523 first uses the camera calibration information 242-1 to convert the position information (image coordinate system) set in the focused entry 2452 from the image coordinate system to the world coordinate system. do. Next, the coordinate conversion unit 2523 sets the location information (world coordinate system) obtained by the above conversion to the entry 25452 of interest.

Next, the tracking unit 2522 of the monitoring unit 252 performs tracking control (step S18). For example, the tracking unit 2522 generates a command for controlling the PTZ value of the camera 11 according to the position information (image coordinate system) set in the focused entry 2452, and sends the command to the camera 11 through the communication I/F unit 21. send to. At this time, for example, the tracking unit 2522 adjusts panning and tilting so that the center of gravity of the circumscribing rectangle of the person to be tracked represented by the position information (image coordinate system) of the focused entry 2452 is displayed at the center of the image. , the zoom may be adjusted so that the entire circumscribing rectangle is within a predetermined angle of view. The camera 11 changes the imaging range by changing the pan, tilt, and zoom in response to the above command.

Next, the tracking unit 2522 acquires an image captured by the camera 11 set to the changed PTZ value, saves it in the image DB 244-1, and displays it on the screen display unit 23 (step S19). Next, the tracking unit 2522 uses various techniques such as pattern recognition and machine learning to detect the person to be tracked from the image saved this time (step S20), and superimposes the circumscribed rectangle of the detected person on the image. The resulting image is generated and displayed on the screen display unit 23 (step S21). Thereby, the operator can confirm the tracking status of the person specified on the image displayed on the screen display section 23 in real time.

Next, the tracking unit 2522 updates the tracked object movement trajectory information 245 (step S22). In updating the tracking target movement trajectory information 245 in step S22, the tracking unit 2522 first secures one empty entry 2452, and after setting the association information between this secured empty entry 2452 and the focused entry 2452, Attention is shifted to the reserved free entry 2452 . Next, the tracking unit 2522 sets the shooting time, person area, position information (image coordinate system), and image features to the entry 2452 of interest newly, and sets the position information (world coordinate system), the movement vector, and the movement speed. Assume that the acceleration is a NULL value.

Next, the coordinate conversion unit 2523 of the monitoring unit 252 converts the position information (image coordinate system) set in the focused entry 2452 from the image coordinate system to the world coordinate system (step S23). The image corresponding to the interested entry 2452 is an image captured by the camera 11 set to a PTZ value different from that of the reference image. Therefore, as described with reference to FIG. 5, the coordinate conversion unit 2523 first calculates a mapping function for converting the position information (image coordinate system) set in the target entry 2452 into position information on the reference image. Then, using the calculated mapping function, position information (image coordinate system) is transformed into position information on the reference image. Next, the coordinate conversion unit 2523 converts the position information on the reference image obtained by conversion from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Further, in step S23, the coordinate conversion unit 2523 calculates the movement vector, movement speed, and acceleration based on the position information (world coordinate system) of the entry 2452 of interest and the entries 2452 preceding it, and converts the entry 2452 of interest to set.

Next, the tracking unit 2522 of the monitoring unit 252 determines whether or not to end tracking (step S24). For example, the tracking unit 2522 may determine to end tracking when it detects that the person to be tracked is no longer present in the monitoring areas 13A, 13B, and 13C. Alternatively, the tracking unit 2522 may end tracking when it detects that the person to be tracked cannot be tracked by the camera 11 . If the tracking unit 2522 does not determine to end the tracking, it returns to step S18 and repeats the same processing as described above. As a result, the tracking of the tracked person by the camera 11 is continued, and the tracked object movement trajectory information 245 is further updated accordingly.

When the tracking unit 2522 determines to end tracking, it determines whether or not to end monitoring (step S25). For example, the tracking unit 2522 determines to end the monitoring when the operator inputs a monitoring end command through the operation input unit 22 . If it is not determined to end the monitoring, the monitoring unit 252 returns to step S11 and repeats the same processing as described above. When it is determined to end the monitoring, the monitoring unit 252 ends the processing shown in FIG.

FIG. 7 is a flowchart showing an example of tracking processing performed using the camera 12. FIG. The tracking process performed by the control device 20 using the camera 12 will be described below with reference to FIG.

First, the matching unit 2531 in the monitoring unit 253 of the control device 20 initializes (step S31). In this initialization, the matching unit 2531 sets the camera 12 to the reference PTZ value. Further, the collation unit 2531 clears all entries of the collated movement trajectory information 246 in the initialization. Next, the matching unit 2531 acquires an image of the monitoring area 13C captured by the camera 12 set to the reference PTZ value, saves it in the image DB 244-2, and displays it on the screen display unit 23 (step S32). Next, the matching unit 2531 detects all persons from the image saved this time using various techniques such as pattern recognition and machine learning (step S33). Next, the matching unit 2531 displays the detection result on the screen display unit 23 (step S34).

Next, the matching unit 2531 updates the to-be-matched movement trajectory information 246 based on the detection result of the person (step S35). Since all the persons detected from the image of the camera 12 first acquired after the initialization are the persons detected for the first time, the collation unit 2531, in step S35, creates one to-be-identified movement trajectory information for each detected person. 246 are assigned, and the following processing is performed for each of the assigned collated movement trajectory information 246 .

First, the collation unit 2531 sets the person ID assigned to the detected person and the camera ID of the camera 12 in the entry 2461 of the movement trajectory information 246 to be collated. Next, the collating unit 2531 secures one free entry 2462, sets information for correlating the secured free entry 2462 and the entry 2461, and then focuses on the secured free entry 2462. FIG. Next, the matching unit 2531 sets the photographing time, the human area, the position information (image coordinate system), and the image feature in the focused entry 2462, and the position information (world coordinate system), movement vector, movement velocity, and acceleration are set to NULL. value.

Next, the coordinate conversion unit 2533 of the monitoring unit 253 performs the following processing on the focused entry 2462 (step S36). First, the coordinate conversion unit 2533 uses the camera calibration information 442-2 to convert the position information (image coordinate system) set in the focused entry 2462 from the image coordinate system to the world coordinate system. Next, the coordinate conversion unit 2533 sets the position information of the world coordinate system obtained by the above conversion as the position information (world coordinate system) of the entry 2462 of interest.

Next, the collating unit 2531 of the monitoring unit 253 collates the tracked trajectory information 245 with one or more pieces of collated trajectory information 246 (step S37). The collation unit 2531 determines which of the persons photographed by the camera 12 and moving in the monitoring area 13C is the person being tracked by the camera 11 by this collation. Next, the matching unit 2531 determines whether or not the person being tracked by the camera 11 has been successfully detected (step S38). If the detection is not successful, the matching unit 2531 returns to step S32 and repeats the same processing as described above. On the other hand, if the above detection is successful, the matching unit 2531 transmits the person ID of the person determined to be the person being tracked by the camera 11 to the tracking unit 2532 .

The tracking unit 2532 focuses on the last entry 2462 of the to-be-matched movement trajectory information 246 in which the person ID received from the matching unit 2531 is set in the entry 2461, and uses the camera 12 to track the tracking target person. For example, the tracking unit 2532 generates a command for controlling the PTZ value of the camera 12 according to the position information (image coordinate system) set in the focused entry 2462, and sends the command to the camera 12 through the communication I/F unit 21. send to. At this time, for example, the tracking unit 2532 adjusts the pan and tilt so that the center of gravity of the circumscribing rectangle of the person to be tracked represented by the position information (image coordinate system) of the focused entry 2462 is displayed at the center of the image. , the zoom may be adjusted so that the entire circumscribing rectangle is within a predetermined angle of view. The camera 12 changes the imaging range by changing the pan, tilt and zoom in response to the above command. Also, in step S39, the tracking unit 2532 acquires an image captured by the camera 12 after changing the PTZ, and performs processing such as detection of a person to be tracked and monitor display on the screen display unit 23. FIG.

Furthermore, the tracking unit 2532 determines whether or not to end tracking (step S40). For example, the tracking unit 2532 may determine to end tracking when it detects that the person to be tracked no longer exists in the monitoring areas 13C and 13D. Alternatively, the tracking unit 2532 may end tracking when it detects that the person to be tracked cannot be tracked by the camera 12 . If the tracking unit 2532 does not determine to end tracking, it returns to step S38 and repeats the same processing as described above. As a result, the tracking of the person to be tracked by the camera 12 is continued. When the tracking unit 2532 determines to end tracking, it determines whether to end monitoring (step S41). For example, the tracking unit 2532 determines to end the monitoring when the operator inputs a monitoring end command through the operation input unit 22 . If it is not determined to end the monitoring, the monitoring unit 253 returns to step S31 and repeats the same processing as described above. If it is determined to end the monitoring, the monitoring unit 253 ends the processing shown in FIG.

As described above, according to the present embodiment, the tracking unit 2522 captures a moving tracking target person from a plurality of images captured continuously while tracking the camera 11 having a tracking function that captures the moving tracking target person within the shooting range. Position information (image coordinate system) is acquired, and each entry 2452 of the tracked object movement trajectory information 245 is calculated. In addition, the coordinate transformation unit 2523 calculates a mapping function from the image corresponding to each entry 2452 captured by the camera 11 to the reference image, and uses the calculated mapping function to convert the tracked object movement trajectory information 245. The position information (image coordinate system) of each entry 2452 of is converted into position information on the reference image, and the position information on the reference image obtained by this conversion is converted to the image coordinate system using the camera calibration information 242-1. to the world coordinate system. The mapping function can be calculated without manual intervention unlike the camera calibration information. Therefore, according to the present embodiment, it is possible to calculate the movement trajectory information, which is the time series of the position of the person in the real space, while tracking the person with a PTZ camera having a tracking function that captures the moving person within the shooting range. can. As a result, the moving trajectory of a person can be collated in real time between a plurality of PTZ cameras, and the moving object can be automatically collated between the PTZ cameras.

Next, a modified example of this embodiment will be described.

The coordinate transformation unit 2523 may calculate a planar projective transformation matrix from an image obtained by photographing with the camera 11 to a reference image based on the PTZ value of the camera 11 when the image was photographed and the reference PTZ value. good.

A computer may be mounted on the camera 11, and all or part of the functions of the monitoring unit 252 may be mounted on the computer. Similarly, a computer may be installed in the camera 12, and all or part of the functions of the monitoring section 253 may be installed on the computer. Alternatively, the camera 11 and a computer connected to the camera 12 via a network may be configured to include the operation input unit 22, the screen display unit 23, and the storage unit 24. FIG.

The monitoring area may be a place other than aisles, such as a store, factory, station platform, ground, or gymnasium.

The target to be tracked may be a moving object other than a person, such as an animal, a car, or a walking robot.

The cameras that share part of the camera field of view are not limited to the two cameras 11 and 12, and may be three or more.

[Second embodiment]
Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 8, the movement trajectory information processing device 30 according to the present embodiment includes calculation means 31 and conversion means 32 .

The calculating means 31 calculates movement trajectory information, which is a time series of positional information on the images of the object, from a plurality of images of the object continuously photographed by a camera that captures the moving object within the photographing range. is configured to The calculation means 31 can be configured, for example, in the same manner as the tracking section 2522 in FIG. 2, but is not limited thereto.

The conversion means 32 calculates a mapping function from the image to the reference image, calculates position information on the reference image corresponding to the position information of the object on the image using the calculated mapping function, and calculates The position information on the reference image thus obtained is converted into position information on the world coordinate system using the camera calibration information. The conversion means 32 can be configured, for example, in the same manner as the coordinate conversion section 2523 in FIG. 2, but is not limited thereto.

The movement trajectory information processing device 30 configured as described above operates as follows. That is, the calculation means 31 calculates the movement trajectory information, which is the time series of the position information of the object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within the photographing range. do. Next, the conversion means 32 calculates a mapping function from the image to the reference image, and uses the calculated mapping function to calculate position information on the reference image corresponding to the position information on the image of the object. , the calculated positional information on the reference image is converted into positional information in the world coordinate system using the camera calibration information.

According to the movement trajectory information processing apparatus 30 configured and operated as described above, a plurality of images of a moving object continuously photographed by a camera that captures the moving object within the photographing range are used to obtain the real space image of the object. Movement trajectory information, which is a time series of positions, can be calculated. The reason is that the mapping function can be calculated without manual intervention unlike the camera calibration information.

Although the present invention has been described with reference to the above-described embodiments, the present invention is not limited to the above-described embodiments. Various changes can be made to the configuration and details of the present invention within the scope of the present invention that can be understood by those skilled in the art.

For example, the movement trajectory of an object calculated by the present invention may be used for the purposes of personal authentication, abnormal behavior detection, etc. based on the movement trajectory, in addition to monitoring and tracking. Also, the camera used in the present invention does not necessarily have a tracking function. For example, the photographer manually changes the orientation of the camera so that the moving object is captured within the shooting range. The present invention can also be applied when

It can be used for systems that detect and track objects from multiple images that are continuously captured by a camera such as a PTZ camera that captures moving objects within the shooting range.

Some or all of the above embodiments may also be described in the following additional remarks, but are not limited to the following.
[Appendix 1]
Calculation means for calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; conversion means for converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
A movement trajectory information processing device comprising:
[Appendix 2]
The transforming means calculates a planar projective transformation matrix from the image to the reference image, and uses the computed planar projective transformation matrix to compute the mapping function.
The movement trajectory information processing device according to appendix 1.
[Appendix 3]
further comprising collation means for collating the trajectory information with other trajectory information;
3. The movement trajectory information processing device according to appendix 1 or 2.
[Appendix 4]
The other movement trajectory information is movement trajectory information calculated from a plurality of images continuously captured by another camera installed so that a part of the camera field of view overlaps with the camera.
The movement trajectory information processing device according to appendix 3.
[Appendix 5]
wherein the camera is a PTZ camera;
5. The movement trajectory information processing apparatus according to any one of Appendices 1 to 4.
[Appendix 6]
calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
A moving trajectory information processing method.
[Appendix 7]
In the conversion, a planar projective transformation matrix from the image to the reference image is calculated, and the mapping function is calculated using the calculated planar projective transformation matrix.
The movement trajectory information processing method according to appendix 6.
[Appendix 8]
Further, matching the movement trajectory information with other movement trajectory information,
The movement trajectory information processing method according to appendix 6 or 7.
[Appendix 9]
The other movement trajectory information is movement trajectory information calculated from a plurality of images continuously captured by another camera installed so that a part of the camera field of view overlaps with the camera.
The movement trajectory information processing method according to appendix 8.
[Appendix 10]
to the computer,
A process of calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; a process of converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
A computer-readable recording medium that records a program for performing

10 tracking system 11, 12 PTZ cameras 13A to 13D surveillance area 20 control device 21 communication I/F section 22 operation input section 23 screen display section 24 storage section 25 arithmetic processing section 30 movement trajectory information processing device 31 calculation means 32 conversion means 241 Programs 242-1, 242-2 Camera calibration information 243-1, 243-2 Reference images 244-1, 244-2 Image DB
245 tracking target movement trajectory information 246 to-be-verified movement trajectory information 251 camera calibration units 252, 253 monitoring unit 2521 detection units 2522, 2532 tracking units 2523, 2533 coordinate conversion unit 2531 matching unit

Claims (10)

1. Calculation means for calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
   calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; conversion means for converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
   A movement trajectory information processing device comprising:
2. The transforming means calculates a planar projective transformation matrix from the image to the reference image, and uses the computed planar projective transformation matrix to compute the mapping function.
   The moving trajectory information processing apparatus according to claim 1.
3. further comprising collation means for collating the trajectory information with other trajectory information;
   The movement locus information processing device according to claim 1 or 2.
4. The other movement trajectory information is movement trajectory information calculated from a plurality of images continuously captured by another camera installed so that a part of the camera field of view overlaps with the camera.
   The movement trajectory information processing apparatus according to claim 3.
5. wherein the camera is a PTZ camera;
   5. The movement trajectory information processing apparatus according to any one of claims 1 to 4.
6. calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
   calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
   A moving trajectory information processing method.
7. In the conversion, a planar projective transformation matrix from the image to the reference image is calculated, and the mapping function is calculated using the calculated planar projective transformation matrix.
   The movement trajectory information processing method according to claim 6.
8. Further, matching the movement trajectory information with other movement trajectory information,
   The movement trajectory information processing method according to claim 6 or 7.
9. The other movement trajectory information is movement trajectory information calculated from a plurality of images continuously captured by another camera installed so that a part of the camera field of view overlaps with the camera.
   The movement trajectory information processing method according to claim 8.
10. to the computer,
    A process of calculating movement trajectory information, which is a time series of position information of the moving object on the image, from a plurality of images of the object continuously photographed by a camera that captures the moving object within an imaging range;
    calculating a mapping function from the image to a reference image; using the calculated mapping function to calculate position information on the reference image corresponding to position information on the image of the object; a process of converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information;
    A computer-readable recording medium that records a program for performing

Images