JP6679349B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing device, an information processing method, and a program that detect an object from a captured image.

There is a technique of detecting an object in a set space by using an imaging device such as a plurality of cameras. This technique includes, for example, a technique of monitoring an object using a plurality of image capturing devices in a monitored space, a technique of tracking an object using a plurality of image capturing devices in the monitored space, and the like.
A method has been proposed in which an object is detected by a plurality of imaging devices and the attributes of the object such as the position of the object at coordinates (for example, world coordinates) common to all the imaging devices are estimated.
Non-Patent Document 1 discloses an arithmetic operation in which an object is imaged by a camera whose world coordinates are known, the coordinates of the object on the image captured by each camera are obtained, and the world coordinates of the object are obtained by the principle of triangulation. Further, for example, Patent Document 1 discloses a method of obtaining a moving body region with each of a plurality of cameras, and assuming that the height of an object is known, estimating three-dimensional coordinates.
Non-Patent Document 2 discloses a method of obtaining coordinates of a certain object on a captured image by a camera. Non-Patent Document 2 discloses a method for identifying, with respect to every partial image of an image captured by a camera, whether or not the partial image is a human body. At this time, the amount of calculation for obtaining the coordinates of the human body on the image captured by the camera is proportional to the number of partial images, that is, the size of the input image. Patent Document 2 discloses a method of limiting an area in which a partial image is selected from an image in order to reduce the calculation amount.

Japanese Patent No. 5263694 JP, 2007-233517, A

Richard Hartley and Andrew Zisserman (2013), Multiple View Geometry in computer vision Second Edition, CAMBRIDGE UNIVERSITY PRESS Navneet Dal, Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR2005

Conventionally, in the detection of an object using a plurality of image pickup devices, for each of the images picked up by the plurality of image pickup devices, the object detection is performed for the entire range of the image. Therefore, the amount of processing load such as the amount of calculation required to detect an object is proportional to the total number of pixels of the captured images of all cameras. Further, the number of imaging devices used for detecting an object is substantially proportional to the size of the space to be detected. Therefore, the object detection process has a problem that the larger the detection target space, the greater the processing load.
An object of the present invention is to reduce the load of object detection processing.

The information processing apparatus of the present invention displays an image of a two-dimensional map in which the imaging ranges of the plurality of image capturing apparatuses are overlooked in a common coordinate system common to the plurality of image capturing apparatuses, and based on user's operation on the images, Input means for inputting an area in which an object first appears in a two-dimensional map, and imaging for acquiring an image in which the object should be detected for the first time among the plurality of imaging devices based on the input area in which the object first appears. A device, an initial region determining means for determining a region in the image where the object should be detected for the first time, and an image captured by the image capturing device determined by the initial region determining means, the image being determined by the initial region determining means. First detecting means for detecting the object from the area where the object should be detected for the first time, Serial based on the object captured image is detected, the acquisition means for acquiring the coordinates representing the position of the object in the common coordinate system, on the basis of the coordinates, the period set by the current point the future course prediction means for predicting a prediction value of the coordinates representing the position of the object in the common coordinate system at the time, and each of the imageable range of said plurality of imaging devices, based on said predicted value, the plurality of imaging Estimating means for deciding an image pickup device capable of picking up the object at the future time point of the apparatus, and estimating an area in which the object can be picked up in the decided image pickup device; and the estimating means at the future time point Detecting the object from the estimated region in the image captured by the imaging device determined by A second detecting means.

  According to the present invention, the load of object detection processing can be reduced.

It is a figure which shows an example of a system configuration etc. of a monitoring system. It is a figure showing an example of hardware constitutions etc. of an information processor. FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. It is a flow chart which shows an example of object tracking processing. It is a figure showing an example of a picked-up image. It is a figure explaining the tracking of a pedestrian. It is a figure showing an example of a picked-up image. It is a figure explaining the area | region predicted that an object exists. It is a figure which shows an example of a designation screen. It is a figure explaining the area where an object can appear.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
In the present embodiment, a process will be described in which the monitoring system uses a plurality of cameras having overlapping fields of view to track an object such as a person moving in the monitoring space. The monitoring system tracks an object by detecting the object from the captured image of each camera, acquiring the coordinates of the object in the world coordinate system, and repeating these to obtain the trajectory of the object. The world coordinate system is an example of a common coordinate system that is common to all cameras, and is a three-dimensional coordinate system in which one axis is perpendicular to the bottom surface and the remaining two axes are parallel to the bottom surface. It is a system.
In this embodiment, it is assumed that the bottom surface in the monitoring space is a flat surface for simplification of description. Further, in the present embodiment, the object to be tracked by the monitoring system is assumed to be a pedestrian (pedestrian), but the object is not limited to a pedestrian, and a person who stops and does some work, a person who runs, a wheelchair. People such as dogs, cats, cows, pigs, chickens, and drones.

FIG. 1 is a diagram illustrating an example of a system configuration and the like of a monitoring system. The monitoring system of this embodiment includes a plurality of cameras such as the information processing device 341 and the cameras 311 to 313. A plurality of cameras such as the information processing device 341 and the cameras 311 to 313 are connected to each other via a network 331. The information processing device 341 acquires the captured image data from each of the plurality of cameras via the network 331.
The information processing device 341 is an information processing device such as a PC or a server device that acquires captured images from a plurality of cameras in the surveillance system, detects an object, and tracks the detected object.
Each of the plurality of cameras in the monitoring system such as the cameras 311 to 313 is a camera that can be connected to a network, and transmits captured image data to an external device such as the information processing device 341 via the network. It is a camera that can. In the present embodiment, the plurality of cameras in the surveillance system have common time information, and the information at the time when the captured image is captured is associated with the information of the captured image and transmitted to the information processing device 341. . The information processing device 341 can confirm when the captured image from each camera is the captured image by confirming the time point information associated with the captured image transmitted from each camera. Further, when the time point information linked to the captured images transmitted from different cameras respectively indicate the same time point, the information processing device 341 can recognize that the images are images of the monitoring space 301 at the same time point.

FIG. 1 shows an installation situation of a plurality of cameras in a surveillance system, a surveillance space 301 which is a surveillance target space, and the like. The monitoring space 301 is a space monitored by the monitoring system, and is shown as a space surrounded by a dotted line in the example of FIG. 1. The plurality of cameras in the surveillance system are installed such that a region in which the fields of view of the cameras are combined includes the surveillance space 301 and the fields of view of the cameras overlap. Further, the monitoring space 301 may be a region in which the imaging ranges of a plurality of cameras in the monitoring system are all combined.
Pedestrians who come in and out of the surveillance space 301 always pass through the area where they can come in and out of the surveillance space 301. The new object appearance space 302, the new object appearance space 303, and the new object appearance space 304 are spaces set in areas that can enter and leave the monitoring space 301, and are surrounded by solid lines in the monitoring space 301 in the example of FIG. 1. It is shown as an open space.
A plurality of cameras in the surveillance system are installed so that their fields of view overlap so as to monitor the surveillance space 301. The field of view of the camera 311 is represented as a fan-shaped region centered on the camera 311 and is shown as a broken-line fan-shaped region 321 in the example of FIG. 1. Similarly, the fields of view of the cameras 312 and 313 are shown as a region 322 and a region 323, respectively.
In the present embodiment, the monitoring system tracks the pedestrian 351 by obtaining the trajectory of the coordinates of the pedestrian 351 moving in the monitoring space 301.

FIG. 2 is a diagram illustrating an example of the hardware configuration of the information processing device 341. The information processing device 341 includes a communication unit 201, a CPU 202, a RAM 203, an auxiliary storage device 204, and a user interface 205. The communication unit 201, CPU 202, RAM 203, auxiliary storage device 204, and user interface 205 are connected to each other via a system bus 206.
The communication unit 201 receives data of captured images and the like from a plurality of cameras in the monitoring system such as the cameras 311 to 313 via the network 331. The communication unit 201 can also send instructions such as an imaging instruction and a stop instruction to each of the plurality of cameras.
The CPU 202 is a central processing unit that executes a program such as a control program stored in the RAM 203, the auxiliary storage device 204, etc., and controls the information processing device 341.
The RAM 203 is a storage device that functions as a work area of the CPU 202 or a temporary save area for data.

The auxiliary storage device 204 is a storage device that stores programs such as control programs, various parameter data, images, various setting information, and the like.
The user interface 205 is an interface used to connect an input device or an output device such as a keyboard, a mouse, a display, and a touch panel that displays information to the user and inputs information from the user. The user interface 205 is connected with an input device and an output device such as a keyboard, a mouse, a display and a touch panel.
In the present embodiment, the CPU 202 executes the process based on the program stored in the RAM 203 or the auxiliary storage device 204, thereby realizing the function of the information processing device 341 described later in FIG. 3 and the process of the flowchart described later in FIG. To be done.

FIG. 3 is a diagram illustrating an example of a functional configuration of the information processing device 341. The information processing device 341 includes an image acquisition unit 101, an attribute acquisition unit 102, an attribute update unit 103, a common attribute estimation unit 104, a common attribute prediction unit 105, a device determination unit 106, a region estimation unit 107, a space input unit 110, and a region determination. Part 111.
The image acquisition unit 101 acquires captured images from a plurality of cameras in the surveillance system such as the cameras 311 to 313.
The attribute acquisition unit 102 detects an object that newly appears in the monitoring space 301 from the captured image acquired by the image acquisition unit 101. In the following, an object that newly appears in the monitoring space 301 is a new object. Then, the attribute acquisition unit 102 acquires the coordinates of the detected object in the captured image in the coordinate system. Coordinates in the coordinate system in the captured image are an example of attributes unique to each camera in the surveillance system.
The common attribute estimation unit 104, based on the coordinates of the tracking target object in the coordinate system in the captured image of each camera acquired by the attribute acquisition unit 102 or the attribute update unit 103 described later, coordinates of the object in the world coordinate system. To estimate. Coordinates in the world coordinate system are an example of common attributes that are attributes common to all cameras in the surveillance system. The attribute value of the object for the common attribute is the common attribute value. That is, the coordinate of the object in the world coordinate system is an example of the common attribute value of the object.

The common attribute prediction unit 105, based on the coordinates in the world coordinate system of the tracking target object estimated by the common attribute estimation unit 104, the coordinates in the world coordinate system of the object at the time when the set period elapses from the current time. Predict. Further, the common attribute prediction unit 105 performs the above prediction process based on the coordinates of the object in the world coordinate system estimated by the common attribute estimation unit 104 and the past coordinates of the object in the world coordinate system. Good. That is, the common attribute prediction unit 105, based on the estimated current coordinates and the past coordinates in the world coordinate system of the object, the world coordinate system of the object at the time when the set period elapses from the current time. The coordinates at may be predicted.
The device determination unit 106 estimates a camera that can image the object among the plurality of cameras in the monitoring system based on the coordinates in the world coordinate system of the tracking target object predicted by the common attribute prediction unit 105. The camera estimated to be able to image the object by the device determination unit 106 performs another imaging, and the object is detected from the captured image.
The area estimation unit 107, for each of the cameras estimated to be able to image the object by the device determination unit 106, based on the coordinates in the world coordinate system of the tracking target object predicted by the common attribute prediction unit 105, Perform processing. That is, the area estimation unit 107 estimates the area in which the object can appear in the captured image. Hereinafter, the area estimated by the area estimation unit 107 will be referred to as a tracking area.

The attribute update unit 103 detects an object to be tracked from the tracking regions estimated by the region estimation unit 107 in the captured image from the camera estimated to be able to capture the object by the device determination unit 106, and The coordinates in the captured image of are acquired.
The space input unit 110 determines a new object appearance space, which is a space through which an object necessarily passes in order to enter the monitoring space, based on a user's operation via the input / output device and the user interface 205. The user performs an operation of inputting a wall, camera installation coordinates and related parameters, a space in which an object newly appears, and the like in the monitoring space through the input / output device and the user interface 205.
The area determination unit 111 detects a new object that has newly appeared in the monitoring space based on the new object appearance space determined by the space input unit 110, and the new object appears on the image captured by the camera. Determine the potential area.
The element group 121 in FIG. 3 is a functional component group that loops and repeats the processing in the present embodiment.

FIG. 4 is a flowchart showing an example of the object tracking process in this embodiment. The process of FIG. 4 will be described by taking as an example a situation where a pedestrian 351 enters the monitoring space 301 from the outside and moves in the monitoring space 301.
In step S401, the image acquisition unit 101 acquires captured images captured at the same time from a plurality of cameras in the surveillance system such as the cameras 311 to 313. In addition, the image acquisition unit 101 is imaged by a plurality of cameras in the monitoring system such as the cameras 311 to 313 within a set period (for example, a period having a width of 0.1 seconds around a certain time point). The captured image may be acquired.

In S402, the attribute acquisition unit 102 detects a pedestrian 351 that has newly appeared in the monitoring space 301 from the captured image acquired in S401, and acquires the coordinates of the pedestrian 351 in the captured image.
In the present embodiment, the attribute acquisition unit 102 performs a process of detecting a new object newly appearing in the monitoring space 301 from a captured image acquired from a preset camera. The attribute acquisition unit 102 may perform a process of newly detecting a new object appearing in the monitoring space 301 from captured images acquired from all the cameras in the monitoring system.
In addition, the attribute acquisition unit 102 performs a process of newly detecting a new object appearing in the monitoring space 301 from a preset area in the captured image of the camera. This preset area is an area in which a new object can appear in the captured image, and is hereinafter referred to as a new object attribute acquisition area. The new object attribute acquisition area corresponds to the area in which the above-described new object appearance space appears in the captured image of the camera. Further, the attribute acquisition unit 102 may perform a process of newly detecting an object appearing in the monitoring space 301 from the entire area within the captured image of the camera.
In the example of FIG. 1, the cameras showing the new object appearance space 302 are the camera 311 and the camera 312. FIG. 5 shows an example of a captured image captured by the camera 311, the camera 312, and the camera 313.

FIG. 5 is a diagram showing an example of images captured by the cameras 311, 312 and 313. The captured image 411, the captured image 412, and the captured image 413 in FIG. 5 are examples of captured images of the camera 311, the camera 312, and the camera 313, respectively. The new object attribute acquisition areas 421 and 422 are new object attribute acquisition areas on the captured images 411 and 412, respectively, and are shown as areas surrounded by broken lines. When the pedestrian 351 who has entered the monitoring space 301 is present in the new object appearance space 302, it is reflected in the new object attribute acquisition areas 421 and 422.
The attribute acquisition unit 102 detects the pedestrian 351 from the new object attribute acquisition area 421 and acquires the coordinates of the pedestrian 351 in the captured image 411. The attribute acquisition unit 102 also detects the pedestrian 351 from the new object attribute acquisition area 422 and acquires the coordinates of the pedestrian 351 in the captured image 412.
In addition, since the camera 313 does not show any of the new object appearance spaces 302 to 304, no new object appears in the captured image 413. Therefore, the attribute acquisition unit 102 does not perform the process of detecting the pedestrian 351 in the captured image 413.
As a method for the attribute acquisition unit 102 to acquire the coordinates of the pedestrian 351 from the image captured by the camera, for example, regarding a partial image in the image captured by the camera described in Non-Patent Document 2, whether the partial image is a human body or not There is a method to identify that. The attribute acquisition unit 102 will select a partial image for identifying whether or not it is a human body (pedestrian 351) from within the range of the new object attribute acquisition region. In the present embodiment, the coordinates on the captured image of the pedestrian 351 are the coordinates of the center when the head of the pedestrian 351 is approximated by a circle.

In the conventional technique, the amount of calculation for detecting an object newly appearing in the monitoring space 301 in obtaining the trajectory of the object moving in the monitoring space 301 is proportional to the area of the monitoring space 301. In the present embodiment, the calculation amount for the attribute acquisition unit 102 to detect an object that newly appears in the monitoring space 301 is a value proportional to the length of the width of the area that the object can pass through when entering or leaving the monitoring space 301. Which is reduced compared to the prior art.
This is because the attribute acquisition unit 102 performs the detection process only on the new object attribute acquisition area, and the size of the new object attribute acquisition area is proportional to the size of the new object appearance space. The new object appearance space is set so that an object newly appearing in the monitoring space 301 always passes through the new object appearance space. That is, the size of the new object appearance space is proportional to the length of the width of the area through which the object can pass when entering and leaving the monitoring space 301. In the example of FIG. 1, the sum of the areas of the new object appearance space 302, the new object appearance space 303, and the new object appearance space 304 is proportional to the length of the width of the area through which the object can pass when entering and leaving the monitoring space 301. To do.

In S403, the common attribute estimation unit 104 estimates the coordinates 362 of the pedestrian 351 in the world coordinate system based on the coordinates in the captured image of the pedestrian 351 acquired in S402 or S408. The coordinate of the pedestrian 351 in the world coordinate system estimated in S403 is an example of the common attribute value of the pedestrian 351 for the common attribute.
As a method for obtaining the coordinates of the pedestrian 351 in the world coordinate system from the coordinates of the pedestrian 351 in the coordinate systems of the images captured by the plurality of cameras, triangulation based on epipolar geometry shown in Non-Patent Document 1 is used. There is a way to do it. It is assumed that the information on the coordinates of each camera in the world coordinate system, the inclination of the optical axis, the angle of view, and the like are known because of the fact that they were measured when the cameras were installed. In the present embodiment, the coordinates of the pedestrian 351 in the world coordinate system are assumed to be the coordinates of the center of the sphere when the head of the pedestrian 351 is approximated by a sphere.
In step S <b> 404, the common attribute estimation unit 104 registers the information on the coordinates of the pedestrian 351 in the world coordinate system estimated in step S <b> 403 in the information on the trajectory of the pedestrian 351. The common attribute estimation unit 104 generates information about the trajectory of the pedestrian 351 by storing the information about the coordinates of the pedestrian 351 in the world coordinate system estimated at S403 in chronological order every time the process proceeds to S404. Of the information about the trajectory of the pedestrian 351, information other than the information about the coordinates of the pedestrian 351 added in the latest process of S404 is information about the coordinates of the pedestrian 351 in the past in the world coordinate system. That is, among the information on the trajectory of the pedestrian 351, the information other than the information on the coordinates of the pedestrian 351 added in the latest process of S404 is an example of the past attribute value of the pedestrian 351 for the common attribute.

In S405, the common attribute prediction unit 105, based on the information of the trajectory of the pedestrian 351 in the world coordinate system, the coordinates in the world coordinate system in which the pedestrian 351 may exist at the time when the set period has elapsed from the current time. Predict the range of. The set period is, for example, a period from the time when a certain captured image is captured by the plurality of cameras in the monitoring system to the time when the next captured image is captured. The range of coordinates in the world coordinate system predicted by the common attribute prediction unit 105 in S405 is an example of a predicted value that is a predicted value of the common attribute value of the pedestrian 351 for the common attribute.
When only the coordinate information registered in S404 is registered in the trajectory information of the pedestrian 351, the common attribute prediction unit 105 performs the following processing. That is, the common attribute prediction unit 105, based on the information of the coordinates of the pedestrian 351 in the world coordinate system estimated in S403, the pedestrian 351 in the world coordinate system at the time when the set period has elapsed from the current time. Would predict the range of coordinates in which may exist.
In addition, the common attribute prediction unit 105 performs the following process when the past coordinate information of the pedestrian 351 is also registered in the trajectory information of the pedestrian 351 in addition to the coordinate information registered in S404. become. That is, the common attribute prediction unit 105, based on the information of the coordinates of the pedestrian 351 estimated in S403 and the information of the past coordinates of the pedestrian 351, the time when the set period has elapsed from the current time. Will predict the range of coordinates in which the pedestrian 351 can exist in the world coordinate system.

For example, in the example of FIG. 1, the common attribute prediction unit 105 calculates the coordinates 363 and the prediction error 371 based on the information on the trajectory of the pedestrian 351 including the coordinates 362. That is, the common attribute prediction unit 105 predicts that the pedestrian 351 at the time when the set period has elapsed from the current time exists in the circular area having the radius of the prediction error 371 with the coordinate 363 as the center. The coordinate 363 is a value predicted as the coordinate of the pedestrian 351 at the time when the set period has elapsed from the current time. The prediction error 371 is an index indicating an error from the value predicted as the coordinate of the pedestrian 351. To obtain the range of coordinates in the world coordinate system in which the pedestrian 351 can exist from the information of the trajectory of the pedestrian 351 at the time when the set period has elapsed from the current time, the information of the trajectory of the pedestrian 351 is used. There is a method using linear regression based on the above.
In addition, when the pedestrian 351 is newly appearing in the new object appearance space and there is no information on the past coordinates of the pedestrian 351 or when the number of information on the past coordinates of the pedestrian 351 uses linear regression. If the value is not enough to do, the following method is used. That is, the world coordinate system in which the pedestrian 351 can exist at the time when the set period elapses from the present time within the range in which the pedestrian 351 can move in the set period around the coordinates estimated in S403. The method of obtaining the range of the coordinates in is used. For example, the common attribute prediction unit 105 calculates the distance by multiplying the speed of the pedestrian 351 set in advance by the set period. Then, the common attribute prediction unit 105 determines that the pedestrian 351 exists in the circular range having the radius of the calculated distance centered on the coordinates estimated in S403 at the time when the set period has elapsed from the current time. The range of coordinates in the world coordinate system to be obtained.

In step S <b> 406, the device determination unit 106 uses the imaged image from which of the imaged images from the plurality of cameras of the monitoring system to detect the pedestrian 351 based on the range in the world coordinate system predicted in step S <b> 405. Decide what to do. Further, the region estimation unit 107 uses which region of the captured image from the camera determined by the device determination unit 106 based on the range in the world coordinate system predicted in S405 to detect the pedestrian 351. Decide
For example, the device determination unit 106 selects, from a plurality of cameras in the monitoring system, a camera that captures an image of the pedestrian 351 existing in the range in the world coordinate system predicted in S405 as an area having a set number of pixels or more. . Then, the device determination unit 106 determines the selected camera as a camera that captures a captured image used for detecting the pedestrian 351. Further, the device determination unit 106 determines, among the plurality of cameras in the monitoring system, a camera that includes the range in the world coordinate system predicted in S405 in the field of view as a camera whose captured image is used for detecting the pedestrian 351. It may be done.

It is assumed that the auxiliary storage device 204 stores information of an area in the world coordinate system that is set in advance for each camera and that can image an object to be detected as an area having a larger number of pixels than the set number of pixels. Hereinafter, this area will be referred to as a trackable area. That is, the trackable area is an area obtained by projecting, on the bottom surface of the world coordinate system, an area in the visual field of a camera in which the object to be detected appears larger than the preset number of pixels. In the present embodiment, the trackable area is assumed to be the same as the area of the field of view of a certain camera in which the object to be detected is projected onto the bottom surface of the world coordinate system, that is, the area of the field of view.
The device determination unit 106 acquires, from the auxiliary storage device 204, information on the trackable area preset for each camera. If the coordinates in the world coordinate system of the movement destination of the object among the plurality of cameras in the monitoring system are inside the trackable area of a certain camera, it can be determined that the camera can track the object. In the example of FIG. 1, since it is estimated that the pedestrian 351 moves to the vicinity of the coordinate 363, the device determination unit 106 determines that the camera 311 and the camera 313 that include the coordinate 363 in the trackable area have the captured image of the pedestrian 351. Determined as the camera used for detection.

For example, the device determination unit 106 uses the even odd rule method to determine whether the coordinates in the world coordinate system are inside or outside the trackable area. The even odd rule method assumes a half line connecting the coordinates in the world coordinate system and a point at some infinity, and is based on the number of times this half line intersects with the fan-shaped arc and two radii of the trackable area. It is a method of determining whether the coordinates are inside or outside the trackable area. If the number of times the assumed half line and the sector-shaped arc and two radii of the trackable area intersect is odd, it is determined as the inside of the sector, and if 0 or even, it is determined as the outside of the sector.
FIG. 6 is a diagram illustrating tracking of a pedestrian 351. In the example of FIG. 7, when it is predicted that the pedestrian 351 moves to the coordinate 363 in S405, the number of times the half line 601 extended from the coordinate 363 and the area 323 which is the trackable area of the camera 313 intersect with each other is 1 Times or odd. Therefore, the device determination unit 106 determines that the coordinate 363 exists inside the area 323 that is the trackable area, and determines that the camera 313 is a camera that can track the pedestrian 351. Since the number of intersections of the half line 601 and the trackable area 621 of the camera 611 is twice, that is, an even number, the device determination unit 106 determines that the camera 611 is a camera that cannot track the pedestrian 351.

The area estimation unit 107, for each of the cameras determined by the device determination unit 106 as a camera that captures the captured image used for detecting the pedestrian 351, is a tracking region that is an area that the pedestrian 351 can appear on the captured image. To estimate.
FIG. 7 is a diagram showing an example of a captured image. The captured image 513 in FIG. 7 is an image captured in the next imaging process performed by the camera 313. The area estimation unit 107 estimates the tracking area of the pedestrian 351 in the captured image 513 like the tracking area 531.
The range in which the pedestrian 351 can exist in the world coordinate system predicted in S405 is spherical with a prediction error of radius. However, spheres are difficult to transform in perspective projection. Therefore, in this embodiment, the region estimation unit 107 approximates the spherical range predicted in S405 with a polyhedron. In the present embodiment, the region estimation unit 107 approximates this spherical range with a rectangular parallelepiped, finds points on the image captured by the camera at each vertex of the rectangular parallelepiped, and obtains the convex hull of this point group as the tracking region. . In the present embodiment, this rectangular parallelepiped is in contact with the floor surface, and its height is set to a value higher than that of the head of the pedestrian 351 being tracked, and the length of a horizontal piece of the rectangular parallelepiped is set to twice the prediction error. It is a figure. FIG. 8 is a diagram illustrating a region in which an object is predicted to exist. The figure in FIG. 8 shows this rectangular parallelepiped whose center is the coordinates 363. The area estimation unit 107 obtains a vertex group on the captured image 513 shown in FIG. 7 by performing perspective projection transformation on each of the vertices of this rectangular parallelepiped, and convexly wraps the vertex group to make a tracking area 531. To get.
A method of obtaining the coordinates of a point in the coordinate system of an image captured by a camera corresponding to a point in a certain world coordinate system is perspective projection conversion according to the pinhole camera model described in Non-Patent Document 1. According to Non-Patent Document 1, coordinates m (u, v) obtained by perspective projection of an object having world coordinates M (x, y, z) on a captured image are obtained by the following Expression 1. However, information on the camera internal parameter matrix A, the camera external parameter matrix [R | t], and the scale s for perspective projection of the world coordinate system onto the coordinate system on the captured image is required for each camera in advance. In the present embodiment, the auxiliary storage device 204 is assumed to store such information in advance.

sm = A [R | t] M (Formula 1)
However, m = [[u] [v] [1]]
A = [[f_x 0 c_x] [0 f_y c_y] [c_x c_y 1]]
[R | t] = [[r11 r12 r13 t1] [r21 r22 r23 t2] [r31 r32 r33 t3]]
M = [[x] [y] [z] [1]]

Further, the device determination unit 106 may not determine which camera, of the captured images from the plurality of cameras of the monitoring system, is used to detect the pedestrian 351. In that case, the region estimation unit 107, based on the range in the world coordinate system predicted in S405, which region of the imaged images from all the cameras in the monitoring system is used to detect the pedestrian 351. May be determined. In this case, the area estimation unit 107 determines not to use the entire area of the captured image for detecting an object for a camera that does not include the range predicted in S405 in the trackable area.
Information about which camera is used to detect the pedestrian 351 among the images captured from the plurality of cameras of the monitoring system determined by the device determination unit 106 is used to detect an object such as the pedestrian 351. It is an example of a detection parameter which is a parameter used. In addition, information indicating which region of the image captured by the camera, which is determined by the region estimation unit 107, is used for detecting the pedestrian 351 is also an example of the detection parameter. In addition to the detection parameters, information indicating which detector is used to detect an object, information indicating which size of a partial image is extracted from a captured image from a camera for detecting an object, etc. There is.
The process of the device determination unit 106 and the region estimation unit 107 in S406 is an example of the process of determining the detection parameter based on the predicted value of the common attribute predicted in S405. In particular, the process of the area estimation unit 107 in S406 acquires the predicted value of the attribute for each captured image of each camera from the predicted value of the common attribute predicted in S405, and uses the acquired predicted value for the attribute for each captured image. It is an example of a process of determining a detection parameter based on the above. The device determination unit 106 and the region estimation unit 107 are an example of a first determination unit that determines a detection parameter used for detecting an object based on the predicted value of the common attribute of the object.

In step S407, the image acquisition unit 101 acquires a captured image group from a plurality of cameras in the surveillance system. In this embodiment, the image acquisition unit 101 acquires, from a plurality of cameras in the monitoring system, a captured image captured by each camera next to the captured image acquired in the immediately preceding process of S401 or S407. In addition, the image acquisition unit 101 acquires, from a plurality of cameras in the surveillance system, a captured image captured by each camera when a set period has elapsed since the processing of S401 or S407 was performed immediately before. It may be that. For example, the image acquisition unit 101 acquires the captured image 513 shown in FIG. 7 from the camera 313 when the pedestrian 351 has moved within the circular range of the prediction error 371 with the radius centered on the coordinate 363.
In step S <b> 408, the attribute update unit 103 performs the following processing based on the captured image from the camera determined as the camera that captures the captured image used for detecting the pedestrian 351 by the device determination unit 106 in step S <b> 406. That is, the attribute updating unit 103 detects the pedestrian 351 from the tracking area determined by the area estimating unit 107 in S406, and acquires the coordinates in the coordinate system of the captured image of the detected pedestrian 351. For example, when the captured image 513 is obtained, the attribute updating unit 103 detects the pedestrian 351 from inside the tracking area 531 and obtains the coordinates of the detected pedestrian 351. The method of detecting the pedestrian 351 and the method of acquiring the coordinates of the pedestrian 351 are the same as the processing in S402. The attribute update unit 103 is an example of a first detection unit that detects an object based on the detection parameter determined in S406.
In S409, the attribute update unit 103 determines whether or not the pedestrian 351 is detected in S408. When the attribute update unit 103 determines in S408 that the pedestrian 351 is detected, the attribute updating unit 103 determines that the tracking of the pedestrian 351 is continued, and proceeds to the process of S403. If the attribute updating unit 103 determines in S408 that the pedestrian 351 has not been detected, the attribute updating unit 103 determines that tracking has ended because the pedestrian 351 has gone out of the monitoring space 301, and ends the processing in FIG.

As described above, by the processing of this embodiment, the monitoring system estimates the coordinates of the pedestrian 351 in the world coordinate system from the coordinates of the pedestrian 351 for each of the captured images of the plurality of cameras. Then, the monitoring system predicts, on the basis of the acquired coordinates of the pedestrian 351 in the world coordinate system, the coordinates of the pedestrian 351 in the world coordinate system at the time when the set period has elapsed from the current time. Alternatively, the monitoring system, based on the coordinates of the pedestrian 351 in the world coordinate system and the past coordinates of the pedestrian 351 in the world coordinate system, the world of the pedestrian 351 at the time when the set period has elapsed from the current time. Predict the coordinates in the coordinate system. The monitoring system determines a camera for capturing a captured image for detecting the pedestrian 351 and an area for detecting the pedestrian 351 in the captured image based on the predicted coordinates. Then, the monitoring system detects the pedestrian 351 in the determined area in the captured image of the determined camera, and tracks the pedestrian 351.
As a result, the monitoring system only needs to detect the pedestrian 351 only in the region where the pedestrian 351 can exist in the image captured by the camera that can image the pedestrian 351. That is, the monitoring system does not have to perform the process of detecting the pedestrian 351 in the entire area of the captured images of all the cameras, and the burden of detecting the pedestrian 351 can be reduced.

Moreover, the monitoring system can track the pedestrian 351 as long as the movement of the pedestrian 351 is within the range predicted by the common attribute prediction unit 105. Further, in the monitoring system, when the attribute acquisition unit 102 detects an object newly appearing in the new object attribute acquisition area in S402 and the object is detected again in S408, the monitoring of the object is performed as follows. You can get the speed. That is, the monitoring system obtains the coordinate of the object in the world coordinate system when first detected and the coordinate of the object in the world coordinate system when detected next, and the difference between the coordinate and the time when the difference is detected. The speed can be calculated from the deviation of the speed. Further, according to the processes of the common attribute prediction unit 105 and the region estimation unit 107, the tracking region of the pedestrian 351 is obtained based on the assumed maximum speed. That is, as long as the speed of the pedestrian 351 is lower than the assumed maximum speed, the attribute update unit 103 can detect the pedestrian 351 because the pedestrian 351 exists in the tracking area on the captured image.
In addition, when the common attribute prediction unit 105 predicts the range in which the pedestrian 351 exists by using linear regression based on the information on the trajectory of the pedestrian 351, the wider the monitoring space 301, the more the attribute update unit 103 operates. The amount of calculation will be reduced. This is because the reliability of the linear regression increases as the information of the coordinates of the pedestrian passing in the information of the locus of the pedestrian 351 increases, and the common attribute prediction unit 105 predicts the estimation error of the range in which the pedestrian 351 exists. As, the error of linear regression can be used. The error of the linear regression becomes smaller as the information of the coordinates included in the information of the trajectory of the pedestrian 351 increases, because the reliability of the regression increases. Therefore, the area estimation unit 107 reduces the tracking area according to the estimation error. As a result, the area to be processed by the attribute updating unit 103 becomes smaller, and the processing load on the attribute updating unit 103 is reduced. As a result, the larger the monitoring space 301 is, the smaller the calculation amount of the tracking process of the attribute updating unit 103 is.

In the present embodiment, it is assumed that the CPU 202 executes the program in the RAM 203 or the auxiliary storage device 204 to implement the processing of the flowchart of FIG. However, some or all of the processing of the flowchart of FIG. 4 may be realized by hardware such as an electronic circuit.
Further, a part or all of the processing of the flowchart of FIG. 4 of the present embodiment may be executed by the camera in the surveillance system including the image sensor. In that case, each of the cameras in the surveillance system includes a CPU and a storage device that stores a program or the like for executing a part or all of the processing of the flowchart of FIG. The CPU of the camera in the surveillance system executes the program stored in the storage device of the camera, so that the function of the camera and part or all of the processing of the flowchart of FIG. 4 are realized.

<Embodiment 2>
In this embodiment, the processing of the space input unit 110 and the area determination unit 111 will be described. The space input unit 110 inputs a new object appearance space. Then, the area determination unit 111 performs the following processing based on the designation of the new object appearance space input by the space input unit 110. That is, the area determination unit 111 determines from which camera the captured image for performing the detection processing of the object newly appearing in the monitoring space 301 in S402 is the captured image. Further, the area determination unit 111 determines the area in the captured image in which the object detection processing is performed in S402. That is, the space input unit 110 and the region determination unit 111 newly determine the detection parameter used when detecting an object that can appear in the surveillance space. The processing of the space input unit 110 and the area determination unit 111 of this embodiment is performed before the processing of FIG.
The system configuration of the monitoring system of this embodiment and the hardware configuration and functional configuration of system components are the same as in the first embodiment.

The space input unit 110 inputs a new object appearance space through which an object always enters to enter the monitoring space 301.
The space input unit 110 displays a designation screen used for designating a new object appearance space on the display unit via the user interface 205. Then, the space input unit 110 receives the designation of the new object appearance space based on the designation screen and the user's operation via the input device. For example, the space input unit 110 displays a plan view of the monitoring space 301 on the display unit as a designation screen. The user specifies the range in which the new object appearance space is to be set in the monitoring space 301 displayed on the specification screen, for example, by dragging with a mouse. In addition, the user selects a range to be determined as a new object appearance space from the ranges displayed as candidates for the new object appearance space in the monitoring space 301 displayed on the designated screen by, for example, a click process using a mouse. It may be that. The space input unit 110 receives the information of the range in which the specified new object appearance space is desired to be set based on the user's operation, and inputs the information of the new object appearance space.
In addition, the space input unit 110 may display, through the user interface 205, not only the new object appearance space but also a designation screen used for designation of walls, camera installation coordinates, related parameters, and the like on the display unit. Good. For example, the space input unit 110 displays a designated screen including a screen overlooking the world coordinate system and a screen for setting related parameters. The user specifies the coordinates by placing icons such as walls and cameras in the coordinate system displayed in the specified screen, and specifies the desired parameters on the related parameter setting screen to set the related parameters. specify. The space input unit 110 receives the designated information via the designation screen.

FIG. 9 is a diagram showing an example of a designation screen displayed by the space input unit 110. An example of the process of designating the new object appearance space based on the user's operation through the designation screen displayed by the space input unit 110 will be described with reference to FIG. In the designation screen of FIG. 9, solid lines, broken lines, dotted lines, etc. are drawn. The solid lines include relatively thick lines (thick solid lines below) and relatively thin solid lines (thin solid lines below). In the example of FIG. 9, the designated screen has a thick solid line indicating a wall, a rectangle indicating a camera, a region surrounded by a thin solid line indicating a space where an object may newly appear, and a dotted line indicating a space where the camera can track the object. Is displayed. The designation screen of FIG. 9 shows an area in which the surveillance space 301, the new object appearance space, and the space in which an object for each camera can be detected are all viewed from above and projected onto the bottom surface. Also, a wall, a new object appearance space, and the like are areas projected on the bottom surface. That is, the designation screen of FIG. 9 is a screen in which the monitoring space in the world coordinate system is represented as a two-dimensional map which is a planar map.
The space input unit 110 estimates a trackable area for each camera and displays the area on a designated screen to present it to the user. For example, when the user designates the installation of the camera 311 in the surveillance space 301 via the designation screen, the region 321 that is the trackable region is estimated and presented to the user. The trackable area has a fan shape. The central angle of the fan-shaped trackable area is the angle of view of the camera. The radius of the sector is the distance at which the object to be tracked is reflected by a certain number of pixels or more, and this number of pixels is the minimum number of pixels by which the attribute updating unit 103 can detect the object.
In the present embodiment, the space input unit 110 determines the radius of the sector of the trackable area based on the characteristics of the attribute update unit 103. As a result, the device determination unit 106 can determine a camera that can track an object as a camera that captures a captured image for performing the detection process, instead of a camera that simply captures the object. As a result, the space input unit 110 can reduce the amount of calculation required for the process of tracking an object that cannot be detected by the attribute update unit 103.

The area determination unit 111 estimates a camera capable of detecting an object newly appearing in the monitoring space 301 and a new object attribute acquisition area on the camera image based on the new object appearance space input by the space input unit 110. To do. For example, the area determination unit 111 obtains the cameras 311 and 312 that include the new object appearance space 302 input on the designation screen of FIG. 9 in the trackable area. Further, the area determination unit 111 estimates the new object attribute acquisition area 421 on the captured image 411 of the camera 311 and the new object attribute acquisition area 422 on the captured image 412 of the camera 312, as shown in FIG.
The area determination unit 111 performs the following processing, for example, in order to obtain a camera that can detect a new object. It is assumed that the auxiliary storage device 204 stores in advance information on a space in which an object can be detected for each camera. The area determination unit 111 acquires, from the auxiliary storage device 204, information on a space in which an object can be detected for all cameras. Then, the area determination unit 111 selects a camera in which the space where the object can be detected and the new object appearance space overlap based on the acquired information. In the present embodiment, a space where the camera can detect an object, a space where a new object appears, and the like are treated as an area projected on the bottom surface. Further, in the present embodiment, since the same method described in Non-Patent Document 2 is applied to the attribute acquisition unit 102 and the attribute update unit 103, the area where an object can be detected by a certain camera and the trackable area of the camera are equal. Therefore, the area determination unit 111 may find a camera in which the new object appearance space and the camera trackable area overlap. For example, the area determination unit 111 determines whether or not the new object appearance space 302 and the area 321 that is the trackable area of the camera or the area 322 that is the trackable area of the camera overlap, and select the cameras that are determined to overlap. Good.
Further, the area determination unit 111 may determine the tracking area for all the cameras without selecting a camera that can detect a new object. In that case, the area determination unit 111 determines an area having an area of 0 as a tracking area for a camera that cannot detect a new object.

Any method may be used to determine whether or not two areas on the same plane overlap each other. For example, after approximating both areas with a plurality of triangles, one area may be inside a triangle. If the vertices of the triangle of the other region are present, the method of making the two regions overlap is used. For example, since the apex 801 of the new object appearance space 302 is included in the area 321 that is the trackable area, the area determination unit 111 uses the camera 311 as a camera that can detect a new object from the new object appearance space 302. select.
Next, the area determination unit 111 determines a new object attribute acquisition area on the image captured by a certain camera. The method for determining the new object attribute acquisition area on the captured image is to calculate the coordinates on the captured image by the camera for each of the vertices of each new object appearance space, and use the area in which the calculated coordinates are convexly enveloped and the camera. It suffices to find the common part with the captured image. However, since the new object appearance space is treated as a region projected on the bottom surface, the region determination unit 111 provides a vertex assuming an appropriate height according to the height at which the object can be seen, and then, on the camera image thereof. It is necessary to find the coordinates of. In the present embodiment, it is assumed that the auxiliary storage device 204 stores in advance information of an appropriate height according to the height at which the object can be seen. The area determination unit 111 acquires, from the auxiliary storage device 204, information on an appropriate height according to the height at which the object can be seen. There is a method of using Expression 1 as a method of obtaining coordinates of a certain coordinate on a captured image by a camera. For example, as shown in FIG. 10, the area determination unit 111 provides a group of vertices on the bottom of the new object appearance space 302 at a height such that even a pedestrian with a sufficiently high height can see them, and forms a cube. To do. The area determination unit 111 obtains the coordinates in the captured image 411 for all the vertex groups of this cube, obtains the convex hull thereof, and obtains the new object attribute acquisition area 421. The area determination unit 111 is an example of a second determination unit that determines the detection parameter used when the object is detected for the first time, based on the area input by the space input unit 110. Here, the information indicating the new object attribute acquisition area 421 is an example of the detection parameter.
Then, the attribute acquisition unit 102 detects the new object based on the detection parameter used when detecting the new object determined by the area determination unit 111. In the present embodiment, in step S402, the attribute acquisition unit 102 walks from the new object attribute acquisition area determined by the area determination unit 111 for the captured image from the camera that can capture the new object determined by the area determination unit 111. The person 351 will be detected. The attribute acquisition unit 102 is an example of a second detection unit that detects an object from a region in which the object first appears, which is set for each of a group of captured images from a plurality of cameras.

  As described above, according to the processing of this embodiment, the monitoring system can set the new object appearance space for each camera. In addition, the space input unit 110 can set the new object appearance space according to the user's wish by presenting the user with a designation screen for designating the new object appearance space. This can reduce the processing load when detecting an object that newly appears in the monitoring space 301.

<Embodiment 3>
In the first and second embodiments, the following processing has been described. That is, this is a process in which the attribute acquisition unit 102 and the attribute update unit 103 acquire the attributes of objects for each camera, and the common attribute estimation unit 104 integrates the attributes and estimates the common attribute between the cameras. Further, the common attribute prediction unit 105 predicts the common attribute value of the object in the next captured image by the camera, and the device determination unit 106 and the region estimation unit 107 determine the detection parameter based on the predicted common attribute value. It is a process to do. Such detection parameters that are collected by each camera, integrated / predicted as a whole, and can be determined based on the predicted attributes are the information of the camera that captures the image of the detection target and the information of the tracking area. Is not limited. Also, the attributes that can be obtained from the object are not limited to coordinates. For example, there are the following.

For example, the height of the pedestrian (height of the object) is an attribute that can be acquired from the object detected by the attribute acquisition unit 102 or the attribute update unit 103.
In the first embodiment, the coordinates of the pedestrian 351 in the world coordinate system are the coordinates of the center of the sphere when the head of the pedestrian 351 is approximated by a sphere. Here, assuming that the coordinates of the pedestrian 351 in the world coordinate system are the coordinates of the point farthest from the floor surface of the sphere, the common attribute estimation unit 104 estimates the height of the pedestrian. Here, the area estimation unit 107 can calculate the number of pixels in the vertical direction in which the pedestrian is reflected by the camera based on the height of the pedestrian, the angle of view of the camera, the number of pixels of the camera, and the like. Then, the area estimation unit 107 determines the number of pixels in the vertical direction in which the pedestrian is reflected, as the detection parameter.
Then, the attribute updating unit 103 has the number of pixels in the vertical direction that is similar to the number of pixels in the vertical direction determined as a detection parameter when selecting a partial image for object detection based on the method of Non-Patent Document 2. Select a partial image. For example, the attribute updating unit 103 selects a partial image having the number of pixels in the vertical direction that is close to the number of pixels in the vertical direction determined as the detection parameter, and performs a process of detecting an object from the selected partial image. The number of pixels close to the number of pixels in the vertical direction determined as the detection parameter is, for example, a range having a width set around the number of pixels (for example, 0.9 times to 1.1 times the number of pixels). (Range up to) can be set as the number of pixels. As a result, the attribute updating unit 103 does not need to perform the object detection processing on the partial image having the number of pixels in the vertical direction that is not approximate to the number of pixels in the vertical direction determined as the detection parameter. As a result, the monitoring system can reduce the load of the detection processing by the attribute updating unit 103.

For example, the attributes that can be acquired from the objects detected by the attribute acquisition unit 102 and the attribute update unit 103 include the orientation of the torso and the face of the pedestrian (the orientation of the object, the orientation of part of the object).
Since the appearance of a pedestrian imaged by the camera is significantly different depending on the image capturing direction, the detection method of Non-Patent Document 2 has poor detection performance when the image capturing direction changes. In order to overcome this, it is assumed that the attribute acquisition unit 102 and the attribute update unit 103 prepare detectors for each combination of face orientation and body orientation, as shown in Non-Patent Document 2.
In S402, the attribute acquisition unit 102 obtains the face direction and body direction of a pedestrian. That is, the attribute acquisition unit 102 obtains the detector that most accurately determines the pedestrian as the true detector from among the prepared detectors. Then, the attribute acquisition unit 102 obtains the face orientation and the body orientation of the pedestrian by acquiring which of the detector orientation corresponds to the face orientation and the body orientation. The common attribute estimation unit 104 integrates the face orientation and the body orientation of the pedestrian in the captured images from each camera, and obtains the face orientation and the body orientation in world coordinates. Then, the common attribute prediction unit 105 predicts the torso and face orientation of the pedestrian at the time when the set period has elapsed from the current time. For example, when the pedestrian walks while gazing at a point in the surveillance space, the common attribute prediction unit 105 predicts that the face direction is the direction in which the point is gazed at. The area estimation unit 107 estimates the face direction and the body direction of the pedestrian on the image captured by the camera based on the predicted face direction and the body direction of the pedestrian in the world coordinate system, and estimates the face direction and the body direction. The orientation information is determined as the detection parameter. Then, the attribute updating unit 103 selects and applies a detector for detecting a pedestrian from a plurality of detectors according to the face orientation and the body orientation of the pedestrian determined as the detection parameter. As a result, the attribute updating unit 103 does not need to perform the process of detecting a pedestrian using a detector that does not correspond to the face direction and the body direction of the pedestrian determined as the detection parameter. Thereby, the monitoring system can reduce the load of the object detection processing by the attribute updating unit 103.
Further, the attribute acquisition unit 102 may acquire the face orientation and the body orientation together with the pedestrian coordinates in S402.

The space input unit 110 and the area determination unit 111 determine the detection parameter used by the attribute acquisition unit 102 to detect an object. Therefore, the space input unit 110 and the region determination unit 111 can set the detection parameters other than the region used for detecting the camera in the captured image used for detection and the object in the captured image. For example, when the space input unit 110 acquires the height and the number of pixels of the pedestrian as the attributes of the object, the area determination unit 111 determines the number of pixels in the vertical direction of the pedestrian reflected in the camera from the average height of the pedestrian. It is calculated and determined as the detection parameter used in S402. Then, in S402, the attribute acquisition unit 102 extracts a partial image based on the calculated number of pixels from the captured image, and detects a pedestrian from the extracted partial image.
Further, when the space input unit 110 acquires the direction of a pedestrian's face or moving body as an attribute of an object, the input of a walking direction or a gazing point is accepted for each new object appearance space via a designation screen. Then, the region determination unit 111 estimates the torso and face direction of the pedestrian from the input walking direction and gazing point, and determines them as the detection parameters used in S402. Then, in S402, the attribute acquisition unit 102 can detect a pedestrian using a detector based on the estimated body direction and face direction.
As a result, the monitoring system can reduce the load of the object detection processing by the attribute acquisition unit 102 in S402.

<Other embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments.
For example, part or all of the functional configuration of the above-described monitoring system may be implemented as hardware in the information processing device 341.

341 Information processing device, 202 CPU

Claims (6)

An image of a two-dimensional map overlooking the imaging ranges of the plurality of image capturing devices is displayed in a common coordinate system common to the plurality of image capturing devices, and an object first appears in the two-dimensional map based on a user operation on the image. Input means for inputting the area to be
An imaging device that acquires an image in which the object should be detected for the first time among the plurality of imaging devices and a region in which the object should be detected for the first time are determined based on the input region where the object first appears. Initial area determining means to
First detection means for detecting the object from the area in which the object determined by the initial area determination means should be detected for the first time in the image captured by the imaging device determined by the initial area determination means,
On the basis of the object captured image detected, an acquisition unit configured to acquire coordinates representing the position of the object in the common coordinate system,
Prediction means for predicting a predicted value of a coordinate representing the position of the object in the common coordinate system at a future time point when a period set from the current time point has elapsed, based on the coordinates ,
Based on the image-capable range of each of the plurality of image-capturing devices and the predicted value, an image-capturing device capable of capturing the object at the future time point is determined from among the plurality of image-capturing devices , and the determination is performed. Estimating means for estimating an area in which the object can be imaged in an image capturing device ;
Second detecting means for detecting the object from the estimated region in the image captured by the image capturing device determined by the estimating means at the future time point;
Information processing device having a. It said predicting means, and the coordinates, the common coordinate and past coordinates of the object in the system, based on the information processing apparatus according to claim 1, wherein predicting the predicted value. It said acquisition means, field of view based on the captured image in which the object is detected within the captured image group captured by the plurality of imaging devices overlapping, acquires the coordinates of the object in the common coordinate system,
Said predicting means, based on the coordinates of the object in the acquired the common coordinate system, the coordinates of the object in the common coordinate system at a future time, according to claim 1 or 2, wherein the prediction as the prediction value Information processing equipment. The initial area determination means may define an area in which the area in which the object appears for the first time in the two-dimensional map input by the input means is projected onto each of the captured image groups captured by the plurality of imaging devices , the information processing apparatus according to claim 1 to 3 any one of claims to determine the object as the first detection should do region. An information processing method executed by an information processing apparatus,
An image of a two-dimensional map overlooking the imaging ranges of the plurality of image capturing devices is displayed in a common coordinate system common to the plurality of image capturing devices, and an object first appears in the two-dimensional map based on a user operation on the image. Input step to enter the area to be
An imaging device that acquires an image in which the object should be detected for the first time among the plurality of imaging devices and a region in which the object should be detected for the first time are determined based on the input region where the object first appears. An initial area determination step to
A first detection step of detecting the object from the area where the object determined in the initial area determination step is to be detected for the first time in the image captured by the imaging device determined in the initial area determination step,
On the basis of the object captured image detected, an acquisition step of acquiring the coordinates representing the position of the object in the common coordinate system,
Based on the coordinates , a prediction step of predicting a predicted value of coordinates representing the position of the object in the common coordinate system at a future time when a period set from the current time has passed,
Each an imaging possible range of the plurality of imaging devices, based on said predicted value, said object Te said future time odor determines the imaging imaging device capable among the plurality of imaging devices, is the determined An estimating step of estimating a region in which the object can be imaged in the image capturing device ,
A second detection step of detecting the object from the estimated region in the image captured by the imaging device determined in the estimation step at the future time point,
Information processing method including. Computer program to function as each unit of the information processing apparatus of claims 1 to 4 any one of claims.

Images