JP7237520B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, surveillance systems using network cameras have become widespread. As such a monitoring system, a monitoring system equipped with a camera linkage function that links a wide-area camera such as an omnidirectional camera capable of imaging a wide area with a PTZ (Pan Tilt Zoom) camera equipped with a pan mechanism, a tilt mechanism and a zoom mechanism. We have a system. The camera linkage function controls the imaging direction of the PTZ camera so that the PTZ camera captures an image with the point of interest selected by the user from the image (video) captured by the omnidirectional camera as the gaze point, and the image captured by the PTZ camera is captured. This is a function that enlarges and displays an image.

In a monitoring system that realizes such a camera linkage function, it is necessary to perform calibration before starting operation, grasp the relative positional relationship of multiple cameras, and align the images captured by each camera. is common. Based on this positional relationship of the cameras, it calculates which position in the coordinate system of the PTZ camera corresponds to the point of interest in the coordinate system of the omnidirectional camera, and issues an instruction to change the imaging direction to the PTZ camera.
However, since depth information cannot be obtained from a captured image that is a two-dimensional plane, when a point of interest is specified from the captured image, the three-dimensional position of the point of interest cannot be calculated. Therefore, for example, when an omnidirectional camera captures an image of a person standing in front of a door from the front of the door, and the face of the person is selected as an attention point in the captured image, the PTZ camera that captures the image from the side of the door captures the face of the person. and the door should be the point of gaze. Also, in the case of a moving object that moves around, such as a person, it is difficult to align the positions before starting operation.
Patent Document 1 discloses a method of using a distance sensor and a method of obtaining the focus position of the camera as a method of obtaining depth information by obtaining the distance from the camera to the object to be imaged.

JP-A-2006-115470

However, the method of obtaining depth information using a distance sensor like the technique described in Patent Literature 1 is costly. In addition, the method of obtaining depth information by acquiring the focus position of a camera is difficult to apply to an omnidirectional lens and the like, and the accuracy is not good.
Accordingly, an object of the present invention is to appropriately derive the three-dimensional position of a point of interest in a captured image at low cost.

In order to solve the above problems, one aspect of an information processing apparatus according to the present invention provides area information for specifying the positions of a floor area and a wall area in an image captured by a first imaging device . a first acquiring means for acquiring; a second acquiring means for acquiring position information indicating a position of a point of interest in the captured image; area information acquired by the first acquiring means; positional information obtained by means for determining whether the target object corresponding to the target point is an object existing on the floor surface or an object existing on the wall surface, and based on the result of the determination; and derivation means for deriving the three-dimensional position of the attention point.

According to the present invention, it is possible to appropriately derive the three-dimensional position of a point of interest in a captured image at low cost.

It is a figure which shows the system configuration example of the imaging system in 1st embodiment. 2 is a hardware configuration example of an imaging device according to the first embodiment; 3 is a hardware configuration example of a client device according to the first embodiment; FIG. 4 is a schematic diagram of an environment in which floor surface setting processing is performed in the first embodiment; 4 is a flowchart of floor setting processing procedures in the first embodiment. FIG. 4 is an explanatory diagram of marker detection in the first embodiment; FIG. 10 is an explanatory diagram of marker position acquisition in the first embodiment; It is an example of a map table in the first embodiment. 4 is a flowchart of camera cooperation operation in the first embodiment. FIG. 4 is a diagram for explaining attribute determination of a gaze point in the first embodiment; FIG. 4 is a diagram illustrating a method of calculating an elevation angle Θ _MAX in the first embodiment; FIG. It is a figure explaining the calculation method of the gaze point in 1st embodiment. 9 is a flowchart of camera cooperation operation in the second embodiment. It is a figure explaining the calculation method of the gaze point in 2nd embodiment. It is an example of a UI screen in the second embodiment. It is an example of a UI screen in the second embodiment. It is an example of a UI screen in the second embodiment. It is an example of a UI screen in the second embodiment. It is another example of a map table. FIG. 11 is a diagram illustrating another example of a method of calculating a point of gaze; It is a figure explaining another example of the floor specification method.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.
The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to the embodiment of

In the present embodiment, a monitoring system using a network camera will be described as an imaging system that images an object by linking a plurality of imaging devices.
Network cameras are used in a wide range of fields as surveillance cameras in large-scale public institutions and mass retailers, and there are various operating methods. In addition, there are network cameras with various functional features in order to match the operation mode. For example, a network camera that can freely change the imaging direction such as pan and tilt, a network camera that cannot change the imaging direction but can perform high-magnification zoom imaging, a network camera that can monitor a wide range at once with a fisheye lens (omnidirectional) camera) exists.
Among such network cameras, an omnidirectional camera has a very wide viewing angle, but has the characteristic that the number of pixels assigned to the camera is small and the resolution drops at a location away from the camera. Therefore, omnidirectional cameras are not suitable for capturing detailed information such as people's faces and vehicle license plates. In order to compensate for such a drawback, there is a camera linkage function that links an omnidirectional camera and a camera (PTZ camera) having pan/tilt/zoom functions. That is, an omnidirectional camera monitors a wide range at once, a PTZ camera captures an image of a point of particular interest (a point of interest) in the image, and the point of interest is enlarged and displayed to obtain detailed information.

In this embodiment, in an imaging system having a plurality of imaging devices, when a point of interest is specified from an image captured by an omnidirectional camera, the PTZ camera is instructed to take an image with the point of interest as the point of gaze. . In this embodiment, a planar area such as a floor surface in the captured image of the omnidirectional camera and the height of the point of interest from the plane in the real space corresponding to the planar area are determined, and based on this information, , to derive the three-dimensional position of the point of interest. Accordingly, in the camera cooperation function, it is possible to instruct an appropriate imaging position to the PTZ camera that cooperates with the omnidirectional camera.

(First embodiment)
FIG. 1 is a diagram showing a system configuration example of an imaging system 1000 according to this embodiment. The imaging system 1000 according to the present embodiment is a camera cooperation system that images an object of interest to be monitored by linking two imaging devices 100A and 100B.
In the imaging system 1000, calibration is performed before starting operation in order to grasp the positional relationship between the imaging devices 100A and 100B. In this embodiment, the imaging system 1000 performs floor surface setting processing using a marker during calibration, and sets a plane area such as a floor surface (ground) or a wall surface in the captured image of the imaging device 100A. After the start of operation, the imaging system 1000 derives the three-dimensional position of the designated point of interest from the image captured by the imaging device 100A, and performs processing for causing the imaging device 100B to capture an image using the derived point of interest as the point of gaze. . Specifically, when the point of interest is the face of a person, the imaging system 1000 calculates a point whose height from the floor is the estimated height of the person as the point of interest (point of gaze), An imaging direction instruction process for controlling the direction is performed.
Note that the attention point is not limited to the person's face. The point of interest may be a part of the person other than the face, or may be the whole body of the person. Also, the point of interest may be an object other than a person (for example, a vehicle or a window).

The imaging system 1000 includes an imaging device 100A, an imaging device 100B, and a client device 200. The imaging devices 100A and 100B and the client device 200 are connected by a network 300 so as to be able to communicate with each other. The network 300 may be of any communication standard, size, and configuration as long as the cameras 100A and 100B and the client device 200 can communicate with each other. Also, the physical connection form to the network 300 may be wired or wireless.

The imaging devices 100A and 100B are network cameras (hereinafter simply referred to as "cameras"). In this embodiment, camera 100A is an omnidirectional camera, and camera 100B is a PTZ camera. The cameras 100A and 100B can be installed on walls, ceilings, or the like, and can be cameras that capture video including one or more images. The cameras 100A and 100B may be compatible with PoE (Power Over Ethernet (registered trademark)), or may be configured to supply power via a LAN cable or the like. Also, in FIG. 1, two cameras 100A and 100B are connected to the network 300, but two or more cameras may be connected to the network 300, and the number of connected cameras is limited to the number shown in FIG. not.
In this embodiment, a case will be described in which the camera 100A operates as an information processing device that executes the floor surface setting process and the imaging direction instruction process described above. In this embodiment, a case where the camera 100A operates as an information processing device will be described, but the client device 200 and the imaging device 100B may operate as the information processing device.

FIG. 2 is a hardware configuration example of the camera 100B, which is a PTZ camera.
The camera 100B includes an imaging optical unit 101, an imaging device unit 102, a CPU 103, a ROM 104, a RAM 105, an imaging system control unit 106, a PT control unit 107, an image processing unit 108, an encoder unit 109, and communication a unit 110;
The imaging optical unit 101 includes an objective lens, a zoom lens, a focus lens, an optical diaphragm, and the like, and collects optical information of a subject onto the imaging element unit 102 .
The imaging element unit 102 includes an imaging element such as a CCD or CMOS sensor that converts light information collected by the imaging optical unit 101 into an electric signal, and acquires color information by combining with a color filter or the like. Further, the image sensor unit 102 can be configured using an image sensor that can set arbitrary exposure time and gain adjustment for all pixels.

The CPU 103 comprehensively controls operations in the camera 100B. The ROM 104 is a non-volatile memory that stores control programs and the like necessary for the CPU 103 to execute processing. Note that the program may be stored in an external memory (not shown) or a removable storage medium. A RAM 105 functions as a main memory, a work area, and the like for the CPU 103 . The ROM 104 and RAM 105 can store set values such as image quality adjustment parameters and network settings, and can be started using previously set values even when restarted.
The CPU 103 loads a necessary program or the like from the ROM 104 to the RAM 105 when executing processing, and executes the program or the like to realize various functional operations. For example, the CPU 103 can instruct the imaging system control unit 106 to perform AE (Automatic Exposure) control for controlling exposure, AF (Autofocus) control for controlling focus, and the like.

In accordance with instructions from the CPU 103, the imaging system control unit 106 controls the imaging optical unit 101 to focus by driving the focus lens, control exposure to adjust the aperture, and control to change the zoom magnification by driving the zoom lens. I do.
The PT control unit 107 controls panning and tilting of the camera 100B by controlling a camera attitude driving unit (not shown). The PT control unit 107 can also control panning and tilting of the camera 100B according to camera operation commands transmitted from the client device 200 and other cameras such as the camera 100A.

The image processing unit 108 receives an image signal output from the image sensor unit 102, performs image processing on the input image signal, and generates a luminance signal and a color difference signal.
The encoder unit 109 converts the image data image-processed by the image processing unit 108 into JPEG or H.264 format. 264 or other predetermined format.
The communication unit 110 processes communication with other cameras such as the camera 100</b>A and the client device 200 . In this embodiment, the communication unit 110 distributes image data converted by the encoder unit 109 to the client device 200 via the network 300 . The communication unit 110 also receives various commands from the client device 200 and the camera 100A, and transmits responses to the received commands and necessary data other than image data.

A part of the functions of each element of the camera 100B shown in FIG. 1 can also be realized by the CPU 103 executing a program.
Also, the camera 100A, which is an omnidirectional camera, is a pan-focus camera without a focus lens. Therefore, the hardware configuration of the camera 100A is the same as that of the camera 100B shown in FIG. 2, except that the zoom lens and the PT control unit 107 are omitted.

The client device 200 can be, for example, a display device that includes a general-purpose computer such as a personal computer (PC) or mobile terminal, and a monitor capable of displaying images.
FIG. 3 is a hardware configuration example of the client device 200. As shown in FIG.
The client device 200 includes a CPU 201 , a ROM 202 , a RAM 203 , an HDD 204 , an operation input section 205 , a communication section 206 and a display section 207 .
The CPU 201 comprehensively controls operations in the client device 200 . A ROM 202 is a non-volatile memory that stores control programs and the like necessary for the CPU 201 to execute processing. Note that the program may be stored in an external memory (not shown) or a removable storage medium. A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 . The CPU 201 loads necessary programs and the like from the ROM 202 to the RAM 203 when executing processing, and executes various functional operations by executing the programs and the like.

The HDD 204 can store various data and various information necessary when the CPU 201 performs processing using a program. In addition, the HDD 204 can store various data, various information, and the like obtained by the CPU 201 performing processing using programs and the like.
The operation input unit 205 includes operation devices such as a power button, a keyboard, and a mouse, and can receive instruction input from the user. Communication unit 206 receives images transmitted from cameras 100A and 100B via network 300 . Further, the communication unit 206 transmits various commands to the cameras 100A and 100B, and receives necessary data other than responses and image data. A display unit 207 has a monitor such as a liquid crystal display (LCD), and can display a graphical user interface (GUI) for inputting various control parameters for the cameras 100A and 100B.
Some functions of the elements of the client device 200 shown in FIG. 1 can also be implemented by the CPU 201 executing a program.

The floor setting process executed in the information processing apparatus will be described in detail below. In this embodiment, a case where the camera 100A executes the floor setting process will be described, but the client device 200 and the camera 100B may acquire various information from the camera 100A and execute the floor setting process. .
The floor setting process in this embodiment is a process of setting a floor area in a captured image using a marker. It is assumed that the relative positional relationship between the cameras 100A and 100B is measured by calibration when the cameras are installed, and the height of the cameras from the floor is obtained from design drawings of the installation environment. .

FIG. 4 is a schematic diagram of a camera installation environment for floor setting processing.
As shown in FIG. 4, two different types of markers, a floor marker 401 and a wall marker 402, are prepared. The floor marker 401 is placed horizontally on the floor surface, and the wall marker 402 is placed horizontally on the wall surface. Each marker has a known pattern drawn as feature points that are easy to detect, and the type of marker (for floors/for walls) can be detected by differences in shape, pattern design, color, etc. .
Information can also be embedded in each of the markers 401 and 402 using a two-dimensional barcode (eg, QR code (registered trademark)). In this case, if type information indicating the type of marker is embedded in the two-dimensional barcode, the type of marker can be determined by analyzing the two-dimensional barcode. Additionally, each marker 401 and 402 may use a pattern of known length. In this case, the distance between the camera 100A and the markers 401 and 402 can be calculated based on the detected pattern length. In this way, in each marker 401 and 402, information indicating the type of marker and information indicating the installation position of the marker can be embedded as marker information.

Camera 100A captures an image of the environment in which floor marker 401 and wall marker 402 are installed, and detects the markers from the captured image. Then, the camera 100A extracts marker information embedded in the detected marker, and acquires area information indicating the floor surface area and area information indicating the wall surface area in the captured image based on the extracted marker information. . Here, the area information is information indicating the position of a planar area such as a floor area or a wall area in the captured image, information indicating the type of the planar area, and indicating the installation position of the marker (distance from the camera 100A, etc.). Contains information.
That is, it can be said that the floor marker 401 and the wall marker 402 are identification information for identifying the floor surface region and the wall surface region in the captured image. In this embodiment, a case where a marker is used as identification information will be described, but the present invention is not limited to the above. The identification information may be any information that can set a planar area in the captured image and acquire area information of the planar area.
After obtaining the area information of the plane area from the captured image, the camera 100A sets the position of the floor surface and the position of the wall surface in the coordinate system of the camera 100A based on the obtained area information.

FIG. 5 is a flowchart of floor setting processing executed by the camera 100A. The processing in FIG. 5 is executed after calibration and before the imaging system 1000 starts operating. However, the start timing of the processing in FIG. 5 is not limited to the above timing. The camera 100A can realize each process shown in FIG. 5 by reading and executing a necessary program by the CPU provided in the camera 100A. Hereafter, the letter S shall mean a step in the flow chart.

First, in S1, the camera 100A captures an image of a marker in the camera installation environment as shown in FIG. Since the camera 100A is an omnidirectional camera with a wide angle of view, it is possible to capture all the markers at once within the angle of view in the camera installation environment. FIG. 6A shows an example of an image captured by the camera 100A in S1. In this way, the camera 100A can capture images of all installed floor markers 401 and wall markers 402 .
When a camera with a narrow angle of view such as a PTZ camera executes the floor surface setting process, the angle of view may be changed as necessary so that all markers can be captured. Here, the angle of view may be changed manually by the user.

In S2, the camera 100A extracts markers from the captured image captured in S1. In this S2, the camera 100A extracts known floor markers 401 and wall markers 402 from the captured image as shown in FIG. 6(a). For example, the camera 100A can detect a marker by pattern matching from a plurality of orientation information that the marker can take, which has been learned in advance. However, the marker detection method is not limited to the above, and any detection method can be used.
After extracting the marker from the captured image, the camera 100A detects the position information of the marker in the captured image. To simplify the explanation, FIG. 6B shows a captured image with only one marker. As shown in FIG. 6B, the position information of the floor marker 401 can be expressed as coordinates (r, Φ) with Oc, which is the center of the captured image, as the origin. Here, the center Oc of the captured image coincides with the center point of the lens of the camera 100A. The camera 100A detects and holds position information (r, Φ) in the captured image of the marker.

Next, in S3, the camera 100A acquires marker information embedded in the marker extracted in S2. Then, the camera 100A determines whether the extracted marker is the floor marker 401 or the wall marker 402 based on the marker type information included in the marker information.
Next, in S4, the camera 100A updates the map table. The map table is a table that stores information indicating the position of the floor surface in the coordinate system of the camera 100A.

First, a method of updating the map table when the floor marker 401 is detected in S3 will be described.
In this case, in S4, the camera 100A sets the floor area based on the detected positional information of the floor marker 401 in the captured image, and updates the map table.
The position of the floor marker 401 in the coordinate system of the camera 100A _is , as shown in _FIG. can be defined by The elevation angle Θ ₁ can be calculated based on the image height r detected in S2 and the projection system of the lens. In this embodiment, the imaging optical unit 101 uses a stereoscopic projection method as a lens projection method. Therefore, the elevation angle Θ ₁ is obtained by the following formula (1) from the image height r.
Θ ₁ =2×arctan(r/2f) (1)
Here, f is the focal length in the imaging optical unit 101 .
In this way, the position information (Θ ₁ , Φ ₁ ) of floor marker 401 in the coordinate system of camera 100A can be uniquely obtained based on the position information of floor marker 401 in the captured image.

FIG. 8 is an example of a map table. As shown in FIG. 8, the map table is a table that associates the angle parameter Φ with the elevation parameter Θ, and the elevation parameter Θ stores the elevation angle range set as the floor area. That is, the map table is a table that describes the elevation angle range of the floor with respect to 360° around the camera 100A.
When the floor marker 401 is detected as a marker and the position of the detected floor marker 401 in the coordinate system of the camera 100A is (Θ ₁ , Φ ₁ ), the angle Φ ₁ is at least 0° to Θ ₁ ° can be determined to be the floor area. Therefore, in this case, the elevation angle range 0 to Θ ₁ is stored in the elevation angle parameter Θ corresponding to the angle parameter Φ=Φ ₁ in the map table.
If the corresponding elevation parameter Θ is already stored, the maximum value of the stored elevation parameter Θ is compared with the elevation angle _Θ1 , and if the elevation angle _Θ1 is larger, the map table elevation parameter Update the maximum value of Θ to the elevation angle Θ ₁ .

Next, a method for updating the map table when the wall marker 402 is detected in S3 will be described.
In this case, in S4, the camera 100A sets the floor area based on the position information of the detected wall marker 402 in the captured image, and updates the map table. First, camera 100A determines the actual distance W between camera 100A and wall marker 402 . As noted above, markers can include patterns of known length as marker information. Therefore, using the focal length f of the imaging optical unit 101 and the length d of the pattern on the captured image, the distance W can be calculated based on the following equation (2). where p is the known pattern length.
W=p×f/d (2)

Next, the camera 100A obtains the distance D in the horizontal direction between the camera 100A and the wall marker 402. FIG. The distance D, as shown in FIG. 7(b), is the distance between the intersection point of the perpendicular drawn from the camera 100A and the floor and the intersection of the perpendicular drawn from the wall marker 402 and the floor.
D=W×sin Θ ₂ (3)
In the above equation (3), Θ ₂ is the elevation angle up to the wall marker 402. Similar to the elevation angle Θ ₁ up to the floor marker 401 shown in FIG. can be calculated based on the projection method of
Finally, the camera 100A determines the elevation angle Θ ₃ Calculate
Θ ₃ = arctan (D/h) (4)

The elevation angle _Θ3 calculated by the above equation (4) is a value indicating the boundary position between the floor surface and the wall surface, and corresponds to the maximum value of the elevation angle parameter Θ indicating the area of the floor surface in the map table shown in FIG. . Therefore, the calculated elevation angle _Θ3 is used to update the elevation parameter Θ in the map table. That is, when wall marker 402 is detected as a marker, and the position of detected wall marker 402 in the coordinate system of camera 100A is (Θ ₂ , Φ ₂ ), elevation angle Θ ₂ at angle Φ ₂ is A location can be determined to be a floor area. Therefore, in this case, the maximum value of the elevation angle parameter Θ corresponding to the angle parameter Φ= _Φ2 in the map table is updated to _Θ2 .
In this embodiment, the map table is a table that describes an elevation angle range with a floor surface with respect to 360 degrees around the camera 100A, but it is not limited to the above. For example, the map table may be a table describing elevation angle ranges of wall surfaces with respect to 360° around the camera 100A. In this case, the minimum value of the elevation angle parameter Θ is the value indicating the boundary position between the floor surface and the wall surface. Also, there may be a plurality of map tables. For example, map tables may be created for each type, such as for floors and walls, or for each area, such as the right end of a wall.

Returning to FIG. 5, in S5, the camera 100A determines whether or not to end the floor setting process. The camera 100A may determine to end the floor surface setting process by receiving an end instruction from the user. It may be determined to end the face setting process. If the camera 100A determines to continue the floor setting process, it returns to S1, and if it determines to end the floor setting process, it ends the floor setting process of FIG.
In this embodiment, the relative positional relationship between camera 100A and camera 100B is measured by calibration at the time of installation. Not limited. For example, the positional relationship can also be measured by placing a plurality of markers at predetermined distances from the cameras 100A and 100B, respectively, and taking images of the markers with the cameras 100A and 100B.

Next, the imaging direction instruction processing executed in the information processing apparatus will be described. In this embodiment, a case where the camera 100A executes the imaging direction instruction process will be described, but the client device 200 and the camera 100B may acquire various information from the camera 100A and execute the imaging direction instruction process. .
The imaging direction instruction processing in the present embodiment calculates the three-dimensional position of the point of interest specified in the captured image captured by the camera 100A, and directs the camera 100B so that the point of interest is captured by the camera 100B as the point of interest. This is processing for issuing an instruction to control the imaging direction.

FIG. 9 is a flowchart of imaging direction instruction processing executed by the camera 100A. The processing of FIG. 9 is executed at the timing when the user designates a point of interest from the captured image of the camera 100A. However, the start timing of the processing in FIG. 9 is not limited to the above timing. The camera 100A can realize each process shown in FIG. 9 by reading and executing a necessary program by the CPU provided in the camera 100A.
First, in S11, the camera 100A acquires position information (position-of-interest information) of the point of interest in the captured image of the camera 100A specified by the user. By transmitting the captured image to the client device 200, the camera 100A performs display control to display the captured image on the display unit 207 of the client device 200, and designates an attention point in the captured image displayed on the display unit 207. accept. When the user designates an attention point on the captured image displayed on the display unit 207 of the client device 200, the camera 100A acquires the position information of the designated attention point in the captured image.

10(a) and 10(b) are diagrams showing an example of an image captured by the camera 100A displayed on the display unit 207 of the client device 200. FIG.
In the captured image shown in FIG. 10A, there is a person 501 standing on the floor. When the user designates the face of the person 501 as the point of interest, the camera 100A acquires the coordinates (r', Φ') as the position information of the point of interest. Also, as shown in FIG. 10B, when the user designates a window 502 installed on a wall as a point of interest, the camera 100A sets the coordinates (r'', Φ'') as the positional information of the point of interest. get.

Returning to FIG. 9, in S12, the camera 100A determines the attribute of the gaze point. The attribute of the point of interest is defined by two types, that is, whether the object of interest existing at the point of interest is an object existing on the floor or any other object. In this embodiment, the attribute of the point of interest is defined by two types, that is, whether the object of interest present at the point of interest is a person present on the floor surface or an object present on the wall surface. The camera 100A determines the attribute of the gaze point using the attention position information and the map table created in the floor surface setting process. The attribute determination method will be described below.

First, the camera 100A calculates the coordinates (Φ', Θ') of the target point in the coordinate system of the camera 100A based on the target position information. Next, the camera 100A refers to the map table, searches for the angle parameter Φ that is closest to the calculated angle Φ', and obtains the maximum value of the corresponding elevation parameter Θ.
Next, the camera 100A calculates an elevation angle Θ _MAX by adding a predetermined height to the maximum value of the elevation angle parameter Θ obtained from the map table.
FIG. 11 is a diagram illustrating a method of calculating the elevation angle Θ _MAX . When the point of interest is a person's face, the maximum value of the elevation angle of the point of interest is equal to the elevation angle when the face of a person standing by a wall is designated as the point of interest. A face position 600 of a person of height h ₀ standing by a wall is a position above the height h ₀ from a position a distance D away from the intersection of the perpendicular drawn from the camera 100A and the floor surface.

The elevation angle Θ _MAX corresponding to the gaze point position 600 shown in FIG. 11 can be calculated by the following equation.
Θ _MAX = arctan ((h×tan Θ ₃ )/(h−h ₀ )) (5)
where _Θ3 is the maximum value of the elevation parameter Θ stored in the map table.
Therefore, if the coordinate value Θ' of the point of interest is smaller than the elevation angle Θ _MAX , it can be determined that the attribute of the point of interest is a person existing on the floor. On the other hand, if the coordinate value Θ' of the point of interest is greater than the elevation angle Θ _MAX , it is determined that the attribute of the point of interest is an object other than a person existing on the floor, that is, an object existing on the wall. can be done.

Returning to FIG. 9, in S13, the camera 100A calculates the three-dimensional position of the point of interest (position of the point of interest). As described above, the user designates one point as the target point on the two-dimensional captured image captured by the camera 100A. Designation of this point of interest is nothing more than designation of a straight line on which the point of interest can exist in the real space. In S13, the camera 100A calculates the three-dimensional position of the point of gaze according to the attribute of the point of gaze determined in S12.
FIG. 12(a) is a diagram for explaining a method of calculating the point-of-regard position when it is determined that the attribute of the point-of-regard is a person existing on the floor. A point designated as a target point from the captured image of the camera 100A is indicated by a straight line 601 in FIG. 12A when viewed from a viewpoint other than the camera 100A. This straight line 601 is obtained from the imaging conditions of the camera 100A and the coordinates (Φ', Θ') of the point of interest in the coordinate system of the camera 100A. Here, the imaging conditions include the imaging direction and the imaging angle of view of the camera 100A.

Then, when the estimated height of the person of interest is set in advance to h ₀ , among the points on the straight line 601, a point 603 at a height of h ₀ from the floor surface 602 is considered to be a suitable point of gaze. Presumed. Therefore, the camera 100A calculates the three-dimensional position with this point 603 as the gaze point.
Note that the estimated height h ₀ of the person may be a preset fixed value, may be obtained from marker information embedded in the marker, or may be obtained by receiving a designation from the user. . Also, when the point of interest is other than the person's face, the height information h ₀ is information indicating the height from the floor of the point of interest other than the person's face in the real space.

FIG. 12(b) is a diagram illustrating a method of calculating the position of the point of gaze when it is determined that the attribute of the point of gaze is an object existing on a wall surface. A point designated as a point of interest from the captured image of the camera 100A is indicated by a straight line 611 in FIG. This straight line 611 is obtained from the imaging conditions of the camera 100A and the coordinates (Φ', Θ') of the point of interest in the coordinate system of the camera 100A, like the straight line 601 in FIG. 12(a).
Here, the object of interest does not exist on the floor surface, but exists on the wall surface. Therefore, among the points on the straight line 611, a point 613 located above a position separated by a distance D from the intersection of the floor surface 602 and the perpendicular drawn from the camera 100A to the floor surface 602 is considered to be a suitable point of gaze. Presumed. Therefore, the camera 100A calculates the three-dimensional position with this point 613 as the gaze point. This point 613 is the intersection of the straight line 611 and the wall surface, and is the point on the straight line 611 whose height from the wall surface is zero.

Here, the distance D from the intersection of the perpendicular drawn from the camera 100A to the floor surface 602 and the floor surface 602 to the wall can be calculated by the above equation (3). Also, the height h ₁ from the floor surface 602 to the gaze point 613 can be calculated by the following equation using the coordinate value Θ' of the gaze point.
h ₁ =h−sin Θ' (6)
In this way, the camera 100A captures an image of interest in the real space based on the imaging conditions of the camera 100A and the coordinates (Φ', Θ') of the attention point calculated based on the position information of the attention point in the captured image. Derive a line on which points can exist. Then, the camera 100A selects, among the derived points on the straight line, the position where the height from the plane (floor surface or wall surface) on which the object of interest exists is a preset height. Derived as a position.

Returning to FIG. 9, in S14, the camera 100A coordinate-transforms the point-of-regard position in the coordinate system of the camera 100A calculated in S13 into the coordinate system of the camera 100B.
Next, in S15, the camera 100A determines the imaging direction of the camera 100B for imaging the point-of-regard position coordinate-transformed in S14. Then, camera 100A transmits a command (parameter) for controlling the imaging direction of camera 100B to camera 100B via network 300, and causes camera 100B to start imaging.
In addition, in S15, camera 100A may determine the imaging angle of view of camera 100B in addition to the imaging direction of camera 100B and instruct camera 100B. In this case, the camera 100A calculates the zoom magnification so that the object of interest is within the imaging angle of view of the camera 100B.

As described above, the information processing apparatus according to the present embodiment acquires area information indicating a plane area in the captured image captured by the camera 100A and position information indicating the position of the point of interest in the captured image. In this embodiment, the plane area includes a floor area and a wall area. Then, the information processing device derives the three-dimensional position of the point of interest based on the area information of the plane area in the captured image, the position information of the point of interest in the captured image, and the preset height information. . Further, the information processing apparatus determines an imaging direction of the camera 100B different from the camera 100A, and instructs the camera 100B of the imaging direction for imaging the attention point as the gaze point. The information processing device may further determine the imaging angle of view of the camera 100B and instruct the imaging angle of view to the camera 100B.

Here, the attention point can be, for example, a person's face. In this case, the user designates a person's face in the captured image of the camera 100A as the gaze point, and the information processing device sets the designated person's face as the gaze point, and the camera 100B moves the gaze point three-dimensionally. location information can be sent. As a result, the camera 100B can control the imaging direction and appropriately capture an image of the gaze point. Therefore, even a person who does not exist at the time of camera calibration can be appropriately imaged by cooperation between the cameras 100A and 100B.

When deriving the three-dimensional position of the point of interest (point of interest), the information processing device uses the imaging conditions (imaging direction and angle of view) of the camera 100A and the position information of the point of interest in the captured image to determine the position of the point of interest in the real space. can derive a straight line on which the point of interest can exist. Then, the information processing device selects a position where the height from the plane in the real space corresponding to the plane region is a preset height among the derived points on the straight line, and the point of interest is three-dimensionally calculated. can be derived as a position.
Specifically, when the point of interest is the face of a person existing on the floor, the information processing apparatus determines that, among the points on the derived straight line, the height from the floor corresponds to the estimated height of the person. It is possible to derive the position that is the height of the point of interest as the three-dimensional position. Further, when the point of interest is an object existing on the wall surface, the information processing apparatus regards the point at which the height from the wall surface is 0 among the points on the derived straight line as the three-dimensional position of the point of interest. can be derived.

That is, the information processing device determines whether the target object existing at the target point is an object (for example, a person) existing on the floor surface or an object existing on the wall surface, and based on the result of the determination, A three-dimensional position of the point of interest is derived. Therefore, the three-dimensional position of the point of interest can be appropriately derived.
In this way, the information processing apparatus can set the three-dimensional position of the point of interest set two-dimensionally in the captured image of one camera 100A to the floor and the height of the point of interest from the floor (for example, It is possible to uniquely calculate by using the estimated height of the person) as a constraint condition. Therefore, there is no need to provide a distance sensor to obtain the three-dimensional position of the point of interest. Therefore, the cost can be reduced accordingly. Furthermore, even if a method of acquiring a focus position and acquiring a distance from the camera to the gaze point cannot be applied like an omnidirectional camera, it is possible to appropriately calculate the three-dimensional position of the gaze point.

Further, the information processing device detects a marker as identification information for identifying a plane area from the captured image, and acquires area information of the plane area based on the marker information embedded in the marker. For example, when type information indicating the type of planar area is embedded in the marker, the information processing device analyzes the marker information embedded in the marker to determine whether the position where the detected marker exists is the floor surface. Whether it is a wall surface can be determined. Further, when information indicating the installation position of the marker is embedded in the marker, the information processing device analyzes the marker information embedded in the marker to grasp the distance between the detected marker and camera 100A. be able to. Accordingly, the information processing apparatus can appropriately grasp the positions of planar regions such as the floor surface and the wall surface in the captured image, and the positions of the planar regions such as the floor surface and the wall surface in the coordinate system of the camera 100A.

In addition, the information processing device performs display control to display the captured image captured by the camera 100A on the client device 200, which is a display device, receives the setting of the attention point in the captured image by the user, and determines the position of the attention point in the captured image. Get information. Therefore, the information processing apparatus can appropriately calculate the three-dimensional position of the gaze point desired by the user and instruct the camera 100B to take an image.
In the imaging system of this embodiment, the camera 100A can be an omnidirectional camera, and the camera 100B can be a PTZ camera. In this imaging system, a wide range is monitored by an omnidirectional camera, and if there is an object that the user wishes to gaze at, the user can specify a gaze point from the omnidirectional image, and the PTZ camera can take an image of the gaze point. This makes it possible to appropriately present a user with a detailed image (for example, an enlarged image) of the gaze point. Therefore, the user can easily confirm detailed information such as a person's face and a vehicle license plate.

Next, a second embodiment of the invention will be described.
In the first embodiment described above, the case where the point of interest is the face of a person has been described. In the second embodiment, it is possible to select a viewpoint mode such as whether the point of interest is the person's face, whether the point of interest is the center of the person and the whole body is being gazed at, or the point of interest is the feet of the person. A case will be described where the gaze point is set at the desired viewpoint.
The system configuration in this second embodiment is the same as in the first embodiment described above. Also, the floor surface setting process executed by the information processing apparatus in this embodiment is the same as in the first embodiment described above.
In this embodiment, the imaging direction instruction process executed by the information processing apparatus is different from that in the above-described first embodiment. Therefore, the following description will focus on the portions of processing that are different from those of the first embodiment.

FIG. 13 is a flowchart showing imaging direction instruction processing executed by the information processing apparatus according to the present embodiment. In this embodiment, a case where the camera 100A executes the imaging direction instruction process will be described, but the client device 200 and the camera 100B may acquire various information from the camera 100A and execute the imaging direction instruction process. .
The process of FIG. 13 is executed at the timing when the user designates a point of interest from the captured image of the camera 100A. However, the start timing of the processing in FIG. 13 is not limited to the above timing. The camera 100A can realize each process shown in FIG. 13 by reading and executing a necessary program by the CPU provided in the camera 100A.

First, in S21, the camera 100A acquires information indicating the viewpoint mode and the estimated height. Camera 100</b>A performs display control to display a GUI on display unit 207 of client device 200 , and receives designation of a viewpoint mode and estimated height via the GUI displayed on display unit 207 . When the user specifies the viewpoint mode and the estimated height on the GUI displayed on the display unit 207 of the client device 200, the camera 100A acquires the information indicating the specified viewpoint mode and the information indicating the estimated height.
In this embodiment, it is assumed that the user can select from three viewpoint modes, ie, a "face close-up" mode, a "whole body" mode, and a "feet" mode. The "face close-up" mode is a mode for gazing at a person's face when the target object is a person existing on the floor, as in the first embodiment described above. The "whole body" mode is a mode in which not only the person's face but also the whole body including the feet is watched. The "feet" mode is a mode in which a person's feet are watched, and a point at a height of 0 from the floor is the point of interest.
A GUI for inputting the viewpoint mode and estimated height will be described later.

The processes of S22 and S23 are the same as those of the above-described first embodiment (S11 and S12 in FIG. 9), so description thereof will be omitted.
In S24, the three-dimensional position of the gaze point (gazing point position) is calculated in consideration of the viewpoint mode determined in S21.
FIG. 14 is a diagram for explaining how to calculate the position of the gaze point in each of the viewpoint modes when the attribute of the gaze point is determined to be a person existing on the floor in the gaze attribute determination in S23. A point designated as a point of interest from the captured image of the camera 100A is indicated by a straight line 621 in FIG. 14 when viewed from a viewpoint other than the camera 100A. This straight line 621 is obtained from the imaging conditions of the camera 100A and the coordinates (Φ', Θ') of the point of interest in the coordinate system of the camera 100A.
When the viewpoint mode is the “face up” mode, the estimated height h ₀ obtained in S21 is defined as the height from the floor of the point of interest, and the height of h ₀ from the floor 622 among the points on the straight line 621 is It is estimated that a point 623 at the point of interest is suitable as a gaze point. Therefore, the camera 100A calculates the three-dimensional position with this point 623 as the gaze point.

On the other hand, when the viewpoint mode is the "whole body" mode, the point of interest is the center of _the person _. A point 624 in 2) is presumed to be a point suitable as a gaze point. Therefore, the camera 100A calculates the three-dimensional position with this point 624 as the gaze point.
Also, when the viewpoint mode is the "feet" mode, it is estimated that among the points on the straight line 621, the point 625 at a height of 0 from the floor surface 622 is suitable as the gaze point. Therefore, the camera 100A calculates the three-dimensional position with this point 625 as the gaze point.
Note that if it is determined in the gaze attribute determination in S23 that the attribute of the gaze point is an object other than a person existing on the floor surface, the method of calculating the gaze point position in S24 is the same as in the above-described first embodiment. Therefore, the explanation is omitted.

The processing of S25 is the same as that of the above-described first embodiment (S14 in FIG. 9), so the description is omitted.
In S26, camera 100A determines the imaging direction of camera 100B for imaging the point-of-regard position coordinate-transformed in S25. In addition, the camera 100A places the point-of-regard position coordinate-transformed in S25 at the center of the imaging angle of view of the camera 100B, and calculates the zoom magnification so that the area to be imaged falls within the angle of view. In other words, when the viewpoint mode is "face up" mode, the face is shown large, when the viewpoint mode is "whole body" mode, the whole body is shown, and when the viewpoint mode is "feet" mode The imaging angle of view is determined so that the feet appear large in the image. Then, camera 100A transmits commands (parameters) for controlling the imaging direction and imaging angle of view of camera 100B to camera 100B via network 300, and causes camera 100B to start imaging.

An example of a GUI for inputting a viewpoint mode and an estimated height will be described below with reference to FIGS. 15 to 18. FIG.
A GUI 700 shown in FIGS. 15 to 17 is displayed on the display unit 207 of the client device 200 and operated by the operation input unit 205. FIG. The GUI 700 can include an image display unit 701 that displays images distributed from the camera 100A, and an image display unit 702 that displays images distributed from the camera 100B. The GUI 700 can also include a mode setting input form 703 for changing the setting of the viewpoint mode, and an estimated height input form 704 for setting the estimated height.
Here, the mode setting input form 703 is composed of, for example, a pull-down, and the user can select a viewpoint mode using an operation device such as a mouse provided in the operation input unit 205 . Also, the estimated height input form 704 is composed of, for example, a pull-down, and the user can select an estimated height using an operation device. Note that the method of specifying the viewpoint mode and the estimated height is not limited to the above. For example, the estimated height may be configured so that the user directly inputs the numerical value of the estimated height. Alternatively, the user can enter a numerical value within a preset range using an operation device from a pop-up 710 as shown in FIG. It may be a configuration of choice.

Using the operation device, the user aligns the pointer 705 on the display screen with a point of interest on the image displayed on the image display unit 701 and designates the point of interest by clicking the mouse or the like. Triggered by the user's specification of the point of interest, the camera 100A executes the imaging direction instruction process shown in FIG. 13 to instruct the imaging direction and imaging angle of view to the camera 100B. Then, the camera 100B changes the imaging direction and the imaging angle of view according to the instruction from the camera 100A. As a result, an image captured by the camera 100B is displayed on the image display unit 702 with the point of interest specified by the user as the point of interest.

When the user selects the "face close-up" mode as the viewpoint mode, as shown in FIG. Also, when the user selects the "whole body" mode as the viewpoint mode, as shown in FIG. Furthermore, when the user selects the "feet" mode as the viewpoint mode, as shown in FIG.
In the "feet" mode, since the height of the gaze point from the floor is fixed at 0, the estimated height input form 704 may be configured not to accept numerical input from the user. In this case, the estimated height specified by the user may be stored before switching the viewpoint mode, and the stored estimated height may be automatically selected when the viewpoint mode is restored.

As described above, the information processing apparatus according to the present embodiment receives viewpoint mode setting by the user, and calculates the three-dimensional position of the point of interest according to the viewpoint mode. Specifically, the information processing device sets the point of interest to one of the person's face, the person's center, and the person's feet according to the viewpoint mode. Then, when the point of interest is the face of a person, the information processing device sets a point at a height of the estimated height h ₀ from the floor as the point of interest, and when the point of interest is the center of the person, the point of interest is estimated from the floor. A point at the height of height (h ₀ /2) is set as the gaze point. Further, when the point of interest is the feet of a person, the information processing apparatus sets the point at which the height is 0 from the floor surface as the point of interest.
In other words, the information processing apparatus receives the setting of the viewpoint mode by the user, receives the setting of height information indicating the height from the floor surface to the point of interest, and calculates the three-dimensional position of the point of interest. As a result, the information processing apparatus can appropriately calculate the three-dimensional position of the gaze point desired by the user and instruct the cooperative camera of the imaging direction. In addition, the information processing apparatus can determine and instruct the imaging angle of view of the camera 100B according to the viewpoint mode, so it is possible to present to the user an image in which an object that the user wishes to gaze at is appropriately captured.

(Modification)
In each of the above-described embodiments, the case where the floor surface is at a constant height has been described. be able to. In this case, the floor surface setting process is performed on the floor surface that is different in height from the floor surface directly below the camera 100A. Therefore, the floor surface immediately below the camera 100A is set as a reference plane, and information indicating the height with respect to the reference plane is embedded as marker information of the floor marker 401 as height correction information, and the difference in floor surface height is calculated by the camera 100A. should be notified.
That is, in the floor surface setting process shown in FIG. 5, the camera 100A also acquires the height correction information when extracting the marker information in S3. Then, when updating the map table in S4, the camera 100A stores the height correction information acquired in S3 in the map table. An example of the map table is shown in FIG.

Also, in the imaging direction instruction process shown in FIG. 9, the camera 100A determines the attribute of the gaze point using a map table storing height correction information as shown in FIG. 19 in S12. A method of determining the attribute of the gaze point will be described below.
In the first embodiment described above, the maximum elevation angle Θ _MAX is calculated based on the above equation (5) using preset height information (estimated height) h ₀ , but the floor surface When the height difference is considered, the maximum elevation angle Θ _MAX ' is calculated by adding the height correction value h _c .
Θ _MAX ′=arctan((h×tan Θ ₃ )/(h−(h ₀ +h _c ))) …………(7)
If the coordinate value Θ' of the point of interest is smaller than the elevation angle Θ _MAX ', it can be determined that the attribute of the point of interest is a person existing on the floor. On the other hand, if the coordinate value Θ' of the point of interest is greater than the elevation angle Θ _MAX ', it is determined that the attribute of the point of interest is an object other than a person existing on the floor, that is, an object existing on the wall. be able to.

Further, in S13 of FIG. 9, the camera 100A calculates the point-of-regard position in consideration of the height correction value h _c .
FIG. 20 is a diagram illustrating a method of calculating the position of the point of gaze when it is determined that the attribute of the point of gaze is that of a person existing on the floor. A point designated as a point of interest from the captured image of the camera 100A is indicated by a straight line 631 in FIG. 20 when viewed from a viewpoint other than the camera 100A. Here, when the estimated height of the person of interest is set to h ₀ in advance, a point 633 at a height of (h ₀ +h _c ) from the floor surface 632 among the points on the straight line 631 is the gaze point. It is presumed to be a suitable point. Therefore, the camera 100A calculates the three-dimensional position with this point 633 as the gaze point. Note that the method of calculating the point-of-regard position when it is determined that the attribute of the point-of-regard is an object other than a person existing on the floor is the same as in the above-described first embodiment.
With the configuration as described above, it is possible to appropriately capture images of persons existing at different positions on the floor surface in cooperation with a plurality of cameras.

In each of the above embodiments, the floor area is set using the floor markers 401 and the wall markers 402. However, the floor area can be specified by the user via the GUI. good too. In this case, as shown in FIGS. 21(a) and 21(b), the user operates the pointer 705 using an operation device such as a mouse, and the floor is displayed on the captured image of the camera 100A displayed on the display unit 207. Specify face area. At this time, the user may specify the range of the floor surface by freehand as shown in FIG. 21(a), or may specify it by selecting a rectangle as shown in FIG. 21(b). The camera 100A receives the setting of the floor area in the captured image and acquires the area information of the floor area.
In this way, the information processing apparatus performs display control to display the image captured by the camera 100A on the client device 200, which is a display device, receives the setting of the planar area in the captured image, and acquires the area information of the planar area. can also This makes it possible to set the floor surface area and the wall surface area more appropriately.
Further, the method of setting the plane area in the captured image is not limited to the above-described method of using the marker or the method of displaying the captured image and specifying it with a mouse or the like by the user. For example, a method of detecting a moving object from a captured image and estimating the floor surface area from its flow line can also be used.

In each of the above-described embodiments, the camera 100A accepts from the user the setting of the point of interest in the captured image displayed on the client device 200, and acquires the position information indicating the position of the point of interest in the captured image. . However, the camera 100A may set a point of interest from the captured image by automatic recognition technology and acquire position information of the point of interest. For example, the camera 100A performs person detection processing for detecting a person from the captured image, sets the position of the attention point in the captured image to, for example, the face of the person based on the detection result of the person detection processing, and sets the position information of the attention point. may be obtained.
In this way, the information processing apparatus sets the position of the point of interest in the captured image based on the detection result of the person detection process, thereby saving the user the trouble of monitoring the image captured by the camera 100A and indicating the point of interest. can be done.

Furthermore, in each of the above-described embodiments, the case where the camera 100A transmits a command for instructing the imaging direction to the camera 100B by the imaging direction instruction processing has been described. However, the camera 100A may transmit the three-dimensional position of the gaze point in the coordinate system of the camera 100B to the camera 100B. Alternatively, camera 100A may transmit the three-dimensional position in the coordinate system of camera 100A to camera 100B, and camera 100B may coordinate-transform the three-dimensional position received from camera 100A into the coordinate system of camera 100B. good. In this way, even when the three-dimensional position of the point of interest is transmitted to camera 100B, camera 100B appropriately controls the imaging direction so that the point of interest specified from the captured image of camera 100A is the point of interest. It is possible to perform imaging with
Further, in each of the above-described embodiments, a camera link system that captures an object by linking a plurality of imaging devices has been described, but the present invention can be applied to systems other than camera link systems. That is, the three-dimensional position of the point of interest in the captured image of the camera 100A derived by the information processing device can also be used for processing other than the camera cooperation processing.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

DESCRIPTION OF SYMBOLS 100A, 100B... Imaging device, 200... Client apparatus, 300... Network, 401... Floor marker, 402... Wall marker, 1000... Imaging system

Claims (16)

a first acquisition means for acquiring area information for specifying the position of the floor area and the position of the wall surface area in the captured image captured by the first imaging device;
a second acquisition means for acquiring position information indicating the position of the point of interest in the captured image;
Using the area information acquired by the first acquisition means and the position information acquired by the second acquisition means, whether or not the target object corresponding to the target point is an object existing on the floor surface and deriving means for deriving the three-dimensional position of the point of interest based on the result of determination whether the object exists on a wall surface . determining an imaging direction of a second imaging device different from the first imaging device for imaging the point of interest derived by the deriving means as a gaze point; 2. The information processing apparatus according to claim 1, further comprising an instruction means for instructing. 3. The information processing apparatus according to claim 2, wherein said instruction means further determines an imaging angle of view of said second imaging device and instructs said second imaging device of said imaging angle of view. The derivation means is
deriving a straight line on which the point of interest may exist in real space based on the imaging conditions of the first imaging device and the position information acquired by the second acquisition means;
When it is determined that the object of interest corresponding to the point of interest is an object existing on the floor surface, among the derived points on the straight line , the height from the floor surface in real space is set in advance. 4. The information processing apparatus according to any one of claims 1 to 3, wherein the height position based on the height information obtained is derived as the three-dimensional position of the attention point. The first acquisition means is
5. The information processing apparatus according to any one of claims 1 to 4 , wherein identification information is extracted from the captured image to set the floor surface area and acquire the area information. The first acquisition means is
2. Markers are detected from the captured image, the floor area is set based on the detection results, and the area information is obtained based on marker information embedded in the markers. 5. The information processing device according to any one of 4 to 4 . 7. The information processing according to claim 6 , wherein the marker information includes at least one of information indicating whether the marker is the floor surface area or the wall surface area, and information about an installation position of the marker. Device. The first acquisition means is
5. The method according to any one of claims 1 to 4 , wherein the area information is acquired by displaying the captured image on a display device and receiving a setting of the floor surface area in the displayed captured image. The information processing device described. Further comprising person detection means for detecting a person from the captured image,
The second acquisition means is
9. The information processing apparatus according to any one of claims 1 to 8 , wherein the position of the point of interest in the captured image is set based on a detection result by the person detection means, and the position information is acquired. . The second acquisition means is
9. The position information according to any one of claims 1 to 8 , wherein the position information is obtained by displaying the captured image on a display device and receiving the setting of the attention point in the displayed captured image. information processing equipment. 5. The information processing apparatus according to claim 4 , wherein the height information is information indicating a height from the floor area in real space to the point of interest. 5. The information processing apparatus according to claim 4 , further comprising third obtaining means for receiving setting of said height information and obtaining said height information. The information processing apparatus according to any one of claims 1 to 12 , wherein the point of interest is a person or a part of a person. obtaining area information for specifying the position of the floor area and the position of the wall surface area in the captured image captured by the first imaging device;
obtaining position information indicating the position of the point of interest in the captured image;
Using the area information and the position information, it is determined whether the target object corresponding to the target point is an object existing in the floor area or an object existing in the wall surface area, and the determination is performed. and deriving the three-dimensional position of the point of interest based on the result of (1). a first imaging device;
a second imaging device different from the first imaging device;
a first acquisition means for acquiring area information for specifying the position of the floor area and the position of the wall area in the captured image captured by the first imaging device;
a second acquisition means for acquiring position information indicating the position of the point of interest in the captured image;
Using the area information acquired by the first acquisition means and the position information acquired by the second acquisition means, it is determined whether the target object corresponding to the target point is an object existing in the floor area. derivation means for deriving the three-dimensional position of the point of interest based on the determination result ;
an instruction means for determining an imaging direction of the second imaging device for imaging the point of interest derived by the deriving means as a gaze point, and instructing the second imaging device of the imaging direction;
An imaging system comprising: A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 13 .

Images