JP2023120758A - Object position estimation display device, method, and program - Google Patents

Abstract

To provide an object position estimation display device which can accurately and easily estimate the position of an object and also can display the estimated position of the object in an easily understandable manner.SOLUTION: An object position estimation display device of the present invention includes target point setting means, target point position estimation means, and display means 12. The target point setting means sets predetermined points of objects A1 and A2 in a captured image 21 as target points 26a and 26b. The target point position estimation means estimates the positions of the target points 26a and 26b on the overhead coordinate system based on a camera on the basis of positional relation information showing the positional relationship between the camera and a plane 31 and inside parameter information of the camera. Display means 12 displays an image 22 of an overhead plane and icons 27a and 27b showing the target points 26a and 26b on the image 22.SELECTED DRAWING: Figure 2

Description

The present invention relates to an object position estimation display device, method, and program for estimating and displaying the position of an object.

2. Description of the Related Art Conventionally, there has been used a device for estimating the position of an object such as a human based on an image obtained by imaging the object (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2020-136700 (claims 1 and 4, FIG. 2, etc.)

However, in the prior art described in Cited Document 1, when estimating the position of a person, it is necessary to make the person walk and obtain image position information in advance, so the estimation work is difficult. Moreover, since the position is estimated using different height data for each person, it is difficult to accurately estimate the position of the person. In addition, since the prior art described in Cited Document 1 does not disclose that an image is displayed on the display means, the user of the device cannot know the exact position of the object. Therefore, it is required to display an image to display the position of an object in an easy-to-understand manner.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object of the present invention is to provide an object position estimation display device capable of easily and accurately estimating the position of an object and displaying the estimated position of the object in an easy-to-understand manner. , an object position estimation display method and an object position estimation display program.

In order to solve the above problems, the invention according to claim 1 includes a camera that captures an image of an object and acquires the image, and display means that displays the image, and estimates and displays the position of the object. A device, wherein the camera captures an image of a marker placed on a plane, and a point on a marker coordinate system based on the captured marker is transformed into a point on a camera coordinate system based on the camera. , the height from the plane to the camera, and the angle between the perpendicular extending from the plane to the camera and the line segment extending from the camera to the marker are calculated to determine the positional relationship between the camera and the plane. positional relationship grasping means for grasping, target point setting means for setting a predetermined point of the object reflected in the captured image as a target point, positional relationship information grasped by the positional relationship grasping means, and internal parameters of the camera an object point position estimating means for estimating the position of the object point on a bird's-eye view coordinate system with the camera as a reference based on the information, and the display means displays an image of the bird's-eye view plane and displays an image on the image The gist of the present invention is an object position estimation display device characterized in that an icon indicating the target point is displayed on the .

According to the first aspect of the invention, the positional relationship comprehension means can determine the positional relationship between the camera and the plane based on the captured marker by simply placing the marker on the plane and taking an image of the placed marker with the camera. Grasp automatically. Further, the target point position estimating means automatically estimates the position of the target point on the overhead coordinate system with the camera as a reference based on the positional relationship information grasped by the positional relation grasping means. In other words, the position of the target point can be estimated only by setting the marker, so the position of the object can be easily estimated based on the estimated position of the target point. Further, the positional relationship grasping means calculates the height from the plane to the camera and the angle formed by the perpendicular extending from the plane to the camera and the line segment extending from the camera to the marker, thereby determining the positional relationship between the camera and the plane. Because of this, the position of the object can be accurately estimated. Moreover, the display means displays the image of the bird's-eye view plane and also displays the icon indicating the target point on the image. This makes it possible to display the estimated position of the object in an easy-to-understand manner.

Note that the object refers to an object that moves on a plane in the space to be imaged, and examples include bipedal humans (claim 2), living things such as quadrupedal animals, and inanimate objects such as automobiles. be able to. Examples of quadrupedal animals include domestic animals such as cows, horses, pigs, sheep, and goats. Further, the object point setting means may automatically set the object points, or may manually set the object points. If the object point setting means automatically sets the object points, the user does not have to set the object points, so that the user's workload can be reduced.

A second aspect of the invention is characterized in that, in the first aspect, the object is a human or an animal, and the object point is set at the feet of the human or the animal.

In the second aspect of the invention, the target points set by the target point setting means are set at the feet of humans or animals. ) are different from each other, the target point can be set to a point that is grounded on the plane. This makes it possible to accurately estimate the position of the target point on the overhead coordinate system with the camera as a reference.

The gist of the invention according to claim 3 is that in claim 1 or 2, an icon indicating the camera is displayed in addition to the icon indicating the target point on the image of the bird's-eye view plane.

In the invention according to claim 3, since the icon indicating the camera is displayed in addition to the icon indicating the target point, the user can not only know the exact position of the object but also which camera captured the object. (In other words, from which direction the image was taken) can be known. You can also know the exact position of the camera.

A fourth aspect of the present invention is characterized in that, in the third aspect, the display means displays the overhead plane image and the actual image captured by the camera side by side.

In the fourth aspect of the invention, since the image of the bird's-eye view plane and the actual image are displayed on the display means, by comparing both images, it is possible to determine whether or not all the objects appearing in the actual image are recognized. can be confirmed.

The invention according to claim 5 is the gist of the invention according to any one of claims 1 to 4, wherein the icon indicating the target point is assigned an identifier for specifying the type of the object. do.

According to the fifth aspect of the invention, even if a plurality of objects move intricately, it is possible to identify the type of object by checking the identifier assigned to each object. Here, characters, numbers, symbols, pictures, and the like can be used as identifiers.

The invention according to claim 6 is based on any one of claims 1 to 5, wherein a plurality of cameras are provided, and the target point position estimation means aligns the bird's-eye view coordinate systems of the adjacent cameras with the marker. The gist of the invention is to perform a process of connecting the images captured by the respective cameras by performing arithmetic processing for connecting via .

In the sixth aspect of the present invention, the bird's-eye view coordinate systems of adjacent cameras are connected to each other via markers that are shared by these cameras. Specifically, for example, points in the overhead coordinate system of one camera are transformed into points in the overhead coordinate system of the other camera to share the overhead coordinate system. This expands the range in which the position of the object can be grasped. As a result, for example, even if the space to be imaged is so large that one camera cannot cover the entire space, by capturing images together with other cameras, blind spots can be eliminated and the position of objects can be reliably determined in the entire space. can be grasped. Also, the accuracy of estimating the position of the object is improved. Moreover, since the bird's-eye view coordinate system is unified in accordance with one camera that serves as a reference, the position of the grasped object can be easily understood by the user.

A seventh aspect of the invention is characterized in that, in any one of the first to sixth aspects, the marker is a two-dimensional marker having a specific geometric feature.

According to the seventh aspect of the invention, since the marker is a two-dimensional marker, shape information is simpler than when the marker is three-dimensional, for example. As a result, the control for grasping the positional relationship between the camera and the plane based on the shape information of the marker is facilitated, so that the load on the positional relation grasping means can be reduced.

According to an eighth aspect of the present invention, an object position estimating display device including a camera for capturing an image of an object and display means for displaying the image is used to estimate and display the position of the object. a marker imaging step of capturing an image of a marker placed on a plane using the camera; and a point on a marker coordinate system based on the captured marker by Converting to a point on the coordinate system, and calculating the height from the plane to the camera and the angle formed by the perpendicular line extending from the plane to the camera and the line segment extending from the camera to the marker, a positional relationship grasping step of grasping the positional relation between the camera and the plane; an internal parameter inputting step of inputting the internal parameter information of the camera into a storage means; Based on the target point setting step to be set, the positional relationship information grasped in the positional relation grasping step, and the internal parameter information stored in the storage means, the A target point position estimation step of estimating the position of the target point; and a display step of displaying an image of a bird's-eye view plane on the display means and displaying an icon indicating the target point on the image. The gist of the present invention is an object position estimation display method. In the eighth aspect of the invention, the same effect as the first aspect can be obtained.

According to a ninth aspect of the invention, the camera is used in a processor for controlling an object position estimation display device comprising a camera for capturing an image of an object and obtaining an image, and display means for displaying the image. a marker imaging step of capturing an image of a marker placed thereon; converting a point on a marker coordinate system based on the captured marker into a point on a camera coordinate system based on the camera; A positional relationship for grasping the positional relationship between the camera and the plane by calculating the height to the camera and the angle formed by the perpendicular line extending from the plane to the camera and the line segment extending from the camera to the marker. a comprehension step; an internal parameter input step of inputting the internal parameter information of the camera into a storage means; a target point setting step of setting a predetermined point of the object appearing in the captured image as a target point; a target point position estimation step of estimating the position of the target point on a bird's-eye view coordinate system with respect to the camera, based on the positional relationship information grasped in and the internal parameter information stored in the storage means; and an object position estimation display program for displaying an image of a bird's-eye view plane on the display means and displaying an icon indicating the target point on the image. In addition, in the invention according to claim 9, the same effect as that of the above claim 1 can be achieved.

As detailed above, according to the first to ninth aspects of the present invention, the position of an object can be easily and accurately estimated, and the estimated position of the object can be displayed in an easy-to-understand manner.

1 is a schematic configuration diagram showing an object position estimation display device according to a first embodiment; FIG. Schematic which shows the display of a display. (a) is an explanatory diagram showing the marker of the first embodiment, (b) and (c) are explanatory diagrams showing the marker of another embodiment. 4 is a flowchart showing processing for estimating and displaying a human position. Explanatory drawing which shows the coordinate transformation by an external parameter. Explanatory drawing which shows the mode of estimation of a floor vector. Explanatory drawing which shows the bird's-eye view coordinate system based on the camera. Explanatory drawing which shows the mode of position estimation. Explanatory drawing which shows the state of derivation|leading-out of a direction vector. FIG. 9 is an explanatory diagram showing how the same marker is imaged by different cameras in the second embodiment; FIG. 4 is a bird's-eye view showing how a marker is captured; FIG. 11 is a schematic configuration diagram showing five cameras and two markers in the third embodiment; A directed graph showing a connection state between cameras (that is, a state in which the bird's-eye view coordinate systems of the cameras are shared). Explanatory drawing which shows the installation aspect of the marker in other embodiment.

[First embodiment]
BEST MODE FOR CARRYING OUT THE INVENTION A first embodiment embodying the present invention will be described in detail below with reference to the drawings.

As shown in FIGS. 1 and 2, the object position estimation display device 1 of this embodiment is a device that estimates and displays the positions of humans A1 and A2 (objects). The object position estimation display device 1 also includes a camera 11 and a display 12 (display means). The camera 11 of this embodiment is a fixed monocular camera, and captures images 21 (see FIG. 2) of people A1 and A2 in the room. The camera 11 then outputs image data of the acquired image 21 .

A marker 32 (see FIG. 3A) is installed on the floor surface 31 (flat surface) of the room. The marker 32 is a two-dimensional square marker (ArUco marker in this embodiment) having specific geometric features and is attached to the floor surface 31 . Note that the marker 32 may be simply placed on the floor surface 31 and may not be moved. Also, the markers 32 may be removed when performing the actual position estimation.

As shown in FIG. 2, on the left side of the display 12, an actual image 21 captured by the camera 11 is displayed. In this embodiment, a person A1 is shown near the center of the image 21, and another person A2 is shown on the left side of the image 21. FIG. Also, on the image 21, the person A2 is positioned slightly closer to the person A1 than the person A1. Further, on the image 21, a rectangular frame 26c for recognizing the person A1 is displayed so as to surround the person A1, and a rectangular frame 26d for recognizing the person A2 is displayed so as to surround the person A2. Object points 26 a and 26 b are displayed at the feet of the humans A 1 and A 2 appearing in the image 21 . The target points 26a and 26b are set on predetermined points of the humans A1 and A2, more specifically, on the boundary line between the humans A1 and A2 and the floor surface 31, and between the left and right feet of the humans A1 and A2. This is the point to be set. In the present embodiment, a green object point 26a is set at the foot of the person A1 and at the center of the lower side forming the frame 26c, and is the foot of the person A2 and forms the frame 26d. A red target point 26b is set in the central portion of the lower side.

On the other hand, on the right side of the display 12, an overhead plane image 22 is displayed. The bird's-eye view plane image 22 is an image showing an image when the floor surface 31 is viewed from directly above. Grid lines 25 formed by a plurality of horizontal lines 23 and a plurality of vertical lines 24 are displayed on the overhead plane image 22 . Each horizontal line 23 extends linearly in the horizontal direction and is provided at regular intervals in the vertical direction. Each vertical line 24 extends linearly in the vertical direction and is provided at regular intervals in the horizontal direction. A square formed by two adjacent horizontal lines 23 and two adjacent vertical lines 24 indicates a square grid of 1 m long by 1 m wide.

Furthermore, circular icons 27a and 27b indicating object points 26a and 26b displayed on the actual image 21 are displayed on the bird's-eye view plane image 22 . In this embodiment, a green icon 27a corresponding to the green target point 26a is displayed slightly above the center of the image 22, and a red icon 27a corresponding to the red target point 26b is displayed slightly left of the center of the image 22. icon 27b is displayed. Furthermore, the icons 27a and 27b are provided with identifiers 28a and 28b for identifying the types of the humans A1 and A2 (that is, who they are). In this embodiment, the green icon 27a is given the green number "1" as the identifier 28a, and the red icon 27b is given the green number "0" as the identifier 28b. In addition to icons 27a and 27b indicating target points 26a and 26b, an icon 29 indicating the camera 11 is displayed on the image 22 of the bird's-eye view plane. The icon 29 of the present embodiment is a triangular blue icon and is displayed in the lower center of the image 22 .

Next, the electrical configuration of the object position estimation display device 1 will be described.

As shown in FIG. 1, the object position estimation display device 1 includes a personal computer (not shown), and the personal computer includes a control device 40 (processor) that controls the entire device. The control device 40 is composed of a well-known computer including a CPU 41, a ROM 42, a RAM 43, and the like. The display 12 of the personal computer and the keyboard 13 of the personal computer are electrically connected to the CPU 41 . In this embodiment, the camera 11 is electrically connected to the CPU 41 by connecting the camera 11 to the control device 40 via a USB (Universal Serial Bus) cable. The image 21 acquired by the camera 11 is stored in the RAM 43 . The ROM 42 also stores a program (object position estimation display program) for controlling the object position estimation display device 1 .

Next, a method of estimating and displaying the positions of humans A1 and A2 using the object position estimation display device 1 (object position estimation display method) will be described.

Next, a method for grasping the positional relationship between the camera 11 and the floor surface 31 in the room will be described. First, the CPU 41 of the control device 40 uses the camera 11 to capture an image of the marker 32 (see FIG. 3A) placed on the floor surface 31 in step S10 (marker imaging step) shown in FIG.

By the way, as shown in FIG. 5, the positional relationship between the marker 32 and the camera 11 is represented by external parameters consisting of a rotation matrix R and a translation vector t. In addition, the marker 32 has a coordinate system centered on itself (marker coordinate system). On the other hand, the camera 11 also has a coordinate system centered on itself (camera coordinate system).

Therefore, in step S20 (positional relationship comprehension step) shown in FIG. Convert to a point (x, y, z) on the system (see FIG. 5). A point (x, y, z) on the camera coordinate system can be calculated by the following equation (1).

Next, the CPU 41 calculates the height h from the floor surface 31 to the camera 11, and also calculates the angle θ _t between the perpendicular line extending from the floor surface 31 to the camera 11 and the line segment extending from the camera 11 to the marker 32. Thus, the positional relationship between the camera 11 and the floor surface 31 is grasped (see FIG. 6). That is, the CPU 41 has a function as "positional relationship grasping means". Note that the positional relationship between the camera 11 and the floor surface 31 is expressed as a floor surface vector F on the camera coordinate system. The length of floor vector F is equal to the distance from camera 11 to floor 31 . The direction of the floor vector F is opposite to the direction of the Z-axis of the marker coordinate system (that is, perpendicular to the floor 31).

The floor vector F is obtained as follows using the geometric properties of the vectors. First, in the camera coordinate system, a vector f in the direction opposite to the Z axis is calculated. Assuming that _emz is a unit vector in the Z-axis direction in the marker coordinate system, the vector f can be calculated based on the rotation matrix R by the following equation (2).

Next, the angle θ _t formed between the translation vector t and a vertical line extending from the floor 31 to the camera 11 and the height h from the floor 31 to the camera 11 are calculated. First, the angle θ _t can be calculated by the following equation (3) from the inner product of the translation vector t and the vector f.

Also, the height h can be calculated by the following equation (4).

Next, the floor vector F is calculated by scaling the vector f. The floor vector F can be calculated by the following equation (5).

Next, a method of estimating and displaying the positions of the humans A1 and A2 with reference to the camera 11 will be described below. In this embodiment, a method of estimating and displaying the position of the person A1 appearing in the vicinity of the central portion of the captured image 21 will be described. Specifically, based on the calculated floor vector F, the position of the target point 26a on the overhead coordinate system with the camera 11 as a reference is obtained (see FIG. 7). The overhead coordinate system is a coordinate system in which the camera 11 and the floor surface 31 are viewed vertically, and is represented as an xy coordinate system in which the y-axis is in the same direction as the camera 11 and the x-axis is in the direction orthogonal to the y-axis. Three-dimensionally, it corresponds to a left-handed coordinate system with the floor vector F as the z-axis.

Note that the internal parameters of the camera 11 are necessary for position estimation. The internal parameters are matrices consisting of focal lengths f _x and f _y and optical centers c _x and c _y of the camera 11 . A point (x, y, z) on the camera coordinate system can be transformed into a point (a, b) on the image plane by the following equation (6).

In addition, the internal parameters can be calculated using the CalibrateCamera function provided by OpenCV (Open Source Computer Vision Library), calculation from the data sheet of the camera 11, or the like. Then, the CPU 41 stores the internal parameter information (internal parameter information) of the camera 11 in the RAM 43 in step S30 (internal parameter input step) shown in FIG. That is, the RAM 43 has a function as a “storage means” for storing internal parameter information of the camera 11 .

In subsequent step S40 (target point setting step), the CPU 41 sets a predetermined point set at the feet of the person A1 appearing in the captured image 21 as the target point 26a. That is, the CPU 41 has a function as "target point setting means". More specifically, the CPU 41 of this embodiment functions as a learning section. Based on the image data of the image 21 acquired by the camera 11, the learning unit learns in advance how to recognize the set point (the feet of the person A1) of the target point 26a, and obtains the learning result. Then, the learning unit stores data indicating the obtained learning result in the RAM 43 . Further, the CPU 41 determines the method of setting the target point 26a based on the learning result stored in the RAM 43, and causes the CPU 41 to set the target point 26a according to the determined content. In this way, the method of setting the target point 26a is optimized based on the learning result of the learning section. can be set. In addition, if the learning unit has more chances to learn, the method of setting the target point 26a is optimized, so that the target point 26a can be set more accurately.

In subsequent step S50 (target point position estimation step), the CPU 41 moves the camera 11 based on the positional relationship information (positional relationship information) ascertained in the positional relationship ascertaining step and the internal parameter information stored in the RAM 43. The position of the target point 26a on the reference overhead coordinate system is estimated. That is, the CPU 41 has a function as "object point position estimation means". Specifically, as shown in FIGS. 8 and 9, the position (x, y) of the target point 26a on the coordinate system with the camera 11 as a reference is calculated as follows. First, an object point (u, v) shown in FIG. 9 is determined by, for example, taking an image of a person A1. This target point (u, v) is assumed to exist on the overhead plane.

なお、図９では、画像平面に映った人間Ａ１の足元を対象点（ｕ，ｖ）としたうえで、対象点（ｕ，ｖ）に対する方向ベクトルｄを求めている。次に、次式（８）により、床面ベクトルＦと方向ベクトルｄとのなす角度θ_ｄを算出する。

Then, a three-dimensional direction vector d for the object point (u, v) is calculated by the following equation (7).

In FIG. 9, the feet of the person A1 reflected on the image plane are set as the target point (u, v), and the direction vector d for the target point (u, v) is obtained. Next, the angle θ _d between the floor vector F and the direction vector d is calculated by the following equation (8).

Furthermore, after calculating the linear distance d to the human A1 by the following equation (9), the distance _ld on the floor surface 31 is calculated by the following equation (10).

なお、ベクトルｇは、カメラ座標系における、俯瞰平面上での原点から対象点２６ａまでの位置を示している。 Then, after calculating D by scaling the direction vector d according to the following equation (11), the vector g is calculated according to the following equation (12).

The vector g indicates the position from the origin on the overhead plane to the target point 26a in the camera coordinate system.

Then, using equations similar to equations (8) to (12), a vector g _y is obtained for the unit vector e _cz in the z-axis direction on the camera coordinate system (specifically, equation (8 ) to replace the direction vector d in equation (12) with the unit vector e _z and the vector g with the vector g _y ). Note that, as shown in FIG. 8, the vector g _y is parallel to the y-axis of the overhead coordinate system in the camera coordinate system.

ここで、Ｒ_Ｆは回転行列であり、床面ベクトルＦを（Ｆ_ｘ，Ｆ_ｙ，Ｆ_ｚ）で示したとき、回転行列Ｒ_Ｆは次式（１４）にて算出することができる。

また、ベクトルｇ_ｘは、カメラ座標系において、俯瞰座標系のｘ軸と平行になっている。 Next, the vector g _x obtained by rotating the vector g _y around the floor vector F by 90° is obtained using the following equation (13).

Here, R _F is a rotation matrix, and when the floor vector F is represented by (F _x , F _y , F _z ), the rotation matrix R _F can be calculated by the following equation (14).

Also, the vector _gx is parallel to the x-axis of the overhead coordinate system in the camera coordinate system.

Next, from the inner product, the angle θ formed by vector g and vector g _y is calculated using the following equations (15) and (16).

As a result, the position (x, y) of the object point 26a was obtained as a polar coordinate display from the distance _ld and the angle θ, so (x, y) becomes x= _ld sin θ and y= _ld cos θ. . As a result, the position of the person A1 with respect to the camera 11 is estimated.

Thereafter, in step S60 (display step) shown in FIG. 4, the CPU 41 displays the bird's-eye view plane image 22 on the display 12 and displays an icon 27a indicating the target point 26a on the image 22. FIG. Thereby, the estimated position of the person A1 is displayed.

Therefore, according to this embodiment, the following effects can be obtained.

(1) In the object position estimation display device 1 of the present embodiment, the marker 32 is placed on the floor surface 31 and the camera 11 captures an image of the placed marker 32 . are automatically grasped using external parameters, and the positional relation between the camera 11 and the floor surface 31 is automatically grasped. Furthermore, the CPU 41 automatically estimates the positions of the target points 26a and 26b on the bird's-eye view coordinate system with the camera 11 as a reference based on the grasped positional relationship information. In other words, the positions of the target points 26a and 26b can be estimated only by setting the marker 32. FIG. Therefore, the positions of the humans A1 and A2 can be easily estimated based on the estimated positions of the target points 26a and 26b. Moreover, since the CPU 41 estimates the positions of the object points 26a and 26b based on the internal parameters of the camera 11 in addition to the positional relationship information which is the external parameter, the positions of the humans A1 and A2 can be accurately estimated. . Furthermore, the CPU 41 determines the height h (see FIG. 6) from the floor surface 31 to the camera 11, and the angle θ _t ( 6), the positional relationship between the camera 11 and the floor surface 31 is grasped. Therefore, the positions of the humans A1 and A2 can be estimated more accurately. In addition, the display 12 displays an overhead view plane image 22 and displays icons 27a and 27b indicating target points 26a and 26b on the image 22. FIG. This makes it possible to display the estimated positions of the humans A1 and A2 in an easy-to-understand manner.

(2) Cited Document 1 discloses a technique for estimating the position of a subject based on the height of the subject (human) in image data captured by a camera (Claim 4 of Cited Document 1). , see FIG. 5, etc.). However, the heights (heights) of subjects are different. Also, if the subject is close, the subject in the image data will be high, and if the subject is far away, the subject in the image data will be low. Therefore, it is difficult to accurately estimate the position of the subject based on the height of the subject.

On the other hand, in the present embodiment, the target points 26a and 26b set by the CPU 41 are set at the feet of the humans A1 and A2. Even if they are different, the target points 26 a and 26 b can be set at locations on the floor surface 31 . As a result, the positions of the object points 26a and 26b on the bird's-eye view coordinate system with the camera 11 as a reference can be accurately estimated.

(3) The display 12 of the present embodiment displays a bird's-eye view image 22, and on the image 22, grid lines 25 forming a grid of 1 m long by 1 m wide are displayed. Therefore, by displaying the icons 27a and 27b indicating the target points 26a and 26b on the grid line 25, the positions of the humans A1 and A2 can be known accurately.

(4) In the present embodiment, the CPU 41 determines the height h from the floor 31 to the camera 11 (see FIG. 6), the vertical line extending from the floor 31 to the camera 11, and the line segment extending from the camera 11 to the marker 32. The positional relationship between the camera 11 and the floor surface 31 is grasped by calculating the angle θ _t (see FIG. 6) formed by the two. Therefore, the positions of the humans A1 and A2 can be easily estimated with high accuracy without using a special dedicated camera. In this embodiment, a relatively inexpensive monocular camera is used as the camera 11 for capturing images of the humans A1 and A2. Therefore, the cost of the device can be reduced compared to the case where an expensive depth camera or the like is used for imaging the humans A1 and A2.

[Second embodiment]
A second embodiment embodying the present invention will be described below with reference to FIGS. 10 and 11. FIG. Here, the description will focus on the parts that are different from the first embodiment. This embodiment differs from the first embodiment in that the object position estimation display device 1 has two cameras 51 and 52 (first camera 51 and second camera 52). .

First, a method of connecting the two cameras 51 and 52 will be described. As a premise, the cameras 51 and 52 to be used need to know the positional relationship with the floor surface 31 in the form of floor surface vectors F ₁ and F ₂ . Moreover, the marker 61 is installed on the floor surface 31, and it is necessary that the type (shape) of the marker 61 can be identified by an identifier (ID in this embodiment).

First, an image of the marker 61 is captured by the respective cameras 51 and 52, and the positional relationship (R ₁ , t ₁ ) between the marker 61 and the first camera 51, ₂ , t ₂ ). At that time, it is confirmed by the ID of the marker 61 whether or not the two cameras 51 and 52 are imaging the same marker 61 .

When the marker 61 is imaged in this way, the direct positional relationship between the cameras 51 and 52 is unknown, but the world coordinate system (marker coordinate system) is shared. Therefore, it is possible to convert points seen from the respective cameras 51 and 52 into points seen from the marker 61 and compare them, or to convert them into points seen from the other camera.

Therefore, the CPU 41 performs processing for connecting the images 21 captured by the respective cameras 51 and 52 by performing arithmetic processing for connecting the overhead coordinate systems of the adjacent cameras 51 and 52 via the markers 61 . Here, "the process of connecting the images 21 captured by the cameras 51 and 52" means that the positional relationship between the cameras 51 and 52 is grasped and one camera (the second camera 52 in this embodiment) is viewed from above. This is a process of transforming a point (x ₂ , y ₂ ) in the coordinate system into a point (x ₁ , y ₁ ) in the overhead coordinate system of the other camera (the first camera 51 in this embodiment). Note that processing may be performed to convert the point (x ₁ , y ₁ ) of the overhead coordinate system of the first camera 51 to the point (x ₂ , y ₂ ) of the overhead coordinate system of the second camera 52. , both the point (x ₁ , y ₁ ) of the bird's-eye coordinate system of the first camera 51 and the point (x ₂ , y ₂ ) of the bird's-eye coordinate system of the second camera 52 are converted to other points of the bird's-eye coordinate system A conversion process may be performed.

As shown in FIG. 11, since the overhead coordinate systems of the cameras 51 and 52 belong to the same floor surface 31, conversion between the overhead coordinate systems is performed using a rotation matrix R and a translation matrix T by affine transformation. can be done. Specifically, using the properties of the marker coordinate system described above, the affine transformation matrix A is obtained by the following procedure.

First, the floor vectors F ₁ and F ₂ of the respective cameras 51 and 52 are inversely transformed by the following equation (17) to be transformed into floor vectors F _M1 and F _M2 viewed from the marker coordinate system.

ここで、squeeze（Ａ） は、次式（１９）によって算出される。

Next, the unit vectors e _z1 and e _z2 of the respective cameras 51 and 52 are converted to the direction vectors d _M1 and d _M2 of the cameras 51 and 52 in the bird's-eye coordinate system viewed from the marker coordinate system using the following equation (18). Convert to

Here, squeeze(A) is calculated by the following equation (19).

ここで、sign（ａ，ｂ）は、次式（２１）によって算出される。

Furthermore, the translation matrix T is calculated as follows. First, let e _My be a unit vector on the marker coordinate system, and let θ _T be the angle formed by the direction vector d _M1 and the unit vector e _My , then the angle θ _T is calculated by the following equation (20).

Here, sign(a, b) is calculated by the following equation (21).

Furthermore, the affine transformation matrix _A1 for transforming the marker coordinate system into a coordinate system having a common origin with the marker coordinate system and a coordinate axis parallel to the overhead coordinate system for the first camera 51 is obtained by the following equation (22). Calculated.

Also, on the above coordinate system, the translation vector t=(t ₁ , t ₂ , t ₃ ) from the second camera 52 to the first camera 51 is calculated by the following equation (23).

Then, the translation matrix T is calculated by the following equation (24).

Furthermore, the rotation matrix R is calculated as follows. First, assuming that the angle formed by the direction vector _dM1 and the direction vector _dM2 is _θR , the angle _θR is calculated by the following equation (25).

Then, the rotation matrix R is calculated by the following equation (26).

なお、第２のカメラ５２の俯瞰座標系上の点（ｘ_２，ｙ_２）から第１のカメラ５１の俯瞰座標系上の点（ｘ_１，ｙ_１）への変換は、次式（２８）によって算出される。

Next, the affine transformation matrix A obtained by synthesizing the rotation matrix R and the translation matrix T is obtained by the following equation (27).

Note that the conversion from the point (x ₂ , y ₂ ) on the bird's-eye coordinate system of the second camera 52 to the point (x ₁ , y ₁ ) on the bird's-eye coordinate system of the first camera 51 is expressed by the following equation (28) ).

Therefore, according to the present embodiment, the bird's-eye view coordinate systems of the adjacent cameras 51 and 52 are connected via the marker 61 that is shared by both cameras 51 and 52 . Specifically, for example, points in the overhead coordinate system of one camera (second camera 52) are converted into points in the overhead coordinate system of the other camera (first camera 51), and commonality. This expands the range in which a person's position can be grasped. As a result, for example, even if the imaging target space is so large that one camera alone cannot cover the entire space (for example, if the area such as the corner of the image 21 cannot be covered), the image can be captured together with other cameras. As a result, blind spots are eliminated, and the position of the person can be reliably grasped in the entire space. Also, the accuracy of estimating the position of a person is improved. Moreover, since the bird's-eye view coordinate system is unified in accordance with one camera that serves as a reference, the grasped position of the person can be easily understood by the user.

[Third Embodiment]
A third embodiment embodying the present invention will be described below with reference to FIGS. 12 and 13. FIG. Here, the description will focus on the portions that are different from the second embodiment. In this embodiment, the object position estimation device 1 includes three or more (here, five) cameras 71 to 75 (first camera 71, second camera 72, third camera 73, fourth camera 74, It differs from the second embodiment in that it has a fifth camera 75).

For example, when connecting three or more cameras, that is, when sharing the bird's-eye view coordinate system of the cameras, if the same marker can be seen from all the cameras, the method of the second embodiment can be applied. is possible. However, such cases are rare, and connection is often impossible due to restrictions on camera positions and angles of view. Therefore, in order to solve this problem, it is necessary to use a plurality of markers to share the bird's-eye view coordinate systems of the cameras.

For example, in the object position estimation display device 70 shown in FIG. 12, two markers 81A and 81B are provided for five cameras 71-75. The first to third cameras 71 to 73 see a marker 81A, and the third to fifth cameras 73 to 75 see a marker 81B.

In this case, for each of the markers 81A and 81B, after checking the connection state between the cameras in which the same marker is visible, the connection is repeated for the combinations for which the connection is known. For example, in FIG. 12, when the connection between the third camera 73 and the first camera 71 and the connection between the fifth camera 75 and the third camera 73 are already known, the fifth camera A method of connecting 75 and the first camera 71 will be described. First, the CPU 41 connects the bird's-eye coordinate system of the fifth camera 75 and the bird's-eye coordinate system of the third camera 73 via the marker 81B, so that the point of the bird's-eye coordinate system of the fifth camera 75 becomes the third 3 into points on the overhead coordinate system of the camera 73 . Next, the CPU 41 connects the bird's-eye coordinate system of the third camera 73 and the bird's-eye coordinate system of the first camera 71 via the marker 81A, so that the converted point of the bird's-eye coordinate system of the third camera 73 is displayed. is transformed into a point of the overhead coordinate system of the first camera 71 . As a result, the bird's eye coordinate system of the fifth camera 75 and the bird's eye coordinate system of the first camera 71 are connected via the markers 81A and 81B, and the range in which the position of the person can be grasped is expanded step by step. .

Such a connection state between the cameras 71 to 75 (that is, a state in which the bird's-eye view coordinate systems of the cameras 71 to 75 are shared) can be easily shown using a directed graph shown in FIG. Each node is a number assigned to each of the cameras 71 to 75, and the path connecting each node (the arrow in FIG. 13) means that the connections between the cameras are known. Each route is weighted by a transformation matrix A corresponding to the connection. In FIG. 13, the first to third cameras 71 to 73 are connected via a common marker 81A, and the third to fifth cameras 73 to 75 are connected via a common marker 81B. It should be noted that some routes connecting nodes are omitted in FIG.

When a directed graph is used, the connection from a specific camera S to a target camera T can be obtained by examining the route from the camera S to the camera T and then multiplying the weights on the route in order. Methods for representing and searching such directed graphs on computers are well known and easy to implement.

Also, depending on how the markers 81A and 81B and the cameras 71 to 75 are installed, there may be cases where there is no connection between specific cameras (when the directed graph is divided), but such cases can be easily detected. be able to.

Note that each of the above embodiments may be modified as follows.

The cameras 11, 51, 52, 71 to 75 in each of the above-described embodiments are video cameras that capture moving images, but they may be cameras that capture still images. Further, although the cameras 11, 51, 52, 71 to 75 in each of the above embodiments are monocular cameras, they may be other cameras such as a compound eye camera. Furthermore, a depth camera or a stereo camera that can measure the depth of an image may be used. Furthermore, an existing camera originally installed there may be used to estimate the object position.

- In each of the above embodiments, the target points 26a and 26b set for the humans A1 and A2 were set at the feet of the humans A1 and A2, but are set at other locations such as the chests and heads of the humans A1 and A2. may have been If the heights (statures) and postures of the humans A1 and A2 are known, the positions of the humans A1 and A2 can be estimated even if the target points 26a and 26b are not set at the feet of the humans A1 and A2. Become.

- Although the object position estimation display devices 1 and 70 of each of the above embodiments estimate the positions of the humans A1 and A2, they may estimate the positions of livestock (objects) such as cows. In this case, the CPU 41 as the target point setting means sets the target point at the feet of the cow shown in the image 21 . For example, when there are three legs of a cow reflected in the image 21, the CPU 41 sets an object point at an intermediate position between the leftmost cow leg and the rightmost cow leg. Then, the CPU 41 estimates the position of the target point on the overhead coordinate system with the camera as a reference. Also, a system for estimating the health condition of livestock may be constructed using the object position estimation display devices 1 and 70 . Specifically, whether or not the cow is sick may be determined based on the speed or the like of the cow shown in the image 21 . Further, based on the image data of the image 21, it may be determined whether or not the cow is in heat by determining whether or not the cow is in contact with another cow. In addition, you may apply to livestock other than a cow.

- The markers 32 , 61 , 81A, and 81B in each of the above-described embodiments are installed parallel to the floor surface 31 . However, if the floor surface on which the livestock is present is uneven, or if the floor surface is stained with livestock manure, it is difficult to place the markers parallel to the floor surface 31 . Therefore, for example, as shown in FIG. 14, the marker 91 may be placed on the floor surface 31 while being inclined by a predetermined angle (here, 90°) with respect to the floor surface 31 . The data of the predetermined angle must be preset in the object position estimation display device 1, 70 (ROM 42).

- As shown in FIGS. 3(b) and 3(c), markers 33 and 34 having different geometric features (patterns) from the marker 32 of the first embodiment may be used. Also, the marker 32 in the first embodiment is a two-dimensional ArUco marker, but it may be a QR code (registered trademark of Denso Wave Co., Ltd.) or another two-dimensional marker such as AprilTag. . Furthermore, other two-dimensional objects such as pictures, letters, symbols, etc. may be used as markers. Also, a three-dimensional object such as a projection that exists on the floor surface 31 may be used as a marker.

Next, in addition to the technical ideas described in the claims, technical ideas grasped by the above-described embodiments are listed below.

(1) In any one of claims 1 to 7, the positional relationship grasping means uses external parameters consisting of a rotation matrix and a translation vector to determine points on the marker coordinate system with the imaged marker as a reference. into a point on a camera coordinate system with the camera as a reference.

(2) In any one of claims 1 to 7, there is provided storage means for storing the internal parameter information, and the target point position estimation means stores the positional relationship information grasped by the positional relation grasping means and the stored and the internal parameter information stored in means for estimating the position of the target point.

(3) In the technical idea (2), the object position estimation display device is characterized in that the target point position estimation means calculates the internal parameter information and stores it in the storage means.

Reference Signs List 1, 70 Object position estimation display device 11, 51, 52, 71, 72, 73, 74, 75 Camera 12 Display 21 as display means Actual image 22 Bird's eye plane images 26a, 26b Object point 27a, 27b... Icons 28a, 28b indicating target points 29... Icons indicating cameras 31... Floor surfaces 32, 33, 34, 61, 81A, 81B, 91... Markers 40... Control device 41 as a processor ... CPU as positional relationship grasping means, target point setting means, and target point position estimating means
43... RAM as storage means
A1, A2... Human as an object h... Height from the plane to the camera θt... Angle between the perpendicular extending from the plane to the camera and the line segment extending from the camera to the marker

In order to solve the above problems, the invention according to claim 1 includes a fixed camera that captures an image of an object and acquires the image, and display means that displays the image, and estimates the position of the object. A device for displaying, wherein the camera is installed on a plane and captures an image of an AR marker, which is a two-dimensional marker having a specific geometric feature, and the marker coordinate system is based on the captured marker. Using extrinsic parameters consisting of a three-dimensional rotation matrix that rotates a point around the origin of the marker coordinates and a translation vector that translates the origin of the marker coordinates to the origin of the camera coordinate system relative to the camera, Using a determinant that transforms a point on the marker coordinate system with reference to the marker into a point on the camera coordinate system with reference to the camera, first, in the camera coordinate system, Z A vector opposite to the axis is calculated based on the three-dimensional rotation matrix, and then a perpendicular extending from the plane to the camera and a line segment extending from the camera to the marker are calculated by the opposite vector and the translation vector. and a height from the plane to the camera is calculated from the reverse vector and the angle. a positional relationship grasping means for grasping the positional relation between the camera and the plane by calculating a vertical vector extending vertically to the camera; means, the vertical vector as the positional relationship information grasped by the positional relation grasping means, and internal parameter information of the camera, estimating the position of the target point on a bird's-eye view coordinate system with the camera as a reference. and object point position estimation means, wherein the display means displays an image of a bird's-eye view plane and displays an icon indicating the object point on the image. do.

In the case of the present invention, since the marker is a two-dimensional marker having specific geometric features, shape information is simpler than when the marker is, for example, three-dimensional. As a result, the control for grasping the positional relationship between the camera and the plane based on the shape information of the marker is facilitated, so that the load on the positional relation grasping means can be reduced.

According to a seventh aspect of the present invention, the position of an object is estimated by using an object position estimation display device that includes a fixed camera that captures an image of an object and acquires the image, and display means that displays the image. a marker imaging step of imaging an AR marker, which is a two-dimensional marker that is placed on a plane and has specific geometric features, using the camera; and A three-dimensional rotation matrix that rotates a point on the marker coordinate system used as a reference around the origin of the marker coordinates, and a translation vector that translates the origin of the marker coordinates to the origin of the camera coordinate system based on the camera. A determinant for transforming a point on the marker coordinate system with the marker as a reference into a point on the camera coordinate system with the camera as the reference, using an extrinsic parameter consisting of A vector opposite to the Z-axis perpendicular to the plane is calculated based on the three-dimensional rotation matrix, and then a perpendicular extending from the plane to the camera and the After calculating the angle formed by the line segment extending from the camera to the marker, and calculating the height from the plane to the camera from the reverse vector and the angle, the reverse vector and the height are calculated. a positional relationship comprehension step of comprehending the positional relationship between the camera and the plane by calculating a vertical vector extending vertically from the camera to the plane based on; and inputting internal parameter information of the camera into a storage means. an internal parameter input step, a target point setting step of setting a predetermined point of the object appearing in the captured image as a target point, the vertical vector as the positional relationship information grasped in the positional relation grasping step, and the storage means a target point position estimation step of estimating the position of the target point on the bird's-eye view coordinate system based on the camera based on the internal parameter information stored in the display means; and displaying the bird's-eye view plane image on the display means and a display step of displaying an icon indicating the target point on the image. It should be noted that the invention according to claim 7 can achieve the same effect as that of claim 1 above.

According to an eighth aspect of the present invention, the camera is used as a processor for controlling an object position estimation display device comprising a fixed camera for capturing an image of an object and obtaining an image, and display means for displaying the image. , a marker imaging step of capturing an image of an AR marker, which is a two-dimensional marker placed on a plane and having specific geometrical characteristics ; Using extrinsic parameters consisting of a three-dimensional rotation matrix that rotates around the origin of the coordinates and a translation vector that translates the origin of the marker coordinates to the origin of the camera coordinate system relative to the camera, Using a determinant that converts a point on the marker coordinate system to a point on the camera coordinate system based on the camera, is calculated based on the three-dimensional rotation matrix, and then the angle formed by the perpendicular extending from the plane to the camera and the line segment extending from the camera to the marker is calculated by the reverse vector and the translation vector. and calculating a height from the plane to the camera from the reverse vector and the angle, and then a vertical line extending vertically from the camera to the plane based on the reverse vector and the height. a positional relationship grasping step of grasping the positional relation between the camera and the plane by calculating a vector ; an internal parameter inputting step of inputting internal parameter information of the camera into a storage means; and the object appearing in the captured image. Based on the target point setting step of setting a predetermined point of as a target point, the vertical vector as the positional relationship information grasped in the positional relation grasping step, and the internal parameter information stored in the storage means an object point position estimating step of estimating the position of the object point on the bird's-eye coordinate system with the camera as a reference; The gist thereof is an object position estimation display program for executing a display step of displaying the . In the eighth aspect of the invention, the same effect as the first aspect can be obtained.

As detailed above, according to the inventions described in claims 1 to 8 , the position of an object can be easily and accurately estimated, and the estimated position of the object can be displayed in an easy-to-understand manner.

In order to solve the above problems, the invention according to claim 1 includes a fixed camera that captures an image of an object and acquires the image, and display means that displays the image, and estimates the position of the object. A device for displaying, wherein the camera is installed on a plane and captures an image of an AR marker, which is a two-dimensional marker having a specific geometric feature, and the marker coordinate system is based on the captured marker. Using extrinsic parameters consisting of a three-dimensional rotation matrix that rotates a point around the origin of the marker coordinates and a translation vector that translates the origin of the marker coordinates to the origin of the camera coordinate system relative to the camera, Using a determinant that transforms a point on the marker coordinate system with reference to the marker into a point on the camera coordinate system with reference to the camera, first, in the camera coordinate system, Z A vector opposite to the axis is calculated based on the three-dimensional rotation matrix, and then a perpendicular extending from the plane to the camera and a line segment extending from the camera to the marker are calculated by the opposite vector and the translation vector. and the height from the plane to the camera is calculated from the translation vector and the angle. a positional relationship grasping means for grasping the positional relationship between the camera and the plane by calculating a vertical vector extending in the direction of the object; an object for estimating the position of the target point on a bird's-eye view coordinate system with respect to the camera, based on the vertical vector as the positional relationship information grasped by the positional relation grasping means and the internal parameter information of the camera; Point position estimating means, wherein the display means displays an image of a bird's-eye view plane and displays an icon indicating the target point on the image.

According to a seventh aspect of the present invention, the position of an object is estimated by using an object position estimation display device that includes a fixed camera that captures an image of an object and acquires the image, and display means that displays the image. a marker imaging step of imaging an AR marker, which is a two-dimensional marker that is placed on a plane and has specific geometric features, using the camera; and A three-dimensional rotation matrix that rotates a point on the marker coordinate system used as a reference around the origin of the marker coordinates, and a translation vector that translates the origin of the marker coordinates to the origin of the camera coordinate system based on the camera. A determinant for transforming a point on the marker coordinate system with the marker as a reference into a point on the camera coordinate system with the camera as the reference, using an extrinsic parameter consisting of A vector opposite to the Z-axis perpendicular to the plane is calculated based on the three-dimensional rotation matrix, and then a perpendicular extending from the plane to the camera and the After calculating the angle formed by the line segment extending from the camera to the marker, and calculating the height from the plane to the camera from the translation vector and the angle, based on the reverse vector and the height a positional relation grasping step of grasping the positional relation between the camera and the plane by calculating a vertical vector extending perpendicularly from the camera to the plane by using an internal parameter for inputting the internal parameter information of the camera into a storage means; an input step, a target point setting step of setting a predetermined point of the object appearing in the captured image as a target point, the vertical vector as the positional relationship information grasped in the positional relation grasping step, and stored in the storage means. an object point position estimation step of estimating the position of the object point on the camera-based bird's-eye view coordinate system based on the internal parameter information stored therein; and displaying the bird's-eye view plane image on the display means, , and a display step of displaying an icon indicating the target point on the image. It should be noted that the invention according to claim 7 can achieve the same effect as that of claim 1 above.

According to an eighth aspect of the present invention, the camera is used as a processor for controlling an object position estimation display device comprising a fixed camera for capturing an image of an object and obtaining an image, and display means for displaying the image. , a marker imaging step of capturing an image of an AR marker, which is a two-dimensional marker placed on a plane and having specific geometrical characteristics; Using extrinsic parameters consisting of a three-dimensional rotation matrix that rotates around the origin of the coordinates and a translation vector that translates the origin of the marker coordinates to the origin of the camera coordinate system relative to the camera, Using a determinant that converts a point on the marker coordinate system to a point on the camera coordinate system based on the camera, is calculated based on the three-dimensional rotation matrix, and then the angle formed by the perpendicular extending from the plane to the camera and the line segment extending from the camera to the marker is calculated by the reverse vector and the translation vector. and calculating the height from the plane to the camera from the translation vector and the angle, and then calculating the vertical vector extending vertically from the camera to the plane based on the reverse vector and the height. an internal parameter input step of inputting internal parameter information of the camera into a storage means; and a predetermined on the basis of the target point setting step of setting the point of as a target point, the vertical vector as the positional relationship information grasped in the positional relation grasping step, and the internal parameter information stored in the storage means, a target point position estimation step of estimating the position of the target point on a bird's-eye coordinate system with the camera as a reference; and displaying an image of the bird's-eye view plane on the display means, and displaying an icon indicating the target point on the image. The gist of the present invention is an object position estimation display program for executing a display step for performing In the eighth aspect of the invention, the same effect as the first aspect can be achieved.

Claims (9)

A device for estimating and displaying the position of the object, comprising a camera for capturing an image of an object and acquiring the image, and display means for displaying the image,
the camera that captures an image of a marker placed on a plane;
Points on a marker coordinate system based on the captured marker are converted into points on a camera coordinate system based on the camera, and the height from the plane to the camera and from the plane to the camera are calculated. positional relationship grasping means for grasping the positional relation between the camera and the plane by calculating an angle formed by an extending perpendicular line and a line segment extending from the camera to the marker;
an object point setting means for setting a predetermined point of the object appearing in the captured image as an object point;
target point position estimating means for estimating the position of the target point on a bird's-eye view coordinate system with respect to the camera, based on the positional relationship information grasped by the positional relationship grasping means and the internal parameter information of the camera; prepared,
The object position estimation display device, wherein the display means displays an image of a bird's-eye view plane and displays an icon indicating the target point on the image. 2. The object position estimation display device according to claim 1, wherein the object is a human or an animal, and the target point is set at the feet of the human or the animal. 3. The object position estimation display device according to claim 1, wherein an icon indicating the camera is displayed in addition to the icon indicating the target point on the image of the bird's-eye view plane. 4. The object position estimation display device according to claim 3, wherein the display means displays side by side the image of the bird's-eye view plane and the actual image captured by the camera. 5. The object position estimation display device according to any one of claims 1 to 4, wherein an identifier for specifying the type of the object is assigned to the icon indicating the target point. A plurality of the cameras are provided,
The target point position estimating means performs processing for connecting the images captured by the respective cameras by performing arithmetic processing for connecting the bird's-eye view coordinate systems of the adjacent cameras via the markers. 6. The object position estimation display device according to any one of claims 1 to 5. 7. The object position estimation display device according to any one of claims 1 to 6, wherein the marker is a two-dimensional marker having specific geometric features. A method for estimating and displaying the position of an object using an object position estimation display device comprising a camera for capturing an image of an object and acquiring an image, and display means for displaying the image, the method comprising:
a marker imaging step of imaging a marker placed on a plane using the camera;
Points on a marker coordinate system based on the captured marker are converted into points on a camera coordinate system based on the camera, and the height from the plane to the camera and from the plane to the camera are calculated. a positional relationship grasping step of grasping the positional relation between the camera and the plane by calculating an angle formed by an extending perpendicular line and a line segment extending from the camera to the marker;
an internal parameter input step of inputting internal parameter information of the camera into a storage means;
a target point setting step of setting a predetermined point of the object appearing in the captured image as a target point;
A target for estimating the position of the target point on a bird's-eye view coordinate system with the camera as a reference, based on the positional relationship information grasped in the positional relation grasping step and the internal parameter information stored in the storage means. a point position estimation step;
and a display step of displaying an image of a bird's-eye view plane on the display means, and displaying an icon indicating the target point on the image. A processor that controls an object position estimation display device that includes a camera that captures an object and acquires an image, and display means that displays the image,
a marker imaging step of imaging a marker placed on a plane using the camera;
Points on a marker coordinate system based on the captured marker are converted into points on a camera coordinate system based on the camera, and the height from the plane to the camera and from the plane to the camera are calculated. a positional relationship grasping step of grasping the positional relation between the camera and the plane by calculating an angle formed by an extending perpendicular line and a line segment extending from the camera to the marker;
an internal parameter input step of inputting internal parameter information of the camera into a storage means;
a target point setting step of setting a predetermined point of the object appearing in the captured image as a target point;
A target for estimating the position of the target point on a bird's-eye view coordinate system with the camera as a reference, based on the positional relationship information grasped in the positional relation grasping step and the internal parameter information stored in the storage means. a point position estimation step;
an object position estimation display program for displaying an image of a bird's-eye view plane on the display means and displaying an icon indicating the target point on the image.

Images