JP2007071570A - Camera attitude grasping method, photogrammetric method using the same, and their program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera attitude grasping method capable of grasping the attitude of a camera by a simple constitution. <P>SOLUTION: The camera attitude grasping method includes at least a first process for setting a plurality of directional reference points Ph1 and Ph2 in a selection plane 22 fixed on a subject 20, setting attitudinal reference parallel lines K1 and K2 intersecting with the selection plane 22 at right angles, and photographing the subject 20 by a camera 10 in a photographing attitude in a state approximately directly facing the selection plane in such a way as to include the selection plane 22 and the attitudinal reference parallel lines K1 and K2 to acquire a single photograph. It is possible to grasp the attitude of the camera on the basis of the inclination of the camera and its axial direction. It is possible to accurately grasp the attitude of the camera since a plane used when the inclination of the camera is determined and a plane used when the axial direction is determined intersect with each other at right angles and are not in the same plane. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method of grasping a camera posture from a photographed image when a subject is photographed (sometimes referred to as imaging) with a camera. The present invention also relates to a photogrammetry technique for accurately measuring a survey target (subject) using this method.

  A technique for accurately grasping (detecting) a camera posture from a photograph of a subject taken by a camera is an important technique with high versatility. For example, when photogrammetry is performed, accurate surveying cannot be performed unless the posture of the camera with respect to the subject is accurately grasped. In recent years, robot technology has been rapidly advanced, and a variety of multifunctional self-supporting robots such as pet robots, transport robots, and cleaning robots have been provided. Such an autonomous robot is generally equipped with a CCD camera or the like in a portion corresponding to the eye, and performs a predetermined operation by grasping the shape of an operation target object (subject). Therefore, it is important to accurately grasp the posture of the camera mounted on the robot. If the camera posture is inaccurate, the robot will erroneously recognize the position, shape, etc. of the subject, which may cause a malfunction. For example, when the robot grips the cup, there arises a problem that the cup cannot be gripped accurately. Furthermore, when the subject is a plane, if the camera posture can be accurately grasped, the position of a point on the plane can be detected accurately, and thus it can be used for an input device or the like. From the above description, it is understood that the technique for grasping the camera posture is highly versatile and an important technique.

  The “camera posture” is a posture of the camera with respect to one plane arbitrarily determined on the subject on which the image is taken. More specifically, when a three-dimensional coordinate system is defined in the space where the subject is placed, it is quantitatively specified by the angle of the optical axis of the camera with respect to the plane and the rotational angle of the camera around this optical axis. is there. Therefore, the camera posture indicates what posture the camera has taken with respect to the plane with respect to the plane. It should be noted that since the plane of the subject and the camera are in a relative positional relationship, it can be seen that the camera orientation indicates the orientation of the plane.

  Conventionally, techniques for grasping the camera posture (image pickup posture) of a camera (image pickup means) have been studied. For example, Patent Document 1 discloses a method for detecting a posture position of a predetermined plane of a subject or a detected position on the plane from information of image data captured by an imaging unit. In Patent Document 1, four points are set as feature points that characterize a rectangular shape on a plane of a subject. When a rectangular shape is set on the plane, there are two sets of parallel lines, and the plane posture is obtained by calculating the vanishing points of these parallel lines. In this patent document 1, if this technique is utilized, it is possible to directly input a position indicated on a display screen such as a screen.

  Further, Patent Document 2 discloses a method in which a predetermined marker is arranged on a plane, image information is obtained by photographing from an oblique direction with an imaging unit, and binarized image processing is performed based on this image information. This marker is displayed side by side in a horizontal direction so that two square parts touch each other at one vertex. The entire image of the marker is captured without distortion in the field of view of photographing such as a CCD camera. This image information is processed to recognize the position and orientation of the camera. When the technique of Patent Document 2 is applied to a self-supporting robot or the like, it is possible to perform an operation such as reliably approaching a target object and accurately gripping the object with a manipulator or the like.

Japanese Patent Laid-Open No. 2001-148025 JP-A-7-98214

  In Patent Document 1 and Patent Document 2, a rectangular shape or marker (square portion) including parallel lines is set on a plane in order to grasp the camera posture. By the way, when detecting the tilt of the camera with respect to the subject, it is preferable that the rectangular shape or the like is inclined, that is, the camera is photographed in an oblique posture with respect to the rectangular shape or the like. On the other hand, when obtaining the coordinates of the reference point on the same plane as the rectangular shape or the like, the above relationship is reversed. In other words, it is desirable to photograph the camera in a state close to the plane, that is, in a posture close to the plane with respect to the plane.

  In order to grasp the camera posture, it is necessary to obtain the tilt of the camera with respect to the plane and the coordinates of the reference point on the plane. However, in the prior art shown in Patent Documents 1 and 2, reference points for obtaining coordinates are arranged on a plane used for obtaining camera tilt. Therefore, the camera position preferable for detecting the camera tilt and the camera position preferable for calculating the coordinates of the reference point are in a contradictory relationship. Therefore, there is a problem that a situation in which the camera posture cannot be detected with high accuracy occurs.

  In addition, as described above, the purpose of grasping the camera posture is to survey the subject captured by the camera and to specify the position and shape of the subject. Therefore, when the camera posture cannot be accurately grasped, as a result, there arises a problem that the measurement, position specification, shape specification, etc. of the subject cannot be executed with high accuracy. In addition, in the conventional method, in order to perform shooting with higher accuracy, it is necessary to make the two pairs of parallel lines closer and deeper, and it is difficult to select the shooting direction. For example, in the case of using a horizontal or vertical parallel line structure or pattern of a building, it is necessary to take a picture from the sky or underground direction in order to search for a place where it can move horizontally and to shoot the vertical parallel line in perspective.

  Therefore, a main object of the present invention is to provide a camera posture grasping method capable of grasping a camera posture with a simple configuration.

  The purpose is to set a plurality of direction reference points on a selected plane defined on the subject, set an attitude reference parallel line orthogonal to the selected plane, and be in a state of being close to the selected plane. Thus, it can be achieved by a camera posture grasping method including at least a first step of capturing a single photograph by photographing the subject with a camera so as to include the selected plane and the posture reference parallel line.

  According to the present invention, the posture reference parallel lines are arranged orthogonal to the selected plane, and the tilt of the camera can be obtained based on the posture reference parallel lines taken in a single photograph. On the other hand, the axial direction with respect to the selection plane of the camera can be obtained based on the direction reference point on the selection plane captured in the single photograph. The camera posture can be grasped by the tilt and the axial direction of these cameras. As described above, the surface used when obtaining the tilt of the camera (the surface including the posture reference parallel line) and the surface used when obtaining the axial direction (selected plane) are orthogonal to each other and are not the same surface. It is not a reciprocal relationship. Therefore, according to the present invention, it is possible to accurately grasp the camera posture using one single photograph.

  In addition, the camera posture grasping method defines the two-dimensional coordinate system on the single photograph taken in the first step and the first step, and on the direction reference point and the posture reference parallel line on the selection plane. A second step of obtaining coordinates by specifying a determined parallel reference point; a third step of obtaining a camera posture with respect to the selected plane based on the direction reference point obtained in the second step and the coordinates of the parallel reference point; It is good also as what further comprises. By processing a single photograph of the subject in this way, the camera posture can be obtained reliably.

  In the first step, an auxiliary plane arranged at an angle α with the selected plane is set, and a plurality of auxiliary points are set on the intersection line of the selected plane and the auxiliary plane. The subject is photographed with a camera so as to include the auxiliary plane and the auxiliary point, and the single photograph is obtained. In the second step, the coordinates of the auxiliary point are further obtained on the single photograph, and in the third step. Further, the camera posture with respect to the auxiliary plane may be obtained by referring to the angle α and the coordinates of the auxiliary point. In this way, the application range of the present invention can be further expanded.

  In addition, the first step may be performed by placing a sign provided with a figure capable of specifying the orientation reference parallel line and the direction reference point of the selected plane on the subject. In this case, the camera posture can be easily grasped even for a subject having no feature in shape or the like.

  In addition, a plurality of survey reference points whose positions are known to the subject including the survey point and dimension reference points whose intervals are known are set, and a single photograph obtained by photographing the subject with the camera is used. A photogrammetry method for obtaining three-dimensional coordinates, comprising a camera posture grasping process for obtaining a camera posture by the above-described camera posture grasping method, wherein the three-dimensional coordinates of the measured point are obtained. You may apply as a surveying method. In this photogrammetry method, since the camera posture is accurately grasped, an accurate survey result can be obtained.

  Further, a photogrammetry for obtaining a three-dimensional coordinate of the measured point using a plurality of single photographs obtained by photographing the subject with a camera from a plurality of directions and including a camera posture grasping process for obtaining a camera posture for each single photograph. It is good also as a method.

  In addition, the object is to set a plurality of direction reference points on a selection plane defined on the subject, set an attitude reference parallel line orthogonal to the selection plane, and be close to the selection plane. First processing for reading an image of a single photograph obtained by photographing the subject with a camera so as to include the selected plane and the posture reference parallel line in a photographing posture; and defining a two-dimensional coordinate system on the single photograph; A second process for obtaining coordinates by specifying parallel reference points determined on the direction reference point and the posture reference parallel line, and a camera for the selected plane based on the coordinates of the direction reference point and the parallel reference point This can also be achieved by a camera posture grasping program that causes a computer to execute the third process for obtaining the posture.

  The program for grasping the camera posture further sets an auxiliary plane arranged at an angle α with the selected plane, and sets a plurality of auxiliary points on the intersection line of the selected plane and the auxiliary plane. The image of the single photograph obtained by photographing the subject with the camera so as to include the auxiliary plane and the auxiliary point is read, the coordinates of the auxiliary point are obtained on the single photograph, and the angle α and the coordinates of the auxiliary point are further acquired. Based on the above, the camera posture with respect to the auxiliary plane may be obtained.

  In addition, a plurality of survey reference points whose positions are known to the subject including the survey point and dimension reference points whose intervals are known are set, and a single photograph obtained by photographing the subject with the camera is used. A photogrammetry program for obtaining three-dimensional coordinates, including a camera posture grasping process for obtaining a camera posture by the above-described camera posture grasping method, and executing a process for obtaining the three-dimensional coordinates of the measured point on a computer If the program is a photogrammetry program characterized in that the subject can be accurately measured.

  Furthermore, a process for obtaining the three-dimensional coordinates of the measurement point is included, including a camera posture grasping process for obtaining a camera posture for each single photograph using a plurality of single photographs obtained by photographing the subject with a camera from a plurality of directions. The program may be a photogrammetry program that is executed by a computer.

  According to the present invention, it is possible to provide a camera posture grasping method capable of grasping a camera posture with a simple configuration. In addition, in order to perform shooting with higher accuracy, it is only necessary to take care to make the pair of parallel line directions deeper and deeper, so selection of the shooting direction is easier than in the past. For example, when a horizon structure or pattern of a building is used, it is sufficient to search for a place where the ground can be moved horizontally.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the embodiment shown here, a subject is photographed with a camera, and the camera posture is grasped based on this photograph (image data). Here, a photograph obtained by photographing a subject from one direction with a normal monocular camera is referred to as a single photograph. Therefore, the embodiment shown below grasps the camera posture from a single photograph of a subject.

  Further, as described above, the camera posture is the posture of the camera with respect to one plane defined on the subject. Hereinafter, this one plane will be specifically referred to as a selection plane. The camera posture indicates a relative positional relationship between the selected plane and the camera. The posture grasped when the selected plane fixed at the reference position is photographed is exactly the camera posture. On the other hand, the posture grasped when the camera is fixed to the reference position and the selected plane is photographed is related to the selected plane. In this specification, such a case is referred to as a camera posture without distinction. However, in the embodiment described below, a case where the position of the camera is fixed and the orientation of the selected plane is obtained is described for easy understanding.

  Prior to describing specific embodiments, the concept of the present invention will be described with reference to FIG. 1 and FIG. 2 in order to facilitate technical understanding. In order to obtain the camera posture, it is necessary to obtain the inclination of the selection plane and the axial direction of the selection plane. By calculating the coordinates of at least two reference points set on the selection plane, it is possible to determine the reference point and the reference direction of the selection plane and obtain the axial direction. Hereinafter, the reference point for determining the axial direction of the selected plane will be referred to as a direction reference point. The direction reference point also has a role of determining the position such as a reference point.

  Here, it will be described that the posture of the camera can be obtained by obtaining the inclination of the selected plane and obtaining the coordinates of the direction reference point for determining the base point and the axial direction on the selected plane. It is sufficient that the camera is fixed and a three-dimensional coordinate system based on the selected plane can be expressed. Note that fixing the camera means defining a three-dimensional coordinate system based on the camera.

  For example, a first three-dimensional coordinate system is defined in which the camera position is the reference point, the optical axis direction of the lens is the Z axis, the horizontal axis direction of the imaging surface is the X axis, and the vertical direction is the Y axis direction. On the other hand, for example, one direction reference point Ph1 is set as a reference point. The direction connecting this direction reference point Ph1 and another direction reference point Ph2 is taken as the X axis. In addition, a second three-dimensional coordinate system is defined in which the direction orthogonal to the X axis on the selection plane is the Y axis, and the direction orthogonal to the selection plane is the Z axis.

  Here, if the coordinate values of the direction reference points Ph1 and Ph2 in the first three-dimensional coordinate system are found, the reference point and the X axis of the second three-dimensional coordinate system are determined. Furthermore, if the inclination of the selection plane represented by the first three-dimensional coordinate system is found, the Y-axis direction orthogonal to the X-axis and the selection plane Z orthogonal to the second three-dimensional coordinate system on the selection plane. The axial direction is fixed. Therefore, the camera posture can be expressed using the inclination of the selected plane and the axial direction as parameters.

  FIG. 1 is a diagram showing a case where a camera posture is obtained from a single photograph by a conventional method. In order to calculate the inclination of the selection plane 1, four reference points A1 to A4 are set. Each of reference points A1-A4 is arrange | positioned at the vertex of a rectangular shape (rectangle). The (A1-A2) line and the (A3-A4) line are a pair of parallel lines, and the (A1-A3) line and the (A2-A4) line are a pair of parallel lines. In addition, two direction reference points B1 and B2 are set on the selection plane 1 in order to obtain the axial direction. FIG. 1 illustrates the case where the points A1 and B1 and the points A2 and B2 are shared.

  The selected plane 1 shown in FIG. 1 is photographed by the camera 10 so as to include all the reference points A and B, and a single photograph is obtained. The inclination of the selection plane 1 is calculated based on the position of the image of the reference point A that appears in the single photograph, and further, the three-dimensional coordinates are calculated based on the image of the direction reference point B that appears in the single photograph. Processing for obtaining the axial direction is performed in order. This grasps the camera posture. Thus, conventionally, all of the points A and B are arranged on the selection plane 1. For this reason, the problem as pointed out as a problem occurs. That is, if the inclination is made larger in order to detect the inclination of the selection plane 1 more accurately, the detection accuracy of the coordinate value of the direction reference point B is lowered, so that the accuracy in the axial direction is deteriorated. On the contrary, if the inclination is made smaller in order to further improve the axial accuracy, the accuracy of detecting the inclination of the selection plane 1 is deteriorated. In the end, it was difficult to grasp the camera posture with higher accuracy.

  FIG. 2 is a diagram showing the case of the present invention in which the camera posture is obtained from a single photograph. Reference points A and B for calculating the camera posture are arranged by role. Reference points A <b> 1 to A <b> 4 arranged on parallel lines for obtaining the inclination of the selection plane 1 are set separately from the selection plane 1. Specifically, the direction reference points B1 and B2 for obtaining the axial direction on the selection plane 1 are set on the selection plane 1, but the reference points (parallel reference points) A1 to A4 are orthogonal to the selection plane 1. Set on parallel lines K1, K2. Hereinafter, the parallel lines K1 and K2 are referred to as posture reference parallel lines.

  Here, the camera 10 is set to a photographing posture close to the selection plane 1. In this case, since the posture reference parallel lines K1 and K2 are orthogonal to the selection plane 1, the optical axis of the camera 10 is almost parallel to the posture reference parallel lines K1 and K2. In this specification, in this specification, the posture reference parallel lines K1 and K2 are particularly referred to as “in a perspective and deep state” with respect to the camera 10, and the arrangement of the selection plane 1 with respect to the camera 10 is “in a shallow perspective”. It will be referred to as “state”.

  As apparent from the above description, in the present invention, the selection plane 1 on which the direction reference points B1 and B2 are placed and the posture reference parallel lines K1 and K2 are set to be orthogonal to each other. Then, camera photographing is performed so as to include the reference points A1 to A4 on the direction reference points B1 and B2 and the posture reference parallel lines K1 and K2. Therefore, the reciprocal relationship that has been a problem in the past can be resolved.

  That is, since the method of the present invention shown in FIG. 2 can make the posture reference parallel lines K1 and K2 for detecting the inclination of the selection plane 1 into a near and deep state, the directions of the reference parallel lines K1 and K2 can be detected with high accuracy. . Here, since the selection plane 1 is orthogonal to the posture reference parallel lines K1 and K2, the inclination of the selection plane 1 can be accurately detected as a result. In addition, direction reference points B1 and B2 for detecting a directional relationship with the camera are arranged on the selection plane 1. Since the selection plane 1 is in a state close to the camera 10, that is, in a shallow state, the three-dimensional coordinates of the direction reference point can be calculated with high accuracy. As described above, the present invention employs a novel configuration in which the posture reference parallel lines K1 and K2 orthogonal to the object selection plane 1 are set, so that the camera posture can be grasped with high accuracy.

  The reference point A may be an arbitrary point on the posture reference parallel line K. That is, it is not necessary to specify the reference point A, and any point on the posture reference parallel line K may be used. Further, the posture reference parallel lines K are not limited to two as illustrated in FIG. That is, the posture reference parallel lines K may be three or more. Further, the posture reference parallel lines K need only be parallel to each other and do not have to exist on the same plane. By referring to more posture reference parallel lines K, the camera posture can be grasped with higher accuracy.

  Two or more directional reference points B are set on the selection plane 1. This selection plane 1 is one surface of the subject. When actually grasping the camera posture, the selection plane 1 where the entire surface can be seen from the camera may not be set depending on the condition of the subject. Thus, even when it is difficult to confirm the selection plane 1 in the subject, it can be dealt with as follows. For example, the direction reference point B can be a point on a line orthogonal to the posture reference parallel line K (a line that is an element of the selection plane 1). The direction reference point B may be a point where the posture reference parallel line K and the selection plane 1 intersect (a point that is an element of the selection plane 1). That is, two or more intersections between the posture reference parallel lines K and one straight line perpendicular to K may be seen even if the entire view of the selection plane 1 is not visible. As described above, the method of the present invention shown in FIG. 2 has simple requirements regarding the arrangement of points to be referred to and is easy to set. In addition, since the posture reference parallel line and the selection plane 1 or straight line orthogonal to the posture reference parallel line can utilize a shape or a pattern that is common in the construction, the camera posture can be easily grasped. The embodiment will be described on the premise of the above description.

  With reference to FIG. 3 to FIG. 11, the processing for grasping the camera posture from a single photograph will be specifically described. However, in this embodiment, a case will be described in which the camera posture is grasped from a single photograph and the subject is photographed. That is, this embodiment describes an example in which the camera posture is applied to photogrammetry. In the following, an example of photogrammetry using a single photo taken from one direction and two (multiple) single photos taken from two directions (multiple directions) will be described.

  FIG. 3 is a diagram showing a state in which the apartment house is photographed with the subject 20 and the survey is performed after grasping the camera posture. In FIG. 3, the subject 20 is photographed by the camera 10 from each of two directions to obtain two single photographs. First, a case where the camera posture is obtained using a single photograph 15a taken by one camera 10a will be described. The lower side of FIG. 3 shows a single photo 15a taken by the camera 10a and a single photo 15b taken by the camera 10b.

  FIG. 4 is a flowchart summarizing the processes from photographing the subject 20 to obtaining the camera posture and further performing photogrammetry. A description will be given along this flowchart with reference to FIG. First, in step S102, the subject 20 on which the measurement point PN is placed is photographed by the camera 10a of FIG. The subject 20 is a four-story apartment house that can be seen as a substantially rectangular parallelepiped. Here, the front surface of the apartment house where the measured point PN is placed (the front surface with a handrail of the veranda) is the selection plane 22. The photographing posture of the camera 10a is set so that the camera 10a is in a shallow state (a state close to the front) with respect to the selection plane 22. A left plane (left wall surface of the house) 21 of the subject 20 is a reference plane in which posture reference parallel lines K1, K2, etc. are set in a horizontal straight line pattern. With respect to this reference plane 21, the shooting posture of the camera 10a is set so as to become deeper and deeper. A single photograph 15a shown in FIG. 3 was taken with such setting. Note that reference numeral 15b in FIG. 3 is a single photograph taken in a different direction with the same setting.

  This apartment house has a front surface (selection plane 22) and a side surface (reference plane 21) at right angles. Therefore, the posture reference parallel lines K1 and K2 which are horizontal linear patterns set on the reference plane 21 are parallel lines orthogonal to the selection plane 22. The posture reference parallel lines K1 and K2 may be set using a linear pattern or the like existing on the reference plane 21. However, the posture reference parallel lines K <b> 1 and K <b> 2 do not necessarily have to be on the reference plane 21 as long as they are parallel lines orthogonal to the selection plane 22. For example, the roof edges K3 and K4 may be used as long as they appear in the single photograph 15a. Two or more plural lines may be set. Similarly, the selection plane 22 may be a plane that is orthogonal to the posture reference parallel lines K1 and K2. For example, in addition to the surface with a handrail of a veranda, the surface with the window 22p in the back may be sufficient.

  Direction reference points Ph 1 and Ph 2 are set on the selection plane 22. The selection plane 22 may be difficult to see on the subject. For example, although the reference plane 21 is visible, the selection plane 22 is difficult to see, and there are cases where a line orthogonal to the posture reference parallel lines K1 and K2 (this line is a part of the selection plane 22) can be confirmed. In such a case, two points Ph3 and Ph4 on one line orthogonal to the posture reference parallel lines K1 and K2 may be used as direction reference points. Further, reference points Ph5 and Ph6 at positions where one line orthogonal to the attitude reference parallel lines K1 and K2 (this line is also a part of the selection plane 22) and the parallel lines K1 and K2 may be used. .

  Note that the posture reference parallel lines K1 and K2 may be used because there are parallel lines such as ceilings, floors, joints on the walls, etc. in the building. Similarly, when there is a square stone pavement in the site, the joint may be used.

  In the selection plane 22, dimension reference points Ps1 and Ps2 used for surveying are set. The distance Ls (reference dimension) between the dimension reference points Ps1 and Ps2 is measured. A single photograph is obtained by photographing the subject 20 with the camera 10a so that the selection plane 22 and the reference plane 21 set for the subject 20 enter. When the camera 10a can store data like a digital camera or the like, the image data is stored in an IC card or the like. If the camera 10a uses a film, the photographed film or photograph is read by a scanner and converted into image data.

  Here, hardware suitable for carrying out this embodiment will be briefly described. FIG. 5 is a block diagram showing an example of a hardware configuration suitable for implementing this embodiment. An information processing apparatus such as a personal computer 30 is used as the apparatus. A central processing unit (CPU) 32 that controls the entire system includes a keyboard 25, a mouse 27, a display device (CRT, LCD, etc.) 29, a printer 31, and an IC card reader via an input / output control unit (interface) 23. 33 etc. are connected. For example, an image taken with a digital camera is input via the IC card 34. The image can be observed by a display device (display) 29. A photograph taken by a normal camera is converted into a digital image by a film scanner 35 and input.

  The computer 30 can store a program for obtaining the camera posture, a program for surveying the subject, data necessary to execute these programs, data measured by surveying, and the like in the HDD 37. The computer 30 reads out necessary data from the HDD 37 and executes predetermined processing. Although not shown, the CPU 32 includes a predetermined memory. A program or the like may be stored in this memory.

  Steps S104 and after are data processing operations executed using the computer 30. In step S104, the image obtained in step S102 is read into the computer 30 by, for example, the IC card reader 33 shown in FIG. The reference dimension Ls is input from the keyboard 25. Each data is stored in the hard disk HDD 37 or the like.

  In step S106, the single photograph 15a read in step S104 is displayed on the display 29. Arbitrary reference points PL1 and PL2 on the straight line image are specified for the posture reference parallel line K1, and reference points PL3 and PL4 are specified for K2. These reference points PL1 to PL4 are referred to as parallel reference points. A two-dimensional coordinate system is defined on the single photograph image displayed on the display 29, the parallel reference points PL1 to PL4 are designated with the mouse 27, and coordinates are input. These coordinates are called pixel coordinates. When each of the parallel reference points PL1 to PL4 is set at the end of the posture reference parallel lines K1 and K2, the detection accuracy can be improved.

  Subsequently, the measured point Pn, the direction reference points Ph1 and Ph2, and the dimension reference points Ps1 and Ps2 are also specified, and pixel coordinates are input with the mouse. Each input pixel coordinate is converted into coordinates on the virtual imaging surface Ff by the camera characteristic data and stored. Note that the virtual imaging plane Ff is a plane that is assumed to be orthogonal to the optical axis of the camera 10 at a certain distance L in front of the camera 10 and is virtually assumed to have an image formed on the virtual imaging plane Ff.

  Here, the relationship between pixel coordinates and coordinates on the virtual imaging surface Ff will be described. The pixel coordinates represent the position of the image when the obtained photographic data is displayed on the screen by defining the horizontal as the Xd axis, the vertical as the Yd axis, and the unit of the coordinate value as the number of pixels (pixels). Two-dimensional coordinates. Since the position of the image on the photograph or the real imaging surface is represented, distortion of the lens of the camera is also included. On the other hand, the virtual imaging plane coordinates are two-dimensional coordinates defined on a virtual imaging plane so that the subject image is formed on the assumption that the optical system is not distorted like a pinhole camera, and the horizontal axis is the Xf axis, The vertical axis is the Yf axis, the focal length is L, and the coordinate value is expressed in the same unit as L (actual dimension mm or the like). The camera characteristic data is a numerical table used for conversion between pixel coordinates and virtual imaging plane coordinates and created based on camera settings and lens characteristics. A more detailed explanation will be omitted because it is not the main subject. In the following description, it is assumed that pixel coordinates that have been identified and input with a mouse have been converted to virtual imaging plane coordinates expressed in actual dimensions without distortion of the optical system.

  In the next step S108, based on the virtual imaging plane coordinates of the parallel reference points PL1 to PL4 on the posture reference parallel lines K1 and K2 and the virtual imaging plane coordinates of the direction reference points Ph1 and Ph2 on the selection plane 22 obtained in step S106. Calculate the camera posture.

  First, the coordinates xv and yv of the vanishing point Pv are calculated from the images of the posture reference parallel lines K1 and K2 on the virtual imaging surface. FIG. 6 is a conceptual diagram thereof. On the photograph of FIG. 6, there are parallel line images of the straight line k1 (PL1-PL2) and the straight line k2 (PL3-PL4). Photographed with the parallel lines tilted, the vanishing point Pv exists where the extension of these line images intersects on the photograph. The vanishing point is a point or intersection where line extensions, which are parallel lines, are collected. The general formula of the straight line on the plane is yfLn = Akn · xfLn + Ckn. Here, xf and yf are the coordinate values of the points on the straight line on the virtual imaging plane, A and C are parameters of the straight line equation, the subscript kn is the distinction of the line image, and Ln is the same as the subscript of the point on the line. Shows the distinction between points. The coordinates of points belonging to each line are substituted as conditions.

For the line k1, the following equations (1), (2)
yfL1 = Ak1.xfL1 + Ck1 (1)
yfL2 = Ak1 · xfL2 + Ck1 (2)
For the line k2, the following equations (3), (4)
yfL3 = Ak2 · xfL3 + Ck2 (3)
yfL4 = Ak2 · xfL4 + Ck2 (4)
The following formula can be established.

When parameters Akn and Ckn in each of the above lines are obtained, the following formulas (5), (6),
Ak1 = (yfL1-yfL2) / (xfL1-xfL2) (5)
Ck1 = (xfL1 · yfL2−xfL2 · yfL1) / (xfL1−xfL2)
(6)
Moreover, about Formula (3) and (4), following formula (7), (8),
Ak2 = (yfL3-yfL4) / (xfL3-xfL4) (7)
Ck2 = (xfL3.yfL4-xfL4.yfL3) / (xfL3-xfL4)
(8)
It becomes.

From the parameters of the equations of the two straight lines on the photograph obtained as described above, the coordinates of those intersections, that is, the vanishing point Pv on the photograph are obtained. Expressions (9) and (10) are established under the condition that yv is the same in the two lines.

yv = Ak1 · xv + Ck1 (9)
yv = Ak2 · xv + Ck2 (10)

When the above two equations are solved, the following equations (11) and (12) are obtained.

xv = (Ck2-Ck1) / (-Ak2 + Ak1) (11)
yv = (− Ck1 · Ak2 + Ck2 · Ak1) / (− Ak2 + Ak1)
(12)

  Based on the coordinates yv and xv of the vanishing point Pv, an equation of the selection plane 22 is obtained. FIG. 7 is a geometric diagram showing the relationship between the vanishing point Pv on the virtual imaging plane Ff and the selection plane 22 and the like. The optical axis direction of the camera 10 is shown as three-dimensional coordinates with the Z axis, the Xf axis of the virtual imaging surface coordinates as the X axis, and the Yf axis of the virtual imaging surface coordinates as the Y axis. 7A and 7B show the respective projection views.

  The virtual imaging surface Ff is a virtual surface that is placed at the focal length L in front of the camera 10 and the subject image is formed without distortion. Conversely, if the straight line connecting the camera 10 and the image on Pf (Pfn, etc.) is extended forward, the actual image is reached. By the way, the vanishing point has no substance, and when a parallel line is photographed, the extension point is at an infinite edge and appears to be converged to one point. That is, the vanishing point here is not a mere point, but indicates directionality. The direction of the vanishing point is common to all the parallel directions that are the same direction. That is, the straight line connecting the camera 10 and Pv is the direction of the parallel line. If this direction is known, the inclination of the orthogonal plane, that is, the selection plane 22 can be obtained. The inclination of the selection plane 22 is represented by the inclination a in the XZ sectional projection and the inclination b in the XY sectional projection.

Therefore, from FIG. 7, the general formula of the selection plane 22 is the following formula (13).

z = Ax + By + C (13)

From FIG. 7, the XZ coordinate plane projection inclination a and the YZ coordinate plane projection inclination b of the selection plane 22 are expressed by the following equations (14) and (15).

a = xv / L (14)
b = yv / L (15)

When the formulas (14) and (15) are applied to the general formula (13), the following formula (16) can be obtained.

z = −ax−by + C (16)
(A = -a, B = -b)

The inclination of the selected plane 22 with respect to the camera 10a is determined by the above equation (16). Here, since C is not yet determined, the model formula is assumed to be “1”. Based on this equation (16), the point image position on the single photograph plane, that is, the three-dimensional coordinates of the point on the selection plane 22 can be obtained from the virtual imaging plane coordinate values xf and yf. That is, x can be obtained from the following formula (17), y from the following formula (18), and z from the following formula (19). (See FIG. 7 for the relationship between Pfn and PN)

  Next, the axial direction of the coordinates of the selection plane 22 is determined. For this purpose, the direction reference points Ph1 and Ph2 are used. For example, the direction of the line segment Ph2-Ph1 on the selection plane 22 is set as the X axis or the Y axis (see FIG. 3). Thus, the axial direction of the coordinates on the selection plane 22 can be determined by obtaining the three-dimensional coordinates of the direction reference points Ph2 and Ph1.

  By substituting the virtual imaging plane coordinates of the direction reference points Ph1 and Ph2 into the above formulas (17) to (19), three-dimensional coordinates are obtained. Thus, the coordinate axis direction on the selection plane 22 is determined. The camera posture is grasped (detected) as described above. However, since it is still grasping as a model, if necessary, C may be obtained based on a method of obtaining actual coordinates as in step S112 described later, and an actual expression may be obtained.

  Then, the case where the result obtained above is applied to surveying will be described. First, photogrammetry using a single photo (that is, a single photo) taken from one direction will be described. For the measured point PN on the selected plane 22 in step S110, the three-dimensional coordinates can be calculated by substituting the virtual imaging plane coordinates of Pn into the equations (17) to (19). In the next step S112, the coordinates handled by the model are converted to the actual size.

The coordinates of the two dimension reference points (Ps1, Ps2) shown in FIG. 3 are obtained by equations (17) to (19). The interval ls is obtained by the following equation (20). The subscripts s1 and s2 indicate model coordinates of the dimension reference points Ps1 and Ps2.

ls = √ {(xs1−xs2) ² + (ys1−ys2) ² + (zs1−zs2) ² } (20)

  Next, a ratio between the interval Ls (reference dimension) actually measured in step S102 and ls is obtained in advance, and this ratio is multiplied by each coordinate value of the obtained model. Through the above steps, the actual three-dimensional coordinate value of the measured point Pn is calculated. As described above, photogrammetry can be performed with a single photograph taken from one direction.

  Further, an example of application to photogrammetry in which surveying is performed using two (a plurality of) photographs taken from two directions will be described below. Each step here is almost the same as the photogrammetry of a single photograph taken from one direction as described above, so the same figure will be used for explanation.

  This will be described with reference to the flowchart of FIG. In step S102, two pictures 15a and 15b are obtained by photographing with the camera 10a and the camera 10b as shown in FIG. The shooting method is the same as that for a single photo. The positions of the camera 10a and the camera 10b are taken from directions that are separated horizontally or vertically. For example, shooting is performed by changing the direction of 30 °. Steps S102 to S108 are performed in the same manner as in the case of a single photograph, and the respective camera postures are obtained.

  In step S110, an example will be described in which two camera postures are merged using the camera posture for each photograph obtained in S108. In step S108, the camera posture is obtained in an independent coordinate system for each photograph. For this reason, the scales of the coordinate system and model coordinates do not match. Therefore, it is necessary to make these correspond. As an outline, the coordinates are appropriately rotated and expanded or contracted so that the model coordinates of the direction reference points Ph1 and Ph2 obtained as the camera postures from the two photographs 15a and 15b taken from two directions are matched. With this rotation and expansion / contraction, the camera positions Cca and Ccb (elements x, y and z of the initial coordinates are 0) and the camera orientation are determined. As the direction of the camera, coordinates of points C0a and C0b (elements x and y of the initial coordinates, 0 and z is Ca or Cb of the original coordinates) at which the optical axis of the camera intersects the plane 22 are used.

  The rotation and expansion / contraction according to the above is performed based on the parameters Aa, Ba, Ca, Ab, Bb, and Cb (the same subscripts a and b as those in the photograph) of the model of the plane 22 obtained as the camera posture.

  FIG. 8A is a diagram conceptually showing the arrangement of the merged points. Here, reference numeral 7 denotes a camera base axis (a line connecting the camera main points Cca and Ccb), and reference numerals 9a and 9b denote the optical axis directions of the respective cameras. FIG. 8B is a diagram showing lines 2 (projection of 9a) and 3 (projection of 9b) obtained by projecting lines 9a and 9b in the optical axis direction of the camera onto orthogonal cross sections of the camera base axis.

This will be described with reference to FIG. In the equation of the model of the plane 22 calculated for each photograph in step S108, the scale is not matched because C is set to 1 for the time being as described above. To match these, the parameter Ca or Cb is multiplied by a coefficient so that the interval between the two coordinate reference points on the model coordinates is aligned between the two photographs. The distances ea and eb on the model between the direction reference points are obtained by the following equations (21) and (22). The subscripts a and b have the same subscripts as the related photos 15a and 15b. The model coordinates of the direction reference point are x, y, and z obtained by the equations (17) to (19). The suffix indicates the distinction between the direction reference points h1 and h2, and the distinction between the suffixes a and b in the photograph.

ea = √ {(xh2a−xh1a) ² + (yh2a−yh1a) ² + (zh2a−zh1a) ² } (21)
eb = √ {(xh2b−xh1b) ² + (yh2b−yh1b) ² + (zh2b−zh1b) ² } (22)

  When the parameter Ca is fixed to 1, it may be corrected by multiplying Cb by the coefficient ea / eb. As a result, the scales of the equations of the two plane 22 models match. The three-dimensional coordinates of the model are calculated using the formula after Cb correction.

  Here, as to what coordinate system is to be integrated, the XY axis is placed on the plane 22 as shown in FIG. 8A, the vertical direction of the plane 22 is the Z axis, and the direction reference point Ph1 is the base point, the direction An example in which the direction of the reference point Ph1 to Ph2 is taken as the X axis is shown below.

  For the coordinates of each point, the following coordinate conversion is performed for each photograph (15a, 15b). The coordinates of each point are Ph1a (xh1a, yh1a, zh1a), Ph1b (xh1b, yh1b, zh1b), Ph2a (xh2a, yh2a, zh2a), Ph2b (xh2b, yh2b, ch2b), hh2b, and hh2b (ch2b) in each camera 10a and 10b. x = 0, y = 0, z = 0), Ccb (x = 0, y = 0, z = 0), C0a (x = 0, y = 0, z = Ca), C0b (x = 0, y) = 0, z = Cb). In addition, the same subscript is attached | subjected regarding the photographs 15a and 15b and the direction reference points Ph1 and Ph2.

Coordinate transformation 1: Use Ph1 as a base point. If the matrix of coordinates x, y, z of Ph1a or Ph1b is [Ph1], the x, y, z matrix of each point before the base point change is [X0], and the post-change matrix is [X1], Become.

[X1] = [X0]-[Ph1] (23)

である。 Coordinate transformation 2: Align the X axis with the surface 22. When the x, y, z matrix of each point before the rotation is [X1] and after the rotation is [X2], the following equation (24) is obtained.

[X2] = [R1] [X1] (24)

Here, [R1] is the following formula (25),

u = tan ⁻¹ (A) (25)
A can be obtained by substituting the parameter Aa or Ab for each photograph.
given that,

It is.

ＡおよびＢは写真別にパラメータＡａ、ＢａまたはＡｂ、Ｂｂを代入する。
とすれば、下記式（２９）となる。

Coordinate transformation 3: Align the Y axis with the surface 22. When the x, y, z matrix of each point before rotation is [X2] and after rotation is [X3], the following equation (27) is obtained.

[X3] = [R2] [X2] (27)
Also,
[R2] becomes the following formula (28),

For A and B, parameters Aa and Ba or Ab and Bb are substituted for each photograph.
Then, the following equation (29) is obtained.

上の式中ｘ、ｙ値は［Ｘ３］における方向参照点Ｐｈ２の座標値である。関係する点と同じ添え字を付している。
とすれば、下記式（３２）を得る。

Coordinate transformation 4: Align X-axis with Ph2-Ph1. If the x, y, z matrix of each point before rotation is [X3], and after rotation is [X4], the following equation (30) is obtained.

[X4] = [R3] [X3] (30)
Also,
[R3] is represented by the following formula (31).

In the above formula, the x and y values are the coordinate values of the direction reference point Ph2 in [X3]. The same subscripts are attached as relevant points.
Then, the following formula (32) is obtained.

  If the above is performed for every two photos, the camera positions Cca and Ccb are in the same coordinate system as shown in FIG. From these results, the following parameters are set to facilitate subsequent photogrammetry calculations. This will be described with reference to FIGS. Note that FIG. 9 shows the relationship between the camera and the subject being photographed for the surveying, and the imaging on the two cameras 10a and 10b and the points on the subject. The camera 10a is described in detail. (A) is the figure which projected the line 9a, 9b of the optical axis of a camera, and the Pn direction line on the orthogonal cross section of the camera base axis 7. FIG. (B) is a diagram conceptually showing the state of shooting in step 102, the image pickup by the camera, and the geometric relationship between the points. (C) is virtual imaging on Ff, and is a diagram obtained by reversing the camera inclination σa and matching virtual imaging of the camera base axis with the X axis. (D) is virtual imaging on the virtual imaging coordinates Ff. Reference numeral 15 denotes an X-axis of the screen, and reference numeral 13 denotes virtual imaging of the camera base axis, which forms a camera inclination σ with respect to the X-axis 15. Pcc is a virtual image (not necessarily within the screen) when the position of the other camera 10 is captured.

  A line connecting the positions of the two cameras that have taken two photographs in FIG. Αa0 and αb0 are angles formed with the camera base axis with respect to the direction of the optical axis of the camera as viewed from the camera 10a to the camera 10b. In FIG. 8B, the optical axis direction 9b of the camera 10b with reference to the optical axis direction 9a of the camera 10a at an angle of rotation about the camera base axis is β. Angles σa and σb (see σ in FIG. 9D) at which the horizontal axis (imaging X-axis) of the camera's virtual imaging plane tilts with respect to the camera base axis by rotating the camera about the optical axis of the camera. Ask. These parameters are called angle constants.

  Αa, αb, and β are obtained by performing the following coordinate transformation on the merged coordinates. The coordinates are converted such that the base point is the camera 10a, the camera base axis is the Z axis, and the angle of rotation about the camera base axis is the optical axis direction of the camera 10a.

Coordinate transformation 5: The base point is the camera 10a. In the merged [X4], if the matrix of Cca coordinates x, y, z is [Cca], the x, y, z matrix of each point before the base point change is [X4], and after the change is [X5], (33).

[X5] = [X4]-[Cca] (33)

上の式中ｘ、ｙ値はＣｃｂの［Ｘ５］における座標値である。Ｃｃｂと同じ添え字を付している。
とすると、下記式（３６）を得る。

Coordinate transformation 6: The X axis is aligned with the XY plane projection direction of the camera base axis (10b-10a). If the x, y, z matrix of each point before rotation is [X5] and the rotation is [X6], the following equation (34) is obtained.

[X6] = [R4] [X5] (34)

[R4] is

In the above formula, the x and y values are the coordinate values of Ccb at [X5]. The same subscript as Ccb is attached.
Then, the following formula (36) is obtained.

Coordinate transformation 7: Align the Z-axis with the direction of the camera base axis (10b-10a). If the x, y, z matrix of each point before rotation is [X6] and after rotation is [X7]

[X7] = [R5] [X6] (37)

上の式中ｘ、ｚ値はＣｃｂの［Ｘ６］における座標値である。Ｃｃｂと同じ添え字を付している。
とすると、

である。 [R5] is

In the above formula, the x and z values are the coordinate values of Ccb at [X6]. The same subscript as Ccb is attached.
Then,

It is.

Coordinate transformation 8: The X axis is aligned with the XY plane projection direction of the optical axis of the camera 10a. When the x, y, z matrix of each point before rotation is [X7] and after rotation is [X8], the following equation (40) is obtained.
[X8] = [R6] [X7] (40)

上の式中ｘ、ｙ値はＣ０ａの［Ｘ７］における座標値である。Ｃ０ａと同じ添え字を付している。
とすると、下記式（４２）を得る。

[R6] is

In the above formula, the x and y values are the coordinate values of C0a at [X7]. The same subscript as C0a is attached.
Then, the following formula (42) is obtained.

上の式中ｘ、ｚ値はＣ０ａの［Ｘ８］における座標値である。Ｃ０ａと同じ添え字を付している。

上の式中ｘ、ｙ、ｚ値はＣ０ｂ、Ｃｃｂの［Ｘ８］における座標値である。関係する点と同じ添え字を付している。

上の式中ｘ、ｙ値はＣ０ｂの［Ｘ８］における座標値である。Ｃ０ｂと同じ添え字を付している。 The parameters to be obtained are obtained by the following equations.

In the above formula, the x and z values are the coordinate values of C0a at [X8]. The same subscript as C0a is attached.

In the above formula, the x, y, and z values are the coordinate values of C0b and Ccb at [X8]. The same subscripts are attached as relevant points.

In the above formula, the x and y values are the coordinate values of C0b at [X8]. The same subscript as C0b is given.

Next, σa and σb. σa is obtained by returning the coordinate of the merged camera 10b to the original coordinate system of the camera 10a. That is, for [X4] of Cca and Ccb, the reverse of the coordinate transformation used to merge the coordinates related to the camera 10a is performed as in the following equation (46).

[X1] = [R1a] ⁻¹ [R2a] ⁻¹ [R3a] ⁻¹ [X4] (46)

The subscript a means the rotation of the expressions (26), (29), and (32) related to the camera 10a.

上式中ｘ、ｙ値は式（４７）におけるＣｃｂの［Ｘ０］の座標値である。Ｃｃｂと同じ添え字を付している。
[X0] = [X1]-[Cca] (47)

[Cca] is a matrix of x, y, z coordinate values of the camera 10a in [X1]. In equation (47), [X1] [X0] need only be Ccb. σa0 is the rotation of the camera about the optical axis of the camera, and can be expressed as an angle at which the camera (X axis of the imaging surface) is inclined with respect to the camera base axis. From the viewpoint of camera imaging, another camera direction (no real image is captured by camera basic imaging) is also an angle formed by the X axis of the imaging surface (although the sign is negative). Therefore, it can be expressed by the following formula (48).

In the above equation, the x and y values are the coordinate values of [X0] of Ccb in equation (47). The same subscript as Ccb is attached.

Similarly, σb is obtained by returning the coordinate of the merged camera 10a to the original coordinate system of the camera 10b. That is, the reverse of the coordinate transformation used to merge the coordinates related to the camera 10b with respect to [X4] of Cca and Ccb is performed as in the following equation (49).

[X1] = [R1b] ⁻¹ [R2b] ⁻¹ [R3b] ⁻¹ [X4] (49)

The subscript b means the rotation related to the camera 10b in the expressions (26), (29), and (32).

[X0] = [X1]-[Ccb] (50)

[Ccb] is a matrix of x, y, z coordinate values of the camera 10b in [X1]. In the above formula (50), [C1] and [X0] need only be performed by Cca.

上式中ｘ、ｙ値は式（５０）におけるＣｃａの［Ｘ０］の座標値である。Ｃｃａと同じ添え字を付している。以上でアングル定数が求まる。つまり、２つのカメラ姿勢が合併されたことになる。 Σb can be expressed by the following equation.

In the above equation, the x and y values are the coordinate values of [X0] of Cca in equation (50). The same subscript as Cca is attached. The angle constant is obtained as described above. That is, two camera postures are merged.

  The above is an example in which the present invention is implemented by a method of merging camera postures. For grasping the camera posture, there are a conventional method using a coplanar condition, a method of eliminating parallax, a method using a collinear condition, and the like. It is also possible to combine the method for obtaining the camera posture by a conventional method and the camera posture grasp for each single photograph according to the present invention. For example, the present invention may be included in a part of the conventional method for obtaining the camera posture to assist in improving accuracy. In short, the calculation method does not matter.

  When the angle constant is obtained as described above, triangulation calculation can be performed based on the angle constant and the virtual imaging plane coordinates of the measured point Pn. First, prepare the data required for 3D triangulation calculation. That is, the angle αan, αbn in the direction of the measured point Pn with respect to the direction in which the camera 10b is viewed from the camera 10a and the angle of the measured point Pn with the camera base axis as the reference direction common to the cameras 10a, b. The rotation angles βan and βbn are obtained. The triangulation calculation is performed for the dimension reference points Ps1 and Ps2 in addition to the measurement point Pn, but the measurement point Pn will be described as a representative. At the time of calculation, pixel coordinates are converted into virtual imaging plane coordinates xfn and yfn using camera characteristic data for Pn as shown in step S106.

αｂｎ、βｂｎについても同様に求められる。 αan and βan can be calculated from the geometric relationship shown in FIG.

αbn and βbn are similarly obtained.

  Regarding the rotation angle β, angles βan and βbn are obtained from the equation (59), and these are rotation angles with reference to the optical axes 9a and 9b of the respective cameras. Which one should be used as a standard should be adopted. In this description, since 9a is used as a reference, βan is adopted.

Based on the above, the three-dimensional model coordinates of the measured point Pn are calculated. For triangulation, the distance E between the cameras 10a and 10b is also necessary, but since it is unknown for the time being, it is assumed to be 1 and a similar three-dimensional coordinate (denoted as a model) is obtained. As shown in FIG. 10, the orthogonal coordinates (x, y, z) of the measured point Pn with the camera 10a as the origin are obtained by the following formula from the principle of triangulation. As shown in FIG. 10B, the camera base axis 7 is in the X-axis direction, the paper surface is in the XY plane, the axis orthogonal to the Z-axis direction, the distance between the cameras 10a and 10b is E, and FIG. Thus, let H be the distance from the X axis to the measured point Pn vertically.

The model coordinates of the measured point Pn and the dimension reference points Ps1 and Ps2 are obtained by the above formula. In step S110, only the model coordinates of the dimension reference point may be calculated by the above formula until E is determined in step S112. In step S112, an actual value is obtained for the distance E as a temporary value. First, the interval ls is obtained from the model coordinate values of the dimension reference points Ps1, Ps2 obtained in step S110 by the following equation (64). The subscripts s1 and s2 indicate model coordinates of the dimension reference points Ps1 and Ps2.

ls = √ {(xs1−xs2) ² + (ys1−ys2) ² + (zs1−zs2) ² } (64)

  Next, a ratio between the reference dimensions Ls and ls actually measured in step S102 is obtained. This ratio is multiplied by each model coordinate value obtained. Alternatively, the coordinates of the actual dimension of the measurement point Pn may be directly obtained by returning to step S110 and multiplying by E in the above equation (60). Through the above steps, the actual value of the three-dimensional coordinates of the measured point Pn with the camera 10a as the origin is calculated.

  Photogrammetry using photographs from two directions can be performed by the above method. For photogrammetry using photographs from two directions, there are conventional methods such as a method using coplanar conditions, a method of eliminating parallax, and a method using colinear conditions. In either case, the posture of the camera is obtained based on the difference in the image of the subject caused by providing a difference between the two shooting directions, that is, the parallax. In this embodiment, the posture of the camera is obtained regardless of the difference between the two directions. The accuracy with which the camera posture is determined is not related to the difference between the two directions. In the conventional photogrammetry method for obtaining the camera posture based on the parallax of the photos in two directions, the accuracy of calculating the camera posture decreases as the difference between the two directions becomes smaller. According to the method of the present embodiment, the camera posture can be calculated with high accuracy even in surveying when the difference between the two directions is small.

  In the conventional method, there is a successive approximation method, that is, an iterative calculation step when calculating the posture of the camera, and the calculation takes time. In the case of this embodiment, there is no repetition, so the calculation becomes faster. Moreover, even if combined with the conventional method, the initial value can be set in the vicinity of the solution, so that the calculation becomes faster.

  As is apparent from the above, if the purpose is to determine the camera posture, steps S102 to S108 in FIG. 4 may be executed. When photogrammetry is executed using this, steps S110 and S112 may be further executed. Since the camera posture can be accurately grasped by steps S102 to S108, the subject can be accurately measured in the survey steps S110 and S112 using this.

  In the above embodiment, the case where the apartment house is set as the subject is exemplified, so that various points can be used as reference points. However, when the subject is a natural shaped object or the like, it is assumed that it is difficult to set the reference point. In such a case, a sign as shown in FIG. 11 may be attached to the subject and photographed. FIGS. 11A and 11B are diagrams showing examples of signs applied to a subject. As shown in (A) and (B), two or more straight line pairs (for example, K1, K2) that are accurate parallel lines or points (for example, PL1 to PL4) that can be used for specifying parallel lines are arranged as shapes or patterns. In addition, the point (for example, Ph1, Ph2) on the straight line (K3) perpendicularly intersecting the pair of parallel lines may be any shape or pattern.

  Further, the posture reference parallel lines K1 and K2 set in the sign are preferably longer in length than the line interval. For example, it should be set longer as long as it is 1.5 times or more and the usability is not deteriorated. If the posture reference parallel lines K1 and K2 are set relatively long as described above, even if the perspective is arranged in a deep state as described above, imaging of the parallel lines is large (long), and thus the direction detection accuracy is improved. Can be raised.

  Further, the direction reference points Ph1 and Ph2 may also be used as the dimension reference points Ps1 and Ps2, and the interval Ls may be measured or specified. It is also possible to make it easy to identify points and lines as marks with high reflectivity. The sign is placed on the ground, the floor, or on the wall so that the sign is reflected as widely as possible on the entire subject, and the posture reference parallel lines K1 and K2 are substantially directed in the perspective direction. The above-described orientation reference parallel lines are the same when using the structure and pattern of the subject.

  In the above embodiment, the case where the camera posture is grasped from a single photograph and applied to surveying has been described. However, as described above, the technique for grasping the camera posture can be widely used in other fields such as robots and input devices in addition to surveying. For example, in a case directly related to a camera, it can be used for posture determination of an object such as a device on which the camera is installed, or posture determination when a subject is an object such as a device. In addition, if combined with a method that automatically detects posture reference parallel lines and various reference points from an image by increasing the brightness, etc., it can dynamically detect the posture of the device or object with high accuracy or automatically detect the device or object. It can also be applied to attitude control.

  Further, an application example of the above-described embodiment will be briefly described with reference to FIG. The operation after confirming the inclination of the selection plane 22 will be described. A second plane that intersects the first plane is set with the selection plane 22 as the first plane. This second plane is referred to as an auxiliary plane 40. FIG. 12 is a diagram showing an arrangement relationship between the selection plane 22 as the first plane and the auxiliary plane 40 as the second plane. The auxiliary plane 40 is arranged with the following relationship with respect to the selection plane 22.

  The auxiliary plane 40 intersects the selection plane 22 at an angle α. And two auxiliary points Pc1 and Pc2 shared on the intersection line of these two surfaces are set. When performing camera shooting, a single photograph is obtained so as to further include the angle α and the auxiliary points Pc1 and Pc2. If the angle α and the auxiliary points Pc1 and Pc2 are used, the inclination of the auxiliary plane 40 with respect to the camera 10 can be calculated from the connection relationship with the selection plane 22. Pc1 and Pc2 can calculate three-dimensional coordinates as points on the selection plane 22. Pc1 and Pc2 can also be used as direction reference points Ph1 and Ph2 on the auxiliary plane 40. As described above, the camera posture of the selection plane 22 can be grasped. Therefore, the camera posture of the auxiliary plane 40 can be grasped.

  By using the auxiliary plane 40, a reference plane for grasping the posture of the camera, that is, a reference coordinate system can be set in addition to the selection plane 22. The method of surveying the measurement points on the selection plane 22 in which the posture relation with the camera is grasped in the example of the photogrammetry using one photograph taken from one direction by the photogrammetry has already been described. Similarly, the measured points can be surveyed on the auxiliary plane 40 in which the posture relation with the camera is grasped. That is, even in photogrammetry using a single photograph taken from one direction, if the auxiliary plane 40 is combined, survey points on a plane other than the selected plane 22 or a plurality of planes can be measured. As shown in FIG. 12, if the auxiliary plane 40 is further set with respect to the selection plane 22, the versatility of the present invention is improved, so the applicable range can be expanded.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims. It can be changed.

It is the figure shown about the case where a camera attitude | position is calculated | required from a single photograph with the conventional method. It is the figure shown about the case of this invention which calculates | requires a camera attitude | position from a single photograph. It is the figure shown about the Example which surveys after taking a camera photo with a common house as a subject, grasping the camera posture. It is the flowchart which put together the process from image | photographing a to-be-photographed object, calculating | requiring a camera posture, and performing photogrammetry. It is the block diagram which showed the example of the hardware constitutions suitable for implementing an Example. It is the figure shown about the concept which calculates the vanishing point of a posture reference parallel line. It is a geometric diagram showing the relationship between the vanishing point on the virtual imaging surface and the selected plane. (A) is the figure which showed notionally the arrangement | positioning of each merged point, (B) is the figure of the line which projected the line of the optical axis direction of the camera on the orthogonal cross section of the camera base axis. It is the figure which showed the relationship between the imaging | photography and the point on a to-be-photographed object in two cameras, and the arrangement | positioning of the camera currently image | photographed for surveying. It is the figure shown about the principle of triangulation. It is the figure which showed the example of the label | marker applied to a to-be-photographed object. It is the figure shown about the application example of the Example.

Explanation of symbols

1 Selected plane 10 Camera 15a, 15b Single photograph 20 Subject (shared apartment)
22 Selection plane K1, K2 Posture reference parallel line Ph1, Ph2 direction reference point PL1-PL4 Parallel reference point PN Measured point Ps1, Ps2 Dimensions reference point

Claims (10)

A plurality of direction reference points are set on a selection plane defined on the subject, and a posture reference parallel line orthogonal to the selection plane is set.
And at least a first step of obtaining a single photograph by photographing the subject with a camera so as to include the selected plane and the posture reference parallel line in a photographing posture in a state of being directly opposite to the selected plane. The camera posture grasp method. The first step;
A second coordinate system is defined on the single photograph taken in the first step, and the direction reference point on the selected plane and the parallel reference point defined on the posture reference parallel line are specified to obtain coordinates. 2. The method according to claim 1, further comprising: a step and a third step of obtaining a camera posture with respect to the selected plane based on the coordinates of the direction reference point and the parallel reference point obtained in the second step. Camera posture grasp method. In the first step, an auxiliary plane arranged at an angle α with the selected plane is further set, and a plurality of auxiliary points are set on the intersection line of the selected plane and the auxiliary plane. And taking the subject with a camera so as to include the auxiliary point to obtain the single photograph,
In the second step, the coordinates of the auxiliary points are further obtained on the single photograph,
3. The camera posture grasping method according to claim 2, wherein in the third step, the camera posture with respect to the auxiliary plane is obtained by further referring to the angle α and the coordinates of the auxiliary point. 4. The first step is performed by placing a sign provided with a figure capable of specifying the orientation reference parallel line and the direction reference point of the selected plane on the subject, and executing the first step. 5. Crab camera posture grasp method. A plurality of survey reference points whose positions are known to the subject including the survey point and a dimension reference point whose distance is known are set, and the three-dimensional of the survey point is obtained using a single photograph obtained by photographing the subject with a camera. A photogrammetry method for obtaining coordinates,
4. A photogrammetry method, comprising, in part, a camera posture grasp process for obtaining a camera posture by the camera posture grasp method according to claim 1 to obtain a three-dimensional coordinate of the measured point. . Using a plurality of single photographs obtained by photographing the subject with a camera from a plurality of directions, including a camera posture grasping process for obtaining a camera posture for each single photograph, and obtaining three-dimensional coordinates of the measured points. The photogrammetry method according to claim 5. A plurality of direction reference points are set on a selection plane defined on the subject, and a posture reference parallel line orthogonal to the selection plane is set, and the selection is performed in a shooting posture in a state close to the selection plane. A first process of reading an image of a single photograph obtained by photographing the subject with a camera so as to include a plane and the posture reference parallel line;
A second process of defining a two-dimensional coordinate system on the single photograph and specifying the direction reference point on the selected plane and the parallel reference point defined on the posture reference parallel line to obtain coordinates;
A program for grasping a camera posture, which causes a computer to execute a third process for obtaining a camera posture with respect to the selected plane based on the coordinates of the direction reference point and the parallel reference point. Furthermore, an auxiliary plane arranged at an angle α with the selected plane is set, and a plurality of auxiliary points are set on the intersection line of the selected plane and the auxiliary plane, and the auxiliary plane and the auxiliary point are also included. Read a single photo of the subject taken with the camera,
Furthermore, the coordinates of the auxiliary point are obtained on the single photograph,
8. The camera posture grasping program according to claim 7, further comprising: obtaining a camera posture with respect to the auxiliary plane based on the angle α and the coordinates of the auxiliary point. A plurality of survey reference points whose positions are known to the subject including the survey point and a dimension reference point whose distance is known are set, and the three-dimensional of the survey point is obtained using a single photograph obtained by photographing the subject with a camera. A photogrammetry program for finding coordinates,
A camera posture grasping process for obtaining a camera posture by the camera posture grasping method according to any one of claims 1 to 3 is included, and the computer is configured to execute a process for obtaining a three-dimensional coordinate of the measured point. A photogrammetry program. Using a plurality of single photographs obtained by photographing the subject with a camera from a plurality of directions and including in part a camera posture grasping process for obtaining a camera posture for each single photograph, a process for obtaining three-dimensional coordinates of the measured points is performed on a computer. The photogrammetry program according to claim 9, wherein the photogrammetry program is executed.

Images