JP2004048555A - Calculation instrument of position and orientation of camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a calculation instrument of a position and an orientation of a camera which can calculate the position and the orientation of the camera by simple work without requiring a special installation target. <P>SOLUTION: A sketch indicating a room, etc. where a video camera by which operations of pan/tilt angles, etc. are enabled is installed and a video image of the camera are displayed and an optional place where it is assumed that the camera is probably installed from the video image is set as an initial value indicating a camera position. A plurality of optional points on the front surface of the camera are specified by using the sketch and a pan angle and a tilt angle of the camera in the case of photographing the points are calculated. Minute fluctuation quantity is calculated from the pan angle and the tilt angle of the camera for photographing the specified optional points, the initial value is corrected and an exact position of the camera is calculated. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a camera position and direction calculation device, and more particularly to a camera position and direction calculation device that obtains a position and a direction of a camera operated using a sketch.
[0002]
[Prior art]
2. Description of the Related Art Photographing of conferences and lectures using a video camera capable of controlling the pan / tilt angle, zoom, and the like from the outside is widely performed.
[0003]
As a method of operating the camera when performing such shooting, for example, the user can use the operation buttons while viewing the screen on which the image shot by the camera is displayed, and use the operation buttons to adjust the pan / tilt angle and zoom of the camera. Operation method for controlling the
[0004]
As a method for easily operating a camera remotely, for example, Japanese Patent Application Laid-Open No. 2000-83245 proposes an operation method using a sketch of a place where a camera is installed.
[0005]
In order to use the operation method of controlling the camera using this sketch, it is necessary to measure the position and direction of the camera in advance and hold it as a parameter, for example, to set a special measurement target at a specified position The position and direction of the camera are determined by installing the camera and imaging the measurement target.
[0006]
[Problems to be solved by the invention]
However, in this method, it is necessary to set a special measurement target for obtaining the position and direction of the camera, and when it is difficult to set such a measurement target, it is necessary to manually acquire and input parameters. To use the sketch-based camera operation method, it was necessary to find the position and orientation of the camera with great effort and time.
[0007]
Therefore, an object of the present invention is to provide a camera position and direction calculating device capable of calculating the position and direction of a camera by a simple operation without requiring a special installation target.
[0008]
[Means for Solving the Problems]
In the camera position and direction calculation device of the present invention, storage means for storing a sketch of the installation location where the camera is installed, means for displaying the sketch, camera position designation means for designating the position of the camera using the sketch, An object designating means for designating the object using the sketch, a camera control angle acquiring means for acquiring a control angle of the camera with respect to the object designated by the object designating means, and a camera designated by the camera position designating means. Calculating means for calculating the position and direction of the camera using the position, the position of the object designated by the object designating means, and the camera control angle acquired by the camera control angle acquiring means;
[0009]
Further, the apparatus further comprises height specifying means for specifying the height of the object specified by the object specifying means, wherein the calculating means obtains the position of the camera specified by the camera position specifying means and the camera control angle obtaining means. The position and direction of the camera are calculated using the obtained camera control angle and the position of the object specified by the object specifying means and the height of the object input by the height specifying means.
[0010]
Here, the object specifying means specifies at least three objects, and the calculating means calculates the position and direction of the camera based on the specification of at least three objects specified by the object specifying means.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of a camera position and direction calculation device according to the present invention will be described in detail with reference to the accompanying drawings.
[0012]
FIG. 1 is a schematic block diagram showing an example of the overall configuration of a camera position and direction calculation device according to the present invention.
[0013]
This camera position and direction calculating device presumes the position of a camera such as a video camera or a digital camera at an arbitrary position on a sketch, and calculates a plurality of arbitrary points and a control angle which is an actual camera inclination with respect to the points. The camera position and direction calculation device 100 is configured to obtain the actual camera position and direction (camera information) from these elements. As shown in FIG. CPU 110 that performs overall control processing, for example, a primary storage device 120 that is a storage device such as a RAM, for example, a secondary storage device 130 that is a storage device such as a hard disk, a communication interface 140, a video capture unit 150, an input / output interface 160, bitmap display 170, operation unit 180 It is provided.
[0014]
Here, as the operation unit 180, for example, a mouse, a touch sensor panel, a keyboard, or the like can be appropriately used.
[0015]
In addition, the secondary storage device 130 has a floor plan of a room where the video camera 200 is installed, and room basic information.
[0016]
The image captured by the video camera 200 is transmitted to the camera position and direction calculation device 100 via the multi-input integrated screen creator 300 capable of integrating a plurality of video signals into one screen.
[0017]
Here, the video camera 200 can perform controls such as pan, tilt, and zoom in accordance with an external signal, and is configured to be operated in accordance with a control signal transmitted from the camera position and direction calculation device 100. .
[0018]
As the control signal line, for example, RS-232C or the like can be used. In the present embodiment, a four-input integrated screen creator that integrates and outputs four video signal inputs into one screen is provided. Is used as the multi-input integrated screen creator 300, but the number of inputs is not limited to four.
[0019]
Next, the operation of the camera position and direction calculation device according to the present embodiment will be described with reference to FIG.
[0020]
In FIG. 2, first, a camera number acquisition command is created by the CPU, and transmitted to each video camera via the communication interface. Then, by receiving a response from the video camera that has received the camera number acquisition command, the number N of video cameras is acquired and held in the primary storage device (step 101).
[0021]
Further, in order to identify each camera, for example, numbers 1 to N are allocated to the respective video cameras, and the video cameras are stored in the primary storage device and the user is notified of the camera having the number i for which the position and the direction are to be detected (step 102). ).
[0022]
Here, as a notification method to the user, for example, as shown in FIG. 3, a video display window 400 for displaying the video of the video camera is displayed on the display, and the video is captured by the video camera corresponding to the camera number i. A process of surrounding the video 401 with a frame 402 is performed.
[0023]
A control signal for causing the video camera corresponding to the camera number i to reciprocate up and down or left and right in the direction of the video camera and a control signal for performing any movement are transmitted to the video camera displayed on the video display window 400. It is also possible to configure so as to notify when the image 401 from the video camera shakes.
[0024]
Then, as shown in an example in FIG. 4, a camera position acquisition screen window 410 is displayed on the display (step 103).
[0025]
Here, the camera position acquisition screen window 410 is provided with a sketch 411 of a place where the video camera is installed, a message display unit 412 for displaying a message to the user, and a confirmation button 413. The sketch 411 is a secondary storage. An appropriate one is selected and displayed from the room plan stored in the device.
[0026]
Then, a message is displayed on the message display unit 412, requesting the user to specify the camera position of the camera number i, and storing the position of the video camera specified by the user.
[0027]
Here, for example, when a mouse is used as the operation unit, the position of the video camera is designated by a click operation on the floor plan 411 by the user, and the designated position (Px, Py) on the floor plan 411 where the click operation is performed is performed. ) Is detected and stored.
[0028]
At this time, it is desirable to request an arbitrary position where the camera is assumed to be installed from the displayed video image of the video camera, and to specify the specified position (Px, Py). It is preferable that the outline is displayed as a red circle on the sketch so that the user can easily confirm it.
[0029]
The user can correct the specified position (Px, Py) by specifying another position on the floor plan 411 again.
[0030]
Then, when the user presses the confirmation button 413 and the position designation of the video camera is determined, the designated position (Px, Py) is changed from the coordinate system on the screen to the actual room based on the room basic information held in the secondary storage device. Convert to a coordinate system in space to find a point (P'x, P'y) in the room space, record P'x in the internal variable Ox [i] and record P'y in the internal variable Oy [i] Then, they are stored in the primary storage device (step 104).
[0031]
Here, Ox [i] is the i-th element of the internal variable array Ox using the internal variable i indicating the camera number, and the same applies to Oy [i].
[0032]
Then, the camera direction designation screen window 420 shown in FIG. 5 is displayed on the display instead of the camera position acquisition screen window (step 105).
[0033]
Here, the camera direction designation screen window 420 is provided with a sketch 421 of a place where the video camera is installed, a message display unit 422 for displaying a message to the user, a confirmation button 423, a camera control unit 424, and a height designation unit 425. The sketch 421 selects and displays an appropriate sketch from the sketches stored in the secondary storage device.
[0034]
Note that the camera control unit 424 can be configured with a direction instruction button, for example, as shown in FIG. 5, or may be configured with a clickable map or a slider. At this time, it is possible to directly specify the direction of the video camera.
[0035]
Then, a message is displayed on the message display unit 422 to request the user to specify the object, and the user operates the camera control unit 424 using the operation unit to change the direction of the video camera with the camera number i. An arbitrary object on the front side of the camera is designated on the sketch 421.
[0036]
At this time, the user is requested to specify an object whose height is known. Examples of the object whose height is known include, for example, a point on the floor where the height is clearly 0 cm, and an appliance whose height is apparent may be specified as the object, In this case, the height of the object is specified by the user using the height specifying unit 425.
[0037]
Here, as the height specifying unit 425, for example, a text box that can be freely input, a select menu or a radio button group that specifies one height from a plurality of pieces of pre-registered height information, a pre-registered A check box for selecting use / non-use of one piece of height information may be appropriately selected as needed.
[0038]
For example, when a mouse is used for the operation unit, the user specifies the click operation on the sketch 421 as the position designation from the user, and specifies the position (Dx, Dy) on the sketch 421 where the click operation was performed. It is detected and stored as the position of the object.
[0039]
At this time, the stored position (Dx, Dy) is displayed on the sketch using a shape or color different from the symbol indicating the designated position of the video camera, such as a green circle, on the sketch, so that the user can easily check it. Preferably, the user can correct the storage location (Dx, Dy) by specifying another location on the floor plan 411 again.
[0040]
Here, when the user presses the confirmation button 423 and the position designation of the object is determined, (Dx, Dy) is converted from the coordinate system on the screen to the actual room space based on the room basic information held in the secondary storage device. The point (D'x, D'y) in the room space is obtained by converting the coordinate system into a medium coordinate system, D'x is assigned to the internal variable ODx [i] [j], and D'x is assigned to the internal variable ODy [i] [j]. The value designated by the height designation unit 425 is recorded in ODz [i] [j], and D′ y is stored in the primary storage device.
[0041]
Note that ODx [i] [j] is the ith in the row direction and j in the column direction of the internal variable array ODx having a two-dimensional array structure indicated by the internal variable i indicating the camera number and the internal variable j indicating the number of trials. ODy [i] [j] and ODz [i] [j] are the same.
[0042]
Then, the CPU generates a camera direction acquisition command, transmits the command to the video camera via the communication interface, and transmits a pan angle response PA and a tilt angle response TA indicating a pan / tilt angle (control angle) of the camera with respect to the object. , The PA is recorded in the internal variable PA [i] [j], and the TA is recorded in the internal variable TA [i] [j], and are respectively stored in the primary storage device (step 106).
[0043]
Here, it is determined whether or not the number of trials has finished the specified number of times (step 107), and when the value of the internal variable j indicating the number of trials is smaller than the predetermined number of times k of the specified number of trials (NO in step 107). Adds 1 to the value recorded in the internal variable j, holds it in the primary storage device as a new internal variable j, and requests the designation of the object again (step 105).
[0044]
The specified trial number k may be arbitrary, but is preferably 3 or more.
[0045]
If the value indicated by the internal variable j is the same as the prescribed trial count value k (YES in step 107), the value indicated by the internal variable i held in the primary storage device is held in the primary storage device. It is determined whether or not the value exceeds an internal variable N indicating the number of video cameras present (step 108).
[0046]
When the value of the internal variable N indicating the number of video cameras is equal to the value indicated by the internal variable i, or the value indicated by the internal variable i is smaller than the value of the internal variable N indicating the number of video cameras (NO in step 108) ) Indicates that the video camera with the camera number i indicated by the new internal variable i is added to the value recorded in the internal variable i indicating the camera number, and is stored in the primary storage device as a new internal variable i. (Step 102).
[0047]
When the value indicated by the internal variable i is equal to or greater than the internal variable N indicating the number of video cameras, the internal variable N, the internal variable array Ox, and the internal variable array Oy held in the primary storage device are stored. From the values recorded in the internal variable array ODx, the internal variable array ODy, the internal variable array ODz, the internal variable array PA, and the internal variable array TA, the position and direction of the video camera are estimated and calculated (step 111).
[0048]
Here, the estimation calculation for calculating the position and direction of the video camera can be performed using a generally known geometric technique. Hereinafter, an example of the estimation calculation method will be described in detail with reference to FIG. Will be described.
[0049]
First, a value 1 is recorded in an internal variable i indicating a camera number and initialization is performed (step 201).
[0050]
Then, the slopes from the internal variable arrays ODx, ODy, ODz, PA, and TA pass through the points (ODx [i] [j], ODy [i] [j], ODz [i] [j]) in the three-dimensional space. (Cos (PA [i] [j]) cos (TA [i] [j]), sin (PA [i] [j]) cos (TA [i] [j]), sin (TA [i] [ j])), and obtain k straight lines 11,..., lk.
[0051]
Here, j represents an integer value from 1 to k. Further, a point M (Mx, My, Mz) in the three-dimensional space where the sum of the distances from the k straight lines 11, 1,.
[0052]
Then, the value of the internal variable Ox [i] is substituted for the internal variable X, the value of the internal variable Oy [i] is substituted for the internal variable Y, the value of the Z coordinate Mz of the point M is substituted for the internal variable Z, and (X, Y, Z) is set as the initial value of the camera position (step 202).
[0053]
Further, two points Pm in a three-dimensional space are determined by using an internal variable i indicating a camera number and m-th and n-th elements of an internal variable array ODx [i], ODy [i], ODz [i]. (ODx [i] [m], ODy [i] [m], ODz [i] [m]), Pn (OD [i] [n], ODy [i] [n], ODz [i] [n ]) Is defined.
[0054]
Here, arbitrary two values m and n are selected from integer values from 1 to k, corresponding points Pm and Pn in the three-dimensional space are defined, and Pm and Pn are set with the camera position (X, Y, Z) as a vertex. Is calculated as the cosine cos θmn of the angle θmn.
[0055]
Also, a direction vector Qm (cos (PA [i] [m]) cos (TA [i] [m]), sin (PA) determined from the m-th element of the internal variable arrays PA [i] and TA [i] [I] [m]) cos (TA [i] [m]), sin (TA [i] [m])) and a direction vector Qn (cos (PA [i] [ n]) cos (TA [i] [n]), sin (PA [i] [n]) cos (TA [i] [n]), sin (TA [i] [n]) The cosine cos θ′mn of θ′mn is calculated.
[0056]
Then, for all combinations of m and n, the sum of squares of the difference between cos θmn and cos θ′mn is S (X, Y, Z).
[0057]
Further, S (X + dX, Y + dY, Z + dZ) is calculated for a nearby point separated from the camera position (X, Y, Z) by a constant value ± a or 0 in the X-axis, Y-axis, and / or Z-axis directions and the position. , S are determined to be the minimum value (dX, dY, dZ) (step 203).
[0058]
Here, dX, dY, and dZ represent the fluctuation amounts in the X-axis direction, the Y-axis direction, and the Z-axis direction, respectively, and take + a, 0, or -a as a value. The value of the constant value a may be, for example, 1.
[0059]
Then, it is determined whether or not to update the camera position (step 204). If the smile fluctuation amount (dX, dY, dZ) is not (0, 0, 0) (NO in step 204), the camera position (X, Y) , Z) to add the minute fluctuation amount (dX, dY, dZ) to update the camera position, and calculate the minute fluctuation amount again (step 203).
[0060]
When (dX, dY, dZ) is (0, 0, 0) (YES in step 205), the camera position (X, Y, Z) and k points Pm (m = 1,..., K), k direction vectors Vm (m = 1,..., K) are determined.
[0061]
Note that the camera position (X, Y, Z) may be obtained with higher accuracy by reducing the value a of the constant value and again obtaining the minute fluctuation amount. At this time, the constant value a is preferably reduced in order from, for example, 1 to 0.1 and 0.01.
[0062]
Then, for each value of m, the difference between the angles of Vm and Qm in the pan direction and the tilt direction is obtained, and the average value dθ of the difference in the pan angle direction is set to the pan direction initial value SP [i of the video camera indicated by the camera number i. ], And the average value dφ of the difference in the tilt angle direction is set as a tilt direction initial value ST [i] of the video camera indicated by the camera number i.
[0063]
Here, the camera position (X, Y, Z) is recorded in internal variables X [i], Y [i], Z [i], respectively, and is stored in the primary storage device. Also, the values of SP [i] and ST [i] are similarly stored in the primary storage device (step 205).
[0064]
Further, it is determined whether or not the measurement has been completed for all the cameras (step 206). If the value recorded in the internal variable i does not exceed the value recorded in the internal variable N indicating the number of cameras (step 206). NO) is obtained by adding 1 to the value recorded in the internal variable i and holding the same in the primary storage device as a new internal variable i, and the minute fluctuation amount of the video camera of the camera number i indicated by the new internal variable i ( dX, dY, dZ) are obtained (step 203).
[0065]
If the value recorded in the internal variable i is the same as the value recorded in the internal variable N indicating the number of cameras (YES in step 206), the estimation calculation of the position and direction of the video camera ends. .
[0066]
Then, the parameters indicating the position of each video camera and the initial directions of the pan angle direction and the tilt angle direction obtained by the estimation calculation are recorded in the primary storage device using the XML format or the like. Note that the result may be transferred to the secondary storage device and recorded.
[0067]
In this embodiment, it is possible to easily specify an initial value assuming the position of the camera from the video and the sketch of the video camera, and to operate the pan and tilt angles of the camera while checking the video of the video camera. Therefore, it is possible to easily specify an object on which to obtain the minute fluctuation amount on a sketch.
[0068]
Then, the position and direction of the camera can be accurately calculated based on the simple operation history, the correspondence between the sketch and the actual place.
[0069]
Note that the camera position and direction calculation device may have a client-server configuration including a client device that is a client-side terminal that views a camera image and a server device that controls the camera and distributes a captured image.
[0070]
At this time, the server device provides information such as room information and a floor plan to the client device. Based on this information, the client device determines an internal variable N indicating the number of cameras and coordinates of the designated position in the actual room space. Internal variable arrays Ox, Oy indicating the system, internal variable arrays ODx, ODy indicating the coordinate system of the object in the actual room space, internal variable arrays ODz indicating the height of the object in the actual room space, pan An internal variable array PA indicating an angle response and an internal variable array TA indicating a tilt angle response are calculated, and the calculated information is transmitted to the server device. Based on the calculated information, the server device determines the position and direction of the camera based on the calculated information. An estimation calculation (step 111) is performed.
[0071]
【The invention's effect】
As described above, according to the present invention, parameters indicating the position and initial direction of the video camera can be easily obtained without using a special measurement target or the like.
[Brief description of the drawings]
FIG. 1 is a schematic block diagram showing an example of the overall configuration of a camera position and direction calculation device according to the present invention. FIG. 2 is a flowchart showing the operation of the camera position and direction calculation device in the present embodiment. Overview diagram showing an example of a window [FIG. 4] Overview diagram showing an example of a camera position acquisition screen window [FIG. 5] Overview diagram showing an example of a camera direction designation screen window [FIG. 6] Flow of estimation calculation in the present embodiment [Description of reference numerals]
100: Camera position and direction calculation device 110: Primary storage device 120: Communication interface 130 ... Secondary storage device 140 ... Communication interface 150 ... Video capture unit 160 ... Input / output interface 170 ... Display 180 ... Operation unit 200 ... Video camera 300 ... Multi-input integrated screen creator 400 ... Video display window 401 ... Video 402 ... Frame 410 ... Camera position acquisition screen window 412 ... Message display unit 413 ... Confirmation button 420 ... Camera direction designation screen window 422 ... Message display unit 423 ... Confirmation button 424 ... Camera control unit 425 ... Designation unit

Claims (3)

Storage means for storing a sketch of the installation location where the camera is installed,
Means for displaying the floor plan;
Camera position specifying means for specifying the position of the camera using the floor plan,
An object specifying means for specifying an object using the floor plan,
Camera control angle acquisition means for acquiring a control angle of the camera with respect to the object specified by the object specification means,
The position and direction of the camera using the camera position designated by the camera position designation unit, the position of the object designated by the object designation unit, and the control angle of the camera acquired by the camera control angle acquisition unit Calculating means for calculating the camera position and direction. The apparatus further includes a height designation unit that designates a height of the object designated by the object designation unit,
The calculating means includes:
The position of the camera designated by the camera position designation unit, the control angle of the camera acquired by the camera control angle acquisition unit, the position of the object designated by the object designation unit, and the input by the height designation unit The camera position and direction calculation device according to claim 1, wherein the position and direction of the camera are calculated using the height of the object. The object specifying means,
Specify at least three objects,
The calculating means includes:
3. The camera position and direction calculation device according to claim 1, wherein the position and direction of the camera are calculated based on designation of at least three objects designated by the object designation means.

Images