JP2017040542A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately calibrate an index and a photographing unit even if communication bandwidth is running short.SOLUTION: An information processing device comprises: image acquisition means for acquiring a first image obtained on the basis of imaging by an imaging device, and a second image with a larger amount of data than the first image; detection means for detecting an index from the first image when a first mode is set, and for detecting an index from the second image when a second mode different from the first mode is set; and derivation means for deriving at least any of three-dimensional position information on the index and a parameter of the imaging device by using a detection result by the index detection means.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program for calibrating an index.

  There is a mixed reality (hereinafter referred to as “MR”) technology for the purpose of integrating real space and virtual space. In an MR system, computer graphics representing a virtual object (hereinafter referred to as “CG”) are superimposed and displayed on a real image captured by a photographing unit, so that it is as if a virtual object exists on the spot. Can be experienced.

  In order to display a virtual object in the position and orientation and size as expected, it is necessary to superimpose CG based on external parameters (position and orientation) and internal parameters (focal length, image center, distortion coefficient, etc.) of the imaging unit. There is. Here, the three parameters of the position coordinates (x, y, z) of the imaging viewpoint position of the imaging unit in the world coordinate system and the three parameters of the posture representing the roll (tilt angle), pitch (elevation angle), and yaw (azimuth angle). Means 6 parameters.

  In Patent Document 1, an imaging unit is used by using a correspondence between a known three-dimensional coordinate of an index in real space and a two-dimensional coordinate of an index in an image obtained by imaging the index (hereinafter referred to as “image coordinates”). It is disclosed that the external parameter of is determined.

  Further, in Patent Document 2, a plurality of indices whose relative three-dimensional coordinates are known are photographed, and internal parameters of the photographing unit are calculated by using a projective geometric relationship between the indices and the photographing unit. .

  Hereinafter, the process of calculating the three-dimensional coordinates of the index is referred to as “index calibration”, and the process of calculating the internal parameters of the imaging unit is referred to as “imaging unit calibration”.

JP 2007-48068 A Japanese Patent No. 5111447

  However, when the data amount of the captured image is compressed due to the convenience of the communication band, the image quality is deteriorated by the compression, so that the calibration accuracy is lowered. On the other hand, when the data amount of the photographed image is not compressed, the frame rate is reduced (frame dropping). In order to perform calibration with high accuracy, it is necessary to finely adjust the arrangement of the indices in the image and the photographing method. However, if the photographed image is dropped, such fine adjustment is difficult. Therefore, even when the data amount of the captured image is not compressed, the accuracy of calibration is reduced.

  The present invention has been made in view of the above problems, and an object of the present invention is to calibrate an index and an imaging unit with high accuracy even when a communication band is insufficient.

  In order to solve the above-described problem, for example, an information processing apparatus according to the present invention includes a first image acquired based on imaging by an imaging device, and a second image having a larger data amount than the first image. When an image acquisition means to acquire and a first mode is set, an index is detected from the first image, and when a second mode different from the first mode is set, Detecting means for detecting an index from the second image; and deriving means for deriving at least one of the three-dimensional position information of the index and the parameters relating to the imaging device based on a detection result by the index detecting means. It is characterized by.

  According to the present invention, it is possible to calibrate the index and the photographing unit with high accuracy even when the communication band is insufficient.

It is a figure which shows an example of a structure of the information processing apparatus which concerns on this invention. It is a figure which shows typically the parameter | index arrange | positioned in real space. It is a figure showing a quadrilateral index typically. It is a figure which shows a parameter | index calibration screen typically. It is a figure which shows the general | schematic functional structure at the time of performing the process of 1st Embodiment. It is a flowchart which shows the flow of a process of 1st Embodiment. It is a figure for demonstrating the process of the parameter | index determination part 1101 in 2nd Embodiment. It is a figure which shows typically the pattern for a calibration of an imaging | photography part. It is a figure for demonstrating the process of the parameter | index determination part 1101 in 3rd Embodiment. It is a figure which shows the general | schematic functional structure at the time of performing the process of 2nd Embodiment. It is a flowchart which shows the flow of a process of 2nd Embodiment.

  Hereinafter, embodiments will be described with reference to the drawings.

(First embodiment)
First, the first embodiment will be described.

  FIG. 1 is a diagram showing an example of the configuration of an information processing apparatus according to the present invention.

  The information processing apparatus 100 includes a CPU 101, a RAM 102, a ROM 103, an HDD 104, an interface 105, and a system bus 106. An imaging unit 107, a display unit 108, and an operation unit 109 are connected to the information processing apparatus 100.

  The CPU 101 executes a program stored in the ROM 103 using the RAM 102 as a work memory, and comprehensively controls each component to be described later via the system bus 106. Thereby, various processes described later are executed.

  The HDD 104 has a role as a secondary storage device. The CPU 101 can read data from the HDD 104 and write data to the HDD 104. The secondary storage device may be a storage device such as an optical disk drive in addition to the HDD.

  The interface 105 exchanges data with external devices such as a photographing unit 107, a display unit 108, and an operation unit 109, which will be described later.

  The imaging unit 107 captures an index placed in the real space. In the present embodiment, it is a camera that captures an index in a real space. For example, a stereo camera mounted on a digital camera or a head-mounted display device HMD (Head-Mounted Display) can be used. The photographing unit 107 has an encoder that compresses the photographed image. Note that the compression method is, for example, JPEG, H.264, or the like as an encoding method used for compression. H.264 / AVC or H.264 H.264 / MVC and the like, but are not limited thereto. The photographing unit 107 has a mode for sending a compressed image via the encoder to the information processing apparatus and a mode for sending the image to the information processing apparatus without compression.

  The display unit 108 is configured by a CRT (Cathode Ray Tube), a liquid crystal display, or the like, and can display various types of information as images, characters, and the like.

  The operation unit 109 is configured with a keyboard, a mouse, and the like, and can input various instructions to the CPU 101. For example, an operation instruction for a GUI (graphical user interface) to be described later is performed by a user using a keyboard or a mouse.

  In addition, although the component of the information processing apparatus 100 exists besides the above, since it is not the main point of this invention, description is abbreviate | omitted.

[Explanation of indicators]
FIG. 2 is a diagram schematically illustrating an example of indices arranged in the real space. In FIG. 2, indicators 201 to 209 having different colors are arranged in the real space. The indicator arranged in the real space is any indicator that can detect the two-dimensional coordinates (image coordinates) of the projected image on the captured image and can identify which indicator is the indicator. It may be a form. For example, a square index 301 as shown in FIG. 3 may be used. The square index 301 has a pattern representing an identifier (identification information) inside and can be uniquely identified. The calibration of the index in the present invention is a process of calculating the three-dimensional coordinates of the indices (in the example of FIG. 2, indices 201 to 209) arranged in the real space.

  In the calibration of the indices 201 to 209, first, the indices 201 to 209 are detected from a plurality of images photographed by the photographing unit 107, and image coordinates are calculated. The detection of the index and the calculation method of the image coordinates are arbitrary. For example, an area corresponding to each marker color is detected from the captured image, and the barycentric position of the detected area is set as the image coordinates of the index corresponding to this area. do it. Then, using the image coordinates of the indicators 201 to 209 and the internal parameters of the photographing unit 107, the three-dimensional coordinates in the real space of the indicators 201 to 209 are calculated by a method called bundle adjustment. Since the bundle adjustment method is not the main point of the present invention, the description thereof is omitted. For example, the method described in Patent Document 1 can be used.

  The accuracy of the calibration of the indicators 201 to 209 differs depending on how the photographing unit 107 performs photographing. For example, in general, the calibration accuracy is higher when the indices 201 to 209 are shown in many captured images. In addition, the calibration accuracy increases as the indicators 201 to 209 appear larger. Further, the calibration accuracy is higher when the indicators 201 to 209 are not blurred or blurred and are clearly reflected. In addition, the accuracy of calibration increases as the same index is photographed from various angles. As described above, in order to calibrate the indicators 201 to 209 with high accuracy, it is necessary to finely adjust the arrangement of the indicators 201 to 209, the imaging method, and the like during imaging.

[Explanation of index composition screen]
FIG. 4 is a diagram schematically showing the index calibration screen. In FIG. 4, an index calibration screen 401 is a GUI for performing a series of index calibration processes. The preview window 402 is a window that displays a preview image. Here, the preview image (first image) indicates an image currently captured by the image capturing unit 107. The capture button 403 is a button for a user to issue a capture instruction. The user adjusts the position and orientation of the photographing unit 107 so as to increase the calibration accuracy while viewing the preview window 402, and presses the capture button 403. When the capture button 403 is pressed by the user, the information processing apparatus 100 acquires a captured image from the imaging unit 107. Configuration is performed using the captured image acquired here (hereinafter, this captured image is referred to as a calibration image). The calibration image display windows 404 to 406 are windows for displaying the calibration images (second images) acquired so far. The calibration button 407 is a button for the user to give a calibration instruction. When the user finishes obtaining a photographed image necessary for calibration, the user presses the calibration button 407. When the calibration button 407 is pressed by the user, the information processing apparatus 100 calibrates the indicators 201 to 209 using the calibration image. By using the GUI as described above, the user can finely adjust the arrangement of the indicators 201 to 209 and the imaging method at the time of shooting, and the calibration of the indicators 201 to 209 becomes highly accurate.

  Here, consider the case of compressing the data amount of the captured image. For example, when a captured image is transmitted wirelessly from the imaging unit 107 to the information processing apparatus 100 in real time, the data of the captured image needs to be compressed because the communication band is limited. When the captured image is compressed, the image quality of the captured image is reduced, so that the calculation accuracy of the image coordinates of the indicators 201 to 209 is reduced, and as a result, the accuracy of the three-dimensional coordinates (three-dimensional position information) of the indicators 201 to 209 is also reduced. Resulting in. Here, since the captured images (calibration images) used for the calibration of the indicators 201 to 209 are only images at the timing when the user presses the capture button 403, it is not always necessary to continuously send the captured images in real time. However, unless the preview image is continuously displayed in real time in the preview window 402, it is difficult for the user to finely adjust the arrangement of the indicators 201 to 209, the shooting method, and the like. That is, in order for the user to finely adjust the arrangement of the indicators 201 to 209, the shooting method, and the like, the shot image needs to be compressed and sent to the information processing apparatus 100 and continuously displayed in the preview window 402. If the image is used for proofreading, the proofreading accuracy is lowered.

  Therefore, in this embodiment, a compressed image is acquired as a preview image in real time from the photographing unit 107 and displayed on the preview window 402. When the user presses the capture button 403, an uncompressed image is acquired again from the photographing unit 107 and used as a calibration image. In the following, an example in which a compressed image is acquired as a preview image and an uncompressed image is acquired as a calibration image will be described, but the present invention is not limited to this, and the calibration image is more than the data amount of the preview image. It is applicable to all cases where the amount of data is larger. For example, a low-resolution image can be used as the preview image, and a higher-resolution image can be used as the calibration image. It is also possible to use a high compression rate image as the preview image and a low compression rate image as the calibration image.

[Processing flowchart]
FIG. 5 is a diagram illustrating an example of a functional configuration of the information processing apparatus 100 according to the present embodiment. Each functional unit configuring the information processing apparatus 100 illustrated in FIG. 5 is realized by the CPU 101 developing a program stored in the ROM 103 in the RAM 102 and executing processing according to each flowchart described below. In addition, for example, when configuring hardware as an alternative to software processing using the CPU 101, it is only necessary to configure arithmetic units and circuits corresponding to the processing of each functional unit described here. FIG. 6 is a flowchart showing the flow of processing of this embodiment. Below, the flow of the process of this embodiment is demonstrated with reference to FIG. 5 and FIG.

  In step S601, the mode setting unit 501 sets the shooting mode to the preview mode (first mode). When the shooting mode is set to the preview mode, a shot image acquisition unit 502 described later acquires a compressed image compressed by the encoder from the shooting unit 107.

  In step S <b> 602, the captured image acquisition unit 502 acquires a compressed image from the imaging unit 107. Then, the captured image acquisition unit 502 outputs the compressed image to the display unit 108.

  In step S603, the index detection unit 503 detects the index from the compressed image acquired in step S602. In this embodiment, the detection of the index means obtaining the image coordinates of the index and the identification information that each index has. Then, the image coordinates of the detected index and identification information (detection result) are output on the display unit 108.

  In step S604, the information processing apparatus 100 causes the display unit 108 to display the compressed image acquired in step S602, the image coordinates of the index, and identification information.

  In step S605, the information processing apparatus 100 determines whether there is a capture instruction from the user, and if there is no capture instruction, repeats steps S602 to S604 until there is a capture instruction.

  In step S606, the mode setting unit 501 sets the shooting mode to the main shooting mode (second mode).

  In step S <b> 607, the captured image acquisition unit 502 acquires an uncompressed image from the imaging unit 107. Then, the captured image acquisition unit 502 outputs an uncompressed image to the index detection unit 503.

  In step S608, the index detection unit 503 detects the index from the uncompressed image acquired in step S607. Then, the image coordinates of the detected index are output to the calibration unit 506. Then, the calibration unit 506 derives the three-dimensional coordinates of the index based on the acquired image coordinates of the index.

[Effect of the first embodiment]
In the present embodiment, the calibration unit 506 calibrates the index using the calibration image acquired by the above processing. This calibration result is more accurate than in the past. The details will be described below.

  Conventionally, when a compressed image is used as a calibration image, the calculation accuracy of the index image coordinates is lowered, and the calibration accuracy is lowered. On the other hand, according to the present embodiment, since an uncompressed image is used as the calibration image, the calibration accuracy is higher than in the conventional case. Conventionally, when an uncompressed image is used as a calibration image, the preview image cannot be displayed in real time (the frame is dropped), and thus the user finely adjusts the arrangement of the indicators 201 to 209 and the shooting method. It was difficult. On the other hand, according to the present embodiment, since the preview image can be displayed in real time, the user can finely adjust the arrangement of the indicators 201 to 209, the photographing method, and the like, and the calibration accuracy is higher than the conventional one.

(Second Embodiment)
Next, a second embodiment will be described. In the first embodiment, an example in which a compressed image is displayed as a preview image in real time, and an uncompressed image is acquired as a calibration image when the capture button is pressed, so that the index is calibrated with high accuracy will be described. did. However, in the first embodiment, a time lag occurs between an image (preview image) at the moment when the user presses the capture button 403 and a calibration image, and a calibration image that is not intended by the user is acquired. There is a possibility. As a means for solving this, in the second embodiment, an example in which a preview image and a calibration image are compared and a warning is given to the user when there is a high possibility that an unintended calibration image has been acquired. Will be explained. Note that a description of the same parts as those in the first embodiment is omitted.

[Processing flowchart]
FIG. 10 is a diagram illustrating a functional configuration when performing the processing of the present embodiment. FIG. 11 is a flowchart showing the flow of processing of this embodiment. Hereinafter, the flow of processing of the present embodiment will be described with reference to FIGS. 10 and 11. In addition, since the process of step S601-S608 is the same as that of 1st Embodiment, description is abbreviate | omitted.

  In step S1101, the index determination unit 1001 compares the index information detected in step S603 with the index information detected in step S608, and determines whether the image acquired in step S607 is suitable for calibration. Details of the processing of the index determination unit 1001 will be described later.

  In step S1102, the warning unit 1002 displays warning information on the display unit 108 based on the determination result of the index determination unit 1001 (warning information output). Details of the processing of the warning unit 1002 will be described later.

[Details of processing of index determination unit 1001]
FIG. 7 is a diagram for explaining processing of the index determination unit 1101 in the present embodiment. Below, with reference to FIG. 7, the detail of the process of the parameter | index determination part 1101 in this embodiment is demonstrated.

  FIG. 7A shows a preview image 701, and FIG. 7B shows a calibration image 702 acquired as a result of the user pressing the capture button 403 while viewing the preview image 701. In the preview image 701 and the calibration image 702, the position and orientation of the photographing unit 107 are shifted due to a time lag, and the number of detected indices is changed. When the number of indices changes in this way, it is highly likely that the calibration image 702 has not been taken as expected by the user, so it is determined that the calibration image 702 is not suitable for calibration. Note that the number of detected indices changes is not limited to the case where the position and orientation of the photographing unit 107 are deviated. For example, if blurring or blurring occurs in a captured image, it may be difficult to detect an index, and the number of detected indices may decrease.

  In the present embodiment, it is possible to determine whether the position and orientation of the photographing unit 107 is deviated, or whether blurring or blurring has occurred by comparing the number of detected indices. Here, the occurrence of blur means that the distance between the photographing unit 107 and the index or the focus distance of the photographing unit 107 has changed. That is, by comparing the number of detected indices, it is possible to detect that the distance between the imaging unit 107 and the index or the focus distance of the imaging unit 107 has changed. It should be noted that a threshold value for the increase / decrease number of the index may be set in advance, and it may be determined that the calibration image 702 is not suitable for calibration only when the increase / decrease number exceeds the threshold value.

  Further, even if the number of detected indexes is the same, if the image coordinates of the indexes are deviated, there is a possibility that the image is not taken as expected by the user. Therefore, when the deviation of the image coordinates of the index is greater than or equal to a preset threshold value, it may be determined that the calibration image 702 is not suitable for calibration. Here, an arbitrary value can be set as the threshold value. For example, 5% of the image width may be set as the threshold value. Furthermore, the identification information of the index may be compared. This makes it possible to know which index is how much and how much, so that a more reliable determination can be made for the calibration image 702.

  FIG. 7C shows a preview image 703 when a square index is used as the index. FIG. 7D shows a calibration image acquired as a result of the user pressing the capture button 403 while viewing the preview image 703. An image 704 is shown. In the preview image 703 and the calibration image 704, the distance between the photographing unit 107 and the index changes with time lag, and the size of the detected index changes. In the case of a square index, not only the image coordinates of the index but also the size on the image can be calculated. Therefore, when the difference in the size of the index on the image is equal to or larger than a preset threshold value, the calibration image 704 is displayed. It can be determined that it is not suitable for calibration. Here, an arbitrary value can be set as the threshold value. For example, 10% of the size of the index in the preview image 701 may be set as the threshold value. Note that the size of the detected index changes not only when the distance between the image capturing unit 107 and the index changes. For example, the size of the detected index also changes when the focal length of the photographing unit 107 changes. In the present invention, it is possible to determine whether the distance between the imaging unit 107 and the index or the focal length of the imaging unit 107 has changed by comparing the size of the detected index. Further, the distance between the photographing unit 107 and the index may be calculated from the size of the index and the internal parameters of the photographing unit 107, and the determination may be performed according to the distance.

  Alternatively, the reliability may be calculated when detecting the index, and if the difference in reliability is equal to or greater than a preset threshold value, it may be determined that the calibration images 702 and 704 are not suitable for calibration. Any method can be used to calculate the reliability. For example, when the index is a point index such as the indices 201 to 209, the difference between the color registered in advance as the index color and the color of the actually detected index area can be used as the reliability. Further, when the index is a square index such as the index 301, the degree of bending of the line detected as a straight line constituting the index can be used as the reliability.

[Details of processing of warning unit 1002]
When the index determination unit 1001 determines that the calibration image is not suitable for calibration, the warning unit 1002 displays (notifies) a warning on the display unit 108 and prompts photographing again. As a warning, for example, a text indicating that the calibration image is not suitable for calibration is displayed. Moreover, it is not restricted to this, You may warn a user by outputting information, such as an audio | voice. Also, the warning may be given by emphasizing a window displaying an inappropriate calibration image among the calibration image display windows 404 to 406. In addition, what is determined to be unsuitable for the calibration may be indicated by characters or icons. Further, it may be shown to the user how to make the correction image suitable for correction. For example, the movement or rotation of the photographing unit 107 may be represented by an arrow to guide the user to move the photographing unit 107.

[Effects of Second Embodiment]
In this embodiment, when a calibration image that is not intended by the user is captured, a warning is displayed to the user and prompts re-photographing. Therefore, the index is calibrated more robustly than in the first embodiment. be able to.

(Third embodiment)
Next, a third embodiment will be described. In the first embodiment and the second embodiment, the example in which the index is calibrated with high accuracy even when the communication band is insufficient has been described. In the present embodiment, an example in which the photographing unit is calibrated with high accuracy even when the communication band is insufficient will be described. Note that a description of the same parts as those in the second embodiment is omitted.

  FIG. 8 is an example of a calibration pattern for the photographing unit 107. Assume that the indices 801 to 820 have a known relative three-dimensional positional relationship between the indices. The present invention is not limited to the case where the calibration pattern of FIG. 8 is used. The relative three-dimensional positional relationship between the indices is known, the two-dimensional coordinates (image coordinates) of the projected image on the captured image can be detected, and the index can be identified. Any form may be used as long as it is an index.

  When calibrating the photographing unit 107, the photographing unit 107 photographs the indicators 801 to 820, and calibrates using information of the photographed indicators. Since the content of the calibration process is not the main point of the present invention, a description thereof will be omitted. For example, the calibration can be performed by the method described in Patent Document 2.

  The processing flow of this embodiment is the same as that of the second embodiment. Since the processing of the index determination unit 1001 is different from that of the second embodiment, this point will be described below.

[Details of Processing of Index Determination Unit 1101 of Third Embodiment]
FIG. 9 is a diagram for explaining processing of the index determination unit 1101 in the present embodiment. Below, with reference to FIG. 9, the detail of the process of the parameter | index determination part 1101 in this embodiment is demonstrated.

  Regions 901 to 924 represent regions on the image. In the calibration of the photographing unit 107, in order to perform distortion correction, it is necessary to measure what kind of distortion is generated in each area on the image. In particular, since large distortion occurs at the edge of the image (for example, areas 901, 906, 919, and 924), a calibration pattern is photographed so that the indicators 801 to 820 appear at the edge of the image, and what kind of distortion occurs at the edge of the image. It is important to measure whether this occurs. However, due to the time lag between the preview image and the calibration image, there is a possibility that the indicators 801 to 820 are disengaged from the image edge and the calibration image is captured. Therefore, an evaluation value that increases as the indices 801 to 820 appear at the edge of the image is calculated, and a warning is issued when the evaluation value of the calibration image is lower than that of the preview image. Note that the processing of the index determination unit 1101 in the second embodiment may be performed instead of the processing of the index determination unit 1101 in the present embodiment. Further, it is possible to combine the processing of the index determination unit 1101 in the second embodiment and the processing of the index determination unit 1101 in the present embodiment.

[Effect of the third embodiment]
According to the third embodiment, in the calibration of the photographing unit 107, the calibration accuracy is increased. In particular, distortion calibration accuracy is increased.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (12)

Image acquisition means for acquiring a first image acquired based on imaging by an imaging device and a second image having a larger data amount than the first image;
An index is detected from the first image when the first mode is set, and an index is detected from the second image when a second mode different from the first mode is set. Detecting means for detecting;
An information processing apparatus comprising: derivation means for deriving at least one of three-dimensional position information of the index and a parameter related to the imaging device based on a detection result by the index detection means.   And determining means for determining whether the second image is suitable for processing by the deriving means based on the index detected from the first image and the index detected from the second image. The information processing apparatus according to claim 1, further comprising:   The determination unit compares the number, position information, and size of the indices detected from the first image and the second image, respectively, so that the second image is processed by the deriving unit. The information processing apparatus according to claim 2, wherein the information processing apparatus determines whether it is suitable.   And a warning information output means for outputting warning information for notifying a user when the determination means does not determine that the second image is suitable for processing by the derivation means. The information processing apparatus according to claim 2 or 3.   The information processing apparatus according to claim 1, further comprising an output unit configured to output information prompting the user to take a picture again when the determination unit does not determine that the second image is suitable for processing by the deriving unit. 5. The information processing apparatus according to any one of 2 to 4.   The information processing apparatus according to claim 1, further comprising a setting unit that sets the first mode and the second mode.   The information processing apparatus according to claim 6, wherein the setting unit sets the first mode or the second mode based on an input from a user.   The information processing apparatus according to claim 1, wherein the first image is a compressed image, and the second image is an uncompressed image.   The information processing apparatus according to claim 1, wherein a resolution of the first image is lower than a resolution of the second image.   The information processing apparatus according to claim 1, wherein a compression rate of the first image is higher than a compression rate of the second image. An image acquisition step of acquiring a first image acquired based on imaging by an imaging device and a second image having a larger data amount than the first image;
An index is detected from the first image when the first mode is set, and an index is detected from the second image when a second mode different from the first mode is set. A detection process to detect;
An information processing method comprising: a deriving step of deriving at least one of the three-dimensional position information of the index and a parameter related to the imaging device using a detection result in the index detection step.   The program for functioning a computer as each means of the information processing apparatus of any one of Claims 1 thru | or 10.

Images