JP2007064808A - Method and device for measuring index position - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reset simply to a correct identification state, when an index having no ID is identified wrongly. <P>SOLUTION: A method has an index detection step for detecting images of one or more index candidates from imaged images imaged by a plurality of imaging devices, and determining the position in a space of the index candidate by performing correlation between the images; an index tracking processing step for tracking the index candidate detected in the index detection step; a timing input step for entering timing of execution of identification reset processing; and an identification reset step for performing re-identification of the index based on a height range in a space defined to each index when input is performed in the timing input step, and on the height in the space of the index candidate. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an index identification technique for detecting and identifying an index placed in a real space or a real object from an image captured by an imaging device.

  A marker placed on a real object moving in space is imaged by a camera, and the position and orientation of the real object in space are measured by detecting the position of the projected image of the marker on the image. Yes. Moreover, the position and attitude | position in the space of the camera currently image | photographing are performed by image | photographing the some marker arrange | positioned in real space. In such a position / orientation measurement method, it is necessary to identify (identify) which marker (detection marker) reflected in an image corresponds to a physical entity of the marker (physical marker) physically disposed. .

  For example, as a method of identifying a detection marker (a marker detected on an image) and a physical marker (an entity of a marker existing in the real space) when a single marker that does not have an identification function is captured by a camera The following methods are used. Markers are identified in some way in the initial frame. In the subsequent frames, each marker is tracked (that is, the markers are associated between the previous and subsequent frames).

  Here, tracking is identified as a physically identical marker by continuously monitoring the marker position on the captured image, and the three-dimensional position of the marker in space or the marker position on the image. Refers to the process of measuring coordinates. For example, there is a problem in that two markers 120A and 120B are imaged by two cameras 110A and 110B, and the three-dimensional positions of the two markers are measured using the captured images (referred to as (a1) and (b1), respectively). Think. Here, it is assumed that the respective markers 120A and 120B are correctly detected and identified in the immediately preceding frame. In FIG. 2, the left side is a captured image of the camera 110 </ b> A, the right side is a captured image of the camera 110 </ b> B, and (a1) and (b1) respectively indicate images that are simultaneously captured. 220A and 220B in the figure indicate projection points at the positions of the markers 120A and 120B in the immediately preceding frame. In the tracking process, a spherical search area centered on the three-dimensional position of the marker in the immediately preceding frame is set, and when a marker exists in the search area, the same marker is tracked. In the example of (a1) and (b1) in FIG. 2, since both the markers 120A and 120B fall within the search area (220A and 220B), the tracking process succeeds and is correctly identified. In this tracking process, there are a case where a marker position on an image taken by one camera is measured and a case where a marker moving in a three-dimensional space is measured by a plurality of cameras. These are collectively called tracking.

  In addition, as another tracking method, as in Non-Patent Document 1, an example of tracking a plurality of markers fixed in space on a captured image using information of a gyro sensor attached to a movable camera is given. It is done.

In the example given above, the markers themselves do not include information uniquely identified as physical information (color, shape, etc.). Such a marker has a relatively good detection rate even if the number of pixels when reflected in an image is small compared to a marker having identification information such as a two-dimensional barcode, color, shape, etc. There is a merit that a marker in an area can be detected.
Hirofumi Fujii, Masayuki Kanbara, Hidehiko Iwasa, Haruo Takemura, Naokazu Yokoya: “Marker Tracking Method for Vision-Based AR Using Gyro Sensors”, IEICE Technical Report, MVE99-59, Dec. 1999.

  However, the index tracking method mentioned in the related art may be erroneously identified. If the marker itself does not have information that can be uniquely identified (for example, a two-dimensional barcode or color), that is, it looks the same when captured in an image, a plurality of markers are placed in space. There is a possibility that a marker that should be tracked may be lost or mistaken.

  This misidentification will be described with reference to FIG. 3, as in FIG. 2, the left column is an image in which the captured image of the camera 110A and the identification information are superimposed, and the right column is an image in which the captured image of the camera 110B and the identification information are superimposed, and (a2) (B2), (a3), and (b3) show images taken simultaneously. The captured images are taken in the order of (a1) and (b1) in FIG. 2, (a2) and (b2), (a3) and (b3) in FIG. 3, and (a3) and (b3) are This shows a state where erroneous identification occurs. For example, in the tracking method as described in Non-Patent Document 1, two markers are photographed with one camera, a certain area is set as a search area from the position of the marker identified in the previous frame, and the markers in the area are the same marker. Consider the case to track as. As shown in (a2) of FIG. 3, when two markers move in directions close to each other on the image and enter each other's search region, erroneous identification of the marker may occur. As shown in (a3) of FIG. 2, if the marker 120A moves out of the search area in the next frame after the two markers 120A and 120B come close to each other, the marker 120B is erroneously identified as the marker 120A. End up. When erroneous identification occurs in this way, the wrong marker is tracked and the wrong position is output as a result.

  Also, when shooting markers with multiple fixed cameras, even if there are multiple markers within the marker search range in the image of one camera, the distance of each marker on the image of the other camera Is larger than the search range, each marker can be correctly identified by using epipolar constraint between fixed cameras. However, when a plurality of markers appear in the search range in all camera images, erroneous identification occurs as in the case of one camera. For example, a case where two markers are tracked by two fixed cameras will be described with reference to FIG. As shown in the states of (a2) and (b2) in FIG. 3, a plurality of markers enter the marker search area simultaneously with the two cameras 110A and 110B, and in the next frame as shown in (a3) and (b3). If the other marker is out of the search area, misidentification occurs.

  As described above, in the conventional method, there are cases in which erroneous identification occurs during marker tracking, and the state where the marker position cannot be obtained correctly continues.

  It is an object of the present invention to easily return to a correct indicator identification state from a state in which an erroneous measurement value is output when an indicator is erroneously identified.

  In order to achieve the above object, the present invention has the following configuration.

  The invention according to claim 1 is an index position measuring method for measuring the positions of a plurality of indices arranged in a space, and an index detection step of detecting an index candidate image from a captured image captured by an imaging unit. And an index tracking process for tracking the index candidate detected in the index detection process, a timing input process for inputting the timing of execution of the identification return process, and an input performed in the timing input process, It has an identification return step of re-identifying the index based on the range of height defined in each of the indices and the height of the index candidate in the space.

  The invention according to claim 3 of the present application is an index position measurement method for measuring the positions of a plurality of indices arranged in a space, and an index detection step of detecting an index candidate image from a captured image captured by an imaging unit. And an index tracking process for tracking the index candidate detected in the index detection process, a timing input process for inputting the timing of execution of the identification return process, and an input performed in the timing input process, An indicator blinking control step for controlling lighting and extinguishing of the indicator, and an identification return step for re-identifying the indicator based on a result of comparing the detection status of the detected indicator candidate and the blinking control information in the indicator blinking control step It is characterized by having.

  The invention according to claim 4 is an index position measuring method for measuring the positions of a plurality of indices arranged in a space, and is an index detection for detecting an index candidate image from a captured image captured by an imaging unit. A process, an index tracking process that tracks the index candidate detected in the index detection process, a timing input process that inputs the timing of execution of the identification return process, and an input performed in the timing input process And an identification return step for re-identifying the index based on the time series information of the relative distance between the index candidate of interest and the other index candidates.

  The invention according to claim 6 of the present application is an index position measurement method for measuring the positions of a plurality of indices arranged in a space, and detects an index candidate image from captured images captured by a plurality of imaging devices, An index detection step for obtaining a position of an index candidate in a space by performing correlation between images, an index tracking processing step for tracking the index candidate detected in the index detection step, and timing for executing an identification return process A timing input step for inputting the index, an index position measurement step for measuring the position of the index using means other than the imaging device, and a measurement in the index position measurement step when input is performed in the timing input step An identification return step for re-identifying the index based on the comparison result between the position of the index and the position of each index candidate detected in the index detection step in the space Characterized in that it has a.

  According to the present invention, a misplacement that occurs when an index placed in a real space or a real object is detected from an image captured by an imaging device and a three-dimensional position is identified can be easily performed without performing initialization processing again. It is possible to return to the correct identification state by the method.

  Hereinafter, the present invention will be described in detail according to preferred embodiments with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating a configuration example of an index position measurement apparatus according to the present embodiment.

  In the present embodiment, the positions of the markers 120A and 120B (hereinafter referred to as the markers 120 when the markers are not distinguished) arranged at the head and wrist positions of the user 196A by the plurality of fixed cameras 110A and 110B. Details of the index position measuring device to be measured will be described.

  The fixed cameras 110 </ b> A and 110 </ b> B are cameras for photographing the marker 120. Cameras 110A and 110B reflect infrared light emitted from an infrared light source (not shown) to marker 120, and images the reflected light of the marker. By imaging infrared light, the false detection rate of detecting something other than the marker decreases. In addition, this invention is not limited to imaging infrared light in this way, It is applicable also when imaging visible light using a normal camera.

  The marker 120 is a spherical marker, and the spherical surface is covered with a retroreflecting material. The marker 120A is fixed to the HMD 195A attached to the head of the user 196A, and the marker 120B is fixed to the wrist of the user 196A. In the following, a physical marker that is physically arranged is referred to as a physical marker, and a detected marker is referred to as a detection marker, and is distinguished as necessary. In addition, each physical marker is assigned a unique ID.

  The image input units 130A and 130B send the captured images input from the cameras 110A and 110B to the storage unit 140.

The storage unit 140
● Registered ID of each physical marker, initial 3D position, return altitude, which are known values
Identification information of each detection marker obtained by the index identification unit 160 (detection marker number (referred to as a detection number), image coordinates and measured three-dimensional position in each camera image, and past detection markers A pointer of a detection number indicating a correspondence relationship, and an ID of an identified physical marker (an example is shown in FIG. 7);
● Past identification information (time-series information in which identification information as shown in FIG. 7 is recorded together with the imaging time),
● Image processing parameters (threshold for marker detection and radius of search area),
● Each captured image,
An image (referred to as a detection image) used in the marker detection process (image-processed) output from the index detection unit 150,
Record information such as. Note that the registration items listed here are merely examples, and other information may be registered or fewer items may be required depending on the index to be used and the identification method of the index.

  The index detection unit 150 detects an area that seems to be a marker for the two captured images input from the image input units 130A and 130B, and determines the position of the center of gravity of the detected marker. Use image coordinates. The index detection unit 150 further associates each detection marker between captured images obtained from different cameras, and calculates a three-dimensional position of the detection marker. The detected image coordinates (differentiated for each captured image) and the three-dimensional position of each marker are stored in the storage unit 140 as identification information of the detected marker.

  The index identification unit 160 inputs identification information (image coordinates and three-dimensional position obtained by the index detection unit 150) of each detection marker stored in the storage unit 140. Then, association with the detection marker in the immediately preceding frame (that is, tracking of the detection marker from the immediately preceding frame) is performed. Further, when the detection marker of the immediately preceding associated frame is identified as any physical marker, the detection marker of the current frame of interest is identified as the physical marker. The associated result is stored in the storage unit 140 as identification information of the detection marker. Note that the detection marker does not always correspond to the physical marker, and there may be cases where camera noise or an object other than the marker is erroneously detected. That is, the detection marker indicates a marker (marker candidate) that can be a candidate for identification of a physical marker.

  The input unit 180 is an input unit for transmitting an instruction for executing the erroneous identification return process to the erroneous identification return unit 170. In the present embodiment, it is assumed that the operator who monitors the display unit 185 recognizes the erroneous identification and gives an instruction to the erroneous identification return unit 170 via the input unit 180. Note that the present invention is not limited to an operator recognizing misidentification and giving an instruction to the index position measurement device, but the user 196A performs input for instructing execution of recovery processing of misidentification. Also good. As an input device, an input device with buttons is assumed. The present invention is not limited to an input device with a button as an input device, but can be applied as long as information can be input to an index position measuring device such as a keyboard, a mouse, and a voice input device.

  The erroneous identification return unit 170 acquires the identification information of the detected marker from the storage unit 140 according to the input result of the input unit 180, and executes an erroneous identification return processing module that returns the marker identification information to the correct identification state. The identification information of the detection marker stored in the storage unit 140 is updated.

  The display unit 185 outputs or displays the current detection marker identification information stored in the storage unit 140 to an operator, a user 196A, or the like. In the present embodiment, an image as shown in (a1) of FIG. 2 using information such as the captured image, marker identification information, past identification information, search area radius, and the like stored in the storage unit 140. Is generated and displayed on a monitor or the like. The operator monitors the video displayed on the monitor. In FIG. 2A1, 120A and 120B indicate the positions of the current detection markers on the image. 230A and 230B indicate physical marker IDs obtained from the identification information of the detection markers, and 220A and 220B indicate image coordinates of the detection markers in the immediately preceding frame obtained from past identification information. When the operator recognizes the erroneous identification and operates the input unit 180, the erroneous identification is determined based on the presented image.

  The output unit 188 is a means for outputting information stored in the storage unit 140 to the outside. In the present embodiment, the physical marker ID and the three-dimensional position are output from the detection marker identification information stored in the storage unit 140. Further, the image coordinates in each of the cameras 110A and 110B may be output simultaneously. In addition, as the identification information, only the identification information in which the ID of the physical marker is set in the ID field (identified by the physical marker) is output.

  Next, processing performed by the index position measurement apparatus having the above configuration will be described with reference to the flowchart shown in FIG.

  Before performing the processing of the flowchart shown in FIG. 4, a marker is placed at a predetermined initial position, initialization processing is performed, and each marker is correctly identified (all physical markers are detected). It is assumed that it is detected as a marker and is correctly assigned an ID).

  In step S410, the image input units 130A and 130B input captured images from the cameras 110A and 110B and output the captured images to the storage unit 140.

  In step S420, the index detection unit 150 inputs captured images of the cameras 110A and 110B stored in the storage unit, and detects a marker reflected in the image. As a method for detecting the marker, any method may be used. For example, a method of binarizing a captured image and labeling a region having high brightness in the obtained binarized image is used. Can do. The barycentric point of the detection marker region obtained by labeling is used as the image coordinate of the detection marker. Further, the binarized image (detected image) at this time is stored in the storage unit 140.

  In step S430, the index detection unit 150 associates images obtained from different cameras with all the detection markers obtained in step S420 in order to estimate the three-dimensional position of the detection marker. And a three-dimensional position is calculated | required from the image coordinate of the corresponding detection marker. Any method may be used for the association between the images and the calculation of the three-dimensional position. For example, the following method is used. First, an epipolar line on another detected image (a detected image of the camera 110B when the detected marker in the detected image of the camera 110A is currently processed) is estimated from the image coordinates of one detected marker of interest. Then, it is confirmed whether or not the detection marker exists within a certain distance from the epipolar line. The epipolar line estimation is a known technique and will not be described. Next, when a detection marker exists near the epipolar line, the three-dimensional position of the detection marker in the space is obtained from the position and orientation of the cameras 110A and 110B in the space and the image coordinates of each detection marker. The calculation for obtaining the three-dimensional position of the detection marker may use any method, but for example, the three-dimensional position may be obtained by using a known triangulation principle. When the three-dimensional position of the detection marker is obtained, each image coordinate of the detection marker and the three-dimensional position of the marker candidate are stored in the marker identification information in the storage unit 140 in ascending order of the detection number.

  In step S440, the index identification unit 160 tracks the marker from the “identification information” including the three-dimensional position of the detection marker obtained in step S430 and “past identification information”. For example, the following method is used for tracking the marker. First, among the past identification information, the three-dimensional positions of the respective detection markers in the immediately preceding frame (220A and 220B in FIG. 2 indicate points obtained by projecting the three-dimensional positions on the imaging surface) are obtained. Next, a spherical search area (210A and 210A in FIG. 2) having a preset radius (the radius of the search area held by the storage unit 140) with the three-dimensional position of each acquired detection marker as a central point. 210B indicates a region obtained by projecting the sphere of the search region onto the imaging surface). Next, when the three-dimensional position of the detection marker calculated in the current frame is included in the set plurality of search areas, the three-dimensional position of the detection marker in the frame immediately before generating the search area is newly set in the current frame. It is determined that the detection marker has been moved to the three-dimensional position. Then, the pointer (detection number) to the detection marker of the previous frame in which the search area is generated is recorded in the tracking history field of the detection marker identification information newly calculated in the current frame. At this time, if the ID number of the physical marker is set in the identification information of the detection marker of the previous frame that generated the search area, the same ID number is set in the ID field of the identification information of the detection marker newly calculated in the current frame. Record.

  In step S460, it is determined whether there is an input from the input unit 180. If there is an input, the process proceeds to step S470. If there is no input, the process proceeds to step S480. The operator inputs an instruction to the input unit 180 when determining that the detection marker is erroneously identified and that the erroneous identification recovery process is ready. Here, the preparation means that each marker 120 attached to the user 196A has a predetermined height as shown in FIG. It is assumed that this predetermined height is preset for each physical marker in the “marker return altitude” storage area stored in the storage unit 140. Note that it is preferable that the return altitude is specified in a high altitude area where each marker 120 is likely to exist in the movement of the user 196A. For example, the operating range of the head marker 120A and the hand marker 120B when the user 196A stands upright and lowers his / her hand may be measured in advance and used. Means for moving the respective markers 120 to a predetermined height to the user 196A may be transmitted by voice, or may be transmitted on the HMD 195A worn by the user by means of instructions or drawings.

  In step S470, the misidentification return unit 170 executes processing defined by the misidentification return processing module. The processing of the erroneous identification return processing module will be described later.

  In step S480, the identification information stored in the storage unit 140 is displayed according to the request. In the present embodiment, an image indicating the marker identification status as shown in FIG. 2 is generated by an image generation unit (not shown) using information in the storage unit 140 and displayed on the display unit 185.

  In step S490, among the identification information stored in the storage unit 140, the three-dimensional position of the detection marker identified as the physical marker and the image coordinates of the cameras 110A and 110B are output.

  FIG. 5 is a flowchart showing details of the process of the misidentification return processing module executed by the misidentification return unit 170 in step S470. What is finally desired in the present embodiment is to return from a state erroneously identified by an operator or the like to a normal identification state when an erroneous identification of the marker occurs. For example, as shown in FIG. 2 (a3), when two IDs are associated with one marker, or when the ID of each marker is changed, the purpose is to reassign the correct ID to the marker. Yes. The process of performing the return process is a process in the erroneous identification return process module described below.

  The erroneous identification return processing module in the present embodiment attempts to return using the altitude information among the three-dimensional positions of the obtained markers. Below, the detailed procedure of the return processing using altitude information is described.

  In step S <b> 510, the misidentification return unit 170 obtains the height of each detection marker from the identification information of the detection marker stored in the storage unit 140. The altitude is obtained from the distance when a straight line is extended in the vertical direction from the ground where the user stands to the three-dimensional position of the marker 120.

  In step S520, the misidentification return unit 170 determines whether the misidentification return process in step S530 has been performed on all the registered ID markers. If not completed, a marker with an ID that has not been processed is selected from the physical markers stored in the storage unit 140, and the process proceeds to step S530. When the erroneous identification return processing is completed for all the registered ID markers, the processing ends.

  In step S530, the misidentification return unit 170 determines the altitude information for the predetermined return process stored in the storage unit 140 and each detection obtained in step S510 for a physical marker of a certain ID. Compare the altitudes of the markers to determine if they match. In the altitude information stored in the storage unit 140, altitude area information as shown in FIG. 6 is recorded. For example, the altitude information of the marker 120A indicates a range from the height of L3 to the height of L4, and if the altitude of the detection marker obtained in step S510 is within this range, a marker that matches the detection marker Judge that. If there is one matching detection marker, the process proceeds to step S540. If there are two or more detection markers in one altitude information, or if there is no matching detection marker, the process proceeds to step S535.

  In step S535, the misidentification return unit 170 presents on the image output from the user's HMD 195A or the output unit 185 that the recovery process from the misidentification cannot be performed, and ends the process.

  In step S540, the misidentification return unit 170 determines that the detection marker matched in step S530 is the correct identification result of the ID (physical marker) currently processed. Then, the ID field of the matching detection marker in the identification information stored in the storage unit 140 is replaced with the ID currently being processed.

  The steps from Step S510 to Step S540 will be described with reference to the state of FIG. Thereafter, at the stage where the images (a3) and (b3) are output, the operator recognizes that the ID of the marker 120A is erroneously associated with the marker 120B, and the user 196A When the operator confirms that the marker has been placed on (in this example, the user 196A is upright and has put his hand down) and performs an input instructing execution of the identification return processing. The return process of the misidentification return unit 170 will be described.

  In step S510, the three-dimensional position of the detection marker 120A registered as the marker identification information in the storage unit 140 (unregistered with the physical marker) and the three-dimensional position of the detection marker 120B (two physical markers) The altitudes (double registered in ID1 and ID2) are calculated.

  Next, step S520 is executed, and the processing after step S530 is executed for ID1.

  In step S530, altitude information is obtained by referring to the return altitude of the marker ID1 stored in the storage unit 140. In this embodiment, it is assumed that the altitude information of the marker ID1 is set in the range of L3 to L4 of ID1, and the altitude of the marker ID2 is set in the range of L1 to L2. In this case, the altitude of the detection marker 120A corresponds to the altitude information of the marker ID1, and no other corresponding detection marker exists. Therefore, in step S540, the marker 120A is associated as the marker of ID1 (ID1 is recorded in the ID field of the identification information of the detection marker 120A stored in the storage unit 140).

  Further, step S520 is executed again, and the processing after step S530 is executed for ID2. Since the altitude of the marker 120B corresponds to the heights L1 to L2, it is correctly identified as the marker of ID2 by the processing in steps S530 and S540.

  When there are only two markers as in the present embodiment, it is not necessary to compare the altitudes of the markers 120B, it is simply determined that the remaining one marker candidate is a marker of ID2, and the marker 120A is ID1. When it is determined that the marker 120B is associated with ID2, the marker 120B may be associated with the ID2. Further, the present invention is not limited to returning using altitude information, and an area centered on a specific three-dimensional position may be used as information for erroneous identification return. However, in this case, it is necessary to move the user to a specific position. On the other hand, the erroneous identification return method using altitude information that changes the altitude of the marker on the spot does not need to move the predetermined user 196A to a predetermined position, and can be said to be a more convenient method.

  Moreover, in step S520 of this embodiment, it sets so that the process of step S530 may be performed with respect to registration ID of all the physical markers. However, for example, the return process may be performed only on the ID specified by the operator.

  As described above, according to the present embodiment, even when erroneous marker identification occurs, it is possible to easily return to the correct marker identification state.

  It should be noted that it is automatically recognized that misidentification has occurred without any manual intervention, and further automatically recognized that it can be recovered from misidentification, and the misidentification returning unit 170 via the input unit 180. You may give instructions. For example, automatic detection of erroneous identification is performed by detecting whether or not there is a situation in which a single detection marker has a duplicate ID or a situation in which there is a detection marker without an ID. Can do. In such a case, the necessity for return processing is presented to the user, and further processing for recognizing that the state can be recovered from erroneous identification is executed. This process can be performed, for example, by detecting whether each marker 120 is in a stationary state for a certain time or more.

[Second Embodiment]
In the first embodiment, the altitude of the marker is used as information for returning from the misidentified state. However, if the marker itself can be turned on with a self-luminous marker, the blinking information of the marker is detected from the misidentified state. It can also be used as information for returning.

  FIG. 8 is a block diagram showing the configuration of the index position measuring apparatus in the present embodiment. In this embodiment, in addition to the configuration of the first embodiment shown in FIG. 1, a lighting control unit 190 that controls blinking of the marker is added. Further, the markers 120 </ b> A and 120 </ b> B are changed to self-emitting markers that emit infrared light according to the lighting control unit 190. The lighting control unit 190 receives, for example, ON / OFF signals of lighting states for all physical markers from the misidentification return unit 170, and controls the light emission of the physical markers according to the lighting states. Further, the camera is equipped with an infrared filter, and is set so as to transmit only infrared light. Further, the erroneous identification return processing module executed by the erroneous identification return unit 170 is also different. Other than the above, the present embodiment can be used without changing the configuration of the index position measuring apparatus of FIG.

  The process performed by the index position measurement apparatus in the present embodiment is the same as the process performed by the index position measurement apparatus in the first embodiment (the flowchart in FIG. 4), and the misidentification return unit 170 performs the misidentification performed in step S470. Only the return processing module is different.

  FIG. 9 is a flowchart showing details of the process of the misidentification return processing module executed in step S470 by the misidentification return unit 170 in the present embodiment.

  In step S520, as in the process of the first embodiment (FIG. 5), it is determined whether the return process has been completed for each registration ID (that is, each physical marker). If not completed, a marker with an ID that has not been processed is selected from the physical markers stored in the storage unit 140, and the process proceeds to step S910. If all are completed, the process proceeds to step S925.

  In step S910, first, the misidentification return unit 170 uses the lighting control unit 190 to blink the light emission state of the physical marker that is currently attempting the return process at regular intervals. The blinking process of the physical marker can be realized, for example, by the misidentification return unit 170 having a counter and sending an ON / OFF signal to the lighting control unit 190 every time a certain time comes. At this time, it is assumed that the misidentification returning unit 170 blinks only one physical marker at a time, and other markers do not blink simultaneously.

  Next, in step S915, the erroneous identification return unit 170 is virtually set so that the input of the input unit 180 is maintained (always ON), and in the next frame identification process, The erroneous identification return process is automatically executed.

  Next, in step S920, at the moment when the misidentification return unit 170 causes the lighting control unit 190 to change the physical marker from the on state to the off state, whether or not the detection marker disappears from the search area of the specific detection marker in the index identification unit 160. Determine if. If there is a detection marker that has disappeared, the process proceeds to step S525, and an internal state called a trigger is turned ON. If the disappearance of the marker cannot be confirmed, the process proceeds to step S530.

  In step S530, the misidentification return unit 170 determines whether the trigger is ON. If it is ON, the process proceeds to step S935. If it is OFF, the process ends.

  In step S935, the misidentification return unit 170 determines whether a new detection marker is detected in the index identification unit 160 at the moment when the lighting control unit 190 changes the marker from the unlit state to the lit state. If a new marker is detected, the process proceeds to step S540. If the marker detection cannot be recognized, the process ends.

  In step S940, the misidentification return unit 170 replaces the ID currently processed in the ID field of the identification information in the storage unit 140 of the detected marker detected in step S935.

  In step S945, the misidentification return unit 170 turns off the trigger, returns to step S520, and starts identifying the next ID.

  In step S950, since the misidentification return processing is completed, the misidentification return unit 170 returns the state of the input unit 180 to the normal state (always OFF).

  In step S955, if there is a marker that is blinking, the misidentification return unit 170 returns to the lighting state and ends the process.

  In step S520 of the present embodiment, the processing of step S530 is set to be executed for all physical marker registration IDs. For example, only the ID specified by the operator is restored. Processing may be performed. Further, the present invention is not limited to repeating blinking at a constant interval, and can be applied as long as the lighting / extinguishing timing is blinking that can recognize the correspondence between a physical marker and a detected marker. Furthermore, the present invention is not limited to performing re-identification based on the detection state of the three-dimensional position of the detection marker, but compares the detection state of the detection marker with the lighting / extinguishing timing on the detection image. You may identify by doing.

  As described above, according to the present embodiment, it is possible to easily return from erroneous marker identification by blinking the marker. Further, with the erroneous identification return method of the present embodiment, the users 196A and 196B themselves do not need to move the marker in response to the instruction, so that the return process can be performed without the trouble of the users 196A and 196B.

[Third Embodiment]
In the first embodiment, the altitude of the marker is used as information when returning from the erroneous identification state. Other methods are also conceivable. For example, as shown in (a3 ′) and (b3 ′) of FIG. 11, when the IDs of two markers (120A and 120B) among a plurality of markers (120A, 120B, 120C, and 120D) are switched, they are switched. It is possible to return to a normal state by specifying the ID and performing replacement again. As a method for specifying the ID of the replaced marker, an operator or the like inputs one ID number from the keyboard serving as the input unit 180, and further estimates another marker that is likely to be replaced by the erroneous identification return unit 170. There is a way to replace it. The detection marker that is likely to be replaced is obtained from the relative distance between the past three-dimensional position of the detection marker associated with the input ID and the three-dimensional position of each detection marker recorded in the past identification information. .

  In the following description, it is assumed that the state in which ID1 is assigned to the marker 120A, ID2 is assigned to the marker 120B, ID3 is assigned to the marker 120C, and ID4 is assigned to the marker 120D is the correct state. FIG. 11 shows a state where the ID of 120A is erroneously identified as 2 and the ID of 120B is identified as 1.

  FIG. 10 is a block diagram showing the configuration of the index position measuring apparatus in the present embodiment. Compared to the configuration of the index position measurement apparatus in the first embodiment (FIG. 1), a user 196B and markers 120C and 120D are added, and the input unit 180 is changed to a device for inputting numerical values such as a keyboard. Different. Further, the erroneous identification return processing module executed by the erroneous identification return unit 170 is also different. About another part, it can utilize for this embodiment, without changing the structure of the parameter | index position measuring apparatus of FIG.

  The process performed by the index position measurement apparatus in the present embodiment is the same as the process performed by the index position measurement apparatus in the first embodiment (the flowchart in FIG. 4), and the misidentification return unit 170 performs the misidentification performed in step S470. Only the processing of the return processing module is different. First, an operator or the like inputs one of the ID numbers of the detection markers replaced among the detection markers 120 to the erroneous identification return unit 170 using the input unit 180. Next, the misidentification return unit 170 includes, in the past identification information stored in the storage unit 140, the three-dimensional position path of the input ID detection marker and other detections associated with the ID. The distance between the three-dimensional positions of the respective markers is calculated by comparing the three-dimensional position paths recorded in the marker identification information with the time going back. Finally, it is determined that the detected marker of the first ID that falls within the preset distance threshold among the calculated distances is the replaced marker, and the two IDs of the target among the identification information in the storage unit 140 are determined. The three-dimensional position and the image coordinates are exchanged.

  With the erroneous identification return method of the present embodiment, it is possible to return only by using the past three-dimensional position information. Therefore, the users 196A and 196B themselves do not need to move the marker in response to the instruction, and the users 196A and 196B can The return process can be done without the hassle.

  Note that the present invention is not limited to performing re-identification based on the history of the three-dimensional position of the detection marker, but based on the history of the two-dimensional position of the detection marker on the detection image on the detection image. You may identify.

[Fourth Embodiment]
In the first embodiment, the altitude of the marker is used as information when returning from the misidentified state. However, a device (position sensor) capable of measuring the position and orientation such as a magnetic sensor is brought close to the physical marker whose identification is to be recovered. By pressing the button, the detection marker close to the three-dimensional position of the position sensor is forcibly associated, so that the correct marker identification state can be easily restored. FIG. 12 is a block diagram showing the configuration of the index position / orientation apparatus according to this embodiment. The input device 1210 with a button fixed to the receiving unit 1230 of the magnetic sensor, the magnetic sensor transmitting unit 1225 fixed in the space, and the position / orientation measuring unit 1220 are added to the block diagram of FIG. 1 in the first embodiment. Has been. Further, the erroneous identification return processing module executed by the erroneous identification return unit 170 is also different. Other than the above, it can be used without changing the configuration of the first embodiment. It should be noted that the present invention is not limited to using a magnetic sensor as the position / orientation measurement apparatus, but can be applied to any apparatus that can measure the position of the input apparatus 1210, as will be apparent from the following description. Moreover, the input device 1210 with a button does not need to be fixed to the receiving unit 1230 of the magnetic sensor, and may be at different positions.

  The button-equipped input device 1210 sends a signal to the input unit 180 by pressing the button. The magnetic sensor receiver 1230 measures the change in the magnetic field generated from the magnetic sensor transmitter 1225 and sends a signal to the position / orientation measurement unit 1220. The position / orientation measurement unit 1220 measures a three-dimensional position / orientation in space according to a signal from the receiver. The measured position and orientation of the input device 1210 are input to the erroneous identification return unit 170 and used as information of the erroneous identification return process. The position and orientation of the input device 1210 may be transmitted to the misidentification return unit 170 at the moment when the button of the input device 1210 is pressed, or may be transmitted to the misidentification return unit 170 at all times. The misidentification return unit 170 may hold the position and orientation at the moment when is pressed.

  Regarding the processing of the index position measuring apparatus in the present embodiment, each detection stored as identification information in the storage unit 140 in step S530 of FIG. 5 in the first embodiment, instead of comparing with the specified height. The distance between the three-dimensional position of the marker and the measured value of the three-dimensional position of the input device 1210 close to the actual marker 120 corresponding to the ID currently being processed is compared, and the closest detection marker is processed. What is necessary is just to match | combine as a marker of ID which has carried out.

  The present invention is not limited to bringing the input device 1210 close to a physical marker. For example, a virtual straight line is created in a specific direction of the input device such as a laser pointer, and the actual marker 120 is created. A rough 3D position of the marker candidate close to the actual marker is selected, such as selecting a marker candidate whose 3D line is closest to the 3D position of the marker candidate when the button is pressed with a virtual straight line facing Any method that can be selected is applicable.

[Modification 1]
In the first embodiment, the second embodiment, the third embodiment, and the fourth embodiment, the marker is correctly returned to the correct identification state. The present invention is not limited to recovering from the misidentification state by individual processing as described above, and performs processing by selecting an appropriate method from the above four methods according to the state where misidentification has occurred. May be.

[Modification 2]
Two or more of the erroneous identification recovery methods of the first embodiment, the second embodiment, the third embodiment, and the fourth embodiment may be combined. For example, a method of using the height information in the first embodiment and the blinking information in the second embodiment twice to restore more reliably may be used.

[Other variations]
Further, it is also included in the scope of the present invention to supply and supply software program codes for realizing the functions of the above-described embodiments via a network.

  In this case, the program code of the software itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer constitute the present invention.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) in which the program code is running on the computer, or other application software, etc. It goes without saying that the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the embodiment.

It is a block diagram showing an example of composition of a marker position measuring device concerning a 1st embodiment. It is a schematic diagram which shows the detection condition of a marker. It is a schematic diagram which shows the detection condition of a marker. It is a flowchart which shows the detail of a process of the parameter | index position measuring apparatus which concerns on 1st Embodiment. It is a flowchart which shows the detail of a process of the misidentification return part 170 which concerns on 1st Embodiment. 3 is a schematic diagram showing altitude information of a marker 120. FIG. It is a table | surface which shows the identification information of a marker. It is a block diagram which shows the structural example of the index position measuring apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the detail of a process of the misidentification return part 170 which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the index position measuring apparatus which concerns on 3rd Embodiment. It is a schematic diagram which shows the misidentification state of the marker in 3rd Embodiment. It is a block diagram which shows the structural example of the parameter | index position measuring device which concerns on 4th Embodiment.

Claims (13)

An index position measuring method for measuring positions of a plurality of indices arranged in a space,
An index detection step for detecting an index candidate image from a captured image captured by the imaging unit;
An index tracking process for tracking the index candidates detected in the index detection process;
A timing input step for inputting the execution timing of the identification return process;
When input is performed in the timing input step, the index is re-identified based on the height range in the space defined for each of the indices and the height of the index candidate in the space. An index position measurement method comprising: an identification return step.   The index position measuring method according to claim 1, wherein a height range in the space defined for each of the indices is a high altitude range where the index is likely to exist. An index position measuring method for measuring positions of a plurality of indices arranged in a space,
An index detection step for detecting an index candidate image from a captured image captured by the imaging unit;
An index tracking process for tracking the index candidates detected in the index detection process;
A timing input step for inputting the execution timing of the identification return process;
When an input is made in the timing input step, an indicator blinking control step for controlling lighting and extinction of the indicator,
An index position measurement method comprising: an identification return step for re-identifying an index based on a result of comparing the detection status of the detected index candidate and the flashing control information in the index flashing control step. An index position measuring method for measuring positions of a plurality of indices arranged in a space,
An index detection step of detecting an image of an index candidate from a captured image captured by the imaging unit;
An index tracking process for tracking the index candidates detected in the index detection process;
A timing input step for inputting the execution timing of the identification return process;
An identification return step for re-identifying the index based on time-series information of the relative distance between the index candidate of interest and another index candidate when input is performed in the timing input process, Index position measurement method to do.   5. The identification return step according to claim 4, wherein the identification return step selects an index candidate whose relative distance is equal to or less than a threshold value at the closest time with respect to the index candidate of interest, and replaces the identification information of each index. Index position measurement method. An index position measuring method for measuring positions of a plurality of indices arranged in a space,
An index detection step of detecting an image of an index candidate from captured images captured by a plurality of imaging devices and performing association between the images to obtain a position of the index candidate in a space;
An index tracking process for tracking the index candidates detected in the index detection process;
A timing input step for inputting the execution timing of the identification return process;
An index position measuring step of measuring the position of the index using means other than the imaging device;
Comparison result between the position of the index measured in the index position measurement step and the position in the space of each index candidate detected in the index detection step when input is performed in the timing input step And an identification return step for re-identifying the index based on the index position measuring method.   The index position measuring method according to claim 6, wherein the index position measuring step measures the position of the index by a position and / or orientation measuring device for performing the measurement.   A computer program for causing a computer to execute the index position measuring method according to any one of claims 1 to 7.   A computer-readable recording medium storing the computer program according to claim 8. An index position measuring device that measures the positions of a plurality of indices arranged in a space,
Index detection means for detecting an index candidate image from a captured image captured by the imaging unit;
Index tracking processing means for tracking the index candidates detected by the index detection means;
Input means for inputting the execution timing of the identification return process;
Identification that performs re-identification of an index based on the height range in the space defined for each of the indices and the height of the index candidate in the space when input is performed by the input unit An index position measuring device comprising a return means. An index position measuring device that measures the positions of a plurality of indices arranged in a space,
Index detection means for detecting an index candidate image from a captured image captured by the imaging unit;
Index tracking processing means for tracking the index candidates detected by the index detection means;
Input means for inputting the execution timing of the identification return process;
When an input is performed in the input step, indicator blink control means for controlling lighting and extinction of the indicator,
An index position measuring apparatus comprising: an identification return unit that re-identifies an index based on a result of comparing the detected status of the detected index candidate and the blink control information in the index blink control process. An index position measuring device that measures the positions of a plurality of indices arranged in a space,
Index detection means for detecting an image of an index candidate from a captured image captured by the imaging unit;
Index tracking processing means for tracking the index candidates detected by the index detection means;
Input means for inputting the execution timing of the identification return process;
An identification return means for re-identifying the index based on time-series information of the relative distance between the index candidate of interest and another index candidate when input is performed by the input means; Index position measuring device. An index position measuring device that measures the positions of a plurality of indices arranged in a space,
Index detection means for detecting an index candidate image from captured images captured by a plurality of imaging devices, performing correlation between the images, and obtaining a position of the index candidate in a space;
Index tracking processing means for tracking the index candidates detected by the index detection means;
Input means for inputting the execution timing of the identification return process;
Index position measuring means for measuring the position of the index using means other than the imaging device;
When an input is performed by the input unit, a comparison result between the position of the index measured by the index position measurement unit and the position in the space of each index candidate detected by the index detection unit An index position measuring device comprising: an identification return means for re-identifying an index based on the index.

Images