JP2006277350A - Information processing method and information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for performing measurement of a virtual object so that an observer can intuitively easily understand it. <P>SOLUTION: When a virtual object of an apparatus 500 is brought close to the vicinity of a virtual object 150 to be measured positional attitudes of a first virtual object and a second virtual object are changed (S202 and S204), measurement for the virtual object 150 is performed based on the positional attitude relation between the first virtual object and the second virtual object changed in positional attitude, and the measurement result is displayed (S207). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a virtual object measurement technique.

The MR and AR technologies are technologies that synthesize a live-action video and CG, and make it appear as if a virtual CG object exists in front of the user. In a typical application of MR technology, a user wears a video see-through type HMD, and sees an image obtained by synthesizing an image from a camera in front of the HMD and a CG image created by a computer. Since the CG image is drawn from the viewpoint of the user measured by the position sensor, it appears to the user as if a virtual CG exists in the real space. An actual application of MR technology is verification of industrial design.
Japanese Patent Laid-Open No. 11-88913

  However, it has been difficult to intuitively and accurately measure the dimensions of the CAD model in the CAD design preview using MR described above. In order to obtain an accurate dimension, it is necessary to specify two points of the CAD model accurately. In the case of CAD using a conventional monitor screen, these two points are indicated with a mouse or the like.

  However, it is not intuitive to specify the two points of the CAD model and measure the dimensions of the CAD model in the case of using the MR technique. When MR technology is used, the CAD model appears to the user as if it exists in front of him. If it is not the CAD model but the real thing that is in front of the user, a measure or the like should be used to measure the dimension between the two points. A more intuitive interface is to measure the dimensions by applying a ruler that can be held by the user to the CAD model.

  When the MR technique is used, it is possible to measure the size by holding the real ruler over the CAD model. However, it is difficult to know the exact size. This is because the CAD model has no substance, and the ruler cannot be applied to the CAD model.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique for measuring a virtual object so that an observer can intuitively understand it.

  In order to achieve the object of the present invention, for example, an information processing method of the present invention comprises the following arrangement.

That is, the first acquisition of the position and orientation of the instrument in which one end of the rod-shaped first member and one end of the rod-shaped second member are supported so that the angle formed by each member can be changed. Process,
A second acquisition step of acquiring an angle formed by the first member and the second member;
An instrument virtual object information setting step for setting virtual object information of the instrument based on the position and orientation of the instrument and the angle;
A third acquisition step of acquiring the position and orientation of the user's viewpoint;
Generation step of generating a virtual object image of the instrument and a virtual object image of the measurement target according to the position and orientation of the viewpoint of the user based on the virtual object information of the instrument and the virtual object information of the virtual object of the measurement target When,
An instrument virtual object information adjusting step of adjusting a position and orientation and an angle of the virtual object of the instrument when the virtual object of the instrument approaches the virtual object to be measured.

  In order to achieve the object of the present invention, for example, an information processing apparatus of the present invention comprises the following arrangement.

That is, the first acquisition of the position and orientation of the instrument in which one end of the rod-shaped first member and one end of the rod-shaped second member are supported so that the angle formed by each member can be changed. Means,
Second acquisition means for acquiring an angle formed by the first member and the second member;
Instrument virtual object information setting means for setting virtual object information of the instrument based on the position and orientation of the instrument and the angle;
Third acquisition means for acquiring the position and orientation of the user's viewpoint;
Generation means for generating a virtual object image of the instrument and a virtual object image of the measurement target according to the position and orientation of the viewpoint of the user based on the virtual object information of the instrument and the virtual object information of the measurement target virtual object When,
And an instrument virtual object information adjusting unit that adjusts a position and orientation and an angle of the virtual object of the instrument when the virtual object of the instrument approaches the virtual object to be measured.

  With the configuration of the present invention, a virtual object can be measured so that an observer can easily understand intuitively.

  Hereinafter, the present invention will be described in detail according to preferred embodiments with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a state in which a system for measuring a virtual object existing in a mixed reality space in which a real space and a virtual space are fused is used. In the figure, reference numeral 10 denotes an observer of the mixed reality space, and an HMD (head-mounted display device) 100 is attached to the head of the observer, and the observer 10 can view an image of the mixed reality space through the HMD 100. Can be seen. In addition, an instrument in which an end of the rod-shaped first member and one end of the rod-shaped second member are supported in the observer's hand so that the angle formed by each member can be changed, that is, A compass-like instrument 500 is held.

  FIG. 4 is a diagram showing a schematic configuration of the instrument 500. The instrument 500 is a position / orientation sensor 510 for measuring the position / orientation of the rod-shaped first member 501, the rod-shaped second member 502, and the first member 501. The sensor 520 measures the angle formed by the first member 501 and the second member 502. A method of using the instrument 500 will be described later.

  Returning to FIG. 1, reference numeral 150 denotes a virtual object printer, which is a virtual object to be measured. Reference numeral 210 denotes a magnetic transmission source, and the HMD 100 and the instrument 500 are provided with sensors that measure their position and orientation in accordance with the magnetism generated from the magnetic transmission source 210.

  These sensors are connected to the magnetic transmission source 210, and the measurement results by the sensors are obtained by the magnetic transmission source 210 and sent to the computer 300.

  Hereinafter, each part which comprises a system is demonstrated.

  FIG. 10 is a block diagram showing a hardware configuration of the system according to the present embodiment.

  The HMD 100 includes an HMD built-in sensor 120, a right eye camera 110, a left eye camera 111, a right eye LCD 130, and a left eye LCD 131.

  The HMD built-in sensor 120 detects a change in the magnetic field according to the position and orientation of the magnetic field generated from the magnetic transmission source 210 and outputs a signal indicating the detected result to the three-dimensional sensor main body 200.

  The right-eye camera 110 and the left-eye camera 111 capture moving images in the real space that can be seen from the right and left eyes of an observer who wears the HMD 100 on the head. Image data of each frame constituting a moving image captured by each of the right eye camera 110 and the left eye camera 111 is output to the computer 300.

  The right-eye LCD 130 and the left-eye LCD 131 display images (mixed reality space images) obtained by superimposing virtual object images generated by the computer 300 on the real spaces captured by the right-eye camera 110 and the left-eye camera 111, respectively. Thus, an observer wearing the HMD 100 on the head can see an image of the mixed reality space corresponding to his / her right eye and left eye in front of his / her eyes.

  FIG. 2 is a diagram showing the appearance of the HMD 100. As shown in FIG. 2, the HMD 100 includes a right-eye camera 110, a left-eye camera 111, a right-eye LCD 130, a left-eye LCD 131, and an HMD built-in sensor 120.

  Returning to FIG. 10, the instrument 500 is held by the observer's hand as described above, and the position and orientation thereof change as the observer's hand moves. Therefore, the device built-in sensor 510 detects a change in the magnetic field according to its position and orientation in the magnetic field generated from the magnetic transmission source 210 and outputs a signal indicating the detected result to the three-dimensional sensor main body 200.

  The three-dimensional sensor main body 200 is based on the intensity of the signals received from the HMD built-in sensor 120 and the device built-in sensor 510. Data indicating the position and orientation of the HMD built-in sensor 120 and device built-in sensor 510 (position and orientation of the first member 501) in the coordinate system with x, y, and z axes is obtained and output to the computer 300.

  The instrument 500 includes a device built-in volume 520, and outputs a signal corresponding to the angle formed by the first member 501 and the second member 502 constituting the instrument 500 to the computer 300.

  Next, the computer 300 will be described.

  The A / D converter 340 performs A / D conversion on the signal output from the device built-in volume 520, and outputs this signal to the memory 302 as data indicating the angle formed by the first member 501 and the second member 502. .

  The serial I / O 310 functions as an interface for receiving data output from the three-dimensional sensor main body 200, and the received data is output to the memory 302. This data is data indicating the position and orientation of the HMD built-in sensor 120 and the device built-in sensor 510 in the sensor coordinate system. In the present embodiment, this position and orientation is composed of data indicating the position (x, y, z) and data indicating the orientation (roll, pitch, yaw). That is, the HMD built-in sensor 120 and the device built-in sensor 510 are measurement sensors with six degrees of freedom. Note that there are other methods for expressing the position and orientation, and the present invention is not limited to this.

  Each of the video capture cards 320 and 321 functions as an interface for receiving an image of each frame input from the right eye camera 110 and the left eye camera 111, and the received images are sequentially output to the memory 302. In the present embodiment, the image of each frame is assumed to conform to NTSC, but the present invention is not limited to this.

  Each of the video cards 330 and 331 functions as an interface for outputting the mixed reality space image corresponding to the right eye and the left eye generated by the processing described later by the CPU 301 to the right eye LCD 130 and the left eye LCD 131. The corresponding mixed reality space images are output to the right eye LCD 130 and the left eye LCD 131 via the video cards 330 and 331 as image signals compliant with VGA, respectively.

  The CPU 301 controls the entire computer 300 and executes each process to be described later that should be performed by the computer 300. Execution of such processing is performed by executing various programs and data stored in the memory 302.

  The memory 302 is a work area used by the CPU 301 to execute various processes, an area for storing programs and data for causing the CPU 301 to execute various processes, a serial I / O 310, and an A / D. An area for temporarily storing various data input via the converter 340 and the video capture cards 320 and 321 is provided. In addition, the position / orientation relationship (right-eye bias) between the right-eye camera 110 and the HMD built-in sensor 120 and the position / orientation relationship (left-eye bias) between the left-eye camera 111 and the HMD built-in sensor 120 are measured in advance. Assume that the right-eye bias and the left-eye bias are stored in the memory 302 in advance.

  The PCI bridge 303 is for electrically connecting a bus connecting the CPU 301 and the memory 302 to a bus connecting the A / D converter 340, the serial I / O 310, the video capture cards 320, 321, and the video cards 330 and 331. Is.

  Next, a process performed by the computer 300 having the above configuration to generate a mixed reality space image will be described below with reference to FIG. 11 showing a flowchart of the process. Note that programs and data for causing the CPU 301 to execute the processing according to the flowchart of FIG. 10 are stored in the memory 302, and the CPU CPU 301 performs processing using this, so that the computer 300 performs each processing described below. Will be executed.

  From the right-eye camera 110 (left-eye camera 111), the captured image (real space image) of each frame is input to the computer 300 via the video capture card 320 (321), so the CPU 301 acquires it (step S100). ) And sequentially stored in the video memory provided on the memory 302 (step S101).

  Next, data indicating “the angle formed by the first member 501 and the second member 502” (hereinafter referred to as “volume value”) is input to the computer 300 via the A / D converter 340. Since the data indicating “position and orientation in the sensor coordinate system of the HMD built-in sensor 120” and data indicating “position and orientation in the sensor coordinate system of the device built-in sensor 510” are input to the computer 300 via the serial I / O 310. The CPU 301 adds the left-eye bias (right-eye bias) to the position and orientation in the sensor coordinate system of the HMD built-in sensor 120 to obtain the position and orientation of the left-eye camera 111 (right-eye camera 110) in the sensor coordinate system. Data indicating the posture is stored in the memory 302 (step S102). Hereinafter, the right eye and the left eye may be collectively referred to as “viewpoint”.

  Note that the data indicating the position and orientation in the sensor coordinate system of the device built-in sensor 510 and the data indicating the volume value are stored in the memory 302 as they are.

  Next, the CPU 301 generates a modeling matrix using the viewpoint position and orientation obtained in step S102, and creates a projective transformation matrix from parameters such as the focal length of the camera measured in advance (projection matrix The creation is performed only once for the first time, and is used again thereafter (step S103). Since the processing for obtaining the modeling matrix and the projective transformation matrix is a well-known technique, a description thereof is omitted here. In short, in this step, processing for generating a matrix used for generating an image of the virtual space that can be seen from the viewpoint is performed.

  Next, the CPU 301 generates a virtual space image that can be seen from the right eye (left eye) using the previously obtained matrix and draws it on the video memory (steps S104 and S105). More specifically, when a virtual object to be measured (virtual object 150 of the printer shown in FIG. 1 in this embodiment) is arranged at a predetermined position and orientation, and the arranged virtual object is seen from the right-eye camera 110 (left-eye camera 111). A visible image is generated and drawn on the video memory (step S104).

  Next, a virtual object of the instrument 500 is arranged. First, a virtual object corresponding to the first member 501 (hereinafter referred to as a first virtual object) is arranged at the position and orientation measured by the device built-in sensor 510. . Next, a virtual object corresponding to the second member 502 (hereinafter referred to as a second virtual object) is arranged. One end of the second virtual object is aligned with one end of the first virtual object. The position and orientation in which the angle formed by the first virtual object and the second virtual object is the angle indicated by the volume value acquired from the device built-in volume 520 (that is, the position and orientation of the second member 502) ) And the second virtual object is arranged at the obtained position and orientation. Thereby, the virtual object of the instrument 500 can be arranged.

  FIG. 19 is a diagram showing an example of a virtual object of the instrument 500. In FIG. 19, reference numeral 1901 denotes a virtual object corresponding to the first member 501, which is arranged with the position and orientation of the first member 501. Reference numeral 1902 denotes a virtual object corresponding to the second member 502, which is arranged with the position and orientation of the second member 502.

  Therefore, in step S105 in FIG. 11, the virtual object of the instrument 500 is arranged as described above, and an image that is seen when the arranged virtual object is viewed from the right-eye camera 110 (left-eye camera 111) is generated, and is stored on the video memory. (Step S105).

  Note that since the real space image acquired from the right-eye camera 110 and the real space image acquired from the left-eye camera 111 are stored in step S101 in the memory 302 in advance, in steps S104 and S105, The virtual space image seen from the right-eye camera 110 and the virtual space image seen from the left-eye camera 111 are drawn on the memory 302, so that the mixed reality space image seen from the right-eye camera 110 and the left-eye camera 111 are viewed on the memory 302. An image of the mixed reality space is generated.

  FIG. 3 is a diagram showing a process of generating a mixed reality space image by drawing a virtual space image on the real space image. As shown in FIG. 3, the virtual space image 301 is displayed on the real space image 301. The mixed reality space image 303 is generated by superimposing (combining) the images 302.

  Next, the CPU 301 transfers the right-eye mixed reality space image and the left-eye mixed reality space image to the frame buffers of the video cards 330 and 331, respectively (step S106). The transferred images are sent to the right-eye LCDs 130 and 131 and displayed on the respective screens. An observer wearing the HMD 100 on the head can observe an image of the mixed reality space according to the position and orientation of the viewpoint.

  Next, processing performed by the computer 300 when the observer measures the measurement target virtual object 150 using the instrument 500 will be described. When the observer measures the measurement target virtual object 150 using the instrument 500, the observer naturally brings the instrument 500 close to the measurement target virtual object 150, but the virtual object of the instrument 500 and the measurement target virtual object 150 Depending on the distance, the position and orientation of the virtual object (1901, 1902 in the case of FIG. 19) constituting the instrument 500 are changed to a position and orientation suitable for measurement. Hereinafter, a case where various measurements are performed will be described. It should be noted that what kind of measurement is performed using the virtual object of the instrument 500 (in other words, what kind of measurement instrument is used as the virtual object of the instrument 500) depends on the operation unit (not shown) by the observer or the computer 300. The operator may operate and give instructions.

<Measure the distance between two points>
First, the case where the distance between two points on the measurement target virtual object 150 is measured using the virtual object of the instrument 500 will be described.

  FIG. 12 shows a case where the virtual object of the instrument 500 is arranged when the distance between the two points is obtained using the virtual object of the instrument 500, and the arranged virtual object is viewed from the right-eye camera 110 (left-eye camera 111). It is a flowchart of the process which produces | generates an image and draws on the said video memory (namely, process in step S105).

  First, the CPU 301 refers to the position and orientation of the first member 501 acquired from the device built-in sensor 510 and the volume value acquired from the device built-in volume 520, and the first virtual object position, the second virtual object The previous position is obtained (step S200).

  The method for obtaining the positions of the first virtual object and the second virtual object is not particularly limited. For example, the position of the first virtual object is obtained as follows. The positional relationship (first positional relationship) between the installation position of the device built-in sensor 510 and the previous position of the first member 501 is obtained in advance. Further, since the positional relationship (second positional relationship) between the previous position of the first member 501 and the previous position of the first virtual object is known in advance, the position measured by the device built-in sensor 510, the first If the positional relationship and the second positional relationship are used, the position of the first virtual object can be obtained. The same applies to the case of obtaining the previous position of the second virtual object. Hereinafter, the first virtual object and the second virtual object may be referred to as “foot”.

  Next, it is determined whether or not the feature point constituting the measurement target virtual object 150 is located within a predetermined distance from the position ahead of one foot (hereinafter referred to as the first foot) (step S201). In this embodiment, it is assumed that the CAD data describing the measurement target virtual object 150 is added with information about vertices and sides that are characteristics of the virtual object. The feature points here are, for the printer shown in FIG. 3, the corners and ridgelines of the outer shape of the body. These feature points and edges may be added manually by the user or automatically by the computer 300.

  If there is a feature point within a predetermined distance from the position of the tip of the first foot, the process proceeds to step S202 via step S201 so that the tip of the first foot matches the position of the feature point. Next, the first leg is moved (step S202).

  FIG. 6 is a diagram for explaining a process for matching the position of the tip of the foot with the feature point, in which 601 is a feature point on the measurement target virtual object 150, and 600 is a first foot. In the figure, since the feature point 601 exists within a predetermined distance from the previous position of the first foot 600, the first foot 600 is moved as indicated by the arrow in the figure and moved to the position indicated by 610.

  Returning to FIG. 12, if it is determined in step S201 that there is no feature point within a predetermined distance from the position of the first foot, the process proceeds to step S203, and the predetermined position is determined from the position of the first foot. It is determined whether or not the characteristic side constituting the measurement target virtual object 150 is located within the distance (step S203). If it exists, the process proceeds to step S204, and the first foot tip position is matched with the intersection point when a perpendicular line is drawn from the first foot tip position to the feature side. The foot is moved (step S204).

  FIG. 13 is a diagram for explaining processing for matching the position of the tip of the foot on the feature side. In FIG. 13, 1301 is the feature side on the measurement target virtual object 150, and 1300 is the first foot. Here, since the characteristic side 1301 exists within a predetermined distance from the position ahead of the first foot 1300, the first foot 1300 is moved as indicated by the arrow in the figure and moved to the position indicated by 1310. The method for moving each foot is not limited to the above, and various methods are conceivable.

  Next, the process proceeds to step S205, and it is determined whether the above process has been performed for the other leg (hereinafter referred to as the second leg). If not, the process returns to step S201. It is determined whether a feature point or a feature side exists in the vicinity of the tip of the second foot. If the feature point or feature side exists, the above processing is performed to move the position of the second foot.

  If the above processing is performed for each foot, the process proceeds to step S206. Here, if the above processing is performed for each foot, each foot moves independently, so that it is originally supported. There arises a problem that one end that has been separated is separated. Therefore, the postures of the respective feet are changed so as to match one end originally supported (step S206). Then, the foot is placed at the position and orientation changed as described above (when each foot is not close to any feature point or feature side, the position and orientation are not changed at all), and the placed virtual object is An image that can be seen when viewed from the right-eye camera 110 (left-eye camera 111) is generated and rendered on the video memory (step S206).

  Then, using the positions of the tips of the feet, the distance between the positions is obtained, and a character string indicating the distance is drawn on the bidet memory in order to display it on the image of the mixed reality space (step S207).

  FIG. 5 is a diagram illustrating a display example of an image of the mixed reality space generated by performing the above processing. In the figure, reference numeral 600 denotes a virtual object of the instrument 500, and the tip of each foot is positioned so as to coincide with the feature points 51 and 52 of the measurement target virtual object 54. Thus, the distance between the feature points 51 and 52 can be accurately measured without the observer performing an operation of matching the tip of the foot with the feature point.

  The measured distance between the two points is displayed as shown in FIG. Thus, the observer can know the measurement result without performing an operation such as reading the scale again.

<Measure the angle between two sides>
Next, a case where the angle formed by two sides on the measurement target virtual object 150 is measured using the virtual object of the instrument 500 will be described.

  FIG. 14 shows the case where the virtual object of the instrument 500 is arranged and the arranged virtual object is viewed from the right-eye camera 110 (left-eye camera 111) when the angle formed by the two sides is obtained using the virtual object of the instrument 500. It is a flowchart of the process which produces | generates an image and draws on the said video memory.

  First, the position of the first foot and the position of the second foot are obtained (step S300). Assuming that the first foot is the first virtual object and the second foot is the second virtual object, the position of the first virtual object is the position of the first member 501, and the position of the first member 501. Is a position obtained from the device built-in sensor 510. Further, as described above, the position of the second virtual object is obtained by a known calculation using the position and orientation of the first virtual object and the volume value.

  Next, it is determined whether there is a feature side within a predetermined distance from the first foot and whether the first foot and the feature side are substantially parallel (step S301). In this step, the distance between the position of the first foot and the feature side is obtained, and it is determined whether the obtained distance is within a predetermined distance. If it is within the predetermined distance, it is next determined whether the first foot and the feature side are substantially parallel. For example, if the first foot is the first virtual object, the device The posture measured by the built-in sensor 510 is compared with the posture of the feature side, and if the respective differences are within a predetermined range, it may be determined that they are substantially parallel. If the first foot is the second virtual object, the posture of the second virtual object is obtained as described above, and the obtained posture is compared with the posture of the feature side, so that the difference between them is within a predetermined range. If so, it may be determined to be substantially parallel.

  If there is a feature side within a predetermined distance from the first foot and the first foot and the feature side are substantially parallel, the process proceeds to step S302, and the posture of the first foot is changed to the feature side. The position and orientation of the first foot are then changed by matching the position of the tip of the first foot with the intersection when the perpendicular is drawn from this position to the feature side (step S302). ). As a result, the first leg is positioned along this characteristic side. Note that the method of positioning the first foot along the characteristic side is not limited to this, and various methods are conceivable.

  Then, the process proceeds to step S303, and it is determined whether the above process has been performed for the other leg (second leg) (step S303). If not, the process returns to step S301 to perform the subsequent processes. repeat.

  On the other hand, if the above process is performed for the second leg, the process proceeds to step S304. FIG. 15 is a diagram illustrating a state in which the position and orientation of each foot is moved. In FIG. 15, the first foot 1501 and the second foot 1502 are moved as indicated by arrows, respectively. , 1520 can be moved along.

  Returning to FIG. 14, next, the respective feet are moved so that the end points on the base side of both feet coincide with the position where the extended lines of the moved feet intersect or the closest position (step S <b> 304). As shown in FIG. 16, this corresponds to making each leg cross at one point. FIG. 16 is a diagram showing legs that meet at one point.

  Returning to FIG. 14, the feet are arranged in the position / posture changed as described above (the position / posture is not changed at all when each foot is not close to any feature side). An image that can be seen when viewed from the right-eye camera 110 (left-eye camera 111) is generated and drawn on the video memory (step S305).

  FIG. 8 is a diagram illustrating a state in which the position and orientation are corrected so that each foot is along the feature side. Then, an angle formed by each leg is obtained, and a character string indicating the angle is drawn on the bidet memory in order to display it on the mixed reality space image (step S306).

  FIG. 7 is a diagram illustrating a display example of an image of the mixed reality space generated by performing the above processing. In the figure, reference numeral 600 denotes a virtual object of the instrument 500, and each leg is located along the characteristic sides 71 and 72 of the measurement target virtual object 54. This makes it possible to accurately measure the angle formed by the feature sides 71 and 72 without performing an operation so that the observer follows his / her foot along the feature side.

  The measured angle is displayed as shown in FIG. Thus, the observer can know the measurement result without performing an operation such as reading the scale again.

<Measure the size>
Next, a case where the size of the measurement target virtual object 150 is measured using the virtual object of the instrument 500 will be described. Note that, as shown in FIG. 20, the virtual object of the instrument 500 according to the present embodiment is such that the first virtual object 2001 and the second virtual object 2002 become parallel as they approach the previous position. ing. FIG. 20 is a diagram illustrating a virtual object of the instrument 500 used in the present embodiment. Accordingly, the shapes of the first virtual object and the second virtual object change each time the volume value changes.

  FIG. 17 is a flowchart of processing for arranging the virtual object of the instrument 500 when the size of the measurement target virtual object 150 is obtained using the virtual object of the instrument 500.

  First, the same processing as in step S300 is performed to obtain the position of the first foot and the position of the second foot (step S400).

  Next, it is determined whether there is a feature side within a predetermined distance from the first foot (step S401). In this step, the distance between the position of the first foot and the feature side is obtained, and it is determined whether the obtained distance is within a predetermined distance.

  If there is a feature side within a predetermined distance from the first foot, the process proceeds to step S402, a vector of the shortest distance between the first foot and the feature side is obtained (step S402), and parallel to this vector. The position of the tip of the foot is moved onto the feature side along a straight line (step S403). At that time, since only the previous position is moved, the foot has a curved shape, but this creates a foot (virtual object) bent with a predetermined curvature as appropriate. FIG. 18 is a diagram illustrating a state in which the first foot is moved along the feature side. Note that the method of moving the position of the tip of the first foot onto the feature side is not limited to this, and various methods are conceivable.

  Then, the process proceeds to step S404, and it is determined whether the above process has been performed for the other leg (second leg) (step S404). If not, the process returns to step S401 to perform the subsequent processes. repeat.

  On the other hand, if the above process is performed for the second leg, the process proceeds to step S405. Then, the foot is placed in the position and orientation changed as described above (when each foot is not close to any feature side, the position and orientation are not changed at all), and the placed foot is placed in the right eye camera 110 ( An image that can be seen when viewed from the left-eye camera 111) is generated and drawn on the video memory (step S405).

  Then, the distance between the positions of the toes of the respective feet is obtained, and a character string indicating the distance is drawn on the bidet memory in order to display it on the mixed reality space image (step S406).

  FIG. 9 is a diagram illustrating a display example of an image of the mixed reality space generated by performing the above processing.

  As described above, when the virtual object of the instrument 500 and the measurement target virtual object 150 instrument approach each other, the position and posture of the virtual object of the instrument 500, the angle between the legs, and the like are adjusted, and the measurement target virtual object A measurement for 150 is made.

  Note that the processing according to the present embodiment is not limited to being applied to a system that generates an image of a mixed reality space, but can be applied to a system that generates only an image of a virtual space. Absent.

  In this embodiment, the video see-through type is used as the HMD, but it goes without saying that a system using an optical see-through type may be used.

  Also, an object of the present invention is to supply a recording medium (or storage medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and a computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved when the MPU) reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the recording medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the recording medium, program code corresponding to the flowchart described above is stored in the recording medium.

It is a figure which shows a mode that the system for measuring the virtual object which exists in the mixed reality space with which real space and virtual space were united is used. It is a figure which shows the external appearance of HMD100. It is a figure which shows the process which produces | generates the image of mixed reality space by drawing the image of virtual space on the image of real space. FIG. 3 is a diagram showing a schematic configuration of an instrument 500. It is a figure which shows the example of a display of the image of a mixed reality space. It is a figure explaining the process which makes the position of the tip of a foot correspond to a feature point. It is a figure which shows the example of a display of the image of a mixed reality space. It is a figure which shows a mode that the position and orientation are correct | amended so that each leg may be along a feature side. It is a figure which shows the example of a display of the image of a mixed reality space. It is a block diagram which shows the hardware constitutions of the system which concerns on embodiment of this invention. It is a flowchart of the process performed in order to produce | generate the image | video of mixed reality space. When obtaining the distance between two points using the virtual object of the instrument 500, the virtual object of the instrument 500 is arranged, and an image that is seen when the arranged virtual object is viewed from the right-eye camera 110 (left-eye camera 111) is generated. FIG. 11 is a flowchart of a process of drawing on the video memory (that is, a process in step S105). It is a figure explaining the process which matches the position of the tip of a foot on a feature side. When obtaining the angle formed by the two sides using the virtual object of the instrument 500, the virtual object of the instrument 500 is placed, and an image that is visible when the placed virtual object is viewed from the right-eye camera 110 (left-eye camera 111) is generated. 4 is a flowchart of processing for drawing on the video memory. It is a figure which shows a mode that the position and orientation of each leg | foot were moved. It is a figure which shows the leg | cross which cross | intersects at one point. 12 is a flowchart of processing for arranging a virtual object of the instrument 500 when the size of the measurement target virtual object 150 is obtained using the virtual object of the instrument 500. It is a figure which shows a mode that a 1st leg | foot is moved along a feature side. 6 is a diagram illustrating an example of a virtual object of the instrument 500. FIG. It is a figure which shows the virtual object of the instrument 500 used in embodiment of this invention.

Claims (6)

A first acquisition step of acquiring a position and orientation of an instrument in which one end of the rod-shaped first member and one end of the rod-shaped second member are supported so that the angle formed by each member can be changed; ,
A second acquisition step of acquiring an angle formed by the first member and the second member;
An instrument virtual object information setting step for setting virtual object information of the instrument based on the position and orientation of the instrument and the angle;
A third acquisition step of acquiring the position and orientation of the user's viewpoint;
Generation step of generating a virtual object image of the instrument and a virtual object image of the measurement target according to the position and orientation of the viewpoint of the user based on the virtual object information of the instrument and the virtual object information of the virtual object of the measurement target When,
An information processing method comprising: an instrument virtual object information adjusting step of adjusting a position and orientation and an angle of the virtual object of the instrument when the virtual object of the instrument approaches the virtual object to be measured. A measurement step of performing measurement on the measurement target virtual object based on a positional relationship between the virtual object of the instrument and the virtual object of the measurement target;
The information processing method according to claim 1, further comprising: displaying the measurement result. When one end of the virtual object of the first member is brought close to the first location of the virtual object to be measured and one end of the virtual object of the second member is brought closer to the second location of the virtual object to be measured ,
In the changing step,
The information processing method according to claim 1, wherein the virtual object of the first member is set along the first location, and the virtual object of the second member is set along the second location. . First acquisition means for acquiring the position and orientation of an instrument in which one end of the rod-shaped first member and one end of the rod-shaped second member are supported so that the angle formed by each member can be changed; ,
Second acquisition means for acquiring an angle formed by the first member and the second member;
Instrument virtual object information setting means for setting virtual object information of the instrument based on the position and orientation of the instrument and the angle;
Third acquisition means for acquiring the position and orientation of the user's viewpoint;
Generation means for generating a virtual object image of the instrument and a virtual object image of the measurement target according to the position and orientation of the viewpoint of the user based on the virtual object information of the instrument and the virtual object information of the measurement target virtual object When,
An information processing apparatus comprising: an instrument virtual object information adjusting unit that adjusts a position and orientation and an angle of the virtual object of the instrument when the virtual object of the instrument approaches the virtual object to be measured.   A program for causing a computer to execute the information processing method according to any one of claims 1 to 3.   A computer-readable storage medium storing the program according to claim 5.

Images