JP2004257872A - Position information acquiring system, position information acquiring device, position information acquiring method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a position information acquiring system for acquiring position information accurately and dynamically by a simple method. <P>SOLUTION: A position display tag 200 acquires own position information and displays it by a graphic. This position information acquiring device 101 photographs the outside world scenery by an imaging means (camera) 102 to thereby acquire an image. An image processing means 103 performs image processing of the image acquired by the imaging means 102, and acquires detection of the position display tag 200 and display information. Information on the position of the imaging means 102 and the photographing direction is calculated based on the display information of the position display tag 200 by a position calculation means 104. The information on the position of the imaging means 102 and the photographing direction acquired from a movement detection means (acceleration sensor) 105 and a relative position calculation means 106 is corrected by information acquired by the position calculation means 104 based on the position display tag 200. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a position information acquisition system, and more particularly to a position information acquisition system, a position information acquisition device, a position information acquisition method, and a program for acquiring position information from imaging.
[0002]
[Prior art]
A system that uses virtual reality technology to superimpose and display virtual information in a form that is superimposed on the actual video has been realized. In this case, it is very important to accurately grasp the position information on the camera side. . If accurate position information cannot be obtained, the virtual information in the video displayed as the superimposition information will not be as intended by the system, causing problems such as display displacement.
[0003]
In particular, in a system using a head-mounted display (head-mounted image display device), the camera is mounted on the head, and the position of the camera changes in real time due to movement of a person or movement of the head. Even if accurate positioning can be performed once, errors occur with the passage of time, and some kind of correction means is required in order to always obtain accurate position information.
[0004]
As a conventional method, there is a method in which sensors (acceleration sensors and gyro sensors) are prepared on the camera side, and the current position is calculated using relative position information obtained from the sensors after the first alignment. However, in this case, there is a problem that the accumulated error becomes very large with the passage of time due to the error of the sensor itself.
[0005]
Further, for example, there is an invention disclosed in Patent Literature 1, in which an object (a marker, an object, a building, or the like) serving as a sign in which position information is registered in advance is recognized by image recognition of a camera image to thereby determine the current position. Although there is a method of performing acquisition and correction, an object serving as a marker is a fixed object whose position is known, cannot move freely, and lacks convenience. Further, it is necessary to have management map data of the object position on the camera side or behind the network, and there is a problem that the system is enlarged.
[0006]
[Patent Document 1]
JP 2001-21403 A
[0007]
[Problems to be solved by the invention]
The present invention has been made to solve such a problem, and the present invention can solve the problems of the above conventional example, and can obtain accurate and dynamic position information by a simple method. It is an object to provide a position information acquisition system, a position information acquisition device, a position information acquisition method, and a program.
[0008]
[Means for Solving the Problems]
SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and a position information acquisition system of the present invention acquires a position display tag for displaying position information of a placement location and an image of the position display tag to acquire position information. A position information acquisition system comprising: a position information acquisition device for acquiring position information of the position display tag itself; and a character string based on the position information. Or a position display information generating means for generating position display information by a graphic, and a position information display means for displaying the position display information, wherein the position information acquisition device captures and captures an external scenery. Image processing means for obtaining the position display tag, performing image processing on the image obtained by the image pickup means, detecting the position display tag, and obtaining display information of the position display tag. Based on the information of the position display tag acquired in the image processing unit, characterized in that it comprises a position calculating means for calculating the position and shooting direction information of said imaging means.
With such a configuration, the position display tag acquires the position information of the position display tag itself by the position acquisition unit, and generates the position display information by a character string or a graphic by the position display information generation unit. Then, the position information display means displays the position display information such as a graphic. In the position information acquisition device, an image of an external scene is captured by an imaging unit such as a camera to acquire an image. Then, the image processing unit performs image processing on the captured image, detects the position display tag, and acquires information on the position display tag. Next, a relative distance and a direction between the position display tag and the imaging unit are calculated based on the information of the position display tag by the position calculation unit, and information on the position and the imaging direction of the imaging unit is calculated.
Accordingly, information on the position of the imaging means (such as a camera) and the imaging direction can be obtained accurately and dynamically without manual intervention. Further, the position display tag can be moved and can be arranged at an arbitrary place. Further, the position information acquisition device side (camera side) does not need to have the management map data of the position of the object serving as a marker unlike the related art, and the system can be configured not to be enlarged.
[0009]
Further, in the position information acquisition system according to the present invention, the position information acquisition device includes: a movement detection unit that detects movement of the imaging unit; and an initial position of the imaging unit based on information detected by the movement detection unit. And a relative position calculating means for calculating a relative movement amount from the camera and calculating a current position and a photographing direction of the imaging means.
With such a configuration, the movement of the imaging unit is detected by movement detection means using an acceleration sensor or a gyro, and the relative movement amount from the initial position is calculated by the relative position calculation means. The current position and the imaging direction of the imaging means are calculated.
Accordingly, even when the display information of the position display tag cannot be obtained (when the position display tag is out of the frame from the imaging), the information on the current position and the imaging direction of the imaging unit can be obtained.
[0010]
Further, the position information acquisition system of the present invention, the information of the current position and the imaging direction of the imaging unit calculated by the relative position calculation unit, the current position of the imaging unit calculated by the position calculation unit and It is characterized by further comprising a position information correcting means for correcting based on the information of the photographing direction.
With such a configuration, the information on the current position and the imaging direction of the imaging unit (such as a camera) calculated using an acceleration sensor or the like is used as the current value of the imaging unit calculated based on the imaging of the position display tag. Is corrected by the information of the position and the photographing direction.
Accordingly, it is possible to correct the accumulated error of the information on the position and the photographing direction calculated by the acceleration sensor or the like on the imaging unit side (camera side).
[0011]
In addition, the position information acquisition system of the present invention further includes, in the position display tag, a position detecting unit that detects a position of the position display tag, and a unit that displays the position information together with the position information of the position display tag. It is characterized by.
With such a configuration, the position display tag displays, for example, information in the normal direction of the position display tag display plane (position display tag posture information) together with the position information.
Thereby, if the virtual image display device (camera side) can capture one position display tag, it calculates the relative distance to the position display tag based on the zoom information, and calculates the position information and the posture information of the position display tag. From the flatness of the position display tag shape, it is possible to calculate the position and imaging direction of the imaging means (such as a camera).
[0012]
Further, in the position information acquisition system according to the present invention, the position information acquisition device is a transmission-type or non-transmission-type virtual image display device.
With such a configuration, information on the position and imaging direction of the imaging means (such as a camera) provided in the virtual image display device is acquired by the position information acquisition method of the present invention.
Thus, the information on the position and the imaging direction of the imaging means (such as a camera) provided in the virtual image display device can be accurately and dynamically obtained without manual intervention. Further, the virtual image display device side does not need to have the management map data of the position of the object serving as the sign unlike the conventional art, and the system can be configured not to be enlarged.
[0013]
Further, the position information acquisition system of the present invention is characterized in that the information displayed by the position display tag is displayed as a character string, a barcode, a QR code, or a combination thereof.
With such a configuration, in the position display tag, the position information and the posture information of the position display tag are displayed by a character string, a barcode, and a QR code ((Quick Response) code) that are often used.
This allows the position information acquisition device (camera) to easily read the display information of the position display tag using a general-purpose reading technique.
[0014]
In addition, the position information acquisition device of the present invention is a position information acquisition system that includes a position display tag that displays position information of a placement location and a position information acquisition device that captures the position display tag to acquire position information. The position information acquisition device, wherein the imaging means for capturing an image of the outside scenery to capture an image, image processing of the image captured by the imaging means, the detection of the position display tag, the detection of the position display tag, Image processing means for acquiring display information, and position calculating means for calculating information on the position and the shooting direction of the imaging means based on the information on the position display tag acquired by the image processing means It is characterized by the following.
With such a configuration, the position information acquisition device captures an image of an external scene by using an imaging unit such as a camera. Then, the image processing unit performs image processing on the captured image, detects the position display tag, and acquires information on the position display tag. Next, a relative distance and a direction between the position display tag and the imaging unit (such as a camera) are calculated by the position calculation unit based on the information of the position display tag, and information on the position and the imaging direction of the imaging unit is calculated.
Accordingly, information on the position of the imaging means (such as a camera) and the imaging direction can be obtained accurately and dynamically without manual intervention. Further, the position display tag can be moved and can be arranged at an arbitrary place. Further, the position information acquisition device side (camera side) does not need to have the management map data of the position of the object serving as a marker unlike the related art, and the system can be configured not to be enlarged.
[0015]
In addition, the position information acquisition method of the present invention is a position information acquisition system that includes a position display tag that displays position information of a placement location, and a position information acquisition device that captures the position display tag to acquire position information. A position information acquisition method, comprising: a position information acquisition procedure for acquiring position information of the position display tag itself by the position display tag; and a position information acquisition process using a character string or a graphic based on the position information. The position display information generating procedure and the position information display procedure for displaying the position display information are performed, and the position information obtaining apparatus obtains an image by an image capturing unit that captures an external landscape. An image processing procedure for performing image processing on the image captured in the image capturing procedure, detecting the position display tag, and obtaining display information of the position display tag, Based on the information of the position display tag acquired in management procedure, wherein the position calculating procedure for position and calculates the shooting direction information of said imaging means is carried out.
In such a procedure, the position display tag acquires the position information of the position display tag itself by a position acquisition procedure, and generates position display information by a character string or a figure by a position display information generation procedure. Then, position display information such as a figure is displayed by a position information display procedure. Further, in the position information acquisition device, an image capturing procedure acquires an image of an external scene by an image capturing means such as a camera. Then, according to the image processing procedure, the imaging is image-processed, the position display tag is detected, and the display information of the position display tag is obtained. Next, a relative distance and a direction between the position display tag and the imaging unit are calculated based on the information of the position display tag by a position calculation procedure, and information on the position and the imaging direction of the imaging unit (camera) is calculated.
Accordingly, information on the position of the imaging means (such as a camera) and the imaging direction can be obtained accurately and dynamically without manual intervention. Further, the position display tag can be moved and can be arranged at an arbitrary place. Further, the position information acquisition device side (camera side) does not need to have the management map data of the position of the object serving as a marker unlike the related art, and the system can be configured not to be enlarged.
[0016]
Further, the computer program according to the present invention is characterized in that the position in the position information acquisition system comprising a position display tag for displaying position information of the placement location and a position information acquisition device for capturing the position display tag to acquire position information. A computer in the information acquisition device, an image capturing procedure for obtaining an image by an image capturing means for capturing an external scenery, image processing of the image captured in the image capturing step, detection of the position display tag, and display of the position An image processing procedure for acquiring display information of the tag, and a position calculation procedure for calculating information on the position and the imaging direction of the imaging unit based on the information on the position display tag acquired in the image processing procedure. Is a program for executing.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
[0018]
FIG. 1 is a diagram showing a configuration example of a unit (function) provided in a position information acquisition device 101 in a position information acquisition system 100 of the present invention. The image capturing unit 102 is a camera, a visual sensor, or the like, and is a unit for capturing an image of an external scenery and acquiring an image. Note that the imaging may include the position display tag 200 or not. Further, the imaging unit 102 holds zoom information of a camera or the like.
[0019]
The image processing unit 103 performs image processing on the image data acquired by the imaging unit 102 such as a camera, and detects whether or not the position display tag 200 is included. When the position display tag 200 is included in the imaging data, the display information of the position display tag 200 is acquired. Then, the information of the acquired position display tag 200 is passed to the position calculating unit 104.
[0020]
The position calculating unit 104 analyzes the display information of the position display tag 200 acquired by the image processing unit 103. Then, the relative distance between the imaging means (such as a camera) 102 and the position display tag 200 is calculated from the zoom information held by the imaging means 102 (the size of the position display tag 200 is fixed). Further, the current position and the imaging direction of the imaging unit (camera or the like) 102 are calculated from the position information and the posture information displayed by the position display tag 200 and the flatness of the position display tag shape in the imaging. In some cases, the posture information may not be displayed on the position display tag 200. In this case, the current position of the image capturing unit 102 and the current position of the imaging unit 102 are determined by simultaneously analyzing the position information displayed on the three or more position display tags 200. The direction is calculated (see FIG. 7).
The movement detection unit 105 is an acceleration sensor, a gyro sensor, or the like, and detects movement of the imaging unit (such as a camera) 102 itself. The output information of the sensor and the like detected by the movement detecting means 105 is passed to the relative position calculating means 106.
[0021]
The relative position calculation means 106 obtains information on the amount of movement detected from the movement detection means (such as an acceleration sensor) 105, calculates the relative movement amount from the initial position of the imaging means 102 at the start of imaging, and The current position and shooting direction of 102 are calculated. When the display information of the position display tag 200 cannot be obtained by the imaging unit 102 (when the position display tag goes out of frame), the relative position calculation unit 106 calculates information on the current position and the imaging direction of the imaging unit 102. .
[0022]
The position information correction unit 107 is obtained from the movement detection unit (acceleration sensor or the like) 105 and the relative position calculation unit 106 based on the information on the position and the imaging direction of the imaging unit 102 calculated based on the display information of the position display tag 200. The information on the position of the imaging unit 102 and the imaging direction is corrected. Accordingly, it is possible to correct the accumulated error of the information on the position and the photographing direction calculated by the acceleration sensor or the like on the imaging unit side (camera side).
[0023]
FIG. 2 is a diagram illustrating a configuration example of a unit (function) included in the position display tag 200. The position information acquisition unit 201 is a unit for acquiring position information of the position display tag 200 itself. For example, position information is acquired by a short-range radar, a GPS, an optical marker, or the like using a wireless technology such as UWB. The position information acquired by the position information acquisition unit 201 is passed to the position display information generation unit 203.
[0024]
The posture detecting unit 202 is a direction sensor, a free gyro, or the like, and acquires information on the normal direction of the display plane of the position display tag 200 (posture information of the position display tag). The posture information detected by the posture detection unit 202 is passed to the position display information generation unit 203. Note that the position display tag 200 has two types of patterns, one with and without the attitude detection means 202.
[0025]
The position display information generating unit 203 generates graphic information (a character string, a bar code, a QR (Quick Response) code, etc.) from the position and orientation information acquired by the position information acquiring unit 201 and the orientation detecting unit 202. The graphic information generated by the position display information generation unit 203 is passed to the position information display unit 204. The position information display unit 204 displays the graphic information generated by the position display information generation unit 203 on, for example, a liquid crystal display panel.
[0026]
FIG. 3 is a diagram for explaining the appearance and display form of the position display tag 200. The position display tag 200 is provided with an antenna 211, and is used to receive radio waves from satellites when acquiring position information by, for example, GPS. The display screen 212 is a screen for displaying the position information and the posture information of the position display tag 200, and is, for example, a liquid crystal display panel. It is assumed that a general two-dimensional barcode is used as the graphic information to be displayed. For example, a QR code, a cyber code (for example, refer to JP-A-2000-148904) can be used. The shape of the position display tag 200 is not limited to the square shape shown in FIG. 3, but may be various shapes.
[0027]
FIG. 4 is a diagram showing an example in which the present invention is applied to a virtual image display device. A virtual image using a head-mounted image display device (head-mounted display) that superimposes and displays virtual information on a real image is displayed. 1 shows an example in which the present invention is applied to a display device. The virtual image display device shown in FIG. 4 includes a camera-side system 110 and a position display tag 200. The camera-side system 110 shown in FIG. 4 corresponds to the position information acquisition device 101 (FIG. 1) shown in FIG.
[0028]
Further, the camera-side system (position information acquisition device) 110 includes a transmission-type or non-transmission-type head-mounted display (Head Mounted Display: head-mounted image display device) 111 with a camera worn by a human on the head, It is composed of a camera side control device 130. The head mounted display 111 is provided with a digital camera 112, a pair of left and right image display units 113 of an eyeglass type, and an acceleration sensor 114.
[0029]
The head-mounted display (HMD) 111 displays an image (real image) captured by the camera 112 on the image display unit 113 (does not display a real image in the case of a transmissive HMD), and according to the position of the camera 112 and the shooting direction. The virtual information stored in the camera-side control device 130 is extracted, superimposed on a real image captured by the camera 112, and displayed.
[0030]
For example, FIG. 8 is a diagram illustrating an example of overlapping display of virtual information, in which a real image (an image of a house) captured by the camera 112 illustrated in FIG. 8A and virtual information (O) illustrated in FIG. This is an example of a guidance system that superimposes and displays the home of Mr. ○. As described above, the virtual image display apparatus to which the present invention is applied is a system for appropriately selecting virtual information in accordance with the position of the camera 112 and the shooting direction and displaying the virtual information in a superimposed manner.
[0031]
The information on the position and the shooting direction of the camera 112 is obtained by first providing the initial position setting information to the camera-side control device 130 or by the camera-side system 110 imaging the position display tag 200. . Further, the HMD 111 is provided with a movement detection sensor such as an acceleration sensor 114 (or a gyro), and detects a relative movement amount of the camera 112 due to the movement of the person wearing the HMD 111. Then, based on the information on the initial position and the relative movement amount, the current position and photographing direction of the camera 112 are calculated, and virtual information corresponding to the obtained position and photographing direction is extracted and superimposed and displayed. As described above, the HMD 111 displays the virtual information according to the camera position and the shooting direction, but a cumulative error gradually occurs in the calculation result of the position of the camera 112 and the movement amount in the shooting direction. Therefore, in the present invention, the information of the camera position and the photographing direction is corrected using the position display tag 200.
[0032]
As described above, the position display tag 200 has a function of detecting a position by itself. For this position detection, for example, a short-range radar, a GPS, an optical marker using a wireless technology such as UWB is used. Such techniques can be used. Further, by providing the position display tag 200 with an orientation sensor and a free gyro, it is possible to have information on the normal direction of the display plane of the position display tag (posture information of the position display tag). Further, the position information and the posture information are converted into graphic information (characters, two-dimensional bar codes, etc.) that can be decoded by the camera-side system 110 and displayed, and each time the position and posture of the position display tag 200 are changed, The displayed content is updated.
[0033]
FIG. 7 is a diagram illustrating the concept of camera position acquisition using position display tags, and illustrates an example in which the camera 112 can capture three position display tags A, B, and C. The camera-side system determines the size of the position display tags A, B, and C in the image pickup and the zoom information held in the camera 112 (the size of the position display tags is fixed), and the camera 112 and the position display tags A and B. , C are calculated. In addition, the position information (for example, the latitude, longitude, and altitude of each position display tag) and the posture information displayed by the position display tag 200 are analyzed. And the shooting direction are calculated. When the display information of at least three or more position display tags can be simultaneously analyzed on the camera 112 side as in this example, the function of displaying the posture information on the position display tag side is not necessarily required. It is possible to calculate the camera position and the shooting direction only by displaying the position information on the tag side. However, a condition is that three or more position display tags are not arranged on one straight line. That is, when the position information of the position display tag includes posture information, the position and the photographing direction of the camera 112 can be calculated if one position display tag can be imaged. If no information is included, it is necessary to image three or more position display tags simultaneously.
[0034]
As shown in FIG. 4, as a typical configuration example, sensors such as an acceleration sensor 114 for acquiring position information are provided in the camera itself, and the accumulated error of the position information calculated by the camera is calculated. The position display tag 200 is used as a correction unit. However, if information on the position display screen of the position display tag 200 can always be obtained on the camera side (when the position display screen on the position display tag side is always included in the camera-side imaging), sensors on the camera side are required. May not be adopted. In this case, the resources required on the camera side can be minimized.
[0035]
By using such an HMD 111 to which the present invention is applied, for example, a guidance system at a street corner can be configured. That is, this is a system used when a user wears the camera-equipped HMD 111 and walks around a street corner, and is a guide system in which guide information of a street corner is superimposed on a real image as virtual information and displayed. Specifically, information such as a place name and a building name is superimposed and displayed. Although the polymerization information is usually simple, it can be applied to advertisements and signs. In addition, it can be used not only at street corners but also at offices and homes. The head mounted display with a camera can be either a transmission type or a non-transmission type.
[0036]
FIG. 5 is a diagram showing a configuration example of a camera-side system (position information acquisition device), and shows a portion directly related to the present invention. The configuration example of the camera-side system 110 shown in FIG. It is shown.
In FIG. 5, a digital camera (imaging unit) 112 including a CCD element or the like in a head-mounted display 111 is a camera that captures an image of the position display tag 200 together with the external scenery. The image display unit 113 is a display device that displays a real image and virtual information in a superimposed manner. The acceleration sensor (movement detecting means) 114 is a sensor for detecting the movement of the head mounted display 111 and calculating the amount of movement.
[0037]
The control unit 131 provided in the camera-side control device 130 is a control unit including a CPU that controls the head-mounted display 111 and the entire camera-side control device 130. The virtual information database 132 is a database that stores virtual information associated with a position and a direction. The processing program unit 140 includes an image processing unit (image processing unit) 141, a position calculation processing unit (position calculation unit) 142, a relative position calculation processing unit (relative position calculation unit) 143, and a position information correction processing unit (position (Information correcting means) 144, a virtual information extraction processing unit 145, and a superimposed image display processing unit 146.
[0038]
The image processing unit 141 is a processing unit for performing image processing on the image data acquired by the camera 112 and acquiring the display information of the position display tag 200, and the display information of the position display tag 200 acquired by the image processing unit 141. Is passed to the position calculation processing unit 142.
[0039]
The position calculation processing unit 142 analyzes the information of the position display tag 200 acquired by the image processing unit 141. Then, the relative distance between the camera 112 and the position display tag 200 is calculated from the zoom information stored in the camera 112 (the size of the position display tag is fixed). Further, the current position and the photographing direction of the camera 112 are calculated from the position information and the posture information displayed by the position display tag 200 and the flatness of the position display tag shape in the imaging. Note that the position information may not be displayed on the position display tag 200. In this case, the position information displayed on the three or more position display tags 200 is analyzed at the same time, so that the current position and the shooting direction of the camera 112 are obtained. Is calculated (see FIG. 7).
[0040]
The relative position calculation processing unit 143 obtains movement detection information from the acceleration sensor 114, calculates a relative movement amount from the initial position at the time of starting imaging of the camera 112, and calculates a current position and a shooting direction of the camera 112. Is calculated. When the display information of the position display tag 200 cannot be obtained by the camera 112 (when the position display tag goes out of frame), the relative position calculation processing unit 143 calculates information on the current position and the shooting direction of the camera 112. .
The position information correction processing unit 144 obtains the position and image of the camera 112 obtained from the acceleration sensor 114 and the relative position calculation processing unit 143 based on the information on the position and imaging direction of the camera 112 based on the display information of the position display tag 200. Correct the direction information.
[0041]
Further, the virtual information extraction processing unit 145 refers to the virtual information database 132 based on the information on the current position and the shooting direction of the camera 112 (zoom information may be added), and processes the virtual information superimposed and displayed on the HMD 111. This is a processing unit for extraction. Further, the superimposed image display processing unit 146 is a processing unit for displaying virtual information extracted from the virtual information database 132 so as to be superimposed on a real image.
[0042]
FIG. 6 is a diagram showing a processing flow in the position display tag 200 and the camera-side system 110 shown in FIG. Hereinafter, with reference to FIG. 6 (and FIG. 5), the position display tag 200 and the camera-side system (position information acquisition device) 110 image the position display tag 200, and the position and the photographing direction of the camera (imaging unit) 112 The operation flow for acquiring and correcting the information will be described. In this example, the flow of processing when the position display tag 200 displays only the position information (does not display the posture information) and three or more position display tags can capture an image is shown.
[0043]
First, regarding the processing on the position display tag 200 side, position information is obtained (step S1), and graphic information corresponding to the obtained position information is generated and displayed (step S2). This display information is updated every time the position of the position display tag is changed. As described above, the position information acquisition method of the position display tag 200 uses a normal position detection technology (a position detection technology using UWB, GPS, or the like), and uses latitude, longitude, and altitude as the position information. . In addition, a general technique is used for a method of converting position information into graphic information. Further, the graphic information includes a case of a simple character string.
[0044]
Next, an image capturing process is performed by the camera 112 of the camera-side system 110 (step S11). In the image captured by the camera 112, the display portion of the position display tag 200 is cut out by the image processing unit 141 (FIG. 5) (step S12). When the position display tag 200 is detected in the imaging (step S13), a process of passing the obtained display information of the position display tag to the position calculation processing unit 142 (FIG. 5) is performed. When the position display tag 200 is not detected in the imaging, the process proceeds to step S16. Note that this system has a system configuration in which the overlapping display processing of virtual information is not started unless the position display tag display is recognized once.
[0045]
When the display information of the position display tag 200 is passed to the position calculation processing unit 142, the position calculation processing unit 142 determines each of the position display tags 200 based on the current zoom state of the camera 112 and the size of each position display tag shape in the imaging. The relative distance to the camera 112 is calculated, and the display information of the position display tag 200 is analyzed to calculate the current position and the shooting direction of the camera 112 (step S14).
[0046]
Next, based on the information on the position and the shooting direction of the camera 112 obtained by the position calculation processing unit 142, the information on the current camera position and the shooting direction is corrected by the position information correction processing unit 144 (FIG. 5). (Step S15). That is, the information of the camera position and the photographing direction calculated by the acceleration sensor 114 and the relative position calculation processing unit 143 (FIG. 5) in the virtual image display device is obtained based on the display information of the position display tag 200. And information on the shooting direction.
[0047]
Next, the virtual information extraction processing unit 145 (FIG. 5) determines whether or not there is virtual information that needs to be superimposed and displayed in the current imaging frame based on the information of the current camera position, the camera imaging direction, and the zoom state. Is confirmed with reference to the virtual information database 132. If there is corresponding information in the virtual information database 132, virtual information is extracted (step S17). Then, the overlapping display of the virtual information is performed by the overlapping image display processing unit 146 (step S18). If there is no virtual information, no display processing is performed.
[0048]
Further, when the camera position and the photographing direction are changed due to the movement of the person and the change of the line of sight (step S19), the flow from the image pickup process (step S11) is repeatedly executed. The change is detected based on information from an acceleration sensor 114 provided on the camera 112 side. When the camera 112 cannot acquire the display information of the position display tag 200 (when the position display tag goes out of frame), the relative position calculation processing unit 143 uses the camera 112 based on the information from the acceleration sensor 114. Is calculated (step S16).
[0049]
Although the embodiment of the present invention has been described above, the processing program unit 140 in the camera-side control device 130 shown in FIG. 5 may be realized by dedicated hardware. A general-purpose information processing device such as a memory and a CPU (Central Processing Unit) is used. A program (not shown) for realizing the function of the processing unit is loaded into the memory and executed to realize the function. It may be something.
[0050]
Further, a program for realizing the function of the camera-side system (position information acquisition device) shown in FIG. 5 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by a computer system. By executing, the processing required for the camera-side system (position information acquisition device) of the present invention may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.
[0051]
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system.
Further, the “computer-readable recording medium” refers to a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, which dynamically holds the program for a short time. In this case, a medium holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client in that case, is also included.
Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in a computer system, that is, a so-called difference file (difference file). Program).
[0052]
As described above, as an embodiment of the position information acquisition system of the present invention, an example of application to a virtual reality system using a head-mounted display has been described. However, the present invention is not limited to this and can be applied to the following applications.
・ Application to the current position grasp system for mobile robots
・ Application to broadcast virtual studio
-In a case where the advertisement information is superimposed and displayed on a mobile camera carried by a photographer, for example, virtual advertisement information can be displayed in the background even during a victor interview during a sports broadcast. In the case of a head-mounted display, headphones may be further provided so that audio and music information can be played according to the virtual information.
[0053]
Although the embodiment of the present invention has been described above, the position information acquisition system of the present invention is not limited to the illustrated example described above, and various changes can be made without departing from the gist of the present invention. Of course.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration example of a unit included in a position information acquisition device.
FIG. 2 is a diagram showing a configuration example of a unit included in a position display tag.
FIG. 3 is a diagram for explaining the appearance and display form of a position display tag.
FIG. 4 is a diagram showing an example in which the present invention is applied to a virtual image display device.
FIG. 5 is a diagram showing a configuration example of a camera-side system (position information acquisition device).
FIG. 6 is a view showing a processing flow in a position display tag and a camera-side system.
FIG. 7 is a view showing the concept of camera position acquisition using a position display tag.
FIG. 8 is a diagram showing an example of overlapping display of virtual information.
[Explanation of symbols]
100 position information acquisition system, 101 position information acquisition device
102 imaging means, 103 image processing means, 104 position calculation means
105 movement detecting means, 106 relative position calculating means, 107 position information correcting means
200 position display tag, 201 position information acquisition means, 202 attitude detection means
203 position display information generation means, 204 position information display means

Claims (9)

A position information acquisition system configured to include a position display tag that displays position information of the arrangement location and a position information acquisition device that acquires the position information by imaging the position display tag,
In the position display tag,
Position information acquisition means for acquiring position information of the position display tag itself,
Position display information generating means for generating position display information by a character string or a graphic based on the position information,
Comprising position information display means for displaying the position display information,
In the position information acquisition device,
Imaging means for photographing the scenery of the outside world and acquiring an image;
Image processing of the image captured by the image capturing means, image detection of the position display tag, and image processing means for obtaining display information of the position display tag,
A position information acquisition system, comprising: a position calculation unit for calculating information on a position and a photographing direction of the imaging unit based on information on a position display tag acquired by the image processing unit. In the position information acquisition device,
Movement detection means for detecting movement of the imaging means;
Based on the information detected by the movement detection means, a relative movement amount from an initial position of the imaging means is calculated, and a relative position calculation means for calculating a current position and a shooting direction of the imaging means. The position information acquisition system according to claim 1, further comprising: Position information for correcting the information on the current position and the imaging direction of the imaging unit calculated by the relative position calculation unit based on the information on the current position and the imaging direction of the imaging unit calculated by the position calculation unit The position information acquisition system according to claim 2, further comprising a correction unit. In the position display tag,
Attitude detecting means for detecting the attitude of the position display tag,
4. The position information acquisition system according to claim 1, further comprising: means for displaying the posture information together with the position information of the position display tag. The position information acquisition system according to any one of claims 1 to 4, wherein the position information acquisition device is a transmission-type or non-transmission-type virtual image display device. The position information acquisition system according to any one of claims 1 to 5, wherein the information displayed by the position display tag is displayed as a character string, a barcode, a QR code, or a combination thereof. A position display tag that displays position information of the arrangement location, and the position information acquisition device in the position information acquisition system that includes a position information acquisition device that acquires the position information by imaging the position display tag,
Imaging means for photographing the scenery of the outside world and acquiring an image;
Image processing of the image captured by the image capturing means, image detection of the position display tag, and image processing means for obtaining display information of the position display tag,
A position information acquiring apparatus, comprising: a position calculating unit for calculating information on a position and a photographing direction of the image pickup unit based on information on a position display tag acquired by the image processing unit. A position display tag for displaying position information of a placement location, and a position information acquisition method in a position information acquisition system including a position information acquisition device that acquires the position information by imaging the position display tag,
By the position display tag,
A position information acquisition procedure for acquiring position information of the position display tag itself,
A position display information generating procedure for generating position display information by a character string or a graphic based on the position information,
Position information display procedure for displaying the position display information is performed,
By the position information acquisition device,
An imaging procedure for acquiring an image by an imaging unit that captures an external landscape;
Image processing of the captured image obtained in the image capturing procedure, detection of the position display tag, an image processing procedure for obtaining display information of the position display tag,
A position calculating step for calculating information on the position and the photographing direction of the imaging unit based on the information on the position display tag obtained in the image processing step. A position display tag that displays the position information of the location, and a computer in the position information acquisition device in the position information acquisition system that includes a position information acquisition device that captures the position display tag and acquires the position information,
An imaging procedure for acquiring an image by an imaging unit that captures an external landscape;
Image processing of the captured image obtained in the image capturing procedure, detection of the position display tag, an image processing procedure for obtaining display information of the position display tag,
A program for executing, based on the information of the position display tag acquired in the image processing procedure, a position calculating procedure for calculating information on a position and a photographing direction of the imaging unit.

Images