JP7001711B2 - A position information system that uses images taken by a camera, and an information device with a camera that uses it. - Google Patents

Description

The present invention relates to a position information system using a camera-captured image and an information device with a camera used thereof. In particular, the present invention relates to a location information system suitable for use in a complex facility including a plurality of service facilities such as an airport, a station, a stadium, and a shopping mall, a store, a restaurant, and an event venue.

Information devices that integrate a wireless network, a camera, a display, a position sensor, and the like into an information device composed of a CPU (Central Processing Unit) and a memory are widely used. Specifically, it is a smartphone or a tablet PC (Personal Computer), and map data can be downloaded via a wireless network and the current position obtained by the position sensor can be superimposed and displayed on the map displayed on the display.

As the position sensor, a GPS (Global Positioning System) sensor is used. The GPS sensor receives radio waves transmitted from GPS satellites and obtains the position of the receiving terminal. On the other hand, reception of radio waves from GPS satellites may become unstable in indoor facilities, and the estimated distance from mobile base stations and access points of IEEE 802.11 standard wireless LAN (Local Area Network) is used as an auxiliary. It may be used. However, even when using a mobile base station or wireless LAN access point, it is a measurement using radio waves whose main purpose is not to measure the position, and the accuracy is about 10 m to 200 m at most due to the influence of the surrounding environment. Has been done.

For this reason, it is not suitable for the purpose of grasping the current location in order to search for individual service facilities, stores, etc. in a complex facility that is an indoor facility, and there is no choice but to rely on the guide map that is a printed matter provided by each complex facility. ..

On the other hand, Patent Document 1 discloses a method of detecting a current position using an image taken by a camera. In the first method disclosed in Patent Document 1, the block information in which the address is described is photographed with a camera, and the current position is grasped. In order to apply this method to indoor facilities, it is necessary to newly install signs corresponding to block information, and there is a problem that many signs need to be installed in order to obtain high accuracy.

The second method disclosed in Patent Document 1 holds landscape images from each point as a database, collates the images taken by the camera with the landscape images in the database, and grasps the current position. According to this method, it is necessary to create and maintain a database of landscape images, but there is a problem that the database becomes enormous in order to obtain high accuracy.

Japanese Unexamined Patent Publication No. 2004-226170

In the above-mentioned Patent Document 1, it is necessary to newly install a sign corresponding to the block information or to create and maintain a database of a huge landscape image, and the construction of the position information system becomes a large scale.

The present invention has been made in view of the above points, and an object thereof is position information using a camera-taken image in an indoor facility without installing a new sign and without requiring a huge landscape database. The purpose is to provide a system and an information device with a camera.

In view of the above background technology and problems, the present invention is a position information system using an image taken by a camera of an information device with a camera, and an object in an indoor facility is used as a mark object to form a mark object. It has a facility map service device that holds a database of datasets consisting of image data and installation data, and has a means to send part or all of the database to an information device with a camera. The information device with a camera is a camera. A means to extract a significant size object from the camera image taken with the camera and compare it with the image data in the database to identify the mark object in the camera image, and multiple mark objects in the camera image. It is provided with a means for detecting the current position of the information device with a camera and the direction of the camera by using the combination of the above.

According to the present invention, there is provided a position information system using a camera-taken image that does not require a new sign or the like to be installed in an indoor facility and does not require a huge database, and an information device with a camera used for the system. be able to.

It is a figure which shows the outline of the position information system using the image taken by a camera in Example 1. FIG. It is a block diagram of the facility map service apparatus in Example 1. FIG. It is a block diagram of the information apparatus with a camera in Example 1. FIG. It is a mark object database in Example 1. It is an image taken by a camera in Example 1. It is explanatory drawing of the method of detecting the position and direction from two mark objects in Example 1. FIG. It is explanatory drawing of the method of detecting the position and direction from three mark objects in Example 1. FIG. It is explanatory drawing of another method of detecting a position and direction from two mark objects in Example 1. FIG. It is a processing flow diagram of the information apparatus with a camera of the position information system using the image taken by the camera in Example 1. FIG. It is a modification of the camera of the information apparatus with a camera in Example 2. It is a block diagram of the facility map service apparatus in Example 3. FIG. It is a block diagram of the information apparatus with a camera in Example 3. FIG. It is a print guide map of the complex facility in Example 4. It is a guide map database in Example 4. It is a processing flow diagram of the position information system using the image taken by the camera in Example 4. FIG. This is an example of displaying the current position on the information device with a camera according to the fourth embodiment.

Hereinafter, examples of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an outline of a position information system using images taken by a camera in this embodiment. In FIG. 1, 1 is a facility map service device, 2 is a network, 3 is an indoor facility complex facility, 4 is an access point, 311 and 312, 313 are the first to third floors of the complex facility 3, 321 respectively. ~ 330 are mark objects which are characteristic objects that serve as landmarks in indoor facilities. Specifically, it is a guide sign of a service facility, a store name such as a store, a signboard or panel on which a logo is displayed, a product displayed on a window, or the like. Further, 5 is an information device with a camera, and 6 is a user.

In the mark objects 321 to 330, 321 and 326 are store logo panels, 322 and 327 are products displayed in the store show window, 323, 325 and 329 are facility service pictogram panels, 328 and 330. Is a store sign, etc., and 324 is a facility facility (in FIG. 1, a security camera is taken as an example). A mark object database is composed of a data set that combines the size of the actual body of these objects and the arranged position data (hereinafter, the database is referred to as DB (Data Base)). In addition, in FIG. 1, in order to avoid congestion in the figure, not all the mark objects in the figure are numbered, and further, the marked objects shown in the figure are examples of those existing in the actual complex facility 3. Not too much.

In FIG. 1, the user 6 on the third floor floor 311 holds the information device 5 with a camera, takes a picture of the inside of the complex facility 3 with the camera built in the information device 5 with a camera, and uses the image taken by the camera to mark an object. The current position is detected by collating with the DB.

Further, the wireless network function built in the camera-equipped information device 5 receives the service of the facility map service device 1, and for example, the guide map of the complex facility is downloaded and displayed on the display built in the camera-equipped information device 5. .. A pointer or the like corresponding to the detected current position information is superimposed and displayed on the display, and the user 6 can know where he / she is in the complex facility 3.

FIG. 2 is a block diagram of the facility map service device 1 in this embodiment. In FIG. 2, the facility map service device 1 is composed of a facility map service server 10 and an external storage 11, and the facility map service server 10 is composed of a network IF (Interface) 101, a CPU 102, a RAM 103, an internal storage 104, and an external storage IF 106. Become. Further, the internal storage 104 holds the facility map service program 105, and the facility map service program 105 is expanded in the RAM 103 and executed by the CPU 102.

Further, the facility map DB 12 and the mark object DB 13 are held in the external storage 11. A part of the facility map DB 12 and the mark object DB 13 is read from the facility map service program 105 as needed, and downloaded to the camera -equipped information device 5 via the network IF 101.

FIG. 3 is a block diagram of the information device 5 with a camera in this embodiment. In FIG. 3, the information device 5 with a camera includes a camera 501, a GPS sensor 502, a gyro sensor 503, an acceleration sensor 504, an orientation sensor 505, a video processor 506, a graphic processor 507, a display with a touch panel 508, a microphone / speaker 509, and mobile communication. It is composed of IF 510, wireless LAN IF 511, CPU 512, RAM 513, FROM (Flash ROM) 514, EXTIF (external IF) 521, and external terminal 522. The mobile communication IF 510 is an interface for mobile communication typified by 4G, and the wireless LAN IF 511 is an interface for wireless LAN of the IEEE 802.11 standard. One of these two interfaces is selected according to the application and communication environment, and services such as downloading the facility map DB 12 and the mark object DB 13 from the facility map service device 1 via the external access point 4 and the network 2 are selected. Receive an offer.

Further, the FROM 514 includes a facility map service application 515 as a program, a camera linkage process 516, a mark object extraction process 517, a position / direction detection process 518, a map information linkage process 519, and a download DB 520 in the lower hierarchy thereof. These programs are expanded in RAM 513 and executed by CPU 512.

The facility map service application 515 first downloads the facility map DB 12 and the mark object DB 13 related to the target complex facility and stores them in the download DB 520. The camera cooperation process 516 takes a picture with the camera 501 and captures the image taken by the camera. The mark object extraction process 517 extracts an object (a mass having a significant size) that may be a mark object from the image taken by the camera, and collates it with the image data of the mark object DB stored in the download DB 520. , Identify the mark object in the camera shot image.

Using a plurality of the specified mark objects, the position / direction detection process 518 detects the current position of the information device 5 with a camera and the direction in which the camera is pointed. The detection method will be described later with reference to FIGS. 5 to 7.

The map information linkage process 519 reads out the facility map DB stored in the download DB 520 and displays the map data on the display unit of the display 508 with a touch panel. Further, the detected current position and the direction of the camera are superimposed on the map data as graphic data such as a pointer.

FIG. 4 is a mark object DB 13 in this embodiment. The database is not limited to the format shown in the figure, and may be any format as long as it has the same function.

In FIG. 4, the mark object DB 13 includes the data of the facility name 131 and the reference point 132 in the facility as reference data, and the data of the mark object is described corresponding to the item shown in 133.

In FIG. 4, the mark objects No. 0001 to No. 0012 are described as an example, but the present invention is not limited to this. No. 0001 is the mark object 329 of FIG. 1, and the image data, the actual size, the position of the mark object 329 on the N (north) -S (south) axis, and the E (east) -W (west) axis. The position of and the position data of H (height) constitute one mark object. The image data is, for example, bitmap data or JPEG data. When the substance is a three-dimensional object such as a product, image data taken from a plurality of directions may be described. The substance size is data indicating the size of the substance of the mark object, and is described by H (vertical) × W (horizontal) with m (meter) as a unit. The position on the NS axis, the position on the EW axis, and the position data of H describe the relative position from the reference point 132 with m (meter) as a unit.

The mark objects 325, 324, and 321 are multiple mark objects existing in the complex facility 3 having the facility name 131. Even if the image data and size data are common, the mounting positions are different, so there are as many position data as there are.

FIG. 5 is an image taken by a camera in this embodiment. The image captured by the camera is displayed on the display 508 with a touch panel. In FIG. 5, the background is omitted because it does not relate to this embodiment. Mark objects 324, 323, 322, and 321 are photographed in the image captured by the camera.

The image data of the mark object may be used in a plurality of places in the complex facility 3, and the current position of the information device 5 with a camera cannot be specified only by collating one mark object. Therefore, in this embodiment, the current position is specified by combining a plurality of mark objects. In the example shown in FIG. 5, the mark objects 324, 323, 322, and 321 are photographed. In the complex facility 3 shown in FIG. 1, the third floor floor 311 is located on the right side of the elevator and is photographed to the left. It can be seen that the image is an elevator. Extraction accuracy is achieved by taking multiple images taken with a camera and adding processing such as removing those that have changed from the mark object and removing mark objects taken at a position lower than the specified height. Can be improved. Further, if the mark object whose position is correctly detected is stored from the result of detecting the current position a plurality of times and the mark object is preferentially used, the detection accuracy can be improved.

As for the direction of the camera-equipped information device 5, the orientation sensor 505 of the camera-equipped information device 5 at the time of camera shooting is used before determining the direction of the camera-equipped information device 5 by the method of FIGS. Orientation information may be used to identify the position.

FIG. 6 is an explanatory diagram of a method of detecting the current position and the direction of the camera from the mark object in this embodiment. In FIG. 6, two mark objects, MO1 and MO2, are used. The center of the mark object MO1 is, for example, at a coordinate position (X1, Y1) with the center of the image captured by the camera as the origin, and the captured size is S1 (= H1 × W1) in terms of area. Similarly, for the mark object MO2, the center coordinate position and size are shown in the figure.

The size of the entity of the mark object MO1 (331) is derived from the item of the entity size of the mark object DB. Assuming that this is SO1, the distance r1 between the information device 5 with a camera and the mark object MO1 can be calculated as a condition that an object having an area SO1 is photographed in an area S1 from a known camera magnification. The point where the distance r1 between the information device 5 with a camera and the mark object MO1 is not one point, but a curved surface Q1 as a set.

Similarly, a curved surface Q2 in which the distance between the information device 5 with a camera and the mark object MO2 is r2 is obtained. The line where the curved surfaces Q1 and Q2 intersect is a candidate for the position of the information device 5 with a camera. At a point on the line, the position A of the information device 5 with a camera can be determined by searching for a point A in which the distance between the mark objects MO1 and MO2 is L12 in the image captured by the camera.

Further, the orientation of the camera of the information device 5 with a camera is determined as a vector v facing the origin of the image captured by the camera from the current position A.

FIG. 7 is an explanatory diagram of a method of detecting the current position and the direction of the camera from the three mark objects in this embodiment. In FIG. 7, the mark object MO3 is added to FIG. For the mark object MO3, as in the case of the mark objects MO1 and MO2 described above, the distance r3 is obtained by comparing the area of the entity with the area S3 of the image taken by the camera, and the curved surface Q3 is further obtained from the distance r3. The point where the curved surfaces Q1 and Q2 and the curved surface Q3 intersect can be obtained as the position A of the information device 5 with a camera.

Further, the orientation of the camera of the information device 5 with a camera is determined as a vector v facing the origin of the image captured by the camera from the current position A, as in FIG.

According to the methods of FIGS. 6 and 7, since the distance and direction of the information device 5 with a camera can be known from the mark object whose installation position is known, the position of the information device 5 with a camera can be detected accurately.

FIG. 8 is an example of detecting the position of the information device 5 with a camera by a process different from the methods shown in FIGS. 6 and 7. In FIG. 8, it is assumed that the mark object MO4 and the mark object MO5 are extracted from the image taken by the camera. Then, the camera-equipped information device can grasp what kind of lens is attached to the camera 501 mounted on the camera-equipped information device 5 and what kind of focus setting is used for shooting. That is, since the angle of view is known, the angle of the object in the captured image can be known. Therefore, in the image taken by the camera 501, the camera-equipped information device 5 can detect in which position of the image the object is in which direction with respect to the camera-equipped information device.

In FIG. 8, it can be detected from the image captured by the camera that the angle between the mark object MO4 and the mark object MO5 is θ12. Further, the direction sensor of the information device 5 with a camera can know the direction of the image taken by the camera. Therefore, the position of the information device 5 with a camera can be obtained from the angles of the two mark objects and their directions. Here, the position may be obtained by using three or more mark objects instead of the direction of the direction sensor, or the position may be obtained by the combination of the direction of the direction sensor and the angle of the plurality of mark objects. The relationship between the setting of the camera 501 and the position of the object in the captured image may be stored in a memory (not shown) of the information device 5 with a camera.

FIG. 9 is a processing flow diagram of the camera-equipped information device 5 of the position information system using the camera-captured image in this embodiment. In FIG. 9, when the information device 5 with a camera is outside the complex facility 3, the facility map service application 515 is stopped, and the current position is grasped by the GPS sensor 502. In this state, when the GPS sensor 502 detects that it has come to the complex facility 3 (S10), the facility map service application 515 is activated (S11). The facility map service application 515 may be manually performed.

When activated, the facility map service application 515 downloads data corresponding to the complex facility 3 from the facility map DB 12 and the mark object DB 13 (the mark object is described as MO in the figure) of the facility map service device 1. (S12).

Next, the camera 501 takes a picture of the surroundings in the complex facility 3 (S13), an object of a significant size is detected from the images taken by the camera (S14), and the detected object is downloaded to the download DB 520. It collates with the image data of the object DB, and recognizes the matching object within a predetermined error range as a mark object (S15).

For the recognized mark object, the current position and the direction of the camera are obtained from the distance between the information device 5 with a camera and the mark object by the method described with reference to FIGS. 5 to 8 (S16).

It is confirmed whether the current position obtained in S16 is in the complex (S17). For confirmation, the positioning result by GPS may be used together. If the current location is outside the complex facility, the facility map service application 515 is terminated (S18).

If the current location is in the complex facility, the map data of the facility guide map from the facility map DB downloaded to the download DB 520 is displayed on the touch panel display 508 of the information device with a camera (S19). Further, graphic data such as a pointer indicating the current position and the direction of the camera obtained in S16 is superimposed and displayed on the facility guide map (S20).

When displaying the movement locus of the information device 5 with a camera (Yes in S21), the movement locus is held by the acceleration sensor 504, the direction sensor 505, etc. from the previously detected position information, and the movement to the current position is performed. The locus is also superimposed and displayed (S22).

From No or S22 of S21, it returns to the front of S13 as a standby state before performing position detection again.

As described above, according to the present embodiment, accurate position detection is possible by detecting the distance and direction from the mark object having known position data using the camera image taken by the information device 5 with a camera. Become. Further, as the mark object, existing signs, signboards, and the like can be applied, and there is an advantage that it is not necessary to install a dedicated object for position detection.

This embodiment describes a form of a camera of an information device with a camera.

FIG. 10 is a modification of the camera of the information device with a camera in this embodiment. In FIG. 10A, the external camera 51 is connected to the external terminal 522 of the information device 5 with a camera to take a picture with the camera. The external camera 51 is a camera capable of wide-angle shooting and is suitable for capturing a mark object. A camera capable of photographing a 360 ° surrounding can also be used as an external camera 51.

FIG. 10B uses a camera-equipped information device 5 including two cameras, an OUT camera 501b and an IN camera 501a. The OUT camera 501b and the IN camera 501a are provided facing in different directions by 180 °, and by using images taken from both cameras, it is possible to capture a mark object in a wide range.

FIG. 10C uses a camera-equipped information device 5 provided with a stereo camera 501c capable of photographing the same subject with two cameras and measuring the distance between the subject and the camera by the parallax of the two cameras. The distance of the mark object can be measured with high accuracy.

As described above, according to this embodiment, there are effects such as facilitation of capturing the mark object and improvement of the measurement accuracy of the distance from the mark object.

This embodiment describes an example in which the process related to the detection of position information is executed not by the information device 5 with a camera but by the facility map service device 1.

FIG. 11 is a block diagram of the facility map service device in this embodiment. In FIG. 11, the same functions as those in FIG. 2 are designated by the same reference numerals, and the description thereof will be omitted. The difference from FIG. 2 in FIG. 11 is that the camera image receiving process 107, the mark object extraction process 108, the position / direction detection process 109, the position information transmission process 110, and the map information transmission process 111 are placed in the lower layer of the facility map service program 105. It is a point having.

In FIG. 11, in the camera image receiving process 107, the camera image taken by the camera-equipped information device 5 of FIG. 12, which will be described later, is received. The mark object extraction process 108 extracts a significant object from the received camera image, collates it with the mark object DB 13 , and extracts the mark object. The position / direction detection process 109 uses the extracted mark object to detect the position of the information device 5 with a camera and the direction of the camera by the detection methods described with reference to FIGS. 5 to 8. The detected position of the camera-equipped information device 5 and the direction of the camera are transmitted to the camera-equipped information device 5 in the position information transmission process 110. Further, the detected map data around the position of the camera-equipped information device 5 is transmitted from the facility map DB 12 to the camera-equipped information device 5 by the map information transmission process 111.

FIG. 12 is a block diagram of an information device with a camera in this embodiment. In FIG. 12, the same functions as those in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted. The difference from FIG. 3 in FIG. 12 is that the facility map service application 515 has a camera image transmission process 523, a location information receiving process 524, a map information linkage process 525, and a map information download DB 526 in the lower layer.

When the user 6 tries to detect the current position, the camera 501 takes a picture of the surrounding area, and the camera image is transmitted to the facility map service device 1 in the camera image transmission process 523. Further, the location information receiving process 524 receives the location information detected by the facility map service device 1. Further, the map information download DB 526 downloads and holds a part of the facility map DB 12 as map data around the position of the detected information device 5 with a camera, and the map data and the current position information are stored in the map information linkage process 525. The superimposed display process is performed with, and the display is displayed on the display 508 with a touch panel.

As described above, this embodiment also has the same effect as that of the first embodiment. Further, since the facility map service device 1 is responsible for the processing process related to the position detection, the information device with a camera suitable for the demand for reducing the processing load of the information device 5 with a camera and reducing the power consumption is provided. Can be provided.

In this embodiment, as the map data of the facility map, an image obtained by taking a printed matter such as a facility guide map with the information device 5 with a camera will be described.

FIG. 13 is a guide map (hereinafter referred to as a print guide map) of the complex facility in this embodiment. The print guide map 7 is distributed, for example, at the information corner of the complex facility 3. In FIG. 13, 70a is a floor reference point indicating an upper left corner of the floor diagram of the print guide map 7, 70b is a floor reference point indicating a lower right corner of the floor diagram of the print guide map 7, and 71 is a QR code of a two-dimensional barcode. (Registered trademark). The QR code 71 describes how to access the URI of the facility map service device 1, which is the connection information of the facility map service device 1.

Generally, the user 6 looks at the print guide map 7, compares the store name written on the print guide map 7 with the signboards of the stores in the vicinity to be visually recognized, and guesses his / her current position to the destination. Make a move. On the other hand, in this embodiment, the print guide map is photographed by the camera and the facility map service device 1 indicated by the QR code 71 is accessed. In addition to the mark object DB 13 described above, the facility map service device 1 holds the guide map DB 14 described below.

FIG. 14 is a guide map DB 14 in this embodiment. In this embodiment, the facility map service device 1 of FIG. 2 described above is used by substituting the facility map DB 12 and the guide map DB 14. The database is not limited to the format shown in the figure, and may be any format as long as it has the same function.

In FIG. 14, the guide map DB 14 includes the facility name 141 and the reference point 142 as reference data, and among the items of 143, the floor of the upper left corner 70a of the print guide map 7 and the lower right corner 70b of the print guide map 7. The actual position data of the reference point is described as the difference from the reference point. The facility name 141 and the reference point 142 are the same as the facility name 131 and the reference point 132 of the mark object DB 13, indicating that they are the DBs of the same complex facility 3.

FIG. 15 shows a processing flow of the information device 5 with a camera in this embodiment. In FIG. 15, the same functions as those in FIG. 9 are designated by the same reference numerals, and the description thereof will be omitted. The camera-equipped information device 5 used in this embodiment is basically the same as the camera-equipped information device 5 shown in FIG. 3, but the processing contents of the map information linkage process 519 and the download DB 520 are slightly different.

In the processing flow of FIG. 15, the facility map service application 515 is started in S11, the print guide map 7 is photographed by the camera 50 (S31), and the facility map service device 1 described in the QR code 71 included in the photographed image is accessed. (S32), the related mark object DB 13 is downloaded and saved in the download DB 520 (S33). Hereinafter, since S13 to S16 are the same as those in FIG. 9, the description thereof will be omitted. It should be noted that the mark object arrangement information may be extracted when the print guide map 7 is photographed by the camera 50, and the mark object DB 13 may be downloaded instead.

In S34, it is confirmed whether the current position obtained in S16 is in the map. For confirmation, the positioning result by GPS may be used together. If the current position is off the map, the facility map service application 515 ends (S18).

If the current position is in the map, the information device 5 with a camera takes a picture, the floor reference points 70a and 70b of the print guide map 7 displayed on the display 508 are read, and the reference position of the display is determined (S35). Hereinafter, since S20 to S22 are the same as those in FIG. 9, the description thereof will be omitted.

FIG. 16 is an example of displaying the current position on the information device with a camera in this embodiment. In FIG. 16, 5 is an information device with a camera, 508 is a display with a touch panel, 7a is a print guide map photographed by a camera, and 70c and 70d are reference points of a print guide map similarly photographed by a camera. Also, 71a is a QR code taken by the camera.

FIG. 16 shows an example in S20 in which graphic data such as a pointer indicating the current position and the direction of the camera is superimposed and displayed on the print guide map 7a on the screen of the display 508. Here, the print guide map 7a is a camera. This is an image that is being shot and is being displayed on the display. In FIG. 16, the pointer 72 indicating the current position and the direction of the camera is superimposed and displayed like AR (Argument Reality).

In this embodiment, the upper left corner 70a of the floor map of the print guide map 7 and the lower right corner 70b of the floor map of the print guide map 7 download the mark object DB 13 by the QR code 71 of the two-dimensional barcode. The method of capturing the arrangement relationship of the mark object is not limited to this, for example, the print guide map is scanned to extract the characters and marks of the sign, and the relative arrangement relationship between them is obtained. This also enables similar processing.

As described above, according to the present embodiment, accurate position detection is possible by detecting the distance and direction from the mark object having known position data using the camera image taken by the information device 5 with a camera. .. Further, as the mark object, existing signs, signboards, etc. can be applied, and it is not necessary to install a dedicated object for position detection. Further, the image of the print guide map taken by the camera can be handled in the same manner as the map data without the need for the facility map DB, and the construction of the system is further simplified. Of course, the guide map is not limited to printed matter, and the same operation can be performed even if the floor map installed in the facility is taken with a camera.

Although the examples have been described above, the present invention is not limited to these, and for example, a part of the configuration of a certain example can be replaced with another example. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these belong to the category of the present invention. Further, the numerical values and messages appearing in the text and figures are merely examples, and even if different ones are used, the effect of the present invention is not impaired.

In addition, some or all of the functions of the invention may be implemented by hardware or software. Moreover, hardware and software may be used together.

1: Facility map service device 2: Network 3: Complex facility 4: Access point 5, Information device with camera, 6: User, 7: Print guide map, 321-330: Mark object, 10: Facility map service server , 12: Facility map DB, 13: Mark object DB, 70a, 70b: Floor reference point, 71: QR code, 14; Guide map DB, 72: Pointer, 105: Facility map service program, 107: Camera image reception process, 108 : Mark object extraction process, 109: Position / direction detection process, 110: Position information transmission process, 111: Map information transmission process, 501: Camera, 502: GPS sensor, 505: Direction sensor , 508: Display with touch panel 510: Mobile communication IF, 511: Wireless LAN IF, 512: CPU, 513: RAM, 514 : FROM, 516: Camera linkage process, 517: Mark object extraction process, 518: Position / direction detection process, 519: Map information linkage process, 520 : Download DB, 521: EXTIF, 522: External terminal, 523: Camera image transmission process, 524: Position information reception process, 525: Map information linkage process, 526: Map information download DB

Claims (12)

It is a position information system that uses images taken by the camera of an information device with a camera.
A means for holding a database of a data set consisting of image data and installation data of the mark object as a mark object in an indoor facility and transmitting a part or all of the database to the camera-equipped information device is provided. Has a facility map service device
The information device with a camera is a means for extracting an object having a significant size from a camera image taken by the camera and comparing it with image data in the database to identify a mark object appearing in the camera image. And a means for detecting the current position of the information device with a camera and the direction of the camera by using a combination of a plurality of mark objects shown in the camera image .
The means for detecting the current position of the information device with a camera and the direction of the camera is a mark object or a predetermined mark object in which a change is observed by taking a plurality of images taken by the camera from a plurality of mark objects shown in the camera image. A position information system characterized in that it can detect the current position and the direction of the camera by excluding mark objects photographed at a position lower than the height . The location information system according to claim 1.
The indoor facility is a complex facility having a service facility, a store, and the like.
The mark object is a position information system characterized by being a guide sign of a service facility, facility equipment, a panel showing a store name or logo, and a product displayed in the complex facility. The location information system according to claim 1 or 2.
The facility map service device holds a database of map data in the indoor facility, and includes means for transmitting a part or all of the database to the camera-equipped information device.
The camera-equipped information device is characterized by comprising a means for superimposing and displaying map data in the indoor facility, the detected current position of the camera-equipped information device, and graphics data indicating the direction of the camera. Information system. The location information system according to claim 3.
The facility map service apparatus includes means for holding a database of reference points of positions in the map data, and means for transmitting a part or all of the database to an information device with a camera.
The camera-equipped information device is a means for photographing a map in the indoor facility and superimposing and displaying graphics data indicating the detected current position of the camera-equipped information device and the direction of the camera on the photographed map. A location information system characterized by being equipped with. The location information system according to claim 1 or 2.
The means for detecting the current position of the information device with a camera and the direction of the camera includes a means for obtaining the distance between the substance of the mark object shown in the camera image and the information device with a camera. Information system. The location information system according to claim 5.
The means for obtaining the distance between the actual size of the mark object shown in the camera image and the information device with a camera is calculated by using the ratio between the size of the mark object in the camera image and the actual size of the mark object. A location information system that is characterized by being a camera. It is an information device with a camera
It has a CPU, a memory for executing programs, a network connection means, a camera, a display, and a memory for storing programs.
The memory for storing the program controls the camera and the process of downloading a database of a data set consisting of image data and installation data of the mark object using an object in an indoor facility as a mark object via a network connection means. The process, the process of extracting an object having a significant size from the image taken by the camera and comparing it with the image data of the mark object to identify the mark object appearing in the camera image, and the camera image . The current position of the information device with a camera is obtained by excluding the mark objects that have been photographed by the camera and changed or the mark objects that have been photographed at a position lower than a predetermined height from the plurality of mark objects shown in. It is characterized by storing a program having a process of detecting the direction of the camera and a process of superimposing and displaying map data, the current position of the detected information device with a camera, and graphics data indicating the direction of the camera. Information equipment with a camera. The information device with a camera according to claim 7.
The memory for storing the program includes a process of transmitting an image taken by a camera via a network connection means , a process of receiving position information of an information device with a camera, and a process of receiving map data around the position information. An information device with a camera, which stores a program having the map data and a process of superimposing and displaying the current position of the received information device with a camera. The information device with a camera according to claim 7.
The map data is map data taken by the camera.
The map data is an information device with a camera, which includes a reference point mark of a position and a mark including connection information of a facility map service connected by a network connection means. The information device with a camera according to claim 7 .
An information device with a camera that is provided with an external connection terminal and is used by connecting a wide-angle camera to the external connection terminal. The information device with a camera according to claim 7.
An information device with a camera that includes a plurality of cameras and identifies the mark object from images taken by the plurality of cameras. The information device with a camera according to claim 7 .
Information with a camera is provided with a camera capable of measuring the distance between the subject being photographed by the camera and the camera, and the current position of the information device with the camera and the direction of the camera are detected by using the distance measured by the camera. machine.

Images